In this project we morph one face into another, find the mean of a population of faces and create caricatures of our own faces using affine warping and some linear algebra.
The first step is to define a set of corresponding points on the two photos we want to morph into each other. I did this using cpselect
. The key to choosing the right points is to select pairs of points that correspond to the same features on each image (e.g. jawline, nose, eyes, hairline). I had to go through a bit of trial and error here to see which areas needed more points to make the morph look more natural. For example, I realized the nose and eyes were very important because without well-selected points in those areas, the resulting midway morph would have slightly misaligned features.
This brings us to our next step, finding this midway morph. To find this, we first take the average of both images' correspondence points and use the average points to find a Delaunay triangulization. Then, we do an inverse warp on each image to warp them both into the average shape and average the colors together. With our two warped images, we can simply cross-dissolve them together (in this case, using weights of .5 and .5 since we want the mid-way face). I realized the importance of adding in the corner coordinates to my matrix -- without the corner points, the resulting image has some artifacts/borders that you can see.
The idea behind this algorithm is to compute the affine transformation for every corresponding pair of triangles in the two images so that we can map every pixel from the source image to the target image. We use the provided mytsearch
function to find which pixels fall within which triangles. We then use this information for interpolation.
Extending the concepts from the mid-way face, now we can completely morph one image into another by using various weights for image 1 and image 2. We do this repeatedly with evenly spaced warp_frac, dissolve_frac
values for our frames in order to create a cool video morph!
Using the morphing function from above, we can generate the average face of an entire population! I used a subset of the Danes dataset. I used only the male faces in pose 1 (non-smiling) and discarded the images that had been corrupted to be only available in greyscale. I found the average shape of all of the faces and warped each individual face to match the geometry of this average face. Here are some examples of the individuals before and after morphing to the average geometry:
Here is the resulting average face!
Now, let's do some warping to see how my face would look if I were a Danish man, and vice versa.
Now that we have the population mean that we calculated in the last part, we can try to exaggerate those features by simply extrapolating. We do this by morphing with ratios that still sum to 1, but include a value > 1. For example, we can morph with the ratios -0.5 and 1.5 to exaggerate the appearance of Danish features in my face. Here are my results:
With this, we can change my gender! Here's what I would look like as a Korean man: