The goal of this project was to generate face morph animations, compute the average face from a population, and create caricatures using the mean we calculated. We did this by defining corresponding points on each face and using Delaunay triangulations and affine transformations to warp from one face to another.
Part 1: Computing the Midway Face
For this project, we used a picture of me for image A and a picture of my younger sibling for image B:
Left: Image A (me); Right: Image B (my sibling), Olive
The first step was to compute a halfway face between these two images. We did this by finding the average shape between the two faces' corresponding points (0.5 * A_pts + 0.5 * B_pts), calculating a Delaunay triangulation for the average shape, solving for the affine transformations for each triangle, morphing each face using an inverse warp, and averaging the colors together. In the end, we obtained the following result:
Halfway face
We can see that most of our facial features align, and that the face width is around the average width of the two images.
Part 2: The Morph Sequence
Next, we generated the actual morph sequence. In order to do so, we generated 46 frames numbered 0-45, where frame 0 is identical to image A and frame 45 is identical to image B. Images 1 through 44 were midway faces that were warped and dissolved by a fraction of i/45, where i is the image number. For example, for image 1, we warped both images to the shape 44/45 * A_pts + 1/45 * B_pts and calculated the color of the image to be 44/45 * A_color + 1/45 * B_color. Below is our final result:
Morph sequence from image A to B
Part 3: The Mean Face of a Population
Next, we used the Danes face dataset to calculate the mean face of the population. We did this by calculating the average shape of all the annotated points, morphing all images in the dataset to this shape, and averaging the color of all the images. Below is our final result:
Mean face of the Danes dataset
Using the same technique as in the previous parts, we morphed some faces in the dataset to this average shape (hover to see the original image):
We also morphed image A to the mean face, and the mean face to my face's shape:
Mean face of the Danes dataset
This didn't seem to work as well since my face shape differed significantly from the average face. Nonetheless, we can see that my face warped to match the mean face's features (longer face, larger lips, wider chin, etc.), while the mean face warped to match my face's features (shorter face, smaller mouth, wider set eyes, etc.)
Part 4: Caricatures
The final part of the project was to generate a caricature of image A by extrapolating from the population mean calculated in part 3. We did this by finding the difference d between the corresponding points in image A and the mean face, scaling d by some factor k, calculating a new face shape by adding kd to image A's points, and finally morphing the face into this shape. We chose k = 0.8, a relatively small value since my face's features were already very different from the mean face. Below is our result:
Caricature for k = 0.8
for which we can see that features that differentiated my face from the mean face (ie: angular and wider set eyes, shorter chin) are exaggerated.
Bells and Whistles
Morphing Music Video
For our first bells and whistles, we created a music video that morphs between fourteen contestants on the second season of the TV show Produce 101. We used the same technique used for the morphing sequence in part 2 and generated 45 frames between each pair of people.
Changing Ethnicity
For our second bells and whistles, we found a population mean for white American women on the web and used it to change my ethnicity. We experimented by naively cross-dissolving both images to only change appearnace, morphing my face without cross-dissolving to only change shape, and changing both appearance and shape to get the following results:
From left to right: Population mean for white American women; original image of me; only changing appearance; only changing shape; changing both shape and appearance
We can see that naively cross-dissolving both images doesn't work well since the two images' facial features aren't aligned. Only changing shape by directly morphing the image without cross-dissolving doesn't work well either since it distorts the shape without blending colors. Thus, the best result was obtained by changing both appearance and shape.
Class Music Video
For my third bells and whistles, I participated in the class video. For my part of the video, I used the same method outlined in part 2 to morph Nick Titterton's face to mine:
