Overview
This project involves producing a "morph" animation of different faces, computing the mean of a population of faces and extrapolating from a population mean to create a face caricature.
A morph is a simultaneous warp of the image shape and a cross-dissolve of the image colors. The warp is controlled by defining a correspondence between the two pictures. Correspondences mapping from eyes to eyes, mouth to mouth, chin to chin, ears to ears and other facial features will enable for smooth transformations.
In addition to creating a morphing animation between faces, I computed the mid-way face between images and calculated the mean face from the face dataset used by MIT's paper on "Style Transfer for Headshot Portraits" which appeared in SIGGRAPH 2014. The images from the faces dataset were taken from the Flickr dataset. Finally, for bells and whistles I created a morph sequence music video featuring student faces from students in the class, changing my gender and appearance with a mean target face, and creating a morphing video themed sequence of a subset of the 2016 democratic presidential candidates.
Defining Correspondences
Using the matlab cpselect tool, I defined pairs of corresponding points between images. Points were then saved once correspondences were defined. Then, these points were triangulated for use in morphing. I used the Delaunay triangulation technique, which is a good choice since it does not produce overly skinny triangles. Below is an example of the Delaunay triangulation:
Computing the Mid-Way Face
I decided to morph between my face and Emma Watson's face. Before computing the whole morph sequence, I first computed the mid-way face of these two images. This would involve: computing the average shape, warping both faces into that shape, and averaging the colors together. The main task in warping the faces into the average shape was implementing an affine warp for each triangle in the triangulation from the original images into this new shape. This involved computing an affine transformation matrix A between two triangles. A set of these transformation matrices were then used to implement an inverse warp of all pixels. Below are the original source image (my face), target image (Emma Watson's face), and the midway face between the source and target images.
source image (Me)
target image (Emma Watson)
midway face
The Morph Sequence
To produce the morphing sequence, I implemented a technique that warped between two images (im1 and im2) using point correspondences defined as described above and the triangulation structure. The technique also controlled shape warping and cross-dissolve, respectively. In particular, images im1 and im2 were first warped into an intermediate shape configuration then cross-dissolved. Below is the morphing sequence between my face to Emma Watson's face.
The Mean Face of a Population
Using a subset of annotated faces from the annotated faces dataset from MIT's paper on "Style Transfer for Headshot Portraits" which appeared in SIGGRAPH 2014 (as indicated on piazza), from keypoints already annotated on the data, I computed the average face shape of the images. The average face shape was computed for the whole population of images that I used, and subsets of the population which included only the female and only the male subsets respectively.
Then, each of the faces from the overall population I used were morphed into the overall average shape. Lastly, I warped my face into the average geometry of the female subset population face and warped the average female subset population face into my geometry. The female subset population was used as it aligns better to my facial geometry compared to the male subset population, or the overall population. Visualizations are depicted below.
image 1
image 2
image 3
image 4
image 5
image 6
image 7
population mean face
male population mean face
female population mean face
Faces morphed to the mean face
image 1 morphed to overall mean face
image 2 morphed to overall mean face
image 3 morphed to overall mean face
image 4 morphed to overall mean face
image 5 morphed to overall mean face
image 6 morphed to overall mean face
image 7 morphed to overall mean face
my face morphed to the female subset population face
female subset population face morphed to my face
Caricatures: Extrapolating from the Mean
I produced a caricature of my face by extrapolating from the population mean that I calculated above. I again used the female subset population as it aligns better to my facial geometry compared to the male subset population, or the overall population. Visualizations with varying warp fractions are depicted below.
female population mean face
me
warping fraction alpha = 1.5
warping fraction alpha = 2.0
warping fraction alpha = 2.5
warping fraction alpha = 5.0
Bells and Whistles
Changing My Gender and Ethnicity
I wanted to see how it would be if I transformed myself into a male NBA basketball player. As the source
image, I used my face. As the target image, I used the average male NBA face taken from 400 NBA players. Below are the
results:
source image (Me)
target image (average male NBA face taken from 400 samples)
