CS 194-26 Project 4: Face Morphing

Warping from Person A to Person B

First, we would like to be able to morph an image of one person's face to another person's face. For example, let us morph this man into this woman.

We do this by defining correspondences on each face, selecting points of each face that corrrespond to matching features of the faces (e.g. the left corner of the right eye, or the bridge of one's nose). On the above images, we would have correspondences like the following.

We then take the mean correspondence, i.e. the set of points that represent the mean between each pair of corresponding points from the two images. For the example correspondences, our means look like the following.

We then calculate a triangulation between these mean points, and in my case I just used the Delauney triangulation.

We use this triangulation from now on for all the following steps. We apply the triangulation to both of our images.

We then, to compute the midway face between the images, create a new image that will have the same geomentry as the midway correspondences / triangulation. For each triangle in each image, we calculate the affine transformation matrix that maps the source image's triangle to the midway image's corresponding triangle. Then, we create the new midway image by iterating through all the pixels in the new image's triangle, and grab its pixel value from the source image where the coordinates of the pixel value we select is determined by performing the inverse affine transform on our destination's coordinates. Because the inverse affine transform does not guarantee integral results, we interpolate pixel values depending on the nearest pixels to the desired source coordinates. We do this for both source images and average the two resulting midway images. We receive the following.

The correspondences and triangulation on the resulting morphed image are the following.

In order to get all the frames of the transition (not just the midway), we need to do multiple morphs. We do this by selecting the number of frames (e.g. 45 frames in my case) and then have target images with correspondence points that represent a weighted average of the two images' correspondence points and do a weighted average of colors as well. The weights of the averages are 0, 1/(n-1), 2/(n-1), 3/(n-1), ..., (n-2)/(n-1), 1 where n is the number of frames. The full morph sequence is displayed below.

I also did another morph from my friend Scott to me!

I had messed up the correspondence points to our photos originally so here's a mess up.

Population Mean

I also used the Martin dataset and calculated the mean face of the women in the set. They also all happened to be white, so this is the mean face of 22 white women. This was done by taking all their correspondence points, averageing them, morphing each face to the average, and then taking the average color of their faces. Here are some examples from the dataset.

And here is the mean of the 22 faces.

Here is me morphed to fit the geometry of the mean.

Here is the mean morphed to fit the geometry of my face.

Caricatures

We can also produce caricatures by taking a face, tagging corresponding points that match the mean face, and instead of morphing the face so that its points match the mean face's points, we morph in the opposite direction, i.e. we extrapolate. In other words, we morph our points to:

our points + alpha * (our points - mean points)

My caricatured face at different alpha levels are below.

Alpha = .5

Alpha = .75

Alpha = 1

Freaky!

Class Face Morph

We also organized a continuous face morph with some of my classmates (24 of us total). We all uploaded our own images, tagged correspondence points for our own images, and created a set of morphing frames from another classmate to ourselves. Here is the final product!

Change Gender / Ethnicity / Celebrity

I also wanted to change my gender, ethnicity, and celebrity status! I did this by finding a "base" image that represented my demographic: the average Taiwanese. Then I found "target" images: the average Taiwanese male (to change gender), the average French woman (to change ethnicity), and the average Chinese actress (to change celebrity). Then I did the following calculation to find my desired destination correspondence points (the points I wanted to morph my own image into):

destination points = my points + (target points - base points)

In a sense, for example, to change my gender, I subtracted the "femaleness" from the male to get "maleness", and added this "maleness" to me (who is female). I then morphed my geometry to the destination geometry.

For each of these changes, I calculated changes in geometry (the above procedure), color (morphing the target image to fit my unchanged geometry and averaging the color), and then both (changing my geometry, and morphing the target image to fit my changed geometry and averaging the color). Here are the results.

Here is me.

Here is the base mean Taiwanese female image I used.

Change of Gender

Here is the target mean Taiwanese male.

Here is my changed geometry.

Here is my changed color.

Here is both.

Change of Ethnicity

Here is the target mean French female.

Here is my changed geometry.

Here is my changed color.

Here is both.

Change of Celebrity

Here is the target mean Chinese actress.

Here is my changed geometry.

Here is my changed color.

Here is both.

Conclusion

For the geometry-changing operations, all of them look slightly strange. This is likely due to having correspondences that aren't the most accurate,base/target images that aren't all cropped the same, and having correspondences on the top of my head which might have been extraneous because the top of the head does not have many features that can signify gender or ethnicity so it adds additional constraints without improving results.