Project 3: Face Morphing

Calvin Tang | calvintang@berkeley.edu | 3031974882

Defining Correspondences

Similarly to the image aligning script from Project 2, I loaded both images and selected a number of corresponding points. I used matplotlib.pyplot.ginput to select the points on the images. The order of the points needed to be maintained between the two photos. I created a sort of map to keep track of the areas of the face I wanted as correspondence points. After some experimenting, I found that having more correspondence points led to better results in the morphing, especially with differing head shapes and hair styles.

After selecting my correspondences, I found the best triangulation for the images using scipy.spatial.Delaunay. Out of the options for triangulation available, I found that the Delaunay algorithm produced triangles of smallest size, which is desirable for the smoothness of morphing. From there, I used the produced triangulation on both images. You can see the triangulation of both images below.

Computing the "Mid-Way" Face

The morph function has a parameter called dissolve_frac, which I use to compute the expected morph shape of the image based on each triangle within the triangulation of the image. For each triangle in the triangulation, I compute the affine matrix between the input image's triangle and the average triangle location. From there, I perform the affine transformation to the triangle and replace the pixels in the input image to produce a new, morphed image. I used this post to reference how to compute the affine matrix. Below is the matrix operation:

Below, you can see the mid-way faces for both of my tests. A mid-way face is the blend of image when dissolve_frac is 0.5, so both images are equally present in the blend. To be honest, the shapes of the blends aren't very perfect. I think the reason for the odd shapes and features are because of the distinct differences between my faces and the other people I chose. You can even see faded versions of both photos where features are very different, such as Spencer's hair around the shape of my head. The morphing definitely looks smoother in animated form, but each intermediate image looks like some degree of the mid-way face.

Calbama

Spencvin

The Morph Sequence

For this part of the project, I created 50 frames for the final morphing animation. For this, I created a loop to generate a new image each time. During each step of the algorithm, I increment dissolve_frac by a small amount, which dictates how much of an image is present in the current morphing. For each step of the for loop, I computed the result of blending the input images based on the dissolve_frac for that step. I saved each image result into a result folder and compiled the frames as a GIF.

Below are my results for the morphing.

Calbama

Spencvin

My friend Spencer saw me working on this project and wanted to be included in my testing. I think the blending of this image wasn't that great because of the drastic differences between our head shapes. His curly hair did not blend very well into my short hair and our facial features are quite different. Nonetheless, he was still really pleased with the result!

The "Mean Face" of a Population

For this part of the project, I used this dataset. I played around with all of the images and observed the corresponding points. Using 200 faces (100 plain, 100 smiling), I was able to find the average faces for the population. On the left, you can see the average plain face for the population. On the right, you can see the average smiling face for the population.

For the sake of interesting discovery, I decided to work further with the female faces. Since the dataset included both normal and smiling image, I was able to get the average faces for both. Similar to the earlier parts, I found the average coordinates of correspondence points for both normal and smiling images. I used these sets of points to morph each of the individual images into this shape. Below, you can see the average faces for both female sets.

The results of these two parts were strikingly similar! I tried to see if they're exactly the same due to some error, but I think there are some slight differences in the two.

Faces Morphed to Average Shape

Here, you can see some of the faces in the dataset morphed to fit the shape of the average face. Some had pretty good results! Others, not so much.

I was pretty surprised to see the quality of both images. I would believe both were actual people, which is pretty funny. After some experimentation with morphing my face into the average female face, the results weren't so pretty! I would attribute this result with the fact that the photo of my face is quite a bit longer than the photos included in the dataset. As you can see below in the morphed image of my face, it seems really squished into shape. The reverse of the average face being morphed into my face is quite elongated as a result, as well. Below, you can find the original images on the right and the morphed images on the right.

Caricatures: Extrapolating from the Mean

Calvin to Average Female

For good measure, I wanted to try with a different image of me. Due to the focal length of my front camera, it didn't turn out quite how I wanted still.

Average Female to Calvin

Bells and Whistles: Music Video

For the extra portion of this project, I decdied to morph all the faces from the alternative album covers of SATURATION III by BROCKHAMPTON. BROCKHAMPTON is known for their many members, so I thought it would be the perfect opportunity to morph their faces in a music video. Even better, their music aids the animation quite cleanly. This was definitely my favorite part of the project!

Conclusion

This was definitely my favorite project so far! It was really funny to see the morphed images and I feel that the topics covered were much more intuitive than the other assignments. Now I feel like I have a pretty good idea of how to find affine matrices and apply transformations to images. I got a wide variety of results and got to play around with different images. I'm looking forward to the next project!