CS194-26 Project 3: Face Morphing

Defining Correspondences

In freshman year, while watching Game of Thrones with a friend, they made a comment that I had the same face shape as Sansa Stark. This was quite possibly the greatest compliment I had ever received. Over the next few days, I went from hating my face shape to really appreciating it, solely because of this comment and because of her character in the show. Therefore, when this project was introduced, I knew immediately that I wanted to morph my face with that of Sophie Turner, so I began the hunt for a similar photo. Before beginning the programming part of the project, I selected a photo of Sophie Turner that was quite similar to mine and resized it using an online image resizer to have the same dimensions.

Original Nancy Original Sophie

I wrote code to get points of each image with Python's ginput and then ran this in terminal so the images would display for point selection. I then selected one point on each image at a time, alternating between the two images until I had selected 39 points. This was the order of my 39 points:

  1. left eye
  2. right eye
  3. outer left eye
  4. outer right eye
  5. inner left eye
  6. inner right eye
  7. top of left ear
  8. top of right ear
  9. bottom of left ear
  10. bottom of right ear
  11. middle of hairline
  12. bottom of chin
  13. tail of left eyebrow
  14. tail of right eyebrow
  15. peak of left eyebrow
  16. peak of right eyebrow
  17. beginning of left eyebrow
  18. beginning of right eyebrow
  19. left point of nose
  20. middle of nose
  21. right point of nose
  22. left of lips
  23. right of lips
  24. top of lips
  25. bottom of lips
  26. top left corner
  27. top right corner
  28. bottom left corner
  29. bottom right corner
  30. left jaw point
  31. right jaw point
  32. left hairline midpoint
  33. right hairline midpoint
  34. left top of neck
  35. right top of neck
  36. left bottom of neck
  37. right bottom of neck
  38. top of hair
  39. collarbone hollow part

After selecting the points, I calculated the midway points (average of each pair of points between the two images) and computed the Delaunay triangulation on these midway points. Here is the same Delaunay triangulation on the two images:

Delaunay triangulation of Nancy Delaunay triangulation of Sophie

Computing the "Mid-Way Face"

I started out writing a for loop over each triangle in the Delaunay triangulation and then getting each pixel inside that triangle using the polygon function. I warped both faces into the midway shape and averaged the colours together to get the final midway face. However, while writing code for the Morph Sequence, I put the entire for loop into one function and that ended up being my morph function. I came back and changed my code for the midway face so that it used the morph function, with values 0.5 for both warp_frac and dissolve_frac. This made sense and gave me the perfect midway face.

Original Nancy The 'Mid-Way' Face Original Sophie

The Morph Sequence

I started out creating the morph sequence by using the midway points for all 45 frames. I soon realized that there was too much of a jump between the first frame and second frame, even though the second frame was supposed to be 44/45 me——so there shouldn't have been that great of a difference. After reading lots of Piazza comments, I realized that I wasn't using warp_frac at all. Instead of taking the midway points for each frame, I changed set warp_frac equal to dissolve_frac, so that at the second timestep, the image was using 44/45 my points and 1/45 of Sophie's points. This made the transition between each frame so much smoother, as all the points were changing slightly each time as well as the cross-dissolve. Here is my final morph sequence gif:

Nancy to Sophie gif

The "Mean Face" of a Population

I loaded in the points (.asf) files of 36 Danes. I used only their images where they were looking straight at the camera. I then put all these points into a large dataframe and took the mean of all the X points and then the mean of all the Y points to find the average points of all the faces in the population. Then, I scaled these points up to match the image size of the Danes (640 x 480) and called a Delaunay triangulation on the resulting points. This gave me a visualization of the average shape of all 36 faces. Here is the average triangulation, first by itself and then shown on a couple of Danes' faces.

Mean Face Shape Mean Face Shape on First Dane Mean Face Shape on Second Dane

Next, I modified my previous morph function and wrote a warp function that does everything except cross-dissolve the colours of the images. Using this warp function made it easier to morph each face in the dataset to the average shape without worrying about doing anything else. I loaded in each image in a for loop and also retrieved its points from the previous dataframe in the same for loop. From there, I called the warp function. Here are some examples.

First Warped Dane Example Second Warped Dane Example Third Warped Dane Example Fourth Warped Dane Example Fifth Warped Dane Example Sixth Warped Dane Example Seventh Warped Dane Example Eighth Warped Dane Example

After morphing each face to the average shape, calculating the average face of the population was really easy, as all I had to do was take the mean of all these warped faces. I gave each a weight of 1/36, and summed them up into one final average face.

Average Face of Dane Population

Next, to warp my face to the average geometry, and the average face to my geometry, I resized my image and the average face image to match each other, and then selected corresponding points on the average face in the same order as my points. I selected the points in terminal, and then pasted the points in Jupyter. I then warped my face to the points of the average face (average face geometry) and warped the average face to the points of my face (my face geometry). Here are the original images followed by the results.

Original Nancy resized Original Dane average resized My face warped to Dane average geometry Dane average face warped to my geometry

Caricatures: Extrapolating from the Mean

To extrapolate from the mean and create a hyper-dane, I used the formula from lecture: -0.5P + 1.5Q. This was easy to do by using the same midpoints formula earlier, but with different weights than 0.5 and 0.5. The weights were -0.5 and 1.5 in this case, per the formula. After finding the new extrapolated points, I warped my face to that shape, and TA-DA! Here is me in the form of a hyper-dane:

Me as a hyper-dane!