Project 3: Face Morphing

Simona Aksman

Contents


Part 1: Defining Correspondences

For the the first part of this project, I took photos of myself and my husband Kylan that were similar in terms of their aspect ratio, background color, and our facial positions. Then I defined 60 correspondences on both images and used Delaunay triangulation to subdivide the images into triangular sections that could later be used for morphing. The images below show the chosen correspondences with Delaunay triangulation applied. (Note, however, that the triangulation shown in the two images below is slightly different from the final triangulation applied, as the final triangulation was computed using the midpoint of the two sets of correspondences.)




Part 2: Computing the "Mid-way Face"

After computing triangulation on the midpoint of the two sets of correspondences, I iterated over each triangle to apply an affine transformation, which was defined on the vertices of the source and destination triangles. Computing the affine transformation involved transposing the correspondence sets, appending a row of ones to each set, and then applying some linear algebra operations: inverting the source image and left multiplying the destination image with the inverted source image. Once I had the affine transformation matrix for each triangle, I applied an inverse warp by inverting the affine matrix and multiplying it by the target image. I applied the warping function to both images separately and then applied alpha blending to cross-dissolve the warped images and get the "mid-way face" (below).

Simona

The mid-way face

Kylan




Part 3: The Morph Sequence

With the code from part 2, it was easy to generate a morph sequence. To do so, I paramaterized the warp and cross-dissolve and then looped over different values of the two parameters. Varying the warp changes the shape of the image, while varying cross-disolve changes the appearance of the image. See below for the final result.




Part 4: The "Mean face" of a population

For the next part of the project, I used images of 7 female Finnish computer scientists from the IMM database to generate an average female Finnish computer scientist face (see mean face, below). In addition, I computed the average facial shape and warped each of the individuals' faces into that shape, as given below.

Female 1

Female 2

Female 3

Female 4

Female 5

Female 6

Female 7

Mean face



Then I warped my face into the mean face's facial geometry and the mean face into my facial geometry.

My face

My face warped into mean facial geometry

Mean face

Mean face warped into my facial geometry




Part 5: Caricatures: Extrapolating from the mean

Using the population mean image I found in the previous portion of the project, I was able to produce caricatures of myself by accentuating the features on my face that differ from the mean. To do so, I first found the difference between my correspondences and the population mean's correspondences. This represents the deviation of my facial shape from the mean shape. Then I applied different multipliers α to this difference and added these back to the population mean. The α = 1 case is just the original photo of me. α < 1 presents a more "average" version of me, and when α > 1, the quirks in my facial geometry are accentuated, producing the caricature effect.

α = 0.5

α = 1

α = 1.5

α = 2




Bells and Whistles

In part 2 of the project I morphed myself into my husband. That morph was an example of both a change of gender and a change of expression. See below for the shape-only morph and appearance-only morph for that example.

Original image

w/ shape morph only

w/ appearance morph only

w/ both



Then I tried "aging" my baby photo.

Original image

w/ shape morph only

w/ appearance morph only

w/ both

And finally, see below for the entire "aging" sequence. People always say I look like my mom so I added a couple of photos of my mom at the end.




Reflections

This project was fun but challenging. I especially found it difficult to label the correspondences in a consistent way. If I could redo this project, I think I would try to write a more convenient utility function for labeling the correspondences. My process ended up being: 1. label correspondences on one face while running a screen recording of the process, and then 2. watch the screen recording to figure out how to label correspondences on the second face.