Face Morphing

Ron Wang

CS 180, Fall 2023, UC Berkeley




Part 1. Defining Correspondences

In this part, I defined pairs of corresponding points on the two images by hand, using an online labeling tool. This process is largely manual, so one of the improvements to be implemented in future projects will be using machine learning methods to automatically select corresponding points.

I asked my friend to send me a professional headshot - because both of our photos are taken with a white background, warping and morphing was very smooth.

Original Photo (Me)

Original Photo (Friend)

Delaunay Triangles (Me)

Delaunay Triangles (Friend)





Part 2. Computing the "Mid-way Face"

For this part, I wrote a computeAffine API that computes an affine transformation matrix A between two triangles. Here are the original A and B images, as well as the mid-way face I got.

Original A

Original B

The "Mid-Way" Face
The results were surprisingly good - you can observe features from both faces in the "mid-way" face.





Part 3. The Morph Sequence

In this part, I created a smooth morphing sequence to show the transition from image A to image B.

This is made possible possible by adjusting the warp_frac and dissolve_frac parameters in the morph API I defined.

Morph Sequence (GIF)





Part 4. The "Mean face" of a population

In this part, I used the freely available IMM Face Database, a dataset consisting of 240 annotated monocular images of 40 different human faces (from Denmark).

This first graph below shows the average face shape computed from the images (as a representation of the Danish population).

Average Face Shape
Here are some examples where each face in the dataset is morphed into the average shape.

01-1m

10-1m

12-1f

25-1m

My face warped into the average geometry:

The average face warped into my geometry:

Correspondences:






Part 5. Caricatures: Extrapolating from the mean

Finally, I can produce a caricature of my face by extrapolating from the population mean calculated in the last step. To make this work better, I selected a gender-specific (male) mean.

You can see a gradual transition as I increase the control parameter, warp_frac. dissolve_frac is hard-coded to 0.6 as it achieves the best results.

warp_frac=0.2

warp_frac=0.5

warp_frac=0.8

warp_frac=1.0

warp_frac=1.2

warp_frac=1.5

Bells & Whistles: Changing Smile & Gender

I wasn't smiling in the photo, so let's try changing that. Here's a shortcut to do this: I utilized the other subset of the IMM Face Dataset, where participants were all smiling. So this means I'm actually computing the Danish version of me smiling. Nevertheless, the results are decent and they are shown below:

warp_frac=0.8

warp_frac=1.0
Oops. Now I'm happy again.

Using the same idea, I can change the gender of (the Danish version of) my face.

warp_frac=0.8

warp_frac=1.0
Here's just one final GIF to showcase the results: