Project 3: Face Morphing

Jiayue Tao, CS194-26-adl



Project Overview

Overview

In this project, I implemented face morphing with non-parametric warps. Extending from this approach, I also found the mean face of a sub-population, created morphes to the mean, and extrapolated from the mean to create caricatures.

Part 1: Mid-way Faces and Morphing Sequence

We start by defining the correspondence points between two face images, labelling key points of face data (eyes, mouth, outline). Then, we create a triangle mesh over these points, ensuring that the topology of the mesh on each image is exactly the same. We then can compute the middle-shape for each triangle. To fill in these middle-triangles, we employ inverse warping using the affine transformation we find using the three pairs of vertex mappings. After doing this procedure with each triangle in the mesh, we have our mid-way face! By changing the ratio between the two images from 0 to 1, we can create a morphing sequence that gradually changes from one face to the other while the facial features remain consistently matched the whole time!

Here is a mid-way face and morphing sequence between me and a Danish computer scientist. The morphing is not really successful because my glasses are not within the subspace of faces. (Sadly I cannot find any picture of myself without glasses on.)

Image of Me
Image of a Danish computer scientist
Mid-way face (scary...)
Morphing

Here is another example. These two faces belong in the same subspace, so the morphing is much more successful.

Triangle Mesh on Face 1
Triangle Mesh on Face 2
Mid-way face between two computer scientists
Morphing

For extra fun, I morphed the faces of my favorite K-pop girl group MAMAMOO. The morphing is not perfect because in these four photos the orientation of their heads are slightly different. Nevertheless, the result is pretty good, and I can watch this gif all day!!

Part 2: Mean of a sub-population & Caricature

Using this approach, I also found the mean face of a sub-population - bearded Danish computer scientists. The beard looks blurry and minimal because every face’s beard shape is slightly different.

Here are some bearded faces morphed to the bearded mean.

One bearded Danish computer scientists
Average face of a bearded Danish computer scientist
Morphed to the Mean
One bearded Danish computer scientists
Average face of a bearded Danish computer scientist
Morphed to the Mean

We can also use this mean bearded face to extrapolate and create caricatures. For examples, I created caricatures of these two bearded computer scientists. I did not use my own photo because I do not have a beard, and the result would not be so great…

Original
Caricature
Original
Caricature

Part 3: Bells & Whistles

One of my favorite painters of all time is Rembrandt. To me, he is the ultimate epitome of craftsmanship and rebellion. He once was the most popular and affluent painters in Amsterdam, owning his own art collection full of eccentric objects, but to pursue his more boisterous artistic style, he squandered his wealth and reputation away. He painted lots of self-portraits throughout his life, and I morphed a few of them in a sequence to showcase his change - from an aspirational youth, to successful bourgeoise painter, and finally to an aged but staunch artist.

In this morphing, due to the presence of hats and various hairstyles, the morphing is not perfect. Nevertheless, we can still get some inspiration from Rembrandt's adamant pursuit of arstistic excellence and creative expression.

Concluding Remarks

This project has been really fun! I got to create some cool morphs and also apply these techniques to themes that I enjoy and value. It definitely consolidated my understanding on warping, affine transformation, and the power of the mean.