• Project Intro
  • Defining Correspondences
  • The Mid-Way Face
  • The Morph Sequence
  • The Mean Face
  • Caricatures

Face Morphing

Karl Cempron: CS194-26-AEG

The goal of this project is to produce a visual morph between two images. In this particular project, I define a correspondence between two faces, mapping facial features such as the eyes, mouth, nose, and hair shape. Along with the morphing of two faces, I find the mean face of a given population to make comparisons with my own facial features.

Can you guess who I morph into?

Finding Correspondences

Given two images, we define the pairs of corresponding points by hand, specifically targetting features that exist in both faces. Ideally, having more corresponding pairs will create a better fit of the morph from image A to image B. In this example, I chose myself as Image A and George Clooney as Image B.

Image A

Image B

Here are the triangular meshes for each image using 60 points. Since the exterior of our face shapes matches already, I decided to focus primarily on the internal features such as the lips and eyebrows.

Image A Mesh

Image B Mesh

The Mid-Way Face

Once the correspondences have been obtained, I compute the Mid-Way Face of the two images by doing the following:

1. Compute the weighted average shape for some given weight t that ranges between [0, 1] and each point in both image A and B. In this case, the value I used was t = 0.5 .
The equation used was: average_shape = t * image_A_points[i] + (1 - t) * image_B_points[i]

2.Given the points for Image A, Image B, and the Average Shape, we use Delauney Triangulation to obtain a set of triangles for the corresponding points and calculate the inverse affine matrices for each triangle.Specifically, computing the affine transform from the target (in the average shape) back to the source image. This was done for each triangle in image A, and then for every triangle in image B. I then warped all the triangles from image A and B to the triangles in the Average Shape to obtain a warped image.

3.The last step was to cross-dissolve the colors by finding the weighted color averages of Image A and Image B. Prior to this, I used Photoshop to alter the colors so that the same weight of t = 0.5 could be used.
The equation used was: mid_way_face[x, y] = t * image_A[xA, yA] + (1 - t) * image_B[xB, yB] for each (x, y) pair in the Average Shape and the corresponding mapped pairs found from step 2.

Image A

Mid-Way Face

Image B

I realized that the morph from myself to George Clooney was too seamless so I wanted to challenge my morph with a face that had different facial features. Take a look at the corresponding morphs along with a special bonus morph on the next section!

Image A

Mid-Way Face

Image B

It's Morphing Time

Using the algorithm used to find the Mid Way Face, we can vary the value of t between [0, 1] by a distribution of 45 frames. After obtaining 45 T Way Faces ,I take each t as a timestep and create a gif of all 45 frames. The results are as follows:

Since I used 60 points for this morph and had similar facial features, it is very difficult to tell exactly at what point I turn into George Clooney and vice versa.

While the two faces in this example were quite different, it goes to show that using 70 points will still create a tight warp. I found it quite interesting how the morphed image's forehead presented the most change.

Here I try to break the laws of physics by morphing myself into Morty Smith. Let's just say that you won't sleep at night after this one.

"There's an infinite number of realities Morty." - Rick Sanchez

The Mean Face

Using the algorithms previously mentioned, we can take a subset of faces of a given population and find the average/mean face. In this section, I chose to use the Danes Set which contained 30 male faces and 7 female faces. To obtain the mean face, I did the following:

1. Compute the average face shape of the set of faces given the set of correspondence points for each face.

Equation: average_face_shape = sum(face_correspondence_points) / number_of_faces

2. Morph each face into the average shape we derived from step 1.

3. Compute the average face as follows: average_face = sum(morphed_faces) number_of_Faces

The result can be seen below. Due to the uneven distributation of male to female faces, the average face appears to have a more male like appearance.

Here are some specific faces from the Dane Set morphed into the average face. Major changes that occurred were in the overall face shape of people. Person A had a thin face and as a result had his face stretched in the morph. Person B had many noticable changes since her face was wider but also her eyes and mouth were reshaped. Person C's changes were more noticable around the hair line.

Person A

Person B

Person C

Averaged A

Averaged B

Averaged C

I also morphed my face to the average shape and morphed the average face to my face shape. Here are the results:

My Face morphed into the Dane shape

Average Dane Face morphed to my shape

Caricatures

By taking the difference of my face shape with that of the average Dane shape and multiplying it by some scalar t, we can then add it back to the average Dane shape to get a new shape that resembles that of a caricature.

The equation for the caricature is as follows: caricature_shape = t * (imA_points - average_points) + average_points.. Notice that when t < 0 my face gets squished while t > 1 will expand my face.

t < 0

t = 0

t > 1