Face Morphing

CS 194-26 Project 4

cs194-26-aek

---

Introduction

For this project, I explore how to produce a morph between two images A and B. Face morphs work particularly well for this purpose, since humans have easily identifiable keypoints on their faces (eyes, nose, mouth, etc). To produce the face morphs, I use affine transforms and Delaunay triangulation to interpolate the geometry between faces. I can also cross-dissolve the actual pixel values in each of the faces to achieve a smoother look. In addition, I compute the mean face of a population using the same affine transformations and try to extrapolate from the mean faces to produce caricatures.

Defining Correspondences

To achieve face morphs, we first need to define pairs of corresponding points. I chose to define these points by hand. I then used these points for triangulations, on which I can perform affine transformations. I chose to use a Delaunay triangulation to avoid overly skinny triangles.

---

Computing the "Mid-Way" Face

For this part of the project, the goal is to compute a "mid-way" face for the two faces. In general, to compute a "t-way" face, we compute a weighted average on each of the faces where t is in the interval [0, 1]. Doing this involves three steps:

1. Computing the average face shape by adding the two correspondence points together using the formula avg_shape = t * imA_pts + (1 - t) * imB_pts
2. For each triangle in the Delaunay triangulation, we must compute the inverse affine transform. This will be used to transform from the target, which in this case is the average shape, back to the source image.
3. Next, we cross-dissolve the colors together, using a similar weighted average from the first step: avg_pixels = t * imA_pixels + (1 - t) * imB_pixels

To compute the mid-way face, we simply carry out these steps for t = 0.5. The results of the mid-way face between these two are below.

---

The Morph Sequence

To create a morph sequence, we can vary the parameter t and create a sequence of images with different values of t. The algorithm is the same as above, but with different values of t. We can use the photos generated to create a gif:

---

The "Mean Face" of a Population

For this portion of the project, I used a collection of images of 38 researchers from Denmark and computed the average of all of their faces. I did this by first computing the mean of all the keypoints of the faces, and then constructing a Delaunay mesh from these points. I then morphed each individual's facial geometry to the mesh, and then averaged all of the faces' pixels together to get the mean face. The face I computed is below:

I also experimented with warping my face onto the average geometry and ended up with the results below. It's interesting to note that my smile in the warped image is a little more subdued because the mean face has a neutral facial expression.

Here are the results of the mean face warped onto my geometry. Notice also that the mean face smiles a bit more when it's warped, since I was smiling in my original image.

---

Caricatures: Extrapolating from the Mean

For this part, I made caricatures of the researchers' faces by extrapolating from the mean image. We can extrapolate by taking the difference between the mean face's correspondence points and the researchers' correspondence points and then magnifying this difference. The alpha we use to scale the difference by can achieve different results. This resulted in some very flattering photos:

---

Bells and Whistles: Changing gender

For this section, I decided to warp Katy Perry's face onto an average male face to see what she would look like as a male. We can use the same techniques as in the previous parts to morph her face into a male face.