Face Morphing

Andrew Vo

UC Berkeley COMPSCI 194-26, Fall 2020

Learn more


The goal of this project is to create a morphing between two images, compute an average face from a population of faces, and create caricatures of images by extrapolating pictures from a population mean. A morph is a warp from one image shape to another and a cross-dissolve of the two images' colors. To warp between two shapes, we define pairs of corresponding points between two images such as eyes to eyes, nose to nose, and chin to chin. These points tell us where the two images should line up to create a smooth transformation.

Defining Correspondences

I used <ginput> from Matplotlib's Pyplot collection to manually define pairs of corresponding points between two images. Using these points, I defined a triangular mesh over the images using a Delaunay triangulation which will allow us to warp between the two images in the next section. An example of this process can be seen on an image of myself and UC Berkeley lecturer Nicholas Weaver. I selected 44 correspondences between the images and subsequently computed the Delaunay triangulation on the set of points.

Andrew Vo
Nicholas Weaver
Delaunay triangulation of myself
Delaunay triangulation of Weaver

Computing the "Mid-Way Face"

The mid-way face of the image morph is a warp from the two input images into an average face shape and an interpolation of the colors. To compute the mid-way image of the morph, I averaged the pairs of correspondence points for the two images and created an average Delaunay triangulation of the averaged points. Next, I warped the image of myself and the image of Weaver into this average shape. The process of warping both images into the average shape involved iterating through every triangle in the average Delaunay triangulation and computing the inverse affine matrix transformation that transforms our averaged points into the original input correspondences. I then used the original input coordinate points and put them through an interpolation function which returns pixel values for all pixels in the average triangle. These values are equally cross-dissolved from my image and Weaver's image. The result of this algorithm is shown below.

Mid-way image

The Morph Sequence

To create a smooth morph animation between Weaver and myself, I used the same algorithm from the previous section and added an additional two parameters <warp_frac> and <dissolve_frac> which take on values from 0 to 1. <warp_frac> controls the intermediate face shape configuration and is a weight on the point correspondences. For example, a <warp_frac> of 0 means that the intermediate face shape is equal to the face shape of Weaver's image. <dissolve_frac> is a controlled weight on how much of the color values from the input images are interpolated into the intermediate image. If <dissolve_frac> is 0.5, the color value is averaged between the image of myself and the image of Weaver. In general, <warp_frac> = <dissolve_frac>. Shown below is a gif of the morphing process containing 50 images where <warp_frac> and <dissolve_frac> take on values from 0 to 1 with a step size of 0.02.

Morph Animation

The "Mean Face" of a Population

In this section of the project, I compute the average face of an image dataset. The dataset that I used was the images of Danish computer scientists which can be found here. Each image in this dataset contains an image of a Danish computer scientist with 58 labeled correspondence points. Using these points, I was able to calculate the average face shape with a similar process as done in the second section and morph all images into a single average face of the population. Shown below are some images morphed into the average shape and the average face of the population itself.

Image 1
Image 2
Image 3
Image 4
Morphed Image 1
Morphed Image 2
Morphed Image 3
Morphed Image 4
Average Danish Computer Scientist

Just for fun, let's see how an image looks warped into the average face and vice versa. Below I warped a picture of Lebron James into the average face and also warped the average face into Lebron's face. I used 43 pairs of correspondence points for the images.

Lebron James
Average Danish Computer Scientist
Delaunay Triangulation of Lebron James
Delaunay Triangulation of Average Danish Computer Scientist
Lebron James morphed into Average Danish Computer Scientist
Average Danish Computer Scientist morphed into Lebron James

Caricatures: Extrapolating from the Mean

Caricatures enhance the differences between an image and a population mean. We can produce a caricature of Lebron James by taking the difference between James' correspondence points and the average shape correspondence, multiply this difference by an alpha value (α), and add it back into the original image of Lebron James. I then morph the original image into the caricature shape to produce the final result. Here are some caricatures of James at different alpha values.

α = 1.25
α = -1.5
α = -1.25

As we can see, the caricatures at different alpha values enhance Lebron's head position. This may be because the original faces are not aligned vertically.

Bells & Whistles

In this part, I will change the gender of my friend's face. I used an image of the average East Asian woman's face and an image of my friend Félix. I will use the same process as shown in the mean face section to morph my friend into a woman. First, I will show just his face shape morphed into the average East Asian woman's face shape. Next, I will only morph the appearances of the images. And finally, I will morph both shape and image for the complete transformation.

Average East Asian Woman
Shape Morph
Appearance Morph
Complete Morph

Final Thoughts

Overall, I found this project to be very enjoyable and had fun producing the weirdly shaped morphed images. Creating the morph animation was the most difficult part for me because I had to derive the entire process of image warping and cross-dissolving the image colors. Despite this, I learned a lot by completing this project and hope to learn what other cool things the field of computer vision has to offer!