Introduction

The fundamental basis of this entire project is focused on image warping. So if I have an image A, which in this case is a face, how can I morph that image to a similar feature image or face B. In order to do so, we can essentially create a mesh around both images and try to warp each piece of the mesh. Mathematically, this can be done using affine transformation matrices. We can essentially change the basis of each triangle of the source to match that of the target.

Defining Correspondences

The first step of this process is defining correspondences. This means we'll need to define points on each face which correspond to important features. This way we can easy form triangles and warp the image. The more points, the better our transformation will be. Here's what the points look like on the face and the respective Delaunay triangulation.

  • Me

  • Kanye

  • Me

  • Kanye

  • Average

  • Me

  • Kanye

  • Average

Mid-Way Face

After getting the correspondence points, we can use them to do Delaunay triangulations as shown above. But instead of using those triangulations, we can instead average the points first, then find the triangulation of the average instead. Then, using that triangulation, we can form triangles on the source and target images so the triangle and points are consistent. Then like above we find the affine transformation needed to tranform the source triangle to the average triangle. In order to do so we can actually do an "inverse warp" by instead initially deciding which pixel we want to modify on the target, and then inverse affine transform to find the original pixel value that we want from the source. That way we can fill in values easily since the warp points won't initially be discrete. Then we can average the source to average and target to average to get a mid-way face.

  • Me

  • Kanye

  • Cody West

Drake and Kanye

  • Drake

  • Kanye

  • Aubrey West

Morph Sequence

Instead of averaging the points and colors, we can actually just do a fraction and instead do 45 frames using the following: alpha * source + (1 - alpha) * target.

  • Cody West

The "Mean face" of a population

Now what we can do is see how my face is compared to the mean population of white males. We use the Danes dataset and only use the non-smiling white males. Then using the same process above, we average the provided points of all the photos and morph them into a single photo which is shown below:

  • Mean Face

Example Morphed Faces

  • 16-1m

  • 20-1m

  • 32-1

Other Morphs

  • Me

  • Average

  • Both

Caricatures: Extrapolating from the mean

We can also show extreme features by instead using the formula -alpha * source + (1+alpha) * target.

Bells and Wistles

Using the mean population above (white males), can we show a full morph into making me white? As an extra we show just morphing the geometries of both the average and myself, morphing just color, and morphing both geometries and color.

  • Average Geometry

  • My Geometry

  • Average Color (Changing my race)

  • My Color (Changing the average race kinda?)

Reflection

The project was incredibly fun and interesting. Definitely the biggest struggles came from figuring out why the morph wasn't working. A serious bug that took awhile to figure out was the flipping of the x and y pixels. The morph looked fine but then once the mid way face was outputed the morphs didnt align. It almost seemed like the morph was going in the wrong direction which is how I realised it was a pixel swap issue. Also the render became a serious issue, which is why the morph sequence video being frames was a pain. The average face also took awhile since there were 30 photos. Overall however there were incredibly fun results.