# Face Morphing

## 1. Morph Sequence

We can morph images together by first averaging the shape, and then the color. Shape averaging is achieved by averaging the coordinates of a set of correspondence points, e.g. nose-to-nose, mouth-to-mouth, etc. These correspondence points are shown below, along with Delaunay triangulation:

After averaging these correspondence point locations, we can perform a Delaunay triangulation, and then apply affine transformations on the triangles in the original images to transform them into the new shape. Note that there is some distortion due to my hair not matching Clooney's very well, as well as covering my eyebrow.

The specific implementation actually uses the inverse transform, and finds the interpolated pixel values from the original image given the coordinates in the average shape. Then we cross-dissolve the colors (a.k.a. average the color values) to get a morphed image.

In addition, we can take a weighted average when warping and cross-dissolving to get a morphed image that is more similar to one or the other of the original images. By computing morphed images with weights of $$(t, 1 - t)$$ for $$t$$ in range $$[0, 1/44, 2/44, \dots, 43/44, 1]$$, we can produce a 45-frame morphing video:

## Mean Face

We can expand upon the method above to calculate a mean face of a population. Using a dataset of Danish faces, choosing the male faces in the same pose (Pose 1: full frontal face, neutral expression, diffuse light), we average the correspondence points of all 33 faces to an average shape, warp each face into that average shape, and then cross-dissolve the pixel-by-pixel color values of all these warped faces.

Some examples of faces warped into the average shape. Note that the foreheads sometimes don't fit well with the rest of the image: this is because the provided annotations don't include many points on the upper half of the face. I tried to mitigate this issue by adding points on the corners of the images as well.

In addition, my own face warped into the average shape, and the average face warped into my shape. Note that this uses my own annotation schema seen in the earlier morph sequence, rather than the schema from the Danish dataset.

## Caricatures

We can also use this shape warping technique to create caricatures: consider the face shape vectors $$\vec u$$ and $$\vec v$$. Then we can define $$d = \vec u - \vec v$$. Normally, to warp, we take the average face $$\vec v + \vec d / 2$$. But we can overemphasize the effects of $$\vec d$$ and instead compute the shape $$\vec v + 3 \vec d / 2$$, to get a caricatured face.

Below, my face caricatured in the direction of the average Danish face.

Note that this is just a shape warp without color cross-dissolving.

## Bells & Whistles

I also aligned my own face to an average Asian female face found on the internet, using the same correspondence point schema as in the morph sequence.