Face Morphing

Face morphing is a very aesthetic and fun tool for general purposes. It was used on Michael Jackson’s “Black and White” music video clip as well as in various fun Instagram Filters. The idea for this algorithm is to draw points that highlight our features and create subdivisions relative to those. Once we have that, we can start morphing two faces, since we have correspondences in both surfaces. The following sections will explain it a little bit better.

For the first two sections, I decided to use an image of myself and Chris Hemsworth. The images are the following:

Once we have these images loaded, I drew a few points on facial features such as the nose, mouth, eyes and ears. Using Delaunay Triangulation, I specify the triangle surfaces for my image and apply to both of them. We end with this configuration:

With these triangles set, we can create the mean configuration between both images and get the mean triangles. Going through each triangle in both images and Transforming each coordinate into a coordinate inside the mean triangle, we can get a fine result. For efficiency purposes, we don’t use a for loop in the transform. Instead of that, we get every coordinate from the specific triangle inside a matrix and apply an Affine transform matrix. The results follow.

Having the Mid-way face process clarified, creating a morphing sequence is pretty straightforward. Setting a number α as interpolation handler, we can simply weight out the triangle averaging and the color blending. Thus, my morphing sequence goes as follows:

Finally, I downloaded the Image database from the Technical University of Denmark - DTU to compute a population’s mean face. To do that, we calculate the average triangles and adjust every picture to it, finalizing by blending all the colors together. This population consists of 40 people.

Now, tweaking the images so they are the same size and ratio as my image, we can get the following transforms:

Now that we have the averages calculated, let’s find out what are my unique features using that as reference. By extrapolating the morphing algorithm, we get this result:

Considering that my face isn’t really aligned to the mean pictures and I used a Delaunay triangulation instead of the given triangulation from the database, we get that triangle artifact due to triangles being on top of each other. I added a quick fix by removing all the signals greater than 1 and replacing them with the original image’s signal.

So to demonstrate this in a better way, we can see that in one of the samples we have from the population:

Of course it comes a point in which even the faces in the mean can have the triangles stacked by extrapolation, which is what happens in the last ones.