face morphing

Nadia Hyder

OVERVIEW

In this project, I produced a morph animation of my face into someone else’s face, computed the mean of a population of faces, and extrapolated from a population mean to create a caricature of myself. 

 

MORPHING

For my first morph, I used a picture of myself and the provided image of George Clooney. I resized the two images to be the same size, defined a correspondence between the two images mapping eyes to eyes, mouth to mouth, chin to chin, etc. for smooth transformations. I then computed the mid-way face, triangulated the images by correspondence points using Delaunay triangulation, and finally warped the images using inverse affine transformations to produce the morphing sequence.

 

DEFINING CORRESPONDENCE

To morph the 2 images together, I first had to define pairs of corresponding points (for facial features) on the two images. I began by selecting points manually using matplotlib.ginput, but this caused issues in later steps (triangulation). To improve performance, I used dlib’s facial landmark detector which estimates the location of 68 (x,y) coordinates that map to facial structures. These are the input images and their correspondences:

 

 

 

I then found the triangulations of the correspondence points using Delaunay triangulation.

 

 

FINDING THE MIDWAY FACE

In order to morph the images together, I had to define an affine warp function to warp one triangle to another. I used inverse warping, applying the inverse affine transformation to every coordinate in the second image. I used inverse/ backward warping because otherwise, forward warping produces “holes” when a pixel lands between 2 other pixels. Inverse warping on the other hand begins from the destination image and uses interpolation to produce better results.

The following are the midway faces produced by applying inverse warping. The final midway face is produced by averaging the two warped images.

 

 

MORPH SEQUENCE

To find the final 45-frame morph sequence, I warped 45 midway images and interpolated at every level, then stitched the images into a gif with a frame rate of 1/30 fps.

 

 

 

 

THE “MEAN FACE” OF A POPULATION

I used the Danes dataset to find the “mean face” of Danish women. I did this by first finding the average shape of all the images, then used my inverse warping function on the triangularized images to the average shape, to create the mean face.

Here is the mean face achieved from morphing:

 

 

 

Here are a few examples of how the input images were warped in the process of creating the mean face:

 

 

 

 

I then computed my face warped into the average geometry, and the average face warped into my geometry:

 

 

 

 

CARICATURES

Next, I computed my face warped with the Danes average from above to find a caricature of myself. Changing the alpha level to be greater than 1 or less than 0 exaggerated my features relative to the Danes average in opposite directions.

 

 

 

 

 

BELLS AND WHISTLES: GENDER CHANGE

Finally, I used my image morphing algorithm on my image and the average Indian male (image taken from google). This was meant to produce a more “masculine” warp. Here are the results:

 

 

I also contributed to a class morph video. Here is my contribution:

 

 

That about sums it up! Needless to say, this was an entertaining project 