Programming Project #4 - Face Morphing

Sharabesh Ramesh


This project fundamentally focused on using affine transformation matrices to morph features on one face to another. We first annotated points of relevance on each of our target faces then used a Delaunay triangulation to divide up each image into a series of triangles. Given the average triangulation over the faces we sought to morph, we computed an affine transformation between each triangle in the source and destination images, determining the output images pixel values using this affine transformation. Using this same triangulation technique we created videos of face morphs as well as computed the population averages for a set of images.

Defining Correspondences

In order to compute correspondences we first selected pairs of corresponding points using the scheme defined in the project specification. The results of this application are shown below (additionally pinning the corner points)

Coordinate Scheme for Triangulation
Triangulation Applied to Image of Mark Zuckerberg
Triangulation Applied to additional face

Computing the Mid-Way Face

Using the technique described above we computed the affine tranformation between two pairs of triangles. We then generated our midway image by for each triangle and for each pixel finding the corresponding point in the original image (using the affine warp) and then mapping the value at that point to the output image.

Triangulation Applied to Image of Mark Zuckerberg
Triangulation Applied to additional face
Halfway Image of Zuckerberg and additional person
The morph sequence followed identical logic, however, we updated the morph and dissolve fractions to be linear weights between 0 and 1 for 45 consecutive images. We then pieced together the 45 images to get a final morphed output.

The "Mean Face of a population"

To compute the mean face of the Danes population using the provided labeling we first iterated through all of the given images and feature annotations and generated an average over all of the labeled points. This avg_coordinate_feature set was used to define the triangulation that each face was subsequently morphed to. We then morphed each face to the average triangulation and computed an average over all the faces in the set. The following shows some of the intermediate results as well as a depiction of the average danes. Note that the averages were only taken over the neutral expression forward facing males.

Individual 1 morphed to the Danes Average
Individual 6 morphed to the Danes Average
Individual 11 morphed to the Danes Average
Average for all the danes
Me Morphed to Geometry of Danes
The average Dane blended to my geometry

Caricatures Extrapolating From the Mean

In this case we simply use a negative alpha in order to morph away from the average instead of towards the average. The result is this:
The image of myself that I will be morphing
Image of the Danes I am morphing against
A caricature of me with the features shifted away from the average dane.
Alphas = {0.0,-0.2,-0.4,-0.6,-0.8}
Note how the eyebrows and smile shift more intensely with greater values of alpha (divergence from the mean)

Bells and Whistles: Generating a Music Video based on a theme!

Here we take images of 12 of the major game of thrones characters and create a morphing video based on the theme. We further add Game of Thrones background music to complete the effect. Here are the results! We went from this original source image:
The images of the GoT characters. These were subsequently processed to produce individual representations for each character