Face Morphing

CS 194-26: Computational Photography, Fall 2018, Project 4

Nathan Petreaca

cs194-26-afq

This project involved a lot of triangles, inverse affine transforms, and Delaunay Triangles. Bells and Whistles: -Changed a person of Asian ethnicity into a person of European ethnicity

1. Correspondences

Here we do 2 things:

  • We manually define a set of correspondence points between our two images
  • And then find a set of Delaunay triangles connecting these points together

  • Initial Images

    Images with Delaunay Triangles and correspondence points overlaid

    2. Mid-Way Face

    Now that we have a set of correspondence points and we know how to find a set of Delaunay Triangles for these points the goal here is to find a set of Delaunay Triangles that are the same for both images so that we can comprehensively combine the two images. Each image has a unique Delaunay Triangle mask associated with it initially.

    A simple way to do this is to simply find an average between the two triangles and then use half of one image and half of the other image through a cross dissolve. What we get here, is the Mid-Way Face connected to a Mid-Way Delaunay Triangle mask.



    As shown, the Delaunay Triangle makes is now somewhere in between the two shown in the previous section

    3. Morph

    An even half and half averaged mask is only one out of an arbitrary amount of possible intermediate masks, each averaged intermediate mask can be weighted more heavily toward one image mask or the other. From this, we can create a morphing from one image to another image by procedurally finding a set of Mid-Way Delaunay Triangle masks that transition smoothly from one image to the other by slowly being weighted closer and closer toward the destination images Delaunay Triangle mask.

    We warp one image across the set of Mid-Way masks by using an affine transformation from each triangle in the initial Delaunay Triangle mask to the respective triangle in the current Mid-Way mask. Here we use 45 different Mid-Way masks for 45 frames in the following gif.



    Another Example

    In this example we do the same thing as above, but with images of a different nature. The source image is an image of an Asian female’s face, and the destination is an image of the average female European's face.

    By transitioning from the Asian females face toward the average European female’s face, we can procedurally change the ethnicity of the initial image to be more European. Below is shown the Mid-Way Face between the 2 images which changes the ethnicity of the Asian females face to European


    Initial images

    Correspondence points and Delaunay Triangles


    Half-way masks. Combining them with a cross dissolve interpolation changes the ethnicity


    4. Mean Face >:-(

    As demonstrated in the above example, average faces can be incredibly useful in morphing.

    In the following segment we use a freely available dataset of annotated faces of Danes with correspondence points. Using this, we can find the mean Danish face. Here, average set of correspondence points and the resulting average Delaunay Triangle mask is calculated from the given correspondence points from all the images, and then, each image is warped into the average shape using the same method used in morphing.


    Initial Images

    Initial Delaunay triangle mask

    Faces transformed to fit average shape


    Without masks overlay

    Average face

    5. Warping to the Mean

    Now that we have the mean, we can do some interesting things.

  • We can see what a Danish person would look like if they had the facial structure of an Asian person. Likewise, we can see what an Asian person would look like with the facial structure of the average Danish person.
  • Instead of finding averages between the two Delaunay masks between images, we can extrapolate to extenuate facial features and create a caricature.

  • Initial Images

    Images with original mask and new mask

    Results of swapping the shapes of the two Initial images

    Image before and after caricature warp

    Oh My, Im in love