Face Morphing

Tianhui (Lily) Yang CS194 Spring 2020

Introduction

short summary of what this was all about :)

Warping faces is so fun they make mirrors that do just that. So how bout let's try to recreate it with some math and some fun photos?
In this project, I morphed between image sequences by splitting up images into a bunch of triangles and changing them via affine transformations.

Part 1: Defining Correspondences

In order for us to split the image into transformable triangles, we need to find points to map features between images. Using ginput, I saved points of correspondence and performed Delaunay triangulation to create the following nets.

triangles

Part 2: Finding the "Mid-way Face"

mid-way

To correctly transform each triangle we perform 2 main operations.
1. Find the affine transformation between the triangle in the start image and the triangle in the result image as seen in the matrix represented with values a, b, ..., above
2. Using known points in the result image, retrieve corresponding points in the start image through inverse mapping with the computed affine matrix. Then, assign pixel values from the start image at the corresponding points to the known points in the result image.

Part 3: Morph Sequence

mid-way morphed

After performing the warp for the mid-way point, I then used weighted averages to cross dissolve between transformations to other weighted warps.

Part 4: The "Mean Face" of a Population

After trying out warps with my own photos, I looked in the collect averages of database images. Here is the dataset from the FEI face database in Sao Paulo, Brazil. I decided to look into the female sub-population of neutral images. We see that some transformations are very wonky for most people when we morph them to the average face shape. The average is the sharpest around the eyes and noisiest in the hair section.

meanface

4.1: Caricatures

caricature-collage

We can think of the differences between the average and other faces as the difference between the distances of their correspondence points. This then gives us the ability to extrapolate beyond the data by adding or subtracting scalar multiples of the differences. In the gif below, I show just that. As you can see, when the coefficient multiplying the differnce is too high, the face ceases to be natural.

caricaturemorph


Bells and Whistles

I love watching kdramas and particularly liked watching Scarlet Heart: Ryeo. They also had really cool posters, so here's a video sequence coordinated to the song: 'Wind (바람)' (Scarlet Heart: Ryeo OST, Part 11) by Jung Seung Hwan.

Themed Morph from Lily Yang on Vimeo.

Participating in the class morph video :). Thanks to Zixian Zang for putting it together!

The most important thing I learned from this project is that complex transformations can always be broken down into smaller parts :). I thought this was really fun and especially liked looking at faces warping in and out between frames.