CS194-26 Project 3 - Face Morphing

Shreyas Patankar

I. Defining Correspondences

The objective of this part was to obtain an image of a face, label its key points, and generate a Delaunay Triangulation for that set of key points. I used a total of 62 key points and are labeled based on the IMM annotation sequence, where the first four keypoints are the four cornenrs of the image. The Delaunay Triangulation used was the one computed using the average of the keypoints from both images.

Original [Image 1]
Keypoints [Image 1]
Delaunay Triangulation at Average [Image 1]
Original [Image 2]
Keypoints [Image 2]
Delaunay Triangulation at Average [Image 2]

II. Computing the "Mid-way" Face

The next step was to compute the mid-way face of me and the hulk. We first had to compute the average shape using the keypoints (average corresponding keypoints), warping both faces to the average shape, then averaging the colors together. Some missing points were then interpolated to smooth the image. The warp involved writing an affine transformation to move from the original to the average shape.

Delaunay Triangulation on Averaged Keypoints
Original [Image 1]
Warped to Average [Image 1]
Colors Averaged
Warped to Average [Image 2]
Original [Image 2]

III. The Morph Sequence

The next step was to use a concept similar to midpoint calculation to find more "in-between" points. This was set by specifying a warp & dissolve fraction. The warp fraction computed intermediate triangulation while the dissolve fraction acted as a sort of blending quantity. The full morph sequence can be seen below:

Turning into the Hulk

IV. The "Mean face" of a population

In this part, I took the existing & labeled IMM Face Database and computed its "mean face." This was done by first generating the average Delaunay triangulation for the population, then warping individual triangulations to the average. Then the average of all images were taken. Please disregard the strange cropping.

The average Delaunay Triangulation
Person A from the IMM dataset
Person A warped into average shape
Person B from the IMM dataset
Person B warped into average shape
The average person from the IMM dataset

I then took this average person, and warped myself to its geometry and vice versa.

Me, unwarped
Me warped into average shape
Average person
Average person warped to my face shape

V. Caricatures

The next part was to create caricatures of my face using the average person found in the last section. I first extrapolated the caricature face shape: avg_shape + alpha * (my_shape - avg_shape), then warped my face to the extrapolated shape. I tried with three different alpha values:

Caricature 1, alpha = -1
Caricature 2, alpha = 1.5
Caricature 3, alpha = 2

VI. Bells & Whistles 1: Changing Gender & Ethnicity

In this part, I wanted to change my face to look more like the average female from the IMM dataset. This part was similar to part IV, but I found the midpoint image of me and the average woman instead. Here are my results:

Me, unedited
Me with feminine features
Average Woman
Face shape change only
Appearance change only

VII. Bells & Whistles 2: Class Face Morph

Below, a group of students have put together a face morph video. Students include: Nadia Hyder, Shreyas Patankar, David Yi, Avni Prasad, Chetana Ramaiyer, Richard Liu, Samuel Lee, Calvin Chen, & Joey Kraut.

Link: https://youtu.be/vCFfYSlLVwk

VIII. Bells & Whistles 3: Grade Face Morph

I also made a video of me as I grew up using images taken during picture day at school. It goes from Kindergarten to 12th grade, with just a couple of missing years in between.

Link: https://youtu.be/ZimvyfXIbys