Project 4: Face Morphing

by cs194-26-abp

Overview

We created a morph of one face transforming into another. Then we found the mean of a population and used it to extrapolate to create a caricature.

Defining Correspondences

My code displayed the image and a human clicked on points in the image to denote a feature. Then, I used the chosen points to create a Delauny triangulation of the image. Selecting the correct points was challenging since you need an artisitic eye to determine where facial structure is the same. In addition to the points on the face, I added points on the four corners.

Human Chosen points with the 4 corners

A Delauny triangulation using the chosen points

Computing the "Mid-way Face"

We use the correspondances created in the previous part. First, we average the correspondance for both images and used the average correspondances to create a Delauny Triangulation. For each triangle in the triangulation, we computed the affine transformation from the average correspondances to the original correspondances. We apply the affine transformation to all pixels in the chosen triangle and then sample the original image at the calculated pixel values using iterp2d. We do this for both images on all the color channels. Then we average the resulting two images to get the Mid-way Face.

Originals


Results


Original Morphed to Average Shape

Original Morphed to Average Shape

Mid-way Face

Original Morphed to Average Shape

Original Morphed to Average Shape

Mid-way Face

The Morph Sequence

We created a Morph sequence by using a similar process to creating the mid-face. When we average the two images correspondances, we take a weighted average where one image is times by warp_frac and the other by (1-warp_frac). When we combine the two resulting images, we again use a weighted average except one image is times by dissolve_frac and the other by (1-dissolve_frac). Warp_frac dictacts the amount of influence the first original image has on the structure of the current mid-face while dissolve_frac dictacts the amount of influence the first original image has on the appearance(color) of the current mid-face. To create the morphs, I set dissolve_frac equal to warp_frac and let them vary from 0 to 1. I picked 45 evenly spaced points between 0 and 1 for my morph.

Results


The "Mean face" of a population

I used the Danes Face set to find the mean face of the female and male population. To get the average face, I first got the average correspondance by averaging all the correspondances of the faces in the subpopulation. This gives me the average correspondance. Then for each original face, I warped the face to the average correspondances by the same process as in the previous parts except I skipped the averaging of 2 images (the dissolve_frac step) and used the average correspondances instead of a weighted average of 2 correspondances (the warp_frac step). Finally, I added the warped version of all original images together and divided by the number of faces in the subpopulation.

Results


Average Male

Average Female

Male faces warped to Average Structure

Female faces warped to Average Structure

I then warped 'my' face to the average geometry and the average face to `my` structure. I do this by setting my warp_frac to 0 and my dissolve_frac to 1. This means the structure/geometry is completely from the second image and the appearance is completely from the first image. I used the Danes correspondace scheme. I found that if I centered both faces, then the forehead (which is not annotated) looks the most realistic and not largely skewed. We see that my chin and nose are larger in the male geometry than in the female geometry. The forehead was not annotated, so it looks unnatural in all images.

Results


Average Male to my geometry

Average Female to my geometry

Me to Average Male geometry

Me to Average Female geometry

Caricatures: Extrapolating from the mean

I created a caricature of my face by extrapolating the correspondances of the average female face. Specifically, I let alpha vary from -2 to 1 and warped my face to carc_pts = avg_pts + alpha*(avg_pts-me_pts). We see that negative alpha exaggerates my unique traits while a positive alpha exaggerates the average traits. When alpha is -1 we get the original back.

Results


Alpha equal to 1

Alpha equal to 0.75

Alpha equal to 0.5

Alpha equal to 0.25

Alpha equal to 0

Alpha equal to -0.25

Alpha equal to -0.5

Alpha equal to -0.75

Alpha equal to -1

Alpha equal to -1.25

Alpha equal to -1.5

Alpha equal to -1.75

Alpha equal to -2

Bells and Whistles

Changing Gender

I tried to make my female face become a male. It worked well on the facial features. However, the background (namely the long black hair) makes the image appear more female than I would have liked. I used the Danes' correponances which only annotated the eyes, nose, mouth, eyebrows and chin. So, the forehead was not annotated and got excessively large during the warping process. I used the same code as in the previous parts but changed which image contributes to the structure and which image contributes to the appearance.
To create my male face, I warped my face to the average male structure and then cross-dissolve my face and the average face with dissolve_frac=.5. This way I still look like me, but I have a more masculine appearance.

Results


My appearance on the average male geometry

My geometry with the average male appearance

My face as a male

A Spooky Morphing Music Video

See A Face morph between different Halloween Costumes that lasts forever. Click below for the song! :D


Song is This Is Halloween Kayzo X Lookas from www.rsymedia.com