CS 194-26 Project 4: Face Morphing

This is the submission of Myron Liu (cs194-26-afp) for CS 194-26 Project 4.

Background Correspondences Midway Face Morph Sequence Mean Face Caricatures Bell's and Whistles 1 Bell's and Whistles 2 Bell's and Whistles 3 Conclusion

Background

In this project, I build a morph animation from my face to another face, compute the mean of a population of faces and extrapolate from the population mean to create a caricature of myself. A morph is a simultaneous warp of image shape and cross dissolve for image colors - the warp is created by defining correspondences between Delauney triangles of different points selected in each image to get the smoothest transformation possible. Using these correspondences, we can define and affine transformation between corresponding triangles used for the morph.

Defining Correspondences

Below are the two faces I used, the respective points I chose as correspondences and the Delauney triangle mesh built from midway shape of the point correspondences. I chose 36 points as correspondences with 4 corner points to act as anchors for the picture. Special thanks to Santhosh Subramanian for letting me take his photo and use it for this morph.

Myron

Santhosh

Myron Correspondence Points

Santhosh Correspondence Points

Myron with Delaunay Triangle Mesh

Santhosh with Delaunay Triangle Mesh

Midway Face

The midway face is made by first calculating the average shape of the faces. This can be done by taking the mean between correspondence points in each image. Using the triagulation mesh built in the previous step, we find the affine transformation between corresponding points in each image to their respective points in the average Delaunay triangle. We can compute the affine transformation matrix TA = X; A is the source image, X is the target image and T is the affine transformation matrix. Below are the equations used:

Myron

Santhosh

Manthosh - Midway face

Morph Sequence

I produced the warp sequence using the point correspondences between the image of my face and Santhosh's face; by controlling the shape warping and cross dissolving fractions, I produce 45 intermediate faces warping from my face to Santhosh's in the GIF shown below. Frame 0 is my face and frame 45 is Santhosh's face. Each frame is shown for 1/30 of a second.

Myron to Santhosh Morph Sequence

"Mean Face" of a Population

For computing the mean face of a population, I chose to look at plain colored faces from the Danes dataset which can be found via this link. The subpopulation I chose involved filtering out all female images and greyscale images - I was left with male Danes in color. First, I computed the average face shape using the annotated points given by the asf files. In total, I morphed 30 male Danes to get the average male Dane face. Below are some examples of the faces morphed into the average shape and the average face of the population.

Dane #9

Average Warped Dane #9

Dane #9 Morph Sequence

Dane #18

Average Warped Dane #18

Dane #18 Morph Sequence

Dane #28

Average Warped Dane #28

Dane #28 Morph Sequence

Myron

Average Dane Warp Myron

Danish Myron Morph Sequence

Average Male Dane Face

Myron Warped to Average Dane

Average Dane Warped to Myron

Midway Danish Myron

Caricatures

Using the average population face calculated in the previous part, we can caricature specific features and add emphasis using the average population face as a basis. The degree of extrapolation and morphing ("caricature") can be adjusted by an α parameter. The formula is as shown as below for warping shape.

Caricature Shape = Mean Shape + α × (Image Shape - Mean Shape)

Here are some caricatures of my face using the average male Danish face.

Normal Myron

Myron Warped α = 2

Myron Warped α = 3

Bells and Whistles #1: Change of age, gender, ethnicity, smile

Now, we can start playing with a person's look of age, gender, and ethnicity

Normal Myron

Older Chinese Man

Warped Old Myron

Old Appearance Transfer (Misaligned)

Old Myron

Normal Myron

Average Serbian Male

Warped Serbian Myron

Serbian Appearance Transfer

Serbian Myron

Normal Myron

Average Chinese Female

Warped Female Myron

Female Appearance Transfer

Female Myron

Bells and Whistles #2: Morphing Music Video

This is a morphing music video that I produced using photos of members of my fraternity Theta Delta Chi; shout out to all the brothers in TDX for letting me use their photo. The song used is So Close by Spencer Stong (what a dank song).

Bells and Whistles #3: Classmate Morphing Music Video

This is a morphing music video that I produced with 15 students in our CS 194-26 class including myself - cs194-26-aam Alex Kassil, cs194-26-afr Annie Wang, cs194-26-afg Bernie Wang, cs194-26-abl Daniel Shi, cs194-26-abr David Xiong, cs194-26-abt Ivan Jayapurna, cs194-26-abp Jackelyn Forsberg, cs194-26-afl Jacob Green, cs194-26-aes Joe Reid, cs194-26-agr Kyla Woyshner, cs194-26-acc Lakshya Jain, cs194-26-afc Lavanya Mittal, cs194-26-afp Myron Liu, cs194-26-aeb Sharabesh Ramesh, cs194-26-afr Samantha Lai. The song used is So Close by Spencer Stong.

Conclusion

Thanks for watching! This project was a lot of fun - finding cool morphs is really enjoyable.