CS294 Project 3 - Jing Yuan

CS194 - 26 Intro to Computer Vision and Computational Photography

Project 3 - Face Morphing

By Jing Yuan [Email: jingyuan10@berkeley.edu]


In this assignment I produce a "morph" animation of my face into Taylor Swifft's face, compute the mean of a population of danish scientist faces and extrapolate from a population mean to create a caricature of yourself. These could be achieved by morphing and dissolving. A morph is a simultaneous warp of the image shape and a cross-dissolve of the image colors. The cross-dissolve is the easy part; controlling and doing the warp is the hard part. The warp is controlled by defining a correspondence between the two pictures.


Part 1: Defining Correspondences

The warp is controlled by defining a correspondence between the two pictures. The correspondence should map eyes to eyes, mouth to mouth, chin to chin, ears to ears, etc., to get the smoothest transformations possible. I start with defining 50 pairs of corresponding points of the two images by hand with the same ordering. Then I produce a set of mean points by averaging the two set from two images, and compute a Delaunay triangulation of the mean points. Delaunay triangulation is good for avoiding overly skinny triangles.

My Face: Original face & Triangle deformations
Taylor Swifft's Face: Original face & Triangle deformations

Part 2: Computing the "Mid-way Face"

The set of mean points from previous part defines a mean shape of the two faces. The next step is to warp both faces into the mean shape. I implement an affine warp for each triangle in the triangulation from the original images into this new shape. In computeAffine(tri1_pts, tri2_pts). I compute the affine transformation matrix T between two triangles defined by vertices tri1_pts and tri2_pts. Finally, I use linear interpolation to dissolve the colors from two images.
Here I morph my face into Taylor Swifft's face.

Midway face: Me + Taylor Swifft

Part 3: The Morph Sequence

Using the same algorithm descibed above, creating a morph sequence is quite simple. All that needs to be changed is scaling quanitities. Specifically the quantities that affect our average set of points and our cross-disolove. We change these to make transitional steps which are combined into a sequence. I morph still my face into still Taylor Swifft's face and produce 50 frames of animation numbered 0-50, where frame 0 is identical to pmy face and frame 50 is identical to Taylor Swifft's face. In the gif, each frame will be displayed for 1/30 of a second.


Morph Sequence: Me to Taylor Swifft

Part 4: The "Mean face" of a population

In this part I compute the average face of a group of Danish computer scientists from an online face dataset. First, I compute the averge geometry of danish scientists' face. Then, I warp all the danish scientists' face into the average geometry. Finally, I compute the average face of danish scientists.


Original danish scientists face

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Warpped danish scientists face

Average face of danish scientists

And I also use subset of female scientists to create average female danish scientists' face. Then I warp my face into the average female face geometry and warp the average female face into my geometry.

My face and Average face of female danish scientists

Warpped face: Me to Female danish & Female danish to Me

Part 5: Caricatures: Extrapolating from the mean

In this part I produce a caricature of my face by extrapolating from the population mean of danish scientists's face. The extrapolation is implemented by adding the ratios of difference between my face and mean face to my face. Following example images are computed by different ratios. Extrapolation: average_mean + alpha * (my_face - average_mean), My face was warped into the caricature shape with alpha = -1 and alpha = 2:


Caricatures:: alpha = -1 and alpha = 2

Bells and Whistles

In this part, I change ethnicity of my face to black woman's face. First, I download average black female's face from website. And do the same thing as in Part 1. Then,I prodeuce results of morphing just the shape, just the appearance, and both.


Original face: Me & Black female

Morphing face: Both, Shape only & Color Only

Learning

I love this project! It is tough to finish this project and I need to think of the mapping algorithm, but I really enjoy it. I am thinking that this morphng technique might be the behind scence algorithm of beauty function for most of the retouching software. I am the one who is the fan of Photography, this project really help me understand how some function inside the Photoshop work.