Project 3: Face Morphing

Jason Zou

In this project, we explore the usage of affine transformations to manipulate face images in various ways. We also look at the mean faces of populations, and interpolations/extrapolations that we can derive from those mean faces.

Morph Sequence

Below are two headshots, one of myself and one of my housemate Alex. Using manually annotated correspondence points for each image, we compute a Delanauy triangulation on the midway shape, which we then use to generate each morph frame by using inverse warping + interpolation. The pixels sampled from both images at each stage are cross-dissolved accordingly.

...
Me
...
Alex (my Housemate)
...
Me (Correspondence)
...
Alex (Correspondence)
...
Midway Face
...
Full Morph Sequence

"Mean Faces" and Extrapolation

For exploring average faces, we used the publicly available FEI Face Database of 100 male and 100 female staff/students which comes with annotated correspondence points. We find the average face of the total face population, and attempt morphs between individual subjects in the population and the average as well as morphs between our own face and the average face. As a final experiment, we attempt to draw a caricature of our face by extrapolating from the mean using the equation extrapl = me + alpha*(me-mean).

...
Mean Face of Total FEI Face Population
...
Example Male Morph
...
Example Female Morph
...
Me to Average
...
Me to Average (Last Frame)
...
Average to Me
...
Average to Me (Last Frame)
...
Alpha = 0.5
...
Alpha = 0.5 (Last Frame)
...
Alpha = 1.0
...
Alpha = 1.0 (Last Frame)

Bells and Whistles

For fun, I attempted to change the gender and ethnicity of my face by morphing both shape and appearance in the direction of the desired characteristic. This "direction" is calculated as a vector (separate for shape and appearance) going from the average face of my "home" subpopulation to the average face of my "target" subpopulatoin. The average faces I used are publically available here, and in my case I chose the average Han Chinese Male face as my base. Note: when morphing appearances, the subpopulation difference vector was calculated after morphing the average images into the shape of the target face.

More Female

Below, we can observe that my eyes and mouth features get curved upward slightly and my skin tone becomes lighter on average, indeed eliciting a slightly more feminine appearance than before.

...
Average Han Male
...
Average Han Female
...
Me
...
Shape Only
...
Appearance Only
...
Both

More Taiwanese

Unlike the example above, there is not much appearance morphing likely because the subpopulations differ mainly by facial structures. It is interesting that my face gets longer, but this might be due to differences in image scale and alignment rather instead of being completely correlated with differences in structure.

...
Average Han Male
...
Average Taiwanese Male
...
Me
...
Shape Only
...
Appearance Only
...
Both

More Samoan

As a final experiment, I try to shift my face toward a more drastic directional change. The results are surprisingly quite reasonable.

...
Average Han Male
...
Average Samoan Male
...
Me
...
Shape Only
...
Appearance Only
...
Both