Project 4: Face Morphing¶
| Author: | Tianrui Guo <aab> |
|---|
Project Spec: https://inst.eecs.berkeley.edu/~cs194-26/fa18/hw/proj4/
Project Writeup¶
Project Overview¶
The purpose of this project is to morph from one face to another face. To accomplish this, we must warp the image shape and then cross dissolve the colors. The cross dissolve is handled by taking a weighted sum of the two images. To warp the image shape, we must first define a set of point correspondences on the two images, and then compute a consistent triangulation on the these points. Then, the problem reduces to simply warping each pair of triangles in the source images.
Face Morphing¶
Defining Correspondences¶
The first thing to do is define pairs of corresponding points on the two images
by hand. While the rest of this project was written in Python, I chose to use
the cpselect tool in Matlab to pick the points because it was easier to
use. I then exported the points to a .mat file and loaded them into Python.
Affine Transformation¶
The next thing we need to do is to compute the affine transformation based off a source and target triangle. Let \(P_1 = \begin{bmatrix} x_1 & x_2 & x_3 \\ y_1 & y_2 & y_3 \\ 1 & 1 & 1 \end{bmatrix}\) be the matrix corresponding to the source triangle, where \((x_1, y_1)\), \((x_2, y_2)\), and \((x_3, y_3)\). We can define a matrix \(P_2\) in a similar manner for the target triangle. Then, we are interested in a transformation matrix \(T\) such that \(T P_1 = P_2\). Thus, we can compute \(T\) with:
Note that for the face morphing problem, the source points are from the
original images, and the target points are obtained using a weighted sum of the
original points. We fill in the pixels in the target image by using the
transformation matrix for the corresponding triangle to find the matching
points in original images. Note that the transformation will result in
non-integer indices in most cases, so we must do some linear interpolation to
get the pixel values. I used scipy.interpolate.RegularGridInterpolator for
this.
Population Mean Face¶
Using the face morphing function, we can manipulate faces from certain datasets. For this project, I chose to use this dataset: http://www2.imm.dtu.dk/~aam/datasets/datasets.html
Mean Face¶
We can see what the average face looks like by finding the average shape and warping the faces to the average shape and then merging them.
Morph to Mean¶
Here are examples of individual faces being morphed to the mean:
We can also morph Jerome Powell to the shape of the average Danish face, or vice-versa
Bells and Whistles¶
Changing Gender¶
We can try to morph Jerome Powell to the average Danish female. We can morph the shape, appearance, or both.
Changing Gender, v2¶
Another thing we can do is to look at the difference between the average male and female faces, and then add that difference to the points on Jerome Powell’s face.
