In this project, we create transformations between faces by manually annotating triangulations of faces and then cross-dissolving after computing piecewise affine transformations using these triangulations.
I first attempted to compute the "Mid-way Face" of my housemate and myself. Here are the original images for reference:
Original Islam
Original Iverson
To do so, I first hand-selected 57 facial keypoints of each image.
Islam Facial Keypoints
Iverson Facial Keypoints
I then added corner points, constructed shape vectors, took the mean of the shape vectors, and computed the Delaunay triangulation, which is depicted below:
Delaunay Triangulation of "Mid-way Face"
Finally, I warped each image to the mean shape using a piecewise affine transformation and cross-dissolved to get the "Mid-way Face."
The "Mid-way Face"
Using the Delaunay triangulation of the "Mid-way face", I did multiple morphings of my housemate and myself, each with a different linear combination of each of our shape vectors and appearances. The result is the following gif:
Morph Sequence
I computed the "Mean face" of the FEI Face Database by first taking the mean of the provided shape vectors, then morphing each image in the database to the mean shape, and finally cross dissolving.
The following three images are sample faces from the database:
These next three images are results from morphing the sample faces to the mean shape:
After cross-dissolving all 400 such warped images, we obtain the "Mean face" of the database.
The "Mean face"
After obtaining the "Mean face," I annotated a new shape vector for it using the same 57 points I used to annotate my face. Having brought both my face and the "Mean face" into the same vector space, I warped my face into the mean shape and warped the "Mean face" into the shape of my face. The results are below:
Islam warped to shape of "Mean face"
"Mean face" warped to shape of Islam
These faces turned out excessively warped, probably because of the slight differences in pose combined with a poor choice of feature points. However, they still generally resemble what I expected.
To caricature myself, I first morphed my face towards a shape vector with value 1.5*u - 0.5*v, where u was the shape vector of my face and v was the shape vector of the "Mean face." I then cross-dissolved my face with the difference between my face and the mean face warped to the same shape. The result was as follows:
Islam Caricature
You can see that my large eyebrows are further exaggerated.
To produce a picture of my face with a neutral expression, I tried to "subtract" my smile. To do so, I first obtained the "Mean neutral face" and the "Mean smile face" of the FEI Face Database. Luckily, the dataset was already divided into these classes of faces. To compute the faces, I used a similar approach to obtaining the "Mean face."
"Mean neutral face"
"Mean smile face"
The results of only morphing shape, only morphing appearance, and morphing both shape and appearance are displayed below:
Morphing only shape
Morphing only appearance
Morphing shape and appearance
The results of morphing the appearance were not that strong, probably because the shape of a facial expression of a smile is more important than the brightness.