hannah
rush
Starting with two input face images rush.jpg and hannah.jpg, roughly
aligned manually using gimp, I generated shape vectors by parsing keypoint locations from manually labeling the faces.
This procedure needed to be done in identical format, such that the mid-eye feature, for example, in one shape vector is
indexed exactly the same as in the other shape vector, in order to properly compute the triangulation and generate affine
transformation matrices.
hannah
rush
hannah keypoints
rush keypoints
I then used the Delaunay algorithm to generate triangulations for both shape vectors, visualized below:
hannah delaunay
rush delaunay
I computed the mid-way face by first averaging the shape vectors for the two images, warping both the images
to the averaged shape vector, then average the images (color) to obtain the mid-way morph.
Warping the images is achieved by computing affine transformation matrices for each triangle in the Delaunay
triangulation of the shape vector (either one, usually the mid-way shape). These matrices take map triangles
in one image to corresponding ones in the other. After computing all the matrices, we construct polygon masks to select
the appropriate pixels in the image in the output image and inverse-warp them using the transformation matrices
to construct the warped image, visualized below:
rush-hannah-mid
The morph sequence is finally constructed using the same morph function used to generate the mid-way face,
only varying warp_frac and dissolve_frac from 0-1 - the relative weighting
of each warped image in each respective frame:
rush-hannah gif
I used the provided Danes dataset, filtered down to include only male faces of image type 5:
Full frontal face, neutral expression, spot light added at person's left side
I parsed the given .asf files to generate shape vectors for each face in the subsample, then averaged all the vectors to find the average shape vector. I then morphed each image in the subsample to the average shape, and averaged all the images to generate the average face among the sub-population. I have visualized a few individual images morphed to the mean; sometimes the morphed image has tearing due to the sharpness of the underlying triangles transform:
mean-warped 08-5m
mean-warped 28-5m
mean-warped 29-5m
Below I have visualized the mean image among the subsample, as well as my face warped into the mean image shape and vise versa. I achieved the face warping by recomputing shape vectors for the mean image using dlib and using a manually aligned similarly lit photo of myself, also vectorized by dlib to generate identical correspondence:
average face
my face warped to average shape
average face warped to my shape
Finally, I extrapolated from the mean image, warping my face to weighted multiples of the difference vector between my face vector and the average face, which seems to warp my eyes to try to match the softness of the average face vector:
rush caricature
average female face
I experimented with morphing gender by generating an average female face (across type 5 images). Next, I morphed the shape and color of my face, first individually then together, to the average female face:
morphing shape only
morphing appearence only
morphing both