Project 3: Face Morphing~
CS194-26 Fall 2021 | Rio Hayakawa
Defining Correspondences
In order to create a pursuasive morph, we must first warp the images to a geometry in between the two images. Here we determine that geometry by manually selecting facial feature points using ginput in both images in a consistent manner. With those points, we can get the corresponding average points between the two. Based on the average points, we complete the geometry by creating a triangulation mesh object using scipy’s delaunay function. Below, 60 feature points are selected.
| feature points and triangulation |
 |
Computing the “Mid-way Face”
With the corresponding points and triangles, we can calculate the affine transformation matrix of the triangle vertex coordinates between the two images. Using that transformation matrix, we can take the inverse matrix to inverse warp the pixel values and reshape the geometry of each image to be the average shape. From there, we simply cross disolve the pixel values in each image to get the ‘mid-way face’. From left to right, below are images of Spencer warped to average shape, the mid-way image, and Rio warped to average shape.
The Morph Sequence
For the mid-way face we took the 50/50 average but by iterating over a small fraction at a time, we are able to create a start to finish morph sequence. :)
“Mean Face” of a Population
We can use a similar technique to generate the mean face of a set population. Here, we use the IMM Face Database and sample a subset of male faces in neutral position.
By importing the image file and their corresponding facial feature points, we can find the average geometry by finding the mean of all points and warping each image to that shape, and adding together the pixel values of all images scaled down by the number of images.
Below is the result of generating the mean male neutral face using 33 samples. Under are results from warping the facial features to the mean geometry for a few of the sample faces.
| Mean neutral male face |
 |
| Sample 1 before warp |
Sample 1 after warp |
 |
 |
| Sample 2 before warp |
Sample 2 after warp |
 |
 |
| Sample 3 before warp |
Sample 3 after warp |
 |
 |
Similarly, below is the result of using an image of myself.
| Me |
Me after warping to average shape |
 |
 |
| Mean face warped to my shape |
 |
By extrapolating from the mean male neutral face, we can generate caricature that extends beyond what the sample population average will look like on my face. By extrapolating on an alpha number under 0 (-0.8) we get what the opposite of what the mean face looks on me. And with an alpha number above 1 (1.4), we get what my face looks when it is even closer to the mean.
| alpha = -0.8 |
alpha = 1.4 |
 |
 |
Bells and Whistles
Morphing music video
I created a music video of a song I resonate with a morphing sequence of myself across various stages of my life. Here is the gif version and a link to the video is below.
Link to music video!
Changed ethnicity of myself to be white
By creating a mid-way image of myself and the mean face of the white male sample population, I created a version of myself that follows the shape and color features of the white male, as seen below!
Project 3: Face Morphing~
CS194-26 Fall 2021 | Rio Hayakawa
Defining Correspondences
In order to create a pursuasive morph, we must first warp the images to a geometry in between the two images. Here we determine that geometry by manually selecting facial feature points using ginput in both images in a consistent manner. With those points, we can get the corresponding average points between the two. Based on the average points, we complete the geometry by creating a triangulation mesh object using scipy’s delaunay function. Below, 60 feature points are selected.
Computing the “Mid-way Face”
With the corresponding points and triangles, we can calculate the affine transformation matrix of the triangle vertex coordinates between the two images. Using that transformation matrix, we can take the inverse matrix to inverse warp the pixel values and reshape the geometry of each image to be the average shape. From there, we simply cross disolve the pixel values in each image to get the ‘mid-way face’. From left to right, below are images of Spencer warped to average shape, the mid-way image, and Rio warped to average shape.
The Morph Sequence
For the mid-way face we took the 50/50 average but by iterating over a small fraction at a time, we are able to create a start to finish morph sequence. :)
“Mean Face” of a Population
We can use a similar technique to generate the mean face of a set population. Here, we use the IMM Face Database and sample a subset of male faces in neutral position.
By importing the image file and their corresponding facial feature points, we can find the average geometry by finding the mean of all points and warping each image to that shape, and adding together the pixel values of all images scaled down by the number of images.
Below is the result of generating the mean male neutral face using 33 samples. Under are results from warping the facial features to the mean geometry for a few of the sample faces.
Similarly, below is the result of using an image of myself.
Caricatures: Extrapolating from the mean
By extrapolating from the mean male neutral face, we can generate caricature that extends beyond what the sample population average will look like on my face. By extrapolating on an alpha number under 0 (-0.8) we get what the opposite of what the mean face looks on me. And with an alpha number above 1 (1.4), we get what my face looks when it is even closer to the mean.
Bells and Whistles
Morphing music video
I created a music video of a song I resonate with a morphing sequence of myself across various stages of my life. Here is the gif version and a link to the video is below.
Link to music video!
Changed ethnicity of myself to be white
By creating a mid-way image of myself and the mean face of the white male sample population, I created a version of myself that follows the shape and color features of the white male, as seen below!