Defining Correspondences
For this part, I used matplotlib's ginput() function to select the set of facial features that I would use to correspond the two images that would morph into each other. I defined these points on paper, so that I could remember the order in which they were selected. To select the points, I focused on ensuring that every facial feature (eyes, nose, mouth, ears, hair, teeth) were well represented in the feature set. Once I found the points, I saved the feature sets as CSV files so I wouldn't have to reselect them every time. Once this was done, I computed a Delaunay triangulation on the average of the two feature sets. When finding the Delaunay triangulation, you cannot compute it on both feature sets, only one so we used the average of the two to ensure that both sets were equally represented.
My Facial Feature Set
Hasan Minhaj Facial Feature Set
Delaunay Triangulation Visualization
Computing the Midway Face
Our goal in this section is to compute the midway (or average) face. We do this by leveraging the Delaunay triangulations computed in the
previous step (recall that this was computed on the average of the two feature sets). We use that average feature set as a benchmark for
the average face. Once this was done, I defined a function to perform an affine transformation on two sets of triangles and another to
compute the inverse warp. The inverse warp took in the image that would be warped (src image), its features, the features of the destination
image and the triangulation set. The inverse warp procedure loops over all the triangles found in the Delaunay triangulation and
finds the corresponding triangles in the src and dest feature sets and uses these to find the affine transformation matrix T.
The inverse of T is then used to project the src triangles onto the dest triangles. We use an interpolation function defined for each of the color
channels to sample the color of the pixel.
One point to note, instead of morphing my face directly to Hasan Minhaj's, I morphed my face to the set of average features computed between
our two feature sets and morphed his to that set as well. To yield the final midway face, I simply added
0.5 * my warped face + 0.5 * his warped face.
Me
Hasan Minhaj
Midway Face
Morphing
This part was pretty straightforward, it mainly consisted of creating a morphing function that would actually morph a face into another based on a specific warp and dissolve ratio. These two ratios control the shape and color of the morphing by dictating how the two faces are combined at each timestep.
Mean Face of Dane Population
This part of the project consisted of taking a dataset of faces and computing the mean face of it. The dataset came pre-labeled with facial features. However, these feature sets were stored in .asf files and so I had to write a function that parsed through these files and extracted the (x,y) coordinates of the points and converted them into a format that my inverse_warp() function from the previous part could handle. Once this was done, I added 4 new features to the existing feature set: the 4 corners of the image. This ensured that the entire image would be morphed and not just the facial region. Once the feature sets were good to go, I calculated the average feature set and used that to find the Delaunay Triangulation. Finally, it was morphing time. I looped through all the dane faces and their corresponding features and warped each of them to the average feature geometry.
I also found the average Dane face, by taking the average of all the warped features found above.
Finally, I warped my face onto the average Dane face (and vice versa) following the same procedure as earlier in this project.
Me to Dane Warp
Dane to Me Warp
Caricatures
This part directly stems from the previous one, using the average Dane face I will create a caricature of my own face as I warp it to this geometry. To create this caricature, I take the difference between mine and the avg Dane face feature set. I multiply this by a hyperparameter alpha, which I vary from 0.25 to 2. Each caricature face is calculated by weighting the difference feature set by alpha and adding it to my feature set.
As you can see, after alpha=1 the warp factor becomes way too extreme and it looks like my face is crumpling in on itself.
Bells and Whistles
For the Bells & Whistles section, I created a morph showing how my face has changed over time! I took school photos from K-12 and morphed them b/w each other to yield the final morph below:
