Cs 194 Project 3:

Part 1: Definining Correspondences

The goal of this part is to select pairs of corresponding points on our two images, which will be used to create a triangulation for affine morphing. I used ginput to select the points in a Jupyter notebook. Then, I computed the average of the point sets and used Delauanay triangulation on the average point set.

Here are my input images are myself, and TFT streamer C9 k3soju:

Triangulation Results:

Part 2: Computing the "Mid-way Face"

We can now use the triangles that we computed from the previous part in finding an affine transformation from corresponding triangles between images. For each triangle, we first compute a transformation matrix that maps from our source triangle to our destination triangle. In our case, the destination will always be that of the midway image. Next, we apply the inverse of this transformation to the polygon of points contained within the destination triangle, to obtain the corresponding source polygon. I used RectBivariateSpline to interpolate the pixel values of the source polygon coordinates, and set the pixel values of the destination polygon in my transformed image to take on those values. I used 3 interpolation functions per image, one for each color channel. I do the same process for each triangle I have, for both of my input images. Last, I take the average of the two morphed images to obtain the final mid-way face.

Here, I show the original two input images of me and C9 k3soju, as well as the mid way image.

Part 4: The "Mean Face" of a Population

In this part, I computed the average face of the Danes dataset and then performed morphing between my face and the average Dane face. In order to compute the average Dane face, I first computed the average of all the labeled keypoints for each image that were provided in the dataset. Then, I used the average keypoints to get a target triangulation. I morphed each image in the dataset to this target triangulation using the morphing methods I implemented in the previous steps. Then, I computed the average of these morphed images to obtain the average Dane face. For morphing my face to Dane, I used the triangulation of the average Dane keypoints, and morphed directly to the Dane face shape. For morphing Dane to my face, I did the same where I directly morphed the average Dane face to my face shape.

Examples of transformed Dane faces

Average Dane

Warp Between Me and Average Dane

Part 5: Caricatures

In order to create a caricature, I first subtracted the keypoints of my face from the keypoints of the average Dane face. Referring to this differences as delta, I then added the vector alpha * delta to my kyepoints, creating target keypoints. I then morph my image to the target keypoints in a face shape transformation to complete my caricature. I used alpha values of -1, -0.5, 1.5, and 2.0 in order to create different variations on my caricatures. Scroll through the carousel below to see how the caricatures change as the alpha values change!

Part 6: Bells and Whistles

For Bells and Whistles, I tried morphing my face onto the average Asian Woman face, in order to see what my features would look like as a female. I also tried morphing myself into the average NBA player face. For each, I tried morphing just the shape, just the appearance, and both shape and appearance.

Average Asian Woman Morph

Average NBA Player Morph