Project 3 - Jai Singh

Firstly to get a hang of things, I decided to do the correspondences with two of Christian Bale's photos. One with him as himself and the other as Patrick Bateman, his character in American Psycho.

Patrick Bateman

Christian Bale

To define the correspondences I used the tool linked in the project specification and calculated the triangulization with the suggested Delauney function.

Patrick Bateman with Correspondences

Christian Bale with Correspondences

Then after that I computed the correspondences for my own face and Christian Bale:

Me

Christian Bale

Me with Correspondences

Christian Bale with Correspondences

To compute the midway face, I choose to fully develop the framework I would need to create the morph sequence later on in Part 3. This involved computing the affine transform required to change both images triangulizations to the averaged one. This could be done using a system of equations and solving them in matrix form for each of the correspondences. Since morphing also involves dissolving, I choose to implement a billinear interpolation function to get the average pixel values in the opposite image, however this turned out to be too slow and thus I also implemented a nearest neighbor dissolve function. Other techniques I used to speed things up include having a bounding box around the triangle so when applying the affine transformation I would only need to look at a certain area of the image. Then with the overall morph function built, I used 0.5 warp_Frac and 0.5 disolve_frac to get the mid-way face as seen below for both sets of images:

Patrick Bateman

Halfway of Patrick Bale

Christian Bale

Me

Halfway of Jai Bale

Christian Bale

Now with 45 steps in between the two gifs we can see it in action with small increments of warp and disolve fractions:

Gif of the Transition

Reset GIF

Me and Christian Bale

Gif of the Transition

Reset GIF

For this part of the project, I used the FEI Face Database with the cropped style of image as that was the one that included the proper alignment as well as correspondences. Below is the average face from the database without any morphing done to each individual image and also the average face shape as calculated by averaging the correspondence points.

Average Face without Morphing

Average Face Shape

Then I attempted to morph each face into the average face shape calculated above, this resulted in a much cleaner average picture as seen below.

Person 10a Morphed

Person 20a Morphed

Person 30a Morphed

Person 40a Morphed

Person 50a Morphed

Person 60a Morphed

Average Face with Morphing

The features are much more defined after morphing each individual photo as there is less variation in major features like the chin, nose, eyebrows and more. Then we can use both the average face and its geometry morphed with mine and vice versa:

Me morphed to the average shape

Average Face morphed to me

This image does look very derpy, this is likely because of the cropping/focal length being very different in the pictures. Thus the externals are much more expanded in my version and contracted in the database.

Now we can calculate caricatures for my face by adding the difference (of my face shape to the average face) ontop of my face shape, to create a caricature that emphasises my features.

alpha = 0.1

alpha = 0.25

alpha = 0.5

alpha = 0.75

Once again, because of the difference in cropping of the images the extension of a vector to create a caricature even with a low alpha results in an extreme caricature. Particularly shrinking my face to a smaller size and expanding the corner triangles.

With the same dataset I decided to compute PCA and below you can see the top 12 principal components:

Eigenfaces selecting top 12 features

It's interesting to note how much more obvious the first few eigenfaces are in capturing data and notice it decreasing later on, this was also touched upon in class. Then I tried to reconstruct faces within the database with the new basis:

Reconstructed 143

Reconstructed 87

Reconstructed 73

These seemed to be quite good for just 12 basis, but wanted to do better. So I experimented with increasing K=32 and K=100, to see how big the difference would be:

K=12

K=32

K=100

Clearly by the time K=100 the reconstruction is quite good and its quite cool to see how good just 100 basis can be in capturing a face. Then I tried reconstructing my own face:

Reconstructed image of me

Clearly much worse this time most likely to the fact that my face was not in the dataset and the not perfect alignment.

Was looking to do the music video on myself at different ages, and saw the Ed Stem post showing that auto correspondences would be considered an extra bells and whistles. So thought I would "save time" of manually clicking for each of my images and set up something to do it automatically:

Auto-correspondences

This was done with the help of the face alignment python library.

I also was curious in seeing myself morph from a younger version of myself, so I applied the auto correspondence to my own photos and created a morph sequence as seen below:

Myself morphing/aging