Sachit Shroff's Project 3

This project focused on morphing images of faces into one another and taking advantages of various weighted averages to produce morph sequences

Part 1: Defining Correspondences and Computing Triangulations

To begin, I resized corresponding pairs of images to the same size (for consistency in keypoint labelling). Then, I used a tool called LabelBox to label the keypoints on my images and export them in a usable JSON format. Finally, I used scipy.spatial.Delaunay to find Delaunay triangulations on keypoints in images.

In order are my original image, resized image, image with keypoints, and image with the triangulation displayed

Original Image:

Tony Stark

Resized Image:

Tony Stark Resized

Image with Keypoints

Tony Stark Keypoints

Image with Triangulation

Tony Stark Triangulation

Part 2: Computing the Mid-Way Face

I used the procedure in the spec to compute the midway face for two pairs of images: me to Tony Stark and Yoda to baby Yoda. I first find a triangulation of the average of the keypoints, then use that traingulation to create a series of affine inverse warps from the target mid-way image to each source image. I then average the colors taken from both source images for each targe pixel (but with vectorized code). Unfortunately the pictures I used aren't perfect, so you'll notice the blending of Iron Man's armor with my shirt and baby Yoda's robes with Yoda's neck is not particularly smooth, but the result is still mostly visually smooth.

In order are: the original image of me, the mid-way image, the original image of Tony Stark, the original image of Yoda, the seocnd mid-way image, and the original image of baby Yoda.

Morphing from me to Tony Stark:

Original Me Mid-Way Image Original Tony Stark

Morphing from Yoda to Baby Yoda

Original Yoda

Mid-Way Image

Original Baby Yoda

Part 3: Computing the Full Morph Sequence

I used largely the same procedure as above to generate my full morph sequence. I just altered the values of warp_frac and dissolve_frac to step from 0 to 1, then saved the sequence of resulting images as a gif. For clarity, I added copies of the still frame at each target image, and ping-ponged the sequence so it can be played continuously in a loop.

Morph Sequence from me to Tony Stark:

me_morph

Morph Sequence from Yoda to Baby Yoda:

This was too large to upload, so please find it at this Google Drive link, I couldn't figure out how to embed it but it looks (in my opinion) better than the other morph. Plus baby Yoda is really cute.

Part 4: Computing the Mean Image and Caricatures

I used the Danish dataset linked in the spec. I computed the mean shape and face for all males in the dataset. I averaged the 58 keypoints for all the scientists (54 from the original dataset + 4 corners I added) to compute the average shape of the male Danish faces. I then morphed each face to this average geometry using the morph function I wrote before. Next, I element-wise averaged these morphed images to get the face of the average male Danish scientist. After that, I cropped and resized the average image so I could label keypoints on it. Then I used the same morph function as above to warp the Danish male to my geometry (by setting warp_frac to 0 and dissolve_frac to 1) and vice versa. Finally, I computed a caricature of myself as the average Danish male. Since the original image of me warped to the Danish male's geometry already looked pretty bad, I also computed a "softer caricature" in which I did cross disolve the image of my face and the average Danish face to get a slightly less ridiculous looking result.

In order are: 5 images of Danish males warped to the average geometry, the mean face of the (warped) Danish males, the cropped and resized mean face, my face (using same source image as above) warped to the average Danish male's geometry, the average Danish male warped to my geometry, a caricature of me as a Danish male, and a softer version of the same caricature.

Danish Scientists:

Danish Morphed 1 Danish Morphed 2 Danish Morphed 3 Danish Morphed 4 Danish Morphed 5

Average Face of a Danish Scientist:

Average Danish Male

Cropped and Resized Face of a Danish Scientist:

C/R Average Danish Male

Me Warped to the Average Danish Male's Geometry:

me_to_dm

The Average Danish Male Warped to My Geometry:

dm_to_me

Caricature of Me Over-warping to the Average Danish Male's Geometry:

caric

"Softer" Caricature of Me Over-warping to the Average Danish Male's Geometry (with some cross-dissolve):

soft_caric

Part 5: Bells and Whistles

I chose to change my perceived ethnicity. I found average images of Chinese and Tibetan males online, then used them to warp my face's shape, appearance, and both.

Warping Myself to the Average Tibetan Male:

me_tib

Warping Myself to the Average Chinese Male:

me_ch