CS 194 - 26: Computational Photography Proj 3

Ritika Shrivastava, cs194-26-afe

Background

For this assignment, we worked on morphing faces! A morph is a warp of the image shape and a cross-dissolve of the image colors. The warp is done by mapping the affine transformation from one image to another. The correspondences are mapped to the same features between images. Example: nose to noes, etc.

Project

The main goal of this project is to morph from one image to another. This morph will be done with 45 frames of animation numbered 0-45, and converted into a gif.

Defining Correspondences

For this part of the project, I defined pairs of corresponding points on the two images by hand (I used 35 points). The labelling needed to be consistent between the two images.

From there I created triangulations fo the points which would be used for morphing. I used Delaunay triangulation and applied it to get the triangulation at the midway shape.

Below you can see the midway shape on the two images.

Computing the Mid-Way Face

Before we can make the morphing video we need to get the midpoint face of the images. To do this I needed to computing the average shape (a.k.a the average of each keypoint location in the two faces), warping both faces into that shape, and averaging the colors together.

In order to do this, I needed to compute the affine transformation matrix between the two triangles of the images.

Show us the original A and B images as well as the image of the mid-way face that you got.

Midpoint image. Notice how this is partially unrealistic due to my glasses.

The midpoint image. I should have added more points for the lips

Morph Sequence

Then I wrote a function that produced the warp between image 1 and image 2 and another function that used this to create a gif of the morphs. The function that produced the warped images had to additional fields for warp_frac and dissolve_frac control shape warping and cross-dissolve. These values were altered to get the images for the gif. They range from [0,1].

Mean-Face

To compute the mean, I used the Danes dataset. Using the key points that were provided I:

Compute the average face shape of the whole population in images where the faces were smiling and facing straight.

This is the mesh for the average face. Projected on my face

2. Morph each of the faces in the dataset into the average shape. Show us some examples.

3. Compute the average face of the population and display it.

Show the mean image that you got, as well as 1) your face warped into the average geometry, and 2) the average face warped into your geometry.

Then I warped my face onto the average geometry and the average face to my geometry

This is the average applied on me. I should have added a point on my hair, but I enjoyed that I became a unicorn when I was averaged 🙂

This is my points being applied to the average. The neck is so long! I should have added points on my shoulder

Caricatures

Then I produce a caricature of my face by extrapolating from the population mean you calculated in the last step. I became a giraffe 😂 Also my nose got sharper.

Bells and Whistles

For the Bells and Whistles, a few classmates and I got together and made a fun video. It was a really great way to meet people, even in a remote setting. Everyone is so good looking!

To organize this we had everyone submit images of the same size and background. We had everyone label key points in a consistent manner with one another.

Here is the link for the video: https://tinyurl.com/194-26-classvid

https://tinyurl.com/194-26-classvid