Project 4: Face Morphing

cs194-26-aeo

Overview: the purpose of this part of the project is to create a sequence of morphed images to smoothly transform one image into another. The final result is a video of that smooth transformation. The following is the procedure I followed to achieve that result.

Defining Correspondences

At first I attempted to write my code in Python. However, that soon proved to be problematic and tedious. So instead I decided to use MATLAB's cpselect() and to export the corresponding points to txt files. I chose 62 reference points on both images.

cpselect()

Computing the "Mid-way Face"

I then calculated the midpoints between the corresponding points in order to build my triangulation. Using Delaunay triangulation I maximized the minimum inner angles of the triangles to reduce the appearance of artifacts. Once I had identified triangles, I found the transformation from the destination shape triangle (the computed middle shape) to its corresponding source image triangle for both img1 and img2. To find affine transformation I solved for the transformation matrix using the three source and destination points (vertices of the triangles). I chose to use an inverse warp (and rectBivariateSpline to interpolate values) to avoid "holes" or overlapping values for the destination image to warp both img1 and img2 into the midway shape. After warping each source/destination to an intermediate state, I cross-dissolved the two warped images to create a hybrid. Below you'll see the Delaunay Triangulation and the Mid-way Face.

The Morph Sequence

I created 46 frames at intermediate stages of the morphing sequence by increasing the warp and the dissolve fractions and applied the same algorithm I used to calculate the mid-way face. I combined them to form a gif. (You may some loss of quality because I resized the gif file to take up less memory.)

You'll notice the bottom of my shoulders break in the middle and morph in a weird way. This is because I didn't assign the propper relative points in that area for a smoother transition. Also my face is overall positioned higher than Anne's, so you'll notice the morph is virtually dragging my face down.

Overview: I picked the Danes dataset of annotated faces to examine the concept of average faces. I chose two separate subsets of all male and all female non smiling forward looking faces. (By the way, there are a lot more males than females which is not at all representative of the human population. #yesIamafeminist)

Computing the average shape.

First, I computed the average shape of the population faces by reading all the corresponding points and computing the triangulation. Right - female average triangulation, left - male.

Morphing images into the average shape.

Then I morphed each image into the average shape of its subpopulation. I used the same technique as in Part 1. Below are some examples.

Computing the average fae of the population

Finally I cross-dissolved all images in their subpopulation to compute the average face.

I also took another picture of myself and created points with cpselect() again that correspond to the annotated points in the Danes dataset to morph my face into the average geometry as well as morph the average face into the shape of my face.

I extrapolate from the mean Dane to make caricatures by changing my face shape towards the mean Dane. Alphas from left to right: 0.3, 0.5, 1, 1.5, 2. Here are the images:

I extrapolate from the mean Dane to make caricatures by changing my face shape further from the mean Dane and more like my face shape. Alphas from left to right: 0, -0.5, -1, -1.3, -2. Here are the images: