CS194-26 Proj 3

Cheng Cao

Overview

This project explores affine transformations and linearity of images and shapes. (Image / Shape as vectors) We use human face as our target in this project and created face morphing, extrapolating, shape changing, etc.

Defining Correspondences

I used a python package called cpselect to select corresponding control points between two images. As we are going to apply delaunay triangulation onto the control points, we need to make sure the control points are in-order and we don't need to define their connectivity.

Computing the "Mid-Way Face"

Mid-way face is obtained by morphing to images into the average shape and then interpolate their color. The average shape is computed through an average of the control points. As the control points are in the same order and correspond to each other, a simple average can do the trick.

Then we compute a delaunay triangulation on the average shape, the triangle indices are then applied to both original control points to get two sets of mesh with the same topology.

We compute the affine transformation between two triangles. Given that $A=TX$, where $A$ and $X$ are two triangles and $T$ is the transformation matrix, we can get $T=X A^{-1}$.

After we get the transformation, we can transform the triangles in the original image into the target triangle. I used OpenCV's warpAffine function to warp the bounding box crop of the original triangle (for efficiency), and then fill the warpped shape onto target triangle using a mask generated by polyFill.

I morphed the two famous professors' image, Denero and Hug.

Because they have very different hair style, and Josh Hug has a nice beard, which causes the interpolation to produce a bad result. However we can see that the eyes and general shape are morphed successfully.

The Morph Sequence

By creating a sequence where the morph factor and dissolve factor interpolates between image A and B, we can create a morphing "video" from A to B.

The "Mean Face" of a Population

I used the "Danes" dataset for computing the mean face. The "Danes" dataset contain multiple faces per person and contains data for both genders. I chose all the male images with the front normal pose (pose "1") and averged their control points. Then these images are morphed into the average shape and an "mean face" is computed by calculating the color mean of all the morphed images.

The mean face of all males in this dataset:

Here's a image of myself:

Then I selected control points in the same order and same location as the dataset standard on my image, and then I can morph my image into the mean shape:

We can also do this the other way around (mean image morphed to my shape):

We can also extrapolate as now iamge and shapes are all just vectors. By computing the shape difference between me and the mean face, I can create a image that is more deviated from the mean (extrapolated by $-0.5$):

Bells and whistles:

The morphing sequence of multiple students (video created by another student): YouTube link