CS194-26: Image Manipulation and Computational Photography
Programming Project #4: Face Morphing
Allyson Koo
Project Overview
This assignment involved creating "morphs" of different faces. This is accomplished via a combination of warping image shapes and cross-dissolving image colors. To create a warp, we first select points on the images that we want to be aligned in the morph (e.g. map eyes to eyes, noses to noses, mouths to mouths, etc.). We then compute Delaunay triangles, and use these to calculate afine transformations between triangles in the source and target images. We use this transformation, in addition to cross-dissolving the colors, to create morphs between pictures.
Implementation
Part 1: Defining Correspondences
This portion of the project involved selecting points on the original image and the target image to align. For example, for faces, it would be best to select points outlining the face shape as well as the different features (eyes, nose, mouth, etc.). Once these points are selected on both images, we find the average points in order to be able to map both of the faces to the average image. We choose the average in order to lessen any possible triangle deformations. Once we have these average points, we calculate Delaunay triangles on the points. These triangles will then be used to map portions of the source image to the corresponding portion of the desired target image.
Triangulation on my image
Triangulation on target image
Part 2: Computing the Midway Face
The purpose of this portion of the project was to create an image that represents the "midway" face between the source and target images. This was accomplished in three steps: computing the average face shape, warping each face into that average face shape, and then averaging the colors across both images together. To compute the average face shape, we first average together the point selections on both images and then calculate the Delaunay triangles on the average points. Then, we warp the corresponding points on each of the source and target images to the corresponding points in the averaged image. This is accomplished by calculating an affine transformation for each of the Delaunay triangles. Then, we perform an inverse warp for each triangle from the desired average image to the source and target. We do an inverse warp (from the average to the original) in order to "eliminate holes" in the resulting image. Finally, we average the colors from the corresponding source and target images. In order to find the corresponding colors, we had to use an interpolation function, as the transformed pixels likely didn't correspond exactly to a pixel in the original image, so we interpolate to find what color should be "between" the original pixels.
Examples
Original source image
Midway image
Original target image
Part 3: The Morph Sequence
The purpose of this portion of the project was to create an animated "morph" between the source and target images. This was accomplished by extending the previous algorithm to calculate the midway face. Instead of computing an exact midpoint face, we calculate many intermediate images according to the warp_frac and dissolve_frac parameters. The warp_frac and dissolve_frac are used to control how much of either the source or target is present in the resulting output. In order to create a morph from the source to the target, you vary the "amount" of source image present in the output from 1 to 0 with step size equal to warp_frac.
Part 4: The "Mean Face" of a Population
The purpose of this portion of the project was to calculate the mean face of a population. This was accomplished by computing the average face shape of the entire population (using the already annotated correspondence points of the population in the Dane faces data). Once this average face shape was determined, we then morph each face in the population into the average face. We then average all these morphs to the average shape and display it.
Average face shape
The average face of the Dane face database (smaller version)
Example morphings of original faces into the average shape.
Original face
Original face warped to average shape
Original face
Original face warped to average shape
Original picture
My face warped to the average Dane face shape
Original picture
The average Dane face shape warped to my face shape
Part 5: Caricatures: Extrapolating from the Mean
The purpose of this portion of the project was to use the previously calculated mean in addition to your own face to produce charicatures. This is produced by subtracting the mean from your face and using the difference to exaggerate your own unique features.
My face plus 0.5 times the difference between mine and the average
My face plus the full difference between mine and the average
Part 6: Bells and Whistles: Change age/gender/ethnicity/smile/etc of your (or your friend's) face
The purpose of this portion of the project was to do something similar to the caricature except with a different feature -- e.g. age, gender, etc. This was accomplished using the average face of the desired feature, subtracting out the average face, and using this difference to add to your source image to change that particular feature.
Average face shape
The average face of a smiling person from the Dane database
Original face
Shape blend: My face plus the full difference between the smiling and neutral Dane average faces.
Shape blend: My face plus the difference times two between the smiling and neutral Dane average faces.
Just color blend (no warping)
Shape and color blend: Plus one difference between smiling and neutral
Shape and color blend: Plus two times difference between smiling and neutral