CS194-26 Project 3
Face Morphing
Sean Kwan, Fall 2021
Defining Correspondences
In order to define the correspondences of the two images I wrote my own tool using ginput and plot. For the first half of this project, I decided to use 40 correspondence points. Initially, I had less but I realized that I needed a few points that covered the edges of the image in order to morph the background as well.
I then used scipy's Delaunay's triangulation in order to get the best and most spaced out triangulation of the points.
Computing the Mid-Way Face
In order to compute the "Mid-Way Face", I had to take the following steps: 1. Average the correspondence points of my image and the George Clooney image 2. Run a Delaunay triangulation on the average points 3. Interpolate the separate color channel values at all the (x,y) coordinates using RectBivariateSpline 4. Compute the Affine Transformation from the average triangle vertices points to the original triangle vertices points 5. Compute all the points within the corresponding triangle using polygon 6. Perform an inverse warp by using the Affine Transformation matrix (from 4) on all the interior points (from 5) then use the interpolation function (from 3) to get back the corresponding colors. 7. For both images, repeating steps 4-6 on all the triangles then gives us the original images transformed into the average image's shape. Then we average the color channels to get the "Mid-Way Face".
Original Sean
Mid-Way Face
Original Clooney
Morph Sequence
Abstracting the "Mid-Way Face" to the morph sequence is pretty straightforward. Instead of averaging the points, we instead use a warp_frac to find the points that we want to morph the image to. Using the formula warp_frac*im1_pts + (1 - warp_frac)*im2_pts. Then at the end for cross-disolving, instaed of averaging the color channels we use the dissolve_frac with dissolve_frac*color1 + (1-dissolve_frac)*color2. Using this premise and looping over the range [0,1] with a step size of 1/40 we are able to create a morph sequence.
Morph Sequence
The "Mean Face" of a Population
I decided to use the images from the FEI Brazilian dataset. In order to get the average image I took the following steps: 1. Wrote a script to get the correspondence points from the subset of a population. 2. Added extra points on the corners of the image to capture the whole image. 3. Averaged all the correspondence points. 4. Ran Delaunay triangulation on the average of all the images. 5. Morphed all the images into the average shape. 6. Averaged all the images in (5) to get the "Mean Face" of the Population. Below are some examples of the morphs of the faces into the average shape:
Original
Averaged
In the above image we see that the man's real face is longer, whereas in the average face it is squished.
Original
Averaged
In the above image we see that the man's real face is wider, whereas in the average face it is narrower.
Below is the final result of the average face of the population.
Average Face of Population
From this we can play around with these values and get my face warped into the average geometry and the average face warped into my geometry.
My face into the average geometry
Average Image into my geometry
Caricatures: Extrapolating from the Mean
Caricatures can be created by extrapolating from the mean. The inspiration behind my caricature is that after every semester, my family always says that I have gotten fatter. I wanted to explore what I would look like if I was extremely skinny by extrapolating the x-coordinates from the mean and shifting the x-coordinates towards the center of the image.
Caricature
Megamind
What turned out is that I ended up looking remarkably similar to Megamind. This occured as the coordinate corners of the image remained the same, and since the correspondence points of the image are from the eyebrow and below, it caused my forehead to remain the same size while the rest of my head got skinnier.
Bells & Whistles
I decided to change the gender of my face. In order to do this, I manually found the images corresponding to women in the Brazilian dataset and found the average correspondence points. From this I calculated the average face of women in the population.
Average Female Face of Population
From this, I was able to change the shape and change the appearance of my face using the above morph function.
Changing Shape
Changing Appearance
Combining the two, gives my face as a female. From the image, we can see that my facial features are softened and rounder as well as my eyebrows being thinner.
My Face as a Female