In the this project, we created a visual morph between two faces. We achieve this
by warping the shape of the imagaes and by cross dissolving the images for color.
To do this, you first need to define "corrispondence points" on both faces that represents
key features of each person such as eyes, mouth, nose, forehead shape, etc.
Along with the above, we computed the "mean" face of a population, and I included
my own face to the mean set, and applied the features of the mean set to my own face.
I also used this "mean" face to make carricatures of myself, and tried to change
the gender of the subject of a photo.
More details of the process used to achieve each are in the parts' respective
sections.
First, we need to identify key points of each picture. It's important to choose
a lot of points around facial features we (humans) tend to use to identify
faces such as the eyes, mouth, chin, etc. For this warp, I chose myself, and
my friend. I used approximately 40 points.
After defining correspondence points, we can compute the "mid-way" of the two
faces. This is done in 3 steps.
1) Compute the average shape
The average shape of 2 images is their x,y coordinates divided by 2 and added
together i.e. avergae_shape[y,x] = imA[y,x] * (1/2) + imB[y,x] * (1/2)
2) Warping both faces into that shape i.e. create warp from imA --> mid and imB --> mid
First, I used the average shape point set to create a Delaunay triangulation over the entire image.
To make sure every point in the image is located a triangle, I added the
corners of the image to my average point set. Now I can start to compute the warp
from imA --> mid and imB to mid. For a given warp (let's say imA --> mid)
for each triangle, I computed an Affine transformation matrix that maps
point transformation from imA --> mid. Ax = x' where A is the Affine matrix,
x is a vector of points in im1, and x' is a vector of points in im2. Then,
I used inverse warping using inverse of A for every x,y coordinate of my image
to determine where to map all points of imA to this warp. I repeated the
above to calculate my 2nd warp of imB --> mid.
3) Averaging the colors together
For both of my warps, warpA and warpB, I cross-disolved the images using a
weight of 1/2. midway = warpA * (1/2) + warpB * (1/2)
The process of creating a morph between imA and imB is essentially using the
same process as creating the mid-way face, just doing it multiple times while
changing certain weights and values. First, we generalize computing the
average shape. Instead of half of each image, we want (1 - i/45) of one
image and i/45 of the other image. Second, we generalize the crossdisolving.
Instead of just 1/2 of the color of each image, we again want (1-i/45) of
one image and 1/45 of the other image where i is in range from 0 to the number
of frames we want, which is 45.
Using a dataset of 37 Danish people, and predefined corrispondence points,
we can compute the "mean face" of a Dane. To compute the "mean Dane",
we first compute the average_shape of the Danes by adding all the
corrispondence points and then dividing by the number of faces in our data set.
Once we have the mean shape, we can morph each face into our poplution into
the average shape. Once we compute all the average shapes for all the Danes,
we can add them all together, and divide by the total number of faces to get
the "average Danish face."
I can make a caricature of myself by taking the difference of my face
corrispondece points (my face shape) with the average Danish face shape,
and then multiply it by a weight t, and then add that product to the
average Danish face to get a carricature of myself.
I got the mean female Danish face set and I can use this to "change the gender"
of my face.
I mean, beauty is in the eye of the beholder, right? But seriously, you can see that certain traits have blended to become more like the average female img avg above, like my very round face has become a little more slender, along with my eyes. Also my skin has become more "fair". It's not perfect, but there's only so much an image can do to me.