The fundamental basis of this entire project is focused on image warping. So if I
have an image A, which in this case is a face, how can I morph that image to a similar
feature image or face B. In order to do so, we can essentially create a mesh around both
images and try to warp each piece of the mesh. Mathematically, this can be done using affine
transformation matrices. We can essentially change the basis of each triangle of the source
to match that of the target.
The first step of this process is defining correspondences. This means we'll need to define points on
each face which correspond to important features. This way we can easy form triangles and warp the image. The
more points, the better our transformation will be. Here's what the points look like on the face and the respective Delaunay
triangulation.
-
Me
-
Kanye
After getting the correspondence points, we can use them to do Delaunay triangulations as shown
above. But instead of using those triangulations, we can instead average the points first, then find the
triangulation of the average instead. Then, using that triangulation, we can form triangles on the source and target
images so the triangle and points are consistent. Then like above we find the affine transformation needed
to tranform the source triangle to the average triangle. In order to do so we can actually do an "inverse warp" by instead
initially deciding which pixel we want to modify on the target, and then inverse affine transform to find
the original pixel value that we want from the source. That way we can fill in values easily since the warp
points won't initially be discrete. Then we can average the source to average and target to average to get a mid-way
face.
Instead of averaging the points and colors, we can actually just do a fraction and instead
do 45 frames using the following: alpha * source + (1 - alpha) * target.
-
Cody West
The "Mean face" of a population
Now what we can do is see how my face is compared to the mean population of white males. We use the Danes
dataset and only use the non-smiling white males. Then using the same process above, we average the provided
points of all the photos and morph them into a single photo which is shown below:
-
Mean Face
Caricatures: Extrapolating from the mean
We can also show extreme features by instead using the formula -alpha * source + (1+alpha) * target.
Using the mean population above (white males), can we show a full morph into making me white?
As an extra we show just morphing the geometries of both the average and myself, morphing just color,
and morphing both geometries and color.
The project was incredibly fun and interesting. Definitely the biggest struggles came from figuring out why
the morph wasn't working. A serious bug that took awhile to figure out was the flipping of the x and y pixels. The morph
looked fine but then once the mid way face was outputed the morphs didnt align. It almost seemed like the morph was going
in the wrong direction which is how I realised it was a pixel swap issue. Also the render became a serious issue, which is why
the morph sequence video being frames was a pain. The average face also took awhile since there were 30 photos. Overall however
there were incredibly fun results.