In this assignment I produced a "morph" animation of my face into my girlfriend Zoe's face, computed the mean of a population of faces and extrapolate from a population mean to create a caricature of myself.
A morph is a simultaneous warp of the image shape and a cross-dissolve of the image colors. The cross-dissolve is the easy part; controlling and doing the warp is the hard part. The warp is controlled by defining a correspondence between the two pictures. The correspondence should map eyes to eyes, mouth to mouth, chin to chin, ears to ears, etc., to get the smoothest transformations possible.
To start with, I took a pictures of yourself on a white background. My image is the same size and aspect ratio as Zoe's face. This will make your morphing result more pleasing to the eye.
I used my photo as Picture A and Zoe's photo as Picture B. I morphed still picture A into still picture B and produce 45 frames of animation numbered 0-45, where frame 0 is identical to picture A and frame 45 is identical to picture B. In the video, each frame is displayed for 1/30 of a second (ie. 30 fps). I created a video from my sequence of frames, as an animated gif.
First, I defined pairs of corresponding points on the two images by hand. In order for the morph to work I have consistent labeling of the two faces. I labelled 81 points in faces A and B in a consistent manner using the same ordering of keypoints in the two faces.
The below images shows the points I labelled:
Then, I created a triangulation of these points that will be used for morphing. I used a Delaunay triangulation since it does not produce overly skinny triangles. I computed the Delaunay triangulation on the midway shape (i.e. mean of the two point sets) to lessen the potential triangle deformations.
With the triangulation on the mid-way shape, I did inverse warpping of each triangle. For each triangle, I calculate the affine transformation from the one in the mid-way face to the corresponding one in the original image. Then I applied the same transformation to each pixels in the triangle to transform each pixel of the original image to the mid-way shape.
For colors, I inverse warp the pixels in the mid-way face back to the original image, then used bilinear interpolation to get the color from neighbors(Cross Dissolve). This is better than forward warpping as it eliminates holes in the mid-way shape.
|
|
|
With the ability to create an intermediate image with two sources images, we can generate a sequence of images(frames) that together become a gif.
Specifically, we have two new variables called warp_frac and dissolve_frac, controlling shape warping and cross-dissolve, respectively. In particular, two source images are first warped into an intermediate shape configuration controlled by warp_frac, and then cross-dissolved according to dissolve_frac. For interpolation, both parameters lie in the range [0,1]. They are the only parameters that will vary from frame to frame in the animation. For the starting frame, they will both equal 0, and for the ending frame, they will both equal 1.
In this part, we computed the average face shape of all the men in Danes facing the front of the camera. We have enough faces(33) to make sure the result is interesting. We first morphed every men's faces into the average shape. Then compute the average face of all men by taking average of all the morphed faces
Below are some examples of morphed faces, as well as the average face.
|
|
|
|
|
|
|
|
I also tried to morph my face into the average face's geometry, as well as the other way around.
First I identified correspondence between my face and the average face. Then similar to how I morph an image in the previous part, I find triangulation in the average face, then I find the affine transformation that transform from my face to the average face for each corresponding triangles. I used the same transformation to transform all the points inside the triangle. Then I used inverse warpping to interpolate the color using cross-dissolve.
I did the same thing to morph the average face into my face. The difference is that we find triangulation in my face and find affine transformation from the average face to my face.
|
|
|
|
I also produced a caricature of my face by extrapolating from the average face above. This is done by adding my points by a scaled difference between my face and the average face. Then morph my face to the new points. This will exaggerate the difference between my face and the average face.
As seen below, I have larger nose and thicker lips than the average Danes. These traits become more obvious in the caricature.
|
For Bells and Whistles, I worked on changing the 'gender' of my face. I first found an image of the average women from Google. Then I morphed my face to the desired mean face by just the shape, just the appearance, and both. For morphing by just shape, no cross-dissolve is done. The color is taken from my face only. We can see that my nose is smaller. For morphing by just appearance, no warpping is done. I only cross-dissolve between the average women image and my face. We can see that I have much whiter and smoothier skin.
As we can see below, morphing with both shape and color gives the best result. cross-dissolve makes my skin whiter and smoothier, which are features more related to females. Also, morphing by shape makes my eyes larger, canthus slightly upwards and nose smaller, which are also characteristics that are more common on females' faces. Indeed, my face looks more femine after the morph.
|
|
|
|
|
|