The third project of the class required me to implement image warping techniques in order to transform one image into another. I was able to make animations and videos of these transformations. I was also able to apply this to a large dataset of faces.
In order to create an image mask, we must manually annotate our 2 input images and specify the points we want to use. I used ginput() to create an interactive tool that allows me to manually input the pts by clicking on the image grid. I decided to use the provided George Clooney face and a picture of myself as the warping images. I first had to make sure my picture had the same size as the Clooney one. I also removed the background from my picture so that the transition would look better. I then manually annotated the points, averaged the 2 point sets and then found the delaunay triangulation. I then was able to apply the triangulation to the point sets of each input image.
In order to compute the midway face, we must warp both input images into an intermediary image. We found the triangulation points above by finding the average point set between the images. Next, we apply the triangulation to each image and then inverse warp each image to the intermediate. In order to inverse warp, we must find the affine transformation matrix by solving a system of equations. Then we can use the transform matrix to transform each individual triangle from the intermediate image to the input image. Since the transformation may lead to decimal pixel locations, we can use a interpolation function to solve the issue. Combine the warping with a halfway cross dissolve results in the midway face.
Now we can make a nice animation of the transformation between George and my face. First, we define a morph function that can take 2 images and their correspondence points and return the intermediate image. Part of this function is to define a warp and dissolve fraction. The warp fraction is used to determine how far between the 2 correspondences we want to warp to. The dissolve value dictates how much of each image appears. Using this morph function and varying the warp and dissolve fractions over 45 values results in a nice animation.
Using the FEI database of black and white frontal face images of 100 people, I was able to find the mean face. In order to find the mean face, I had to extract the point correspondences from the .pts files. Once they were extracted, I had to find the mean correspondence point set and then use that for the triangulation. Then I individually warped each photo to the average point set. Finally, I averaged all the warped photos and was left with the mean face of the population.
Here are a few images of some faces being warped to the average (original is on the left):
I also warped my face to the mean face found (left image) and warped the mean face to my face (right image). This involved a similar warp procedure as earlier
Now that we have a method of finding the mean face and then warping to and from it, the natural next step is to see what happens if we overwarp an image towards the mean. This results in a caricature. Since I was warping my own face to the mean face shape, I wanted to exaggerate the warping in the direction of the mean. I did this by altering my input image point set. I took the difference between the average point set and my image point set and then scaled this delta by a constant. I then added this scaled value to the average point set and used that as my new image point set.
I decided to take 5 photos from various points of my childhood and create an animation video of my faces morphing. The morph turned out decent. It was quite difficult creating a decently realistic warp due to the various facial sizes and angles I had worry about. But the most challenging aspect is that my glasses keep changing size and shape. The facial warping becomes more obvious with glasses. I uploaded the video to Youtube: https://www.youtube.com/watch?v=Og12fgot8qc