The goal of this project is to create a morphing between two images, compute an average face from a population of faces, and create caricatures of images by extrapolating pictures from a population mean. A morph is a warp from one image shape to another and a cross-dissolve of the two images' colors. To warp between two shapes, we define pairs of corresponding points between two images such as eyes to eyes, nose to nose, and chin to chin. These points tell us where the two images should line up to create a smooth transformation.
I used <ginput> from Matplotlib's Pyplot collection
to manually define pairs of corresponding points between two images.
Using these points, I defined a triangular mesh over the images using a
Delaunay triangulation which will allow us to warp between the two
images in the next section. An example of this process can be seen on an
image of myself and UC Berkeley lecturer Nicholas Weaver. I selected 44
correspondences between the images and subsequently computed the
Delaunay triangulation on the set of points.
The mid-way face of the image morph is a warp from the two input images into an average face shape and an interpolation of the colors. To compute the mid-way image of the morph, I averaged the pairs of correspondence points for the two images and created an average Delaunay triangulation of the averaged points. Next, I warped the image of myself and the image of Weaver into this average shape. The process of warping both images into the average shape involved iterating through every triangle in the average Delaunay triangulation and computing the inverse affine matrix transformation that transforms our averaged points into the original input correspondences. I then used the original input coordinate points and put them through an interpolation function which returns pixel values for all pixels in the average triangle. These values are equally cross-dissolved from my image and Weaver's image. The result of this algorithm is shown below.
To create a smooth morph animation between Weaver and myself, I used the
same algorithm from the previous section and added an additional two
parameters <warp_frac> and
<dissolve_frac> which take on values from 0 to 1.
<warp_frac> controls the intermediate face shape
configuration and is a weight on the point correspondences. For
example, a <warp_frac> of 0 means that the
intermediate face shape is equal to the face shape of Weaver's image.
<dissolve_frac> is a controlled weight on how much of
the color values from the input images are interpolated into the
intermediate image. If <dissolve_frac> is 0.5, the
color value is averaged between the image of myself and the image of
Weaver. In general,
<warp_frac> = <dissolve_frac>. Shown below is a
gif of the morphing process containing 50 images where
<warp_frac> and
<dissolve_frac> take on values from 0 to 1 with a
step size of 0.02.
In this section of the project, I compute the average face of an image dataset. The dataset that I used was the images of Danish computer scientists which can be found here. Each image in this dataset contains an image of a Danish computer scientist with 58 labeled correspondence points. Using these points, I was able to calculate the average face shape with a similar process as done in the second section and morph all images into a single average face of the population. Shown below are some images morphed into the average shape and the average face of the population itself.
Just for fun, let's see how an image looks warped into the average face and vice versa. Below I warped a picture of Lebron James into the average face and also warped the average face into Lebron's face. I used 43 pairs of correspondence points for the images.
Caricatures enhance the differences between an image and a population mean. We can produce a caricature of Lebron James by taking the difference between James' correspondence points and the average shape correspondence, multiply this difference by an alpha value (α), and add it back into the original image of Lebron James. I then morph the original image into the caricature shape to produce the final result. Here are some caricatures of James at different alpha values.
As we can see, the caricatures at different alpha values enhance Lebron's head position. This may be because the original faces are not aligned vertically.
In this part, I will change the gender of my friend's face. I used an image of the average East Asian woman's face and an image of my friend Félix. I will use the same process as shown in the mean face section to morph my friend into a woman. First, I will show just his face shape morphed into the average East Asian woman's face shape. Next, I will only morph the appearances of the images. And finally, I will morph both shape and image for the complete transformation.
Overall, I found this project to be very enjoyable and had fun producing the weirdly shaped morphed images. Creating the morph animation was the most difficult part for me because I had to derive the entire process of image warping and cross-dissolving the image colors. Despite this, I learned a lot by completing this project and hope to learn what other cool things the field of computer vision has to offer!