This project seeks to create face morphing algorithm that allows smooth animated transitions from one image to another. First, the start and end images are annotated with matched keypoints. From these keypoints an “average geometry” is computed that breaks the image into non-overlapping triangles. The triangulation is such that each triangle in the first image is paired with a triangle in the last image that has the same keypoints as vertices. Note these keypoints are in different locations in these images so the triangle’s geometries are different. The animation is generated by gradually warping triangles in the first image into the shape of the last while cross-dissolving the pixel values. Thus, a smooth animation is created by gradually warping the geometry and blending the pixels simultaneously.
Matching keypoints are manually annotated on both the start and final image. Below are two examples of such matched annotations.
|
|
To generate a Mid-Way image the average geometry of the image has to be computed, and the keypoints have to broken into triangles. The average geometry is found by averaging the corresponding keypoints in the first and last frames. A Delaunay triangulation is computed on the averaged points. This triangulation "groups" vertices into non-overlapping triangles, in such a way that eliminates skinny long triangles (which could cause visible artifacts). Note that while the triangulation is kept constant through the entire animation, the vertices' position in each frame changes! This change over time captures the change in geometry during the animation. Below is a visualization of the triangulation overlaid on both start and end frames.
|
|
The Mid-Way face is created by fitting an affine transformation that warps the original triangles from start/end images into their location in the Mid-Way image. After the geometric warping is complete the two triangles (one from start frame and one from last frame) are cross-dissolved 50%. This creates the perception of blended geometry AND blended coloration, which will be crucial to create the final animation. Below is an example of a generated half way image, with average geometry overlaid on top.
|
All the framework is laid to generate image morphs! Instead of generating just the mid-way image, N images are generated that vary (in our case linearly) from start to end. I set N to 45 and playback the animation at 30 FPS. Below are some cool GIFS I generated!
|
|
Using the image morphing algorithm I created a themed music video of morphs between animals in the cat family. Here is the video link. Enjoy!
In this section data driven methods for computational photography are explored. The mean face of a human population is computed, and used to create a caricature of my face. The Dane dataset was used for all computations, and only the neutral faces (photo 1) were used.
The Dane dataset is composed of pictures of Danish computer scientists. Each picture is annotated with the same set of keypoints. The average face is generated by computing the mean geometry of the dataset (average of all keypoints), warping each face in the dataset to the mean geometry, and then averaging all the warped faces.
|
|
|
|
|
|
|
|
|
|
Here is the generated average face.
|
After generating the sample mean image and sample mean geometry, I took a picture of myself and annotated it with the same keypoints used in the Dane Dataset. I then morphed my image into the geometry of the Dane set and morphed the Dane image into my geometry.
|
|
|
Caricatures are generated by warping my face into the average geometry, calculating the difference between my face and the average, and then adding back the difference. At this point I both warp back into my face’s old geometry, and also warp the image into my shape plus the difference between my face’s geometry and average face’s geometry (with scaling factor of course). Below are the results.
|
|
|
Comparing the caricature in my geometry to the original image the most noticeable differences are due to color. Obvious the background is different due to changes in background, but additionally my skin tone is darker than original image! This is expected as my skin tone is darker than the sample mean’s.
After warping into the caricaturized geometry changes in my face shape are apparent. My nose is noticeable larger and my face is longer! This seems to match the qualitative difference between my picture and the average Dane picture.