In this part of the project, we take two images A and B, pick a certain number of corresponding points that define key features of the image, and compute a "midway" face of the two based on this. To do this, we first find the mean of the keypoints for each pair of corresponding points and then generate a Delaunay triangulation on those "mean keypoints". Following this, we have a triangulation scheme that we apply to both image A and B. Then, we calculate the affine transform of each triangle of A and B to the corresponding triangle in the mean triangulation. This allowed us to create a mapping of sorts between the mean triangulation and the original triangulation, which allows us to retrieve the transform associated with each triangle. Then, I performed an inverse transformation on each pixel to retrieve the appropriate pixel value and used sciPy's interp2d function to help with this (in case the inverse landed us on a point between pixels). I then averaged the two warped images to compute a midway face (cross-dissolved with factor 0.5). Below is the midway face computed for me and Barack Obama, along with the original images.
Here is another example, with me (left), my roommate (center), and our midway face (right).
4.2 - Image Morphing
In this part of the project, we take two images A and B, and morph image A into image B. This is done by using the "mean points" used to compute the midway face in section 4.1 and generating a triangulation based on those. Following this, we generate "intermediate points" that are linear combinations of image A's keypoints and image B's keypoints, which is a parameter that we call "warp factor" -- the points computed are A_points * warp_factor + B_points * (1 - warp_factor). We apply the triangulation scheme from the mean points in 4.1 to the intermediate points calculated for the warp factor and then calculated the affine transform needed to warp the triangles from image A and image B towards their corresponding triangles in the intermediate triangulation. Then, I performed an inverse transformation on each pixel to retrieve the appropriate pixel value and used sciPy's interp2d function to help with this (in case the inverse landed us on a point between pixels). Following this, we can cross-dissolve the resulting intermediate faces of A and B into one image -- for mine, I kept the cross-dissolve parameter identical to the warp factor. To morph A into B, I computed intermediate faces of warp factors between 0 and 1, spaced by 0.02 (so 0.02, 0.04, 0.06, etc) as my warp factors, generated the intermediate faces for each warp factor, and then used an online GIF creator to make a gif out of all those images. Below are the results (note that I started and ended with the same images as shown in 4.1).
4.3 - Mean Faces
In this part of the project, I took the Danes dataset given to us in the project spec and computed the mean face of all the Danish scientists. First, I averaged the corresponding keypoints for each image and generated a triangulation based on this. I applied this triangulation scheme to the keypoints of each image in the dataset and computed the affine transform required to turn each triangle from the dataset's image into the triangle of the mean image -- kind of like what we did in 4.1. Then, for each image, I performed an inverse transformation on each pixel to retrieve the appropriate pixel value and used sciPy's interp2d function to help with this (in case the inverse landed us on a point between pixels). This step essentially allowed me to compute the face of each Dane warped to the average face's structure. Following this, I cross-dissolved the images onto each other, and this gave me the mean face of all the Danish Scientists.
This is the mean face computed of all the Danish scientists.
Here are some examples of individual faces warped to the mean face (mouse over the images to see the warps).
I also warped my face into the structure of the face of the Danes through a similar process as the one described above. Then, I warped the average Danish face to the structure of my face. Mouse over the images to see the warps.
4.4 - Image Caricature
In this part of the project, I created a caricature of my face. I did this through taking the keypoints for my face and for the mean face of the danes and taking the difference between the two. Then, I added some scalar multiple alpha of that difference back on to my original keypoints, generated a triangulation, and warped my face to that structure using a similar process to the ones above. Below, the original is on the left, and you see the result with alpha = 0.25, alpha = 0.5, and alpha = 0.75.
4.5 - Bells and Whistles
I created a morph of the Worcester Sharks for this part.
