Project 2 - Brian Agustino

Part 1 - Defining Correspondences

As our goal in this project is to morph images from one face to another. A key part of this is to choose the corresponding points of the features of each face by using the ginput function. When choosing this list of points, we must maintain a similar order of choosing points (ex: Choosing to start on the left ear for both images).

After choosing approximately 40 corresponding points from each image, we could then find the average or midway shape of these points by averaging the points and create a midway triangulation by using the Delaunay to create the triangulation

Part 2 - Computing the “Mid-Way Face”

As we have successfully gotten the triangulation of the mid-way face, we would like to morph the images into the mid-way face.

Warp Images to Mid-Way Triangulation

The first step to do this is to warp the images into the mid-way triangulation. To do this we would first calculate the set of transformation matrixes by using numpy.solve on the mid-way points and the source image points. We are able to do this as:

Note: We add a row of ones for each point matrixes.

Inverse Warp

Once we have the set of transform matrixes from the source points and the mid-way points,

1. We would then first create triangle masks from the triangulation points
2. We would apply a dot product with the inverse of the transform matrixes for each set of transform matrixes and points within the triangle mask.

This would implement the inverse warp on the pixels within the triangle mask we created.

Interpolation

We would then create the interpolation function (scipy.interpolate.RectBivariateSpline) and feed the interpolation function with the inverse warped points to interpolate all the color of the source image. We could then sum up this warped triangle mask into the resulting "canvas" creating a mid-way image

Cross-Dissolve

After implementing all these process for both images, we would then define an alpha for cross-dissolving (averaging) the images with the equation result = (1-a)image1 + aimage2 Defining alpha as 0.5 would give us the resulting mid-way image where the resulting image consists 50% of warped image1 and 50% warped image2

The Morph Sequence

As we now are able to apply the average morphing between two images, we could then apply this morphing multiple times with ordered equally-spaced alpha values into and combine them into a morphing sequence.

We can use this over several seconds to produce a GIF showing a morphing "process" between the first and second image

Morphing Sequence in 10 seconds with 45 frames

The "Mean Face" of a Population

We would now like to find a "Mean Face" of a population from a set of face images. For this I chose the Danes set of images and selected the first 40 person in the set of images (ex: 01-1m.jpg, 02-1m.jpg,..., 40-1m.jpg)

These are the set of images that I selected to find the "mean face" of the population

After extracting and parsing the given correspondence points for each image, we should also include the 4 corner points of each images.

We could then compute the average of all these images into a single target average triangulation points.

Where we could then apply the morph we implemented in the previous parts on each of the 40 face images.

We can then compute the mean of all the warped danes images by using numpy.mean and get the resulting "mean-face" of the population

Mean-Face Population of the 40 Danes images

We could then try to morph the mean face population to our face.

Caricatures: Extrapolating From the Mean

As we have limited the warping_fraction and the cross_dissolve_fraction to between 0 to 1 or in the range of the 2 images, we can get the "caricature" of an image by extrapolating or amplifying the warp_franction and cross_dissolve_fraction to be less than 0 or greater than 1.