CS 194-26 (Computational Photography) Project 4 - CS 194-26-ael


Transitioning between images is a useful technique for making a sequence of images more dynamic. A crossfade, for example, is a very useful way to transition between two scenes in video production. One can apply a similar approach to static images to combine them: by averaging pixel intensities, a "halfway" image can be created. For example, let's say we wanted to transform an image of Chris Allsman (me) to a picture of Chris Hemsworth.

We could make a sequence of images where each pixel was created by some weighted average of the picture of Hemsworth and Allsman. By changing the weights at each step, we get a crossfade.

However, such a technique only allows us to transition the color of the image - it would be much more effective if we could update the shape as well. There are multiple ways to warp an image - we could use an affine or projective transformation, for example. However, by applying a transformation to the entire image, we wouldn't be able to shape individual parts. For example, my overall face is wider than Chris Hemsworth's, but my eyes, nose, and mouth are not proportionally larger, so we can't stretch every feature by the same amount. Instead, we break the image up into smaller shapes by defining correspondences between the two images and creating a Delaney triangulation. Then, we warp each triangle using an affine transformation, which allows more granular control of features.
Here is an example of one such triangulation (the triangulation of the "halfway" image between Allsman and Hemsworth)

Fully Warping

By computing an affine warp on each triangle, we can the shape of each triangle at some point between the initial and final image. We can also find the "average intensity" by finding the pixel intensity at each image corresponding to some point in the triangle and averaging it. Doing so gives us an "average image" like so:

This is using the shape and intensity "halfway" between the initial and final image. We can also find such an image closer to the original image:

And closer to the final image:

Notice how not only do the pixel values change, but also the shape of the face and individual features on the face.
By computing many such images and putting them sequentially, we can see an animated morph between the two images.

Fun with averages

In the previous example, we saw how we might calculate an average image between two images. But there's nothing stopping us from doing so with many images! Here is the result of averaging the pixel intensities and (triangulated) shape of roughly 40 Danish men:

With this we can do interesting things: for example, we might warp the face of one of the subjects that created that image to the shape of the average face. Here are some examples (original images on the left)

We could also warp the shape of another individual to the average shape

Or the average face to another shape (the shape of my face)

Notice how warping to the shape of another image causes the image to take on those features - my face is more rounded and has a larger nose, for example. In order to exaggerate these features, we can interpolate across the difference between the two images. Doing so gives a characature like

Bells and Whistles

In the previous part, we saw how we could change the shape of images to fit the shape of another. Building off of that, it is possible to move in the direction of the shape/value of another image to get some interesting effects. For example, here is an example of interpolating towards the features of the "average woman" (pictured below)

Clearly, the orientation of these two images is insufficient for the task, but it can be seen how the features are tweaked to produce some similarities.