CS194-26 Project 3 - Eric Leong

Overview

In this project, I learned how to use affine transformation, triangulation, and color interpolation to average, warp, and morph images.

Defining Correspondences

In this section, I used Python matplotlib's ginput tool to select corresponding points between 2 images, which I later saved. Then I computed the mean of the 2 sets of points then I utilized the scipy package to compute a Delaunay triangulation of the mean points.

apple
Me
apple
George Clooney
apple
Point Selection
apple
Triangulation
apple
Triangulation

I selected the points on each image manually and added the 4 corners to each set of points. The triangulation should be the exact same for both images.

Computing the "Mid-way Face"

In this section, I computed the mid-way face by iterating through each triangle in the triangulations, find the set of points that fall into the triangle, computing the inverse transformation from the target triangle to the source, then applying this transformation to the set of coordinates. Finally, I found the interpolated values for each color channel and set them to resulting image. After repeating this process for both images, I combined them by cross dissolving. I repeated this process for all triangles to form the mid-way face. Note: since we are working with triangles, we can perform affine transformation, which will always be able to transform any triangle to another.

apple
Me
apple
George Clooney
apple
Mid-way Face

The Morph Sequence

In this section, we have 2 parameters, warp fraction and cross dissolve fraction. Warp fraction determines the relative strength of our warps; the higher the warp fraction, the more we warp each triangle in the image. Dissolve Fractions is primarily related to how we combine the colors of each of the images. For our morphing sequence, we iterated from [0, 1] for both fractions and kept them equal to each other. We then used our functions for computing the mid-way face to compute numerous frames of our morphing sequence. I used 50 frames for the following morph sequence.

apple
Face Morph (Me to George)

The "Mean face" of a population

Here, we utilized the Brazilian Face dataset which included annotated points, to produce the mean face of the data set. The procedure involved using the mid-way face functions but instead of incorporating just 2 images, we incorporated the entire data set. I decided to find the mean face for both the smiling and neutral faced images, which is why the mean face appears to be both smiling and neutral. I then found the morph sequences for some images from the same data set.

apple
Brazilian Dataset Mean Face
apple
Brazilian Face 1
apple
Face 2 to Mean Face
apple
Brazilian Face 2
apple
Mean Face to Face 2

Warp into geometry

Here, we start off with blurring an already sharp image using a gaussian filter then applying our sharpening filter on it. As we can see, the resulting image is less "sharp" than the initial image and looks like it might've even lost some critical detail. Once we removed the high frequency pixels by blurring the image, the sharpening filter is unable to bring it back and we lose some essential information about the image. The sharpening filter only acts to strengthen the remanining high frequency pixels that weren't removed.

apple
My Face Warped from Mean Face
apple
Mean Face Warped from My Face
apple
My Face morphed to Mean Face
apple
Mean Face morphed to My Face

Caricatures: Extrapolating from the mean

In this section, extrapolating from the mean to form caricatures meant going over the warp fraction of 1 when finding the warp from the mean face. Moreover, warping from an image just meant only taking into account the image's geometry and not the color, therefore the dissolve fraction was set to 0.

apple
My Face Warped (warp frac .5)
apple
My Face Warped (warp frac .8)
apple
My Face warped (warp frac 1)
apple
My Face warped (warp frac 1.2)

Bells & Whistles


Changing Gender and Ethnicity

I warped and morphed my face into Kim Kardashian, enabling me to have some luscious eye lashes.

apple
My Face
apple
Kim Kardashian
apple
Mid-way Face
apple
Kim's Face Warped
apple
My Face Warped
apple
Morphing Sequence

A Morphing Music Video

I made a music video, morphing some of my peers faces of different orientation, gender, and ethnicity. I also displayed some of the mid-way faces which I found pretty interesting!

apple apple apple apple apple