Introduction

In this project, I will produce a morph animation of my face into someone else's face. Also I will compute the mean of a population of faces and extrapolate from a population mean to create a caricature of myself.

A morph is a simultaneous warp of the image shape and a cross-dissolve of the image colours. The cross-dissolve is the easy part whilst controlling and doing the warp is the hard part. The warp is controlled by defining a correspondence between the two pictures. For example, mapping eyes to eyes, mouth to mouth, chin to chin, ears to ears, etc., to get the smoothest transformations possible. After these correspondences are defined, we will create a mean structure. Then I apply Delaunay triangulation to the mean structure. After that, we compute the transformation between each corresponding triangles. Finally, we extrapolate the colour from the source image to the target image.

1 Mid Way Face & The Morph Sequence

1.1 Theory

First we find correspondence between two faces


Defining Correspondence

Then we find the Delaunay triangulation by using the mean correspondence


Delaunay triangulation on given points

Next, we need to find the transformation for each corresponding triangles.


Delaunay triangulation on given points¹

Transform Equation

To find the above transformation, we used Inverse Transform - meaning that given a target image, we need a transform to find that where in the source the pixels are coming from. This is because for Forward transform, the target image might not get all filled in and further computation is needed. As shown in the following diagram, getting each pixel from resulting image g(x’,y’) from its corresponding location (x,y) = T^-1(x’,y’) in the original image


Inverse Transform¹

Pixel Interpolation: Using the Inverse Transform, when we transform resulting pixel location back to location in original image, it might not and most likely not land exactly on the grid as shown in the above diagram. This is where interpolation comes in where we interpolate the pixel from its neighbours.

1.2 Results

Here are 3 git animated results. Midway are each individual faces warped into the average face. 2 of the final results are successful and one of them failed. This is due to folding. Since the two faces are facing into different directions, some of the points create folds. This can be fixed if we had 3D data and morphing can be generalised to 3-Dimension.

Midway	Midway	Morphing

Me	Someone	Someone Like Me


Leo	Agent Smith	Agent Leo


Harry	Sahai	Exhibiting Folding

1.3 Folding Issue Demonstration


Folding¹

This is because the point selected was trying to do something 3D-ish. We can remove the points that are causing the fold. Or morphing can be generalised into 3D. So with 3D data, this can be resolved.


Example showing the folds

For example, we can see the folds near the chin of Harry Potter.

2 The 'Mean face' of a Population

2.1 Theory

I computed the average face shape of a set of Denmark women from the IMM Face Database² dataset. Then warped everyone in the population into that shape. Finally, I averaged all the faces. The theory is already explained in 1.1. The only difference in this part is that instead of computing the mean face form 2 images, we compute the mean from a bigger population.


Before and After Warping into Averaged Face

2.2 Results

Averaged Face


Averaged Face

warpped Faces


My face warped into averaged face	Averaged face warped into my face

3 Caricatures: Extrapolating from the mean

3.1 Theory

Caricature is the idea of making one's feature more expressive. Algorithmically, we can subtract a face from the mean to get the features in the face that stands out from the mean. Then to make the caricature, we add the difference back on the original face to make those features more expressive.


Deviation from the mean

In this section, I will produce a caricature of my face by extrapolating from the population mean I calculated in the last step. To extract the features that make me me, I do the following:

∆X = X - Y_mean; where
    X is the image of my face
    Y_mean is the image of averaged faces like the one from previous part

Then, I can make a caricature of myself by the following:

Z = X + α ∆X; where
    Z is the caricature of myself
    X is the original image of my face
    α is caricature factor

3.2 Results

Caricature


Original Face	Averaged Face of a Population	Delta (note negative values)


α = 0.1	α = 0.3	α = 0.5

4 Change Gender and Race (Bells & Whistles)

4.1 Theory

The theory is pretty much the same as morphing. However, in this part, we will explore both warping and colour changing and combining them.

In this section, I will produce a gender and race change of my face by extrapolating from the population mean I calculated in the previous step. The population consists of Danish women. To make more more like Danish women, I find the features that make me me, then subtract this from my original as opposed to add from last section.

∆X = X - Y_mean; where
    X is the image of my face
    Y_mean is the image of averaged faces like the one from previous part

Then, I can make myself more resemble Danish women by the following:

Z = X + α * ∆X; where
    Z is the 'anti-caricature' of myself
    X is the original image of my face
    α is anti-caricature factor (should be -1<α<0)
    ∆X is the caricature of my face

Furthermore, I will warp my face into the Danish women mean shape to a certain fraction.

Z_coor = (1 - β) * X_coor + β * Y_coor; where
    Z is the result coordinates of myself warped into Danish women
    X is the original coordinates of my face
    Y is the averaged coordinates of Danish women faces 
    β is warping factor factor

4.2 Results

Below shows an animated result. After that you will see the decomposition of each step.

Animated result

Colour Shift to Danish Women


α = -0.3	α = -0.5	α = -0.7	α = -0.9

Structural Shift to Danish Women


β = 0.3	β = 0.5	β = 0.7	β = 0.9

Combining Colour and Structural Shifts to Danish Women


α, β = -0.3, 0.3	α, β = -0.5, 0.5	α, β = -0.7, 0.7	α, β = -0.9, 0.9

5 Regeneration of Doctor Who

5.1 Theory

Regeneration was the process by which Time Lords and others renewed themselves, causing a complete physical and often psychological change. It could happen because of severe illness, old age/fatigue or injury. It could also be invoked by choice, whether voluntary or involuntary. Conversely, regeneration could be prevented by choice, although choosing not to regenerate from fatal damage was essentially an act of suicide. Although, a Time Lord could delay the actual change in appearance, precise damage to a Time Lord could also cause a delay in the regeneration starting.