Image of myself

This assignment explored morphing animation between images' corresponding points, computing the mean between images, and playing around a bit with caricatures.

In order to compute a morph between image A and image B, first we must find a "Mid-Face". This image is a half-way mix between two images. In order to create it I had to find the mid points between the two images' correspondence points and compute triangulation. I used scipy's Delaunay function to compute the triangulations. Then for each point in each triangulation in each image, I computed the barycentric coordinates in respect to the mid-face triangles and used those to linearly interpolate the corresponding points in image A and image B. Finally, in order to account for points that are not integers, I used scipy's interpolate function to find the best integer pixel value from both image A and image B. Finally I added the corresponding pixel values between image A and image B and averaged them to get a final mean result.

Here is a mid-face morph between Adrien Brody, photographed by Martin Shoeller, and an image of myself.

Next I worked on creating a morph sequence between two images. This was simply executing the `morph`

function I wrote for the mid-face above, except that now I use a `warp constant`

that goes from 1 to
0 decrementing in steps of `1/(frames per second)`

. Some results I got are shown below.

I thought it would be fun to experiment on showing someone throughout time (in this case myself). So the first image is a picture of me when I was a kid, and the second picture is an image of me now. The result is kind of like a time-lapse!

In the class slides this morph function is said to need two for loops. Prof. Efros mentioned in class that if you could do it in a single for-loop he would give extra points. Well, I made it my goal to vectorize the #$%@ out of my morph function, and by using numpy the way its meant to be used, I achieved morphing my images using no for loops at all! Both images and every thing done to either is done at the same time. The same goes for the mean face below. I will talk about it and show some results soon, but keep in mind that the average of each face is done all concurrently.

I implemented two different methods for achieving image morphing. The first was demonstrated
above, using barycentric coordinates taken from the mid-image and using them to interpolate each
point in it's original image given it's triangle's vertices. The other method was creating a change
of basis transformation matrix that transforms a point from the mid-face into a point in the original image. This
matrix is composed of the product between the basis of the mid-face (which can be calculated by
using the three points to create your two coordinates), lets call that T0, and the basis of the original image,
lets call that T1. Then you multiply each pixel-point in the mid-face by `inverse(T0) * T1`

and you
get the corresponding pixel-point in the original image.

From inspection and testing, it turns out that both methods return the exact same results. Also, against my initial intuition, the inverse-transform method is about 10% faster than the barycentric method. Although, I still rather using barycentric coordinates because the code itself is shorter, more clear, and easier to work with. Here are two results comparing the two methods, demonstrating that they give the same results:

You might notice in some of the videos some "jumping" effects between frames. I believe that many of these issues are due to the number of points created. For example, in the image below both images have different patterned background but I didn't add corresponding points to it, therefore you see this "jumping" effect happen between some frames due to those point lying on top of a triangulation's edge:

Note that the black-and-white sandy noise is due to downscaling my video and converting it into a gif. If you'd like to see the high-res video please email me!

The task for this part of the project was to select a face dataset with attributes and use it to compute a "mean" face. Being Brazilian, I naturally used FEI Face Database. Below you can see the results for the generic mean face, the male mean face, and the female mean face.

Just for fun I decided to also calculate the mean face between my siblings and I (all brazilian too).

Playing around with my siblings faces made me wonder how close we are to the mean face between our parents and grandparents. So I took an image of my four grandparents and my parents when they were all near a similar age (around 20) and starting taking the mean between all of them. So I calculated the mean between each pair of my grandparents, then I took the mean between each pair with their child (my mom and my dad). Then I took the mean between those two final means and got my final family mean! Check it out:

What do you think? Are my siblings mean similar to our "expected" mean based on our genes? Either way, this was a fun little investigation.

Here I played around a bit to see how "off" my own face is from my people! I used the female brazilian mean face to try and get more accurate images. The results are pretty funny, which just shows how diverse us Brazilians really are. The warping factor below is how much I applied to my face, and (1 - warping) is how much I applied to the mean face.

In the images below you can see me playing around with extrapolating the warping constant to above and below zero.

This is a class warp that I participated in! We each worked on a little chunk of it (from the person before us morphed into ourselves).

❤