Image Warping


Note: if there are just blank squares, then your browser might no support HTML5 video, I've provided youtube links to mirrors for convenience. Should work in at least Chrome and Firefox.

Youtube Mirror

Context and Method

This project was all about using correspondence points in order to morph from one shape to the next. For my example I will be morphing my face to a picture of Joseph Gordon Levitt. First one defines points on each image that correspond to some feature of both images, such as the outline of the eyes, mouth, or hairline. Next a triangulation is found and used across both images. This triangulation is found by taken the midway of all corresponding points between both images and then using the delaunay function in Matlab to find an appropriate triangulation. These triangles are important as it lets us locally warp parts of the image to the same shape. The triangulations were found as shown below:

JGL

Me


JGL Triangulation

Me Triangulation

Using the corner points was important as it allowed for smooth transitions without noticing the edges of the main triangles used on the face.

The Mid-way Face

Next, using the triangulations we found, we compute the mid-way face. This is done by again averaging the correspondence points together, but this time, we need to morph each triangle from both images towards this average triangulation. This is done by finding the affine transformation of each triangle to move them towards the result. I used some linear algebra to figure out the coefficients for the affine transform, then took the inverse of this transformation. This way I go through each pixel within the mid-way faces triangles and use the inverse transformation to figure out which pixels I need to pull from the original to get the value. After warping both images to this mean shape I then averaged the values of the two warped images together to produce:

Mid-way face

Now we can apply this idea to create a smooth morph between my face and JGL's, but rather than taking a regular average we use a weighted average from one face to the next. At each time t, with pictures a and b, with some function f(t) that generates the weight, we now do:
a(f(t)) + b(1-f(t))
In order to calculate the shape and cross dissolve at the time. By doing this across steps between 0 and 1 we can then create a warping video:

Youtube Mirror

I'm pretty happy with the result! I had some unsuccessful ones where it was clear where the transform was and it looked more like just cross-fading, and this was due to some features such as glasses or open mouth versus a closed mouth, where smooth transitions don't exist. As said in lecture they don't span a subspace. It's either glasses or no glasses, no halfway point.

Mean face of a Population

The idea behind this part of the project was to take a population and figure out the average shape and color in order to get the 'mean face', or what the average person of the population looked like. I used the images from Here as my population, it includes pictures of a Danish CS department, and they are provided by Mikkel B. Stegmann for educational purposes. So the mean face was found by taking the 80 images, finding the average shape from the given correspondence points, and morphing all faces to that shape and averaging the colors together. The result was this:

Mean danish face

Here are some results where the faces where warped towards the mean, with the original followed by the warp:

The next step was to morph myself towards this Danish mean, and warp the mean towards myself:

Me warped towards the mean

The mean warped to me

It seems that I have bigger eyes and a smaller perceived forehead than the mean. The differences in face position also played a roll in how the

Caricatures

This portion of the project asks the question, well what if you go beyond or farther away from the mean? Well you get an exageration of the mean features or your own features! So to do this I took the difference in shapes between my face and the mean and then either added or subtracted this to my face to get something that was "more me" or "more danish". In the end it emphasizes my eye size more as well as my mouth and nose. I didn't think they could get more clown sized!

Original

More Me (add difference)


Me warped to the mean

More Danish (subtract difference)

Bell & Whistle: Music Video

One of the possible bells and whistles was to make a music video on a theme. One animation studio known as SHAFT is known for doing a particular scene in anime that they work on. This is called the SHAFT head tilt, and it involves the character looking back over their shoulder and it happens in mundane sitcom anime to ones with high level action and plot. From this I decided to morph between various head tilt scenes and put them to an ending theme of Bakemonogatari, one of the series they have worked on. While some of the transformations were iffy, generally transforms between those of the same series or art style were quite smooth.

Youtube Mirror


On that ordinary day
You suddenly stood up and said,
"Let's go stargazing tonight!"

Acknowledgements

  • Song: Kimi no Shiranai Monogatari by Supercell
  • What I Learned

    Triangles truly are the strongest shape! This was by far one of my favorite projects and the results were pleasing. By using simple linear algebra and traingles one can do local morphs and do things such as find averages to morphing to even another ethnicity. It's a samll transition you see a lot in videos, and it's used because it's simple and powerful how much you can do with affine transformations, and just treating things like features and color as vectors to be morphed into different a different basis. This is by far one of my favorite projects.