In this project, we aim to be able to perform face morphing of from my face into someone else's face, compute the mean of a population of faces and extrapolate from a population mean to create a caricature of yourself. Using rather painful techniques of morphing various shapes and points, we are able to apply transformations that morph every pixel into a targeted shape.

Probably the easiest bit of the entire project, this part we simply create a short function or script which allows us to plot points on an image by hand and export the resulting points to some file to be retrieved later. Then, after plotting the said points, we generate a triangulation with some library functions getting us a set of simplicies which we can then perform the transformations on.

The function is pretty simply in itself, we use pyplot to first display the image, and then ginput to record the plotted points of the image and save them to a text file. We then use the Delunay triangulation library function to get our set of triangles from the points that we have defined.

When plotting, we decided to show the path for which we have taken in yellow such that it would be easy to repeat the same path when plotting the same corresponding points for the second

Computing the average will be needed as we will aim to fit both our images of faces into the average shape during the warp. This particular part was fairly simple, as we quite literally just took the average of the two face shapes.

Performing the actual warp was a little trickier as it requires us to derive the transformation matrix. Now after extensive googling, I have arrived at a function of which I wasn't even sure how it worked, but it does by solving the linear algebra equation of turning one set of simplicies into another. Then I simply derived the transformation matrix for every triangle in my overall shape.

Lastly, the colour averaging part was probably the easiest, as after we generated a morph of both images shape-wise, it was simply a matter of adding a different ratio of the colour of each image together.

That being said, this part was a lot easier said than done, as conducting the actual warping of one image to the shape of another drew tons of strange bugs and errors, mostly to do with my somehow morphing of points within one triangle outside of bounds, even though the simplicies of the target triangle lie within bounds of the image. nevertheless, this resulted in some neat looking modern-art-like artifacts

Also, messing with shapes is rather hilarious

Consequently, having implemented the simple image morph, it was fairly easy to implement generation of the entire morph sequence, which was simply a matter of repeatedly applying the morph_image function over the same two images, with increasing values for warp_factor and dissolve_factor.

I noticed that the face and facial feature morphing was pretty solid, but the neck and shoulders kinda just dissappear. This is probably attributed to my lack of plotted points in those areas which can be rectified if I had simply also plotted the outline in those regions, but I didn't and I wouldn't because I'm just too lazy.

Here, we are asked to compute the mean-face of a popuation, which is simply the most average face-shape, with the most average colours.

Being too lazy to get my own faces and plot my own points, I simply pulled from one of the provided datasets of roughly 50 images, and removed all the black-and-white images because those skewed my colours quite severely.

Then, I simply divided all the shapes by the number of images that I have and summed them all up to comptue the average shape, and did the same with the colours after warping each image into the average shape.

Warped faces into mean shape

And finally, throwing my own face into the mix :D

Notice that for this portion, the triangulation of the mean-face is different because the plotted points that were provided as a part of the libraray do not match the way that I plot the points manually, and I didn't want to spend the time to figure out in what order it was done, so I just replotted the mean face :P

Here are some outtakes caused by exhaustion and very badly plotted points

Here we are asked to generate a caricature from extrapolating from the population mean of our dataset. What does this mean? Well, we simply subtracted our shape from the mean shape to obtain the "outstanding features" of my face relative to the mean shape, and then multiplied it and added it back.

This part was particularly painful, not in implimentation, nor the tediousness of replotting points, but rather as a shot to my ego. Let's just say that the displayed image was the most flattering of all the caricatures, and that counselling may be required for my self-image.

Now yes, the angle of which the selfie was taken isn't exactly matching, and also my hair is a little longer than usual, leading to a rather voluptuous mane. But damn, the f*$%ing algorithm went for my nose and mouth. Spared no hesitation in crucifying it's very creator hmmm? Ungrateful prick.

Having played around with faces for a bit, I was curious as to how well this works for drawings. Not brave enough to attempt the algorithm on something too unfamilar, I thought relatively untextured outlines of faces would come out alright

So I went digging, and from the below image set, I cropped out the faces, performed a bit of cleanup and created a looping morphing sequence.

Note that the original gif was too large, so it's actually uploaded to google drive

I found it rather neat that because of the higher detail for the keypoints I set around the eyes, the nicer morphs would make it look as if the eyes were actually moving during the transformation.

And here is the video as per the music video requirement