face morphing and modeling a photo collection

Tony Zhao, CS 194-26 Spring 2020

triangles

I'm always looking to become Tiktok famous, but I don't have that viral energy on my face. If I could some how combine my face with a famous face, I believe I'd be able to drop out of UC Berkeley and pursue a fulltime Tiktok career. Time to use some computer science!

My face

Famous face (George Clooney)

Obviously we can't just layer the two photos on top of each other to create the midway face. Instead our first step is to mark up each picture and roughly figure out what part of the face goes where. We do this by adding points for the four corners, because corners should always be in the corners. I also then manually picked 45 points on the each face, making sure each point corresponded its pair. (i.e. one dot on each face's left pupil)

My face with dots

Famous face with dots

Now that we got our dots, we can use them to split up our image into triangles. We could obviously just randomly connect dots until everything is a triangle, but we get much nicer triangles and save some headache if we just use Delaunay triangulation. The only thing we need to watch out for is that the simplices (triplets) of points are consistent across both images.

My face with triangles

Famous face with triangles

midway

Now that we got our setup done, let's combine! We first compute where the points in the midway face should be. This is pretty straightforward as we just take the average position of corresponding points, and apply the simplices.

Midway face triangles

Now here comes the mathy part. We have to find an affine transformation over the triangles to project them into the midway face positions. We can solve a systems of equations, detailed in this online post. We then use this transformation matrix to inverse warp the pixels of each face onto the midway face. Let's combine the face of a very handsome man with George Clooney!

Yours Truly

George Clooney

Our love child

morphing

I played a lot of video games in high school. The one I spent hundreds of hours on was Oblivion. I still distinctly remember the character screen at the beginning, where there was sliders to decrease and increase aspects such as age or nose size.

The Elder Scrolls IV: Oblivion

I would entertain myself by playing around with these sliders and watch the face grotesquely morph. Well now I get to see my own face do that! When computing the midway face, we simply took the average between my face and George Clooney's face. However, we can instead try by using essentially a weighted average from 0 to 1, with my face being at 0, Clooney's face being at 1, the midway face at 0.5. We can linearly interpolate this value from 0 to 1 and create a sequence of faces gradually morphing from mine to Clooney's.

My face to George Clooney's

the average professor

Let's have some fun with this. I took the portraits of all the CS professors I have either worked for or taken a class and averaged them using the methods outlined above. I present to you, the average UC Berkeley CS professor...

The average CS professor

I know, as a whole, CS professors tend to be pretty attractive, but this image of the average professor takes it to a whole new level. Although somewhat ethnically ambigous, the average CS professor is distinctly male. 👀

I had an interesting thought about the theory that the average face is attractive because ugliness is noise. Wouldn't that make foundation and concealer essentially analog equivalents of low-pass filters? 🤔

Anyways, we can take the average professor face and project other faces onto it.

Professor Denero

More averaged

Daddy DeNero's face seems to be swollen on the right side. My theory is that the average professor faces slightly left in the photograph, while Professor DeNero faces slightly towards the right. Since we don't actually use a 3D mesh of the face, our algorithm compensates by stretching the right side.

Professor Efros

More averaged

We see that Professor Efros, when projected onto the average face shape, has larger eyes and lips. The deformation of the head in the upper left is likely due to inaccurate correspondense in labeling points.

Professor Sahai

More averaged

Professor Sahai seems to have gone through some problems, likely due to the deviation of his portrait from the others.

We can also see what I would look like as the average professor and vice versa.

Me as the average professor

The average professor as me

I feel as if I looks a little bit smarter...

a caricature and a bonus video

When linearly interpolating, we can think of it as taking the points of my face and moving them towards the average professor. We can go the opposite direction by extrapolation. In effect, I get a caricature of my face that should look very not-computer-sciencey.

Me

My caricature

Finally, here is a video of all the professors I've studied or worked with morphing into each other.