CS194-26 Project 3 (something about faces)¶
Hello, welcome to my webpage :).
I mostly used images of members of FlyQuest's League of Legends team, since I knew that the LCS had nicely positioned profile photos of the players, and FlyQuest's players are just wearing white, which provides for nice results.
(Credits to https://lol.gamepedia.com/ and Riot Games for providing the images.)
Part 1 (Defining Correspondances)¶
Ok I literally just copied the align_image_code.py file from the last project so I could use the get_points method (with some slight tweaking for # of points and removing the timeout). This just prints the corresponding points, and then I paste the printed output into my notebook and assign it to some variable.
For the triangulation part, I just used the built in Delaunay triangulation. Here are some visualizations of it for the two main images I used for the early parts.
Part 2 (Mid-way Face)¶
For computing the mid way face, I followed the steps laid out in lecture: Find an "average" geometry by averaging the corresponding keypoints of each image, calculate the transform from the averaged triangles to the original triangles in each image (calculating said transforms by multiplying the original triangle points by the inverse of the averaged points, i.e. using some linalg), and then use said transforms to figure out the pixel values within the averaged triangles of the resulting image. Here's the result for midway, i.e. taking half of each geometry and of each color.
Also, thanks to the people on piazza for advising me not to use interp2d after spending far too long trying to get it to work at a reasonable rate :)
(Image A -> Mid-Way Image -> Image B)
Part 3 (Morph Sequence)¶
Here's the youtube video for it. I think the frame rate is slightly off, but it's because the converter I was using didn't like decimal rates, but hopefully it's close enough. This was just created by changing the Warp/Dissolve factors (well, they were the same constant for these) from 0 to 1, where 0 corresponded to interpolating "closest" to image to image A (i.e. all image A), and 1 corresponded to "closest" to image B (i.e. all image B), with 0.5 in the middle (the midway face above.
Part 4 (Mean Face)¶
I calculated the average face shape similar to the above, except I just averaged over all of the images. I chose the brazilian dataset (https://fei.edu.br/~cet/facedatabase.html), and only included the 'a' files (non-smiling). Here are a few random faces (1, 6, 11) morphed into the average shape.
And here's the mean face of the (non-smiling) population, followed by a selected face, followed by the selected face warped into the average geometry, and the average face warped into the selected face's geometry. I chose the face of my friend Jayanth (with his permission) since his face seemed to behave better than the LCS players' by far, likely because of the head shape and general positioning
Part 5 (Caricatures)¶
Here, I extrapolated using the previous mean photo (leftmost in the prior row) and the photo of Jayanth. The extrapolation here involved finding the difference between the images, and then re-adding it to the original image to emphasize those differences. Here, I varied the weight on the difference "image" when re-adding it to the original to see the various effects. We can see that when we increase the weight (i.e. emphasize the difference), it does in fact highlight some of the key differences between Jayanth and the mean, notables examples of which include the lighting color difference, where the right side (from the camera pov) is noticeably darker, and continuously gets darker, while the original mean was basically symmetrical, and the growing prominence/bushiness of the eyebrows. On the other hand, we can see his face lighten up and become smoother similar to the mean as we decrease the weight (move towards the mean).
Here are the weight values for the following images: (-1.5, -0.5, 0 (original image), 0.5, 1.5)
Part 6 Bell and Whistle¶
(singular cause I only did 1 thing - trying to get the photo of Jayanth to smile - I hope this is considered "changing [his] smile" i.e. the first bullet point of the B&W section)
Here, I took advantage of the second part of the Brazilian face dataset that'd been unused - the smiling faces. I took a similar approach to the extrapolation part with a slight variation. I first averaged the smiling faces to get the average smiling face, and then subtracted the average smiling face from the average not-smiling face to get an "image" that represented the difference of smiling vs not smiling.
Here is a visualization of that: Smiling -> Not Smiling -> Difference Image
Once I had this, I tried a few things with blending it and the Jayanth's face (summing, averaging, morphing), and what ended up having the best results was morphing after playing around with the Warp/Dissolve parameters (specifics in code) and re-aligning the two. The ending result does successfully give him a smile (with teeth!), but ended up changing his face a good amount as well (I considered just only morphing certain triangles, but then I figured I might as well just be cropping the image and processing it that way, and that kind of defeated the purpose of processing whole faces). I suppose one might argue that his ethnicity is no longer clear as well, so I may have achieved that too :).
Here are some of the results:
Order:
Original Smile Difference Image -> Original Jayanth Image -> Only Shape Morph -> Only Appearance Morph -> Morph Both -> Morph Both + Post-process to achieve closer color scheme to original
