In this project, we will be morphing images of faces together, computing mean shapes and appearances of a population of faces (in my case, the Danish scientists in the Danes dataset), and caricaturizing photos by extrapolating from a population mean.
I started off with a couple of images of Professor Denero and Professor Rao and aligned them in Photoshop. The results of the alignment are my starting images here. Credit to the original images from this site: https://www2.eecs.berkeley.edu/Faculty/Lists/faculty.html?_ga=2.177849510.2025357903.1633659746-209332397.1633659746
Then I defined correspondence points using the cpselect function in Python and wrote down each pair of coordinates in a JSON file. Here were the features I used to inform my correspondence points:
1. Right part of left eye
2. Left part of left eye
3. Right part of right eye
4. Left part of right eye
5. Left jaw crease
6. Right jaw crease
7. Bottom of left eye
8. Top of left eye
9. Bottom of right eye
10. Top of right eye
11. Left center Pupil
12. Right center pupil
13. Left eye nose arch
14. Right eye nose arch
15. Bottom left nose crease
16. Bottom right nose crease
17. Left of left nostril
18. Right of left nostril
19. Left of right nostril
20. Right of right nostril
21. Left eye cheek crinkle
22. Right eye cheek crinkle
23. Left mouth crease
24. Right mouth crease
25. Mid big teeth line
26. Top of mid mouth
27. Bottom of mid mouth
28. Center chin
29. Left dimple
30. Right dimple
31. Bottom of left lobe
32. Left ear midpoint
33. Left ear top of arch
34. Bottom of right lobe
35. Right ear midpoint
36. Right ear top of arch
37. Left top of head
38. Right top of head
39. Left head arch in
40. Right head arch in
41. Presumed top of forehead
42. Image Corner
43. Image Corner
44. Image Corner
45. Image Corner
After specifying these correspondences, I computed a triangular mesh of the average shape of the two faces and visualized it. That's visible here:
Next, I warped the images of Prof Denero and Prof Rao's faces to the shape of the average, used the Delanauy triangular mesh and the skimage.draw polygon function to get the area of each "triangle" in the resulting (desired) shape, used an inverse transform matrix to encode the affine transformation necessary to "warp" one image to the other, and finally used an interpolation function to derive the desired color value for each particular pixel once the shape warping was complete to then morph the color channels as well. The result were the following "morphed" Professors Denero and Rao.
In this part, I morph the two images (of Professor Denero and Rao) together using a cross-dissolve and staging the warp to happen over time. That is, we start with a low degree of warp and as we approach the middle of the set of frames for the GIF (which I set to 45), we have a high warp. Then we phase back into low warp as we reach the end of the 45 frames and the image of the other professor. The effect is a morphing look between the two images of the professors over the course of the 45 frames. Throughout all of this, we have a cross-dissolve effect which impacts the "percentage" we see of each image, which we set to complementary factors and likewise stage over time to generate our morphing effect.
In this part, we start off by taking the dataset of Danes, who have conveniently provided us with the correspondence points / shape of each of their faces and then average the shapes together. The result gives us the "average shape" of a Danish scientist per here. The first thing we'll do is morph a couple of the Danes to this average shape we computed (as we did with Denero and Rao in the midway face part). You can see dane 1 and dane 40 below morphed to the average shape of the Dane dataset. Then, we'll actually go and morph every face in the Dane dataset to the average shape and then average those to also get the average APPEARANCE in the dane dataset. We see this below, under "Average Dane". Finally, I manually defined correspondence points between the average Dane face and Professor Denero to create an average shape between the two of them. What follows is then Prof Denero and the avg Dane face warped to the avg shape between these 2.
Finally, we explore extrapolating from the mean in order to essentially "bias" our images in certain way. In this part, I aimed to compute the average face of a male Dane and of a female Dane so that I could take the difference between the Danish female and Danish male (thus extracting some features that characterize female faces) and add this to a morphed version of Prof Denero's face (essentially making him look more female). Unfortunately, I ran into some errors here and wasn't able to complete this part. Still, below I include what my results were from computing the average male Dane face and the average female Dane face. I also defined a list of correspondence points between the avg male and avg female Dane faces so that I could subtract the male from the female and then morph this (correspondingly) with my Denero face to feminize prof Denero. Like I mentioned, however, I ran into bugs. Still, this approach I believe would've worked, as it would've used the correspondences I defined to first warp the two images (Denero and female extrapolation) into the same shape and then blend the appearances as I'd done in previous parts through interpolation.
Here is the bell and whistle feature I implemented: