|
|
|
The purpose of this project was to explore the concept of face warping: Creating a video or gif that shows a seamless transition between faces depicted in two different images. To create these transitions there are two different things we have to worry about: the color and shape of the faces in the images.
Shape:
To create transitions between the shapes of the images we first have to define correspondences between the two images. We define correspondences as points between the two images that are associated with the same object: for example the the tip of the nose, the eyes, the overall jawline should correspond to the same correspondence points on both faces. Once we have these points we want to calculate a shape that is part way between the shapes of the two images. Here we can control the fraction(shape_fraction) of the face shape that the final image takes from both faces and to control the transition and calculate the "midway shape" of the two faces: final shape = shape_fraction*[correspondences in im1] + (1 - shape_fraction)*[correspondences in im2]
Once we compute the shape of the face partway, we have a small number of points that correspond to our two source images in our final image. We now want to use these points to understand where the points in the final image should come from in the source image. To do so we compute a delaunay triangulation on the final or partway shape. We then use this same triangulation on the source images: this tells us that a certain triangle in the final image corresponds to a triangle using the same triangulation in the final image. In order to understand the exact relationship between the triangles we compute a transformation for every triangle in the image that maps a pixel from the final image to a pixel from the source images by solving a system of equations with the vertices of the triangle. With this transformation we can then map where each pixel in the final image should come from and use it to construct midway images, slowly tuning our shape_fraction to create a smooth transition from one image to another.
Color:There is one more thing to consider when transitioning between faces: color! This however is much easier to deal with. When creating "midway images" we can simply crossdissolve the pixels in the two images. To control the transition between the two images' color we have a color_fraction that we use to control what fraction of the color of each image the final midway image takes: final color = color_fraction*[pixel color in im1] + (1 - color_fraction)*[pixel color in im2]. We repeat this for each color channel to cross dissolve the images.
In order to create transitions between the faces I enabled the user to manually select correspondence points between faces in the two images. The program then adds points at the corners of the image in order to allow the final image to incorporate the whole image properly.
|
To compute the midway image we follow the procedure defined above in computing transitions between the shape and color of the two images. Here while doing so we set the shape_fraction(warp_frac in the spec) to 0.5 and the color_fraction to 0.5, allowing the algorithm to compute an effective midpoint between the two images.
|
|
|
|
|
|
To compute the morph sequence we find multiple intermediate images by varying the shape_fraction and the color_fraction. In the case of this project we computed 45 frames to transition from image 1 to image 2 varying the fractions slowly creating the gifs shown below.
|
|
I computed the average face of the population by taking the average of the pre defined correspondences on the images given in the data set. Then I morphed the people in the data set to the average face making the shape_fraction such that the shape was purely the average shape and the color_fraction such that all the color was from the original image.
|
|
|
|
|
|
I computed the average face of the dataset for the group of 33 Danish males facing forward. By using the procedure described above I changed the shape of every image to the shape of the average Danish male and then took the average of the pixel values to blend the colors of all the individual images.
|
|
I then took the average face generated above and mapped it to the correspondences defined on an image of myself.
|
|
|
I also computed the image resulting from mapping my own image to the geometry of the average Dane computed above.
|
|
We can also use the techniques we have established so far to create caricatures of ourselves. A caricature is created simply by exaggerating one's features. To create a caricature we first find delta:the difference in correspondence points of our own face and those of the mean face of some population we calculated. This delta is our deviation from the mean. We can add this to the correspondence points of the mean scaled by some factor. final correspondences = average correspondences + scaling_factor*(individual face correspondences - average correspondences). Thus if the scaling_factor = 0 we just change shape to the average image shape. For scaling_factors greater than 1 the final image shows emphasized characteristics of the individual's face shape. For scaling_factors less than 0 the final image shows emphasized characteristics of the the average face shape. Thus by adjusting the scaling_factor we cna change the degree of our caricature going towards and away from the a certain set of corresopndneces that we define (in this case the mean).
|
|
|
|
|
|
I changed my ethnicity by looking up a precomputed image of the average of a certain ethnic population. In this specific case I looked up the average faces of korean females, european females and asian males. I morphed my own face to their shape, morphed just the appearance by cross dissolving and then combined both approaches to calculate a proper midway image.
|
|
|
|
|
|
|
|
|
|
|
|
I was able to implement automatic face detection using the python library dlib's frontal face detector. Once the faces were detected I looked at the regions of interest and used dlib's landmark predictor to isolate facial landmark on people's faces which were then used to be correspondence points. This allows the user to just submit an image and the correspondence points to all be automatically chosen. Automating this process allows the user to get better face morphs due to consistency in assigning points.
|
|
Use of Dlib inspired by: https://www.pyimagesearch.com/2017/04/03/facial-landmarks-dlib-opencv-python
Website template inspired by: https://inst.eecs.berkeley.edu/~cs194-26/fa17/upload/files/proj1/cs194-26-aab/website/