Jonathan Fung, CS194-26-abs
In this project, we aimed to use morphing in order to smoothly transition one image (faces) into another. We acheived smooth transitions by triangulating images based off a few correspondance points. The Delaunay Triangulation was used in order to not obtain too overly narrow images. Affine transformations were used in order to transform pixels from the source image to the destination image.
Part 1.1: Defining Correspondances
For this part, I made a graphical user interface that could be used to select correspondance points that were used to make a triangulation. The user can select as many points as possible, then press esc in order to exit. The four corners of the image were automatically added as points and saved to disk as a .pkl file.
Example points being selected.
Part 1.2: Midpoint Images
Now that we have our points, we can compute a triangulation of the image. We want to create a midpoint image given face A and face B. Both of these points have correspondance points generated from part 1. In order to create a triangulation, we average together all the points to create a "midway" correspondance, then compute a Delaunay triangulation based off of this midpoint image.
Example Delaunay Triangulation
My Face.
Alex Wu's Face.
With this midpoint triangulation, we then compute an Affine Transformation between each face's points and the midpoints. With this transformation, we then morph face A and face B into the midpoint image triangle by triangle, and then average together the corresponding pixels in order to gain a midway image. Note that when we cycle through all the pixels in order to generate a midway image, we need to do reverse interpolation or else there will be some artifacts. This is because if we do forward interpolation, there might not be pixels in the resulting image that will be filled in. Reverse interpolation guarantees every pixel to be filled in. Images below display the artifacts.
If we use forward interpolation.
My Face, midway.
Alex Wu's Face, midway.
Average of the two, the final midway face.
Part 1.3 Facial Morphing Sequence
Now that we can do midway morphs, we now aim to create a smooth transition from Face A to face B. We generate 45 frames between Face A and Face B. Instead of using a 50-50 balance to create a midpoint face, for every frame, there will be a weighted average of the points based on the (1/45) factor. As such, each frame will have its own Delaunay triangulation. Also, the pixel values will be weighted the same way as the points, creating a smooth transition between the two faces.
Frame 15 of Morph.
Animation of the face morph.
Frame 33 of Morph.
Beginning frame from the chain video morph.
Middle frame from the chain video morph.
End frame from the chain video morph
Part 2.1 Mean Face of Population and Caricatures.
Here, we try to compute the average face of a population employing the same technique as we have employed in the midpoint face. This time, we use an average of ALL the faces in order to create a mean face. I used data from the Danes scientist data set.
Example of one of the Danes
The Mean Dane
The Danish Scientist Morphed to Mean
Caricature of Dane 1, alpha = -0.8
Note how the caricature hasn't changed too much, because this face was in the dataset.
My Face, Caricatured. Face changed a lot. My eyes seem to be a lot smaller than the average Dane. The tilt in my head might have caused the shift.
Part 3 Bells and Whistles
I embarked on two different bells and whistles, the morphing music video chain between me and my friends, and changing my ethnicity + gender.
Part 3.1 Morphing Music Video
Here, I gathered images of 5 of my friends and chained them together. I overlayed music from the song "Why Can't We Be Friends?"
Emily
Jonathan
Rene
Brian
Yi
May
Part 3.2 Changing My Ethnicity/Gender
I obtained images of the Average Asian Female in order to female-ify myself. My theory was that I would align better to the average asian female than, for example, an average Ukranian Female. Let's test it out.
Average Asian Female
Me, an Asian Male.
Average Ukraninan Female.
Midpoint of me and Average Asian Female. Looks a bit more natural than the ukranian picture.
Midpoint of me and Average Ukranian Female.
In this project, we aimed to use morphing in order to smoothly transition one image (faces) into another. We acheived smooth transitions by triangulating images based off a few correspondance points. The Delaunay Triangulation was used in order to not obtain too overly narrow images. Affine transformations were used in order to transform pixels from the source image to the destination image.
Part 1.1: Defining Correspondances
For this part, I made a graphical user interface that could be used to select correspondance points that were used to make a triangulation. The user can select as many points as possible, then press esc in order to exit. The four corners of the image were automatically added as points and saved to disk as a .pkl file.
|
|
Part 1.2: Midpoint Images
Now that we have our points, we can compute a triangulation of the image. We want to create a midpoint image given face A and face B. Both of these points have correspondance points generated from part 1. In order to create a triangulation, we average together all the points to create a "midway" correspondance, then compute a Delaunay triangulation based off of this midpoint image.
|
|
|
|
With this midpoint triangulation, we then compute an Affine Transformation between each face's points and the midpoints. With this transformation, we then morph face A and face B into the midpoint image triangle by triangle, and then average together the corresponding pixels in order to gain a midway image. Note that when we cycle through all the pixels in order to generate a midway image, we need to do reverse interpolation or else there will be some artifacts. This is because if we do forward interpolation, there might not be pixels in the resulting image that will be filled in. Reverse interpolation guarantees every pixel to be filled in. Images below display the artifacts.
|
|
|
|
|
|
|
|
Part 1.3 Facial Morphing Sequence
Now that we can do midway morphs, we now aim to create a smooth transition from Face A to face B. We generate 45 frames between Face A and Face B. Instead of using a 50-50 balance to create a midpoint face, for every frame, there will be a weighted average of the points based on the (1/45) factor. As such, each frame will have its own Delaunay triangulation. Also, the pixel values will be weighted the same way as the points, creating a smooth transition between the two faces.
|
|
|
|
|
|
|
|
Part 2.1 Mean Face of Population and Caricatures.
Here, we try to compute the average face of a population employing the same technique as we have employed in the midpoint face. This time, we use an average of ALL the faces in order to create a mean face. I used data from the Danes scientist data set.
|
|
|
|
|
|
|
Part 3 Bells and Whistles
I embarked on two different bells and whistles, the morphing music video chain between me and my friends, and changing my ethnicity + gender.
Part 3.1 Morphing Music Video
Here, I gathered images of 5 of my friends and chained them together. I overlayed music from the song "Why Can't We Be Friends?"
|
|
|
|
|
|
|
|
Part 3.2 Changing My Ethnicity/Gender
I obtained images of the Average Asian Female in order to female-ify myself. My theory was that I would align better to the average asian female than, for example, an average Ukranian Female. Let's test it out.
|
|
|
|
|
|
|