Blending between images cleanly can be a complicated process. For some
images, simply playing with the opacity of an overlaying image can produce
a very natural looking blend.
A timelapse that demonstrates how blending can look if an image is aligned.
However for others, this technique fails - primarily because the images
are not aligned properly - causing "ghost" artifacts to appear during
the blending process.
A quick solution to this problem would be to align the two images
prior to running the cross-dissolve. This works via a
simple translation for the images above. However, this technique
fails to work in general, especially for images like faces.
Failed blending because features are not aligned.
What if instead of trying to find a single transformation that
works for every single pixel, we allow for many transformations
to be applied across the image.
This method looks to modify the shape of an image
along with the color while performing the cross-dissolve and
produces visually satisfying results.
The process works as follows: a user annotates keypoints for both
images. These points act as the correspondence between the two images.
For morphing between faces, the user might want to outline the
eyes, nose, and other important features using the keypoint
correspondence.
Then from there, the program defines a triangulation of the image.
An efficient method that I used for this project is known as Delauney
triangulation.The triangulation should be identically constructed in each of the
images - such that each image has corresponding triangles.
Between each of these triangles, we can calculate a transformation
based on the vertices. Thus the number of transformations is equivalent
to the number of triangles in the resulting triangulation. The
transformation is then applied to each of the pixels within the triangle,
giving us a visually appealing morph between images.
Correspondence UI
To build a coherent correspondence, I hacked together a fairly simple
correspondence picker using matplotlib. The basic motivation was that
I wanted to make sure the correspondence between two images was roughly
aligned and wanted an easy way to ensure this.
The UI allows a user to define a correspondence with a set number of
points, visualizing past points with a marker.
Then, the UI allows a user to define a second correspondence, using
the first as a reference. The user can toggle between sequential
mode annotating one point at a time and batch mode - annotating all
points at once.
Mid-Way Face
The first step in making a cohesive morph is to create
a "mid-way" face. This requires us to find a face that
has the average shape between two images. For this part
of the project I morphed an image of myself and an image
of george clooney
The resulting midway face simply requires us to morph each
image to the "average" geometry, then we perform a simple
cross-dissolve -taking the average of the colors of each image
Morph Sequence
With this technique in hand, we can then make a cool
morph sequence that demonstrates a gradual transition from one image
to another. Basically, we gradually change the
weighting of shape and color between frames, creating a subtle
transition between the images.
"Mean Face" of a Population
With this new technique in hand, we can go further and make an
average face for an entire population. I used
this dataset
containing a whole collection of Danish computer scientists.
Average face of danes
Now we can then morph images from this population into the mean face
Original image
Image mapped to average danish shape
Original image
Image mapped to average danish shape
Original image
Image mapped to average danish shape
Original image
Image mapped to average danish shape
Additionally, we can isolate different portions of this population such as
the men and the women, giving mean populations for these specific subgroups.
As a note, the dataset is leaning much more towards men than woman (33 vs 7),
which is why the average image looks more manly and looks really close to
the image of the average male.
Average female danish computer scientist.
Average male danish computer scientist.
Original image
Image mapped to average danish female geometry
Image mapped to average danish male geometry
Original image
Image mapped to average danish female geometry
Image mapped to average danish male geometry
Original image
Image mapped to average danish female geometry
Image mapped to average danish male geometry
Original image
Image mapped to average danish female geometry
Image mapped to average danish male geometry
To try and see what a balanced population might look like, I also
made an average face balancing both genders in the subpopulation used
to generate the mean face.
Average danish computer scientist - genders balanced
Original image
Image mapped to average danish shape
Original image
Image mapped to average danish shape
Original image
Image mapped to average danish shape
Original image
Image mapped to average danish shape
My original face
My face mapped to average danish face
My face mapped to average female face
My face mapped to average male face
My face mapped to average balanced face
Extrapolating from the Mean
Not only can we map to the means, we can actually extrapolate away from these means to create
caricatures of our images. We run this process by simply taking the image and subtracting or adding
the mean by some constant factor such that
$$X_{new} = X + \alpha * mean$$
Original image
Image extrapolated to female geometry with $\alpha = -0.5$
Image extrapolated to male geometry with $\alpha = -0.5$
Original image
Image extrapolated to female geometry with $\alpha = -0.5$
Image extrapolated to male geometry with $\alpha = -0.5$
Original image
Image extrapolated to female geometry with $\alpha = -0.5$
Image extrapolated to male geometry with $\alpha = -0.5$
Original image
Image extrapolated to female geometry with $\alpha = -0.5$
Image extrapolated to male geometry with $\alpha = -0.5$
Bells and Whistles
Crazy Face Warps
Face Blow Up
Growing a new head
My objective for this B&W was to make a morph that looked like I regrew my head. To start,
I cropped out the background of the original image (it caused problems in a few of my tests)
and then also cropped out my head for one of the pictures.
The source images for this morph and the headless image's correspondences
Although I did get a neat effect, I wasn't very satisfied with the result (seen in the image on the left below).
However, I did discover a variation that I thought was very amusing,
especially when run at high speeds.
The original intention was to make a creepy-looking head regrow.