Project 1

Overview

In this project, I learned how to use three versions of the same image with only single RGB color channels to create an image that overlays the images to colorize the original black and white appearance.

Single-Scale Align

For smaller, low res images, such as the jpgs, I used a single-scale implementation that simply looped through possible displacements and used the Sum of Squared Differences formula to calculate how well the images matched. Specifically, I found green and red displacements by comparing them on the blue channel. After looping through 30 displacement options, I selected the best one, and aligned the images using that displacement.

Multiscale Align

In order to handle larger images efficiently, I used a pyramid search procedure so that the alignment is done at smaller scales (coarse) and increasing and updating as the same method is called again and again using recursion. This technique is called a "coarse-to-fine pyramid speedup," as the smaller coarser versions are built up and refined to a larger scale.

Example Images

Before After After Cropping Output
cathedral: green (5, 2) red : (12, 3)
monastery: green (-3, 2) red : (3, 2)
tobolsk: green (3, 3) red : (6, 3)
castle: green (33, 2) red : (98, 5)
emir: green (48, 23) red : (68, 40)
harvesters: green (60, 18) red : (124, 17)
icon: green (40, 18) red : (89, 24)
lady: green (48, 8) red : (112, 10)
melons: green (86, 0) red : (180, 12)
onion_church: green (50, 26) red : (108, 36)
self_portrait: green (77, 29) red : (175, 37)
three_generations: green (49, 17) red : (108, 13)
train: green (42, 6) red : (85, 32)
workshop: green (53, 0) red : (105, -12)

Chosen Images

Before After After Cropping Output
lake: green (0, 0) red : (-10, -8)
rock: green (4, 4) red : (9, 6)
mosque: green (30, 6) red : (80, 8)
woods: green (-21, 10) red : (-33, 12)

Issues

Some of the lower res photos, such as lake and cathedral look a little less clear, because its more difficult to distinguish the granularity of the picture when calculating the "best" displacement. Because the picture is less distinguishable with less granular objects and more background, it can appear as "blurry" to the eye when colorized, because the picture is probably not at its exact best alginment.

Bells and Whistles

Auto Crop

I tried to implement automatic cropping, specifically for the tiff images, because they tended to have larger, more distinct borders to be cropped. I detected borders of different colors by taking a difference between the color on each pixel and the color we are detecting, and used this link as a reference for how to create a mask and remove the detected borders. In addition to white, I also implemented modifications for a blue, yellow pink and black borders.
This implementation works well for the most part, however, for some more broken up/faded borders, such as in castle and workshop, detecting the borders did not work as well. Also, potentially, if there were an image with a white, blue, pink or black border in the middle, it might crop incorrectly.