Colorizing the Prokudin-Gorskii photo collection

Exhaustive Search

In order to form a single RGB color image, I needed to align the three provided color channel images. Over a user-given range (ex: (-10, 10), (-15, 15), (-20, 20)), I displaced the R and G color channels along the x and y axis, then scored each displacement by comparing the Sum of Squared Differences (SSD) distances as an image matching metric, against the B color channel. I also tried using Normalized Cross-Correlation (NCC), an alternate image matching metric, but ultimately found that that implementation was slower/less efficient.

The initial results were, unfortunately, pretty mis-aligned. I attributed this to the large discrepancies in the outer edges of the image, at the glass plate borders.

Displacement: R (-1, 1), G (-1, 7)

Displacement: R (0, -6), G (1, 9)

Displacement: R (2, 3), G (3, 6)

So, I cropped 10% off the edges of each color channel before finding the optimal displacements, then applied those displacements to the uncropped images. That way, I could maintain the full size of my color channel images, while still allowing my displacements to be optimized for the center, focus subject of the image. This method yielded much better results, as shown.

Displacement: R (2, 5), G (3, 12)

Displacement: R (2, -3), G (2, 3)

Displacement: R (3, 3), G (3, 6)

Pyramid Search

For high resolution images (such as tiff files), exhausive search would require a much larger pixel displacement range, and would thus become much more expensive. In order to combat this, I implemented a faster search procedure, where alignment is done through a combination of displacements, calculated at various downscaled image resolutions. I start by downscaling the image channel to 1/8th of its original size, then running the exhaustive search on this smaller image. I can then use this course displacement to align the original image channel, scaling appropriately (in this case, since I downscaled the channel to 1/8th of its size, I apply a scaled version (x8) of the yielded displacement). I repeat this method three more times, downscaled at 1/4, 1/2, and 1. By using this pyramid method, I can maintain that my pixel displacement range is relatively small, since I am able to add scaled alignments at less expensive search resolutions.