In the early 20th century, a Russian chemist and photographer named Sergey Prokudin-Gorskii journeyed around the Russian empire, capturing bucolic scenes and still-lifes with his camera. Although B&W photography was still the norm of his time, he took 3 photos of each scene, one through each of red, blue, and green filters. He understood that these images could be later superimposed--say, if, they were projected through their respective filters--to form color images. In this project, B&W image triplets archived by the Library of Congress are used to digitally form three-channel color images.

Once the image triplets are sectioned into their respective channels, they need to be precisely aligned to avoid color fringing. I aligned the green and red channels to the blue, and used the L2 norm to evaluate the alignments. I sought to minimize the L2 norm of the blue channel subtracted from the aligned other channel--the smaller the norm, the more similar the two images. This approach is not perfect because ultimately the images to be compared are different channels and have different pixel intensities for the same image features, and so it is possible to find pathological examples where certain misalignments may result in small norms simply due to unfortunate pixel overlap, relative to whatever the "true" alignment's norm is.

The JPG images are small enough that I computed the norm for all xy displacements in (-15,-15)x(15,15) from the origin. The TIF images are very large and take too long to process, so I used an image pyramid to expedite the search for the best alignment. The steps of an image pyramid are as follows:

• Start with a coarse resolution and find the best alignment starting from the origin.
• Double the resolution, and find the best aligment, starting from (2r,2c), where (r,c) is the best alignment from the previous iteration
• Recurse until the full resolution is reached

Optionally you can reduce the search window on each iteration to further speed the search, since the window only needs to be large initially and can shrink as the best alignment is approached (i.e. coarse resolution pixels represent more of the image than fine resolution pixels).

Below are the results on various images for the collection. The numbers in brackets are the xy displacements from the origin found to best align each channel, indicated by g or r. You will notice that the red component of the image of the emir (siiting man in blue robe) failed to align properly. This is most likely because of the intensity issue mentioned earlier. Because his robe is so blue, the red image's robe pixels are very dark, so when the red and blue channels are subtracted, the alignment prefers overlapping those dark pixels with the back wall because it results in a lower norm than if those pixels were subtracted from such bright blue values. The other images lack large, saturated regions of red/blue/green, which is why they succeeded in alignment. Images with lots of color mixing, low saturation, and equal representation from red/blue/green would fare the best under this approach. Relying on edge detection to match features instead of pixels would fix the emir's photo.