Introduction
Sergei Mikhailovich Prokudin-Gorskii (1863-1944) [Сергей Михайлович Прокудин-Горский, to his Russian friends] was a man well ahead of his time. Convinced, as early as 1907, that color photography was the wave of the future, he won Tzar's special permission to travel across the vast Russian Empire and take color photographs of everything he saw including the only color portrait of Leo Tolstoy. And he really photographed everything: people, buildings, landscapes, railroads, bridges... thousands of color pictures! His idea was simple: record three exposures of every scene onto a glass plate using a red, a green, and a blue filter. Never mind that there was no way to print color photographs until much later -- he envisioned special projectors to be installed in "multimedia" classrooms all across Russia where the children would be able to learn about their vast country. Alas, his plans never materialized: he left Russia in 1918, right after the revolution, never to return again. Luckily, his RGB glass plate negatives, capturing the last years of the Russian Empire, survived and were purchased in 1948 by the Library of Congress. The LoC has recently digitized the negatives and made them available on-line.
Aligning Color Channels - Single Scale Approach
The first thing I did was implement a single exhaustive search which would shift the red and green filters over the blue filter. I calculated the ssd (sum of squared differences) between the alignments of the red or green filter over the blue filter for shifts of -15 to 15 pixels in the vertical and horizontal directions. to improve the alignment, I cropped the borders of the channels I fed into the algorithm. I spent some time tweaking parameters and leveraged an edge detection algorithm (canny's) to get better SSD values which got me some very precise results on the lower resolution JPGs.
Green Offset: 5, 2
Red Offset: 12, 3
Green Offset: -3, 2
Red Offset: 3, 2
Green Offset: 3, 1
Red Offset: 7, 0
Green Offset: 7, 0
Red Offset: 14, -1
Image Pyramid - Multi Scale Approach
The TIF images were larger than the JPGs so we needed a faster way to search large displacements of pixels. The resolution being a lot higher also means there would be more SSD calculuations. I used a scaling factor of 2 to recursively created images of lower resolution. I then took the lower resolution images, started to find the best displacement on the coarsest one, and then leveraged that to figure out where to search on higher resolution images. This is a dynamic programming problem, by reducing this to a smaller subproblem, I made it easier to do the alignment search on the color channels. However, it took time to get the algorithm with good results, and many small parameters affected it. This includes what interval to search over, how many levels the image pyramid should have, how much to crop an image after finding the right alignment and how should we use a best displacement from a low res image to better search on the higher resolution ones. The follow images are of the aligned TIFs and their best displacements.
Green Offset: 77, 29
Red Offset: 173, 33
Green Offset: 49, 15
Red Offset: 108, 12
Green Offset: 56, 22
Red Offset: 117, 29
Green Offset: 64, 13
Red Offset: 136, 23
Green Offset: 48, 23
Red Offset: 106, 41
Green Offset: 59, 19
Red Offset: 123, 18
Green Offset: 40, 18
Red Offset: 89, 24
Green Offset: 42, 6
Red Offset: 85, 32
One thing I could have taken advantage of was aligning on the green channel instead of the blue one on the Emir photo. I also played around with a few edge detectors to see what differences they had.