Between approximately 1909 and 1915, Sergey Prokudin-Gorskii traveled around the Russian Empire, capturing black and white images with colored lenses. He documented scenes ranging from landscapes to villages to castles. Here, I design a computer program to automatically align, merge, and adjust the negatives, restoring color to these early 20th century moments.
The collection features three-image sets where each image stores either the red, green, or blue channels. The primary challenge is aligning the manually captured negatives to avoid blurring. Once aligned, I combine the RGB channels to create a color image. Afterwards, I crop each image's borders and remove residue left behind by the imprecise negatives. Lastly, I use contrast stretching to highlight details and sharpen the picture.
I aligned images by comparing the red and green negatives to the blue negative. I shifted the red and green negatives up, down, right, and left by up to 15 pixels and chose the best matching shift. The strategy worked well with low-resolution images but failed to find sufficient shifts for high-resolution images, causing colored blurring.
I tested both Sum of Squared Distances (SSD) and Normalized Cross Correlation (NCC) when comparing images. Ultimately, NCC seemed to align images better, so I stuck with it for both the naive approach and the pyramid speedup. I optimized both approaches by ignoring a percentage of the negatives' borders where there are inconsistent frames and other imperfections. Instead I compared the centers of the negatives where the images were more consistent.
For high-res images, I used pyramid scaling to speedup the alignment. Instead of merely widening the [-15, 15] interval from the naive solution, I rescaled the high-res images by halving their resolution repeatedly until their width or height was under 100 pixels. Then, I used a [-15, 15] interval to find an alignment for the low-res negatives. I then repeatedly scaled back up and shifted around the (scaled) previous alignment to find a better alignment for high-res images without naively widening the interval. At each "scaled-up" shift, I tested a [-5, 5] pixel interval around the previous alignment. Overall, this found effective alignments for each sample image from the collection.
Some images were still somewhat off. The harvesters in particular had some blurring around a few of the workers. Since our shifts were purely translational in the x and y directions, any misalignment caused by rotation was left unsolved. Moreover, impurities in the negatives themselves can contribute to inaccuracies in the alignment process.
Although well-aligned, the pyramid scaling outputs leave border residue from slight mislignment and black borders on the colored negatives themselves. To crop the residue, I followed four steps:
My automatic cropping has issues with over or undercropping under certain circumstances. Since it considers any straight line near the border of an image to be part of the color residue, straight lines that are actually part of the image can be cropped. For instance, the Emir is overcropped since the door creates a natural vertical line that mimics a border, causing the program to crop it. On the other hand, tilted borders are spilled over multiple rows or columns. Since we only look at individual rows or columns when checking for lines, these titled lines escape dection. For instance, the Prokudin-Gorskii self portrait does not get cropped on the left since it's edge is tilted.
The cropped images still have issues with capturing a wide range of colors. Since the computer can display a finite number of colors and the residue borders can often be lighter or darker than the rest of the image, the pictures may reside in a narrow color spectrum, especially now that the bright borders have been cropped. To widen this spectrum and increase contrast, I implemented contrast stretching. First, I split each cropped image back into the RGB channels. Then, I found the 5th and 95th percentiles on the intensity historgram for each channel and rescaled the overall intensities based on these percentiles. The results were somewhat varying. In images like the Icon, the contrast stretching brought out a sense of realism. However, in images like the Fossil, it seems to add too much brightness and obscure details.
To see more of the Prokudin-Gorskii Photo Collection, please visit https://www.loc.gov/pictures/collection/prok/