Images of the Russian Empire: Colorizing the Prokudin-Gorskii Photo Collection
1 Overview
Sergei Mikhailovich Prokudin-Gorskii (1863-1944), a color photographer pioneer, traveled across the Russian Empire and captured color photographs of his surroundings. He was a man ahead of his time and envisioned that the future would have color photographs. He captured the images by recording three exposures of every scene onto a glass plate with the Red, Green, and Blue filters. His RGB glass plate negatives survived and the Library of Congress (LoC) purchased them and recently made the digitized versions available online.
2 Project Goal
The goal of this project is to use image processing techniques to automatically produce a color image from the digitized glass plate images captured by Prokudin-Gorskii.
3 Data
The digitized glass plate images provided are a set of hi and low-resolution images. See below for examples.
Example of Low Resolution Images
4 Method
The digitized glass plate images were colorized by firstly splitting the images into three color channels, Red (R), Green (G) and Blue (B), and then aligned and stacked to form a single RGB color image. The B channel was used as the base channel to align the R and G channels. The use of the B channel as the base channel will be later investigated. This investigation was inspired by Saurav Mittal’s Fall 2020 project submission. Saurav experimented on using different base channels and obtained interesting results.
Alignment of the images can be achieved in many ways. The exhaustive search approach was used to align low-res images and the image pyramid approach was used on hi-res images.
4.1 Matching Metric
To align the RGB channels, the following matching metrics were considered:
Sum of Squared Difference(SSD) or \(L_2\) norm: \[ SSD = \sum_{i=1}^{n}\sum_{j=1}^{m} (Image_1(i,j) - Image_2(i,j))^2 \]
Normalized Cross-Correlation (NCC): \[ NCC = \frac{Image_{1}}{||Image_{1}||} \cdot \frac{Image_{2}}{||Image_{2}||} \]
SSD appears to align the images better than when using NCC and the program runs faster when using SSD. Hence SSD will be the matching metric used in this project to align the images.
4.2 Exhaustive Search
Exhaustive search is a brute force method for aligning images. It involves splitting the image into RGB channels, then searching over a window of possible displacements, using a range of [-20,20]. Each displacement is scored using a matching metric to identify the best displacement vector (x,y) that will result in the lowest SSD score. The identified best displacement vector (x,y) is used to shift the original image to align with the base channel.
The images below were aligned using the Blue channel as the base channel.
Exhaustive Search Output of Low Resolution Images.
Running an exhaustive search on the hi-res images took a long time to complete. This is why using the Image pyramid approach for hi-res images is recommended because it efficiently tries to align the channels from low level to high-level scales of the image.
Exhaustive Search Output of a High Resolution Image.
4.3 Image Pyramid
The Image Pyramid approach involves rescaling the channels by a factor of 2 and then, starting with the smallest scale level pyramid, the best displacement vector (x,y) for alignment is identified by minimizing the matching metric at each level. This displacement vector is then accumulatively applied to shift the original image.
Image Pyramid Output of High Resolution Images.
4.4 Image Adjustments
There are many ways that the images can be visually improved. Two ways of improving the appearance of the images were considered, namely, base channel selection and cropping. These adjustments were implemented from scratch in python.
4.4.1 Base Channel Selection
As mentioned, this adjustment was inspired by Suvvatal’s project submission to test the appearance of the images for different base channels. From the images below, aligning the Red and Blue channel to the Green channel improved some of the images’ appearance. It can be seen from the monastery image below that using the Green channel as the base channel has drastically improved the image’s appearance.
a)Red Base Channel b)Green Base Channel (After) c)Blue Base Channel (Before)
4.4.2 Cropping
The black and white border around the images makes the image less appealing. As seen below, removing the border improves the images’ alignment and also the image’s appearance. 20% of the border was cropped out of the RGB channels and after the final image’s alignment. This has improved the images’ appearance.
a)Uncropped Green Base Channel (Before) b)Cropped Green Base Channel (After)
This Emir image below further supports the selection of the Green base channel for alignment and cropping channel images during and after alignment. The image demonstrates the effects of cropping the borders of images with different base channels. It is observed that using the Green Base Channel dramatically improved the image’s appearance.
a)Blue Base Channel(Before) b) Green Base Channel (After) c) Red Base Channel
5 Results
5.1 Low Resolution Coloured Images
Red Channel: (1,7) ; Blue Channel: (-2,-5)
Red Channel: (1,6) ; Blue Channel: (-2,3)
Red Channel: (1,4) ; Blue Channel: (-3,-3)
5.2 High-Resolution Coloured Images
Red Channel: (-8,33) ; Blue Channel: (-4,-25)
Red Channel: (17,57) ; Blue Channel: (-24,-49)
Red Channel: (-3,65) ; Blue Channel: (-16,-60)
Red Channel: (5,48) ; Blue Channel: (-17,-41)
Red Channel: (3,63) ; Blue Channel: (-9,-50)
Red Channel: (4,96) ; Blue Channel: (-10,-82)
Red Channel: (10,57) ; Blue Channel: (-26,-52)
Red Channel: (8,98) ; Blue Channel: (-29,-79)
Red Channel: (-3,58) ; Blue Channel: (-14,-53)
Red Channel: (27,43) ; Blue Channel: (-6,-43)
Red Channel: (-11,52) ; Blue Channel: (0,-53)
5.3 Additional Photos from the Prokudin-Gorskii Photo Collection
These are additional photos I ran through the multiscale algorithm.
Red Channel: (-6,45) ; Blue Channel: (-18,-22)
Red Channel: (-16,107) ; Blue Channel: (11,-33)
Red Channel: (-19,90) ; Blue Channel: (18,-25)
6 Conclusion
Aligning images using the green base channel drastically improved the appearance of the images. To the point where minimum visual adjustment is required for some of the images. Cropping the channels before and after alignment for both low and high-resolution images improved the appearance of the images. Some of the images have blemishes that can be fixed by denoising the images, and some are dull and flat and white balancing and contrasting will help make the color more realistic and pop. As a challenge for myself, I will look into creating these image adjustments from scratch.