Images of the Russian Empire
Colorizing the Prokudin-Gorskii photo collection
Sergei Mikhailovich Prokudin-Gorskii (1863-1944) is an Russian photographer.
As early as 1907, he started to travel across Russian Empire and take color photographs.
His idea was to record three photographs of every object with a red, a green and a blue filter.
Our objective is to generate a color photograph from each set of his RGB plate negatives.
Approach
The general idea is simple. We split each set of RGB negatives to three photos in red, green and blue channels. Then we align three photos together to eliminate the minor deviation among them. In the end, we stack these channels together to create a color photograph. |
|---|
| Original Picture |
Split
The general idea is simple. We split each set of RGB negatives to three photos in red, green and blue channels. Then we align three photos together to eliminate the minor deviation among them. In the end, we stack these channels together to create a color photograph.Single-Scale Alignment
For low resolution photos, we could just search over a window of possible displacements, and choose the displacement with best score, and then roll the image according to the best displacement. Here, we try to calculate score with both the Sum of Squared Differences (SSD), and the Normalized Cross-Correlation (NCC). In this case, we choose the displacement window to have a size of 30px * 30px. |
 |
|---|---|
| Cathedral (SSD) Green Offset: (1, 0), Red Offset: (8, 0) |
Cathedral (NCC) Green Offset: (1, 0), Red Offset: (8, 0) |
 |
 |
|---|---|
| Monastery (SSD) Green Offset: (9, 0), Red Offset: (9, 1) |
Monastery (NCC) Green Offset: (9, 0), Red Offset: (9, 1) |
 |
 |
|---|---|
| Nativity (SSD) Green Offset: (3, 1), Red Offset: (7, 1) |
Nativity (NCC) Green Offset: (3, 1), Red Offset: (7, 1) |
 |
 |
|---|---|
| Settlers (SSD) Green Offset: (7, 0), Red Offset: (14, 0) |
Settlers (NCC) Green Offset: (7, 0), Red Offset: (14, 0) |
Multi-Scale Alignment
For high resolution pictures, exhaustive search may become unreasonable because it takes too much time. So instead, we use a search method called image pyramid. By rescaling the high-res photos to a reasonably small size (200px * 200px here), we could apply the single-scale alignment method recursively from smaller size photos to larger size. |
 |
|---|---|
| Emir (Pyramid SSD) Green Offset: (48, 16), Red Offset: (0, 464) |
Emir (Pyramid NCC) Green Offset: (48, 16), Red Offset: (0, 464) |
 |
 |
|---|---|
| Harvesters (Pyramid SSD) Green Offset: (64, 16), Red Offset: (128, 16) |
Harvesters (Pyramid NCC) Green Offset: (64, 16), Red Offset: (128, 16) |
 |
 |
|---|---|
| Icon (Pyramid SSD) Green Offset: (48, 16), Red Offset: (96, 16) |
Icon (Pyramid NCC) Green Offset: (48, 16), Red Offset: (96, 16) |
 |
 |
|---|---|
| Lady (Pyramid SSD) Green Offset: (48, 16), Red Offset: (112, 16) |
Lady (Pyramid NCC) Green Offset: (48, 16), Red Offset: (112, 16) |
 |
 |
|---|---|
| Self Portrait (Pyramid SSD) Green Offset: (80, 16), Red Offset: (176, 32) |
Self Portrait (Pyramid NCC) Green Offset: (80, 16), Red Offset: (176, 32) |
 |
 |
|---|---|
| Three Generations (Pyramid SSD) Green Offset: (48, 16), Red Offset: (112, 16) |
Three Generations (Pyramid NCC) Green Offset: (48, 16), Red Offset: (112, 16) |
 |
 |
|---|---|
| Train (Pyramid SSD) Green Offset: (48, 0), Red Offset: (96, 16) |
Train (Pyramid NCC) Green Offset: (48, 0), Red Offset: (96, 16) |
 |
 |
|---|---|
| Turkmen (Pyramid SSD) Green Offset: (48, 16), Red Offset: (112, 16) |
Turkmen (Pyramid NCC) Green Offset: (48, 16), Red Offset: (112, 16) |
 |
 |
|---|---|
| Village (Pyramid SSD) Green Offset: (64, 16), Red Offset: (128, 16) |
Village (Pyramid NCC) Green Offset: (64, 16), Red Offset: (128, 16) |
Edges
You must have noticed that the alignment of the photo called "Emir" is not successful at all. Because the feature we used is color. If we detect edges with Canny Edge Detector and use edges as a feature, the result would be much better. |
 |
|---|---|
| Emir (Pyramid SSD) Green Offset: (48, 16), Red Offset: (0, 464) |
Emir (Edge Detector) Green Offset: (48, 16), Red Offset: (112, 32) |
Automatic Cropping
Many color photos we generated have very annoying white, black or other color borders. With the canny edge detector, we can deal with this. First of all, we get the edge for the greyscale photo. If the sum of all number in a row is larger than 0.3 * width, or sum of all number in a column is larger than 0.3 * height, it is extremely possible that there is a straight line in the photo, which means it could be a border. Here, I set the threshold to be 0.3, but we could change this according to different situations. |
 |
|---|---|
| Chapel (Before Automatic Cropping) |
Chapel (After Automatic Cropping) |
|
 |
|---|---|
| Cathedral (Before Automatic Cropping) |
Cathedral (After Automatic Cropping) |
 |
 |
|---|---|
| Samarkand (Before Automatic Cropping) |
Samarkand (After Automatic Cropping) |
 |
 |
|---|---|
| Stork (Before Automatic Cropping) |
Stork (After Automatic Cropping) |
Other Photographs
|
 |
|---|---|
| Chapel (SSD) Green Offset: (5, 1), Red Offset: (10, 2) |
Chapel (NCC) Green Offset: (5, 1), Red Offset: (10, 2) |
|
 |
|---|---|
| Stork (SSD) Green Offset: (4, 0), Red Offset: (10, 0) |
Stork (NCC) Green Offset: (4, 0), Red Offset: (10, 0) |
|
 |
|---|---|
| Samarkand (SSD) Green Offset: (2, 1), Red Offset: (10, 2) |
Samarkand (NCC) Green Offset: (2, 1), Red Offset: (10, 2) |