Images of the Russian Empire
Project Overview
The goal of this assignment is to take the digitized Prokudin-Gorskii glass plate images and, using image processing techniques, automatically produce a color image with as few visual artifacts as possible.
General Approach
In order to do this, I extracted the three color channel images, placed them on top of each other, and aligned them so that they formed a single RGB color image.
To find the best alignment, I used the image alignment technique described in the spec. I minimized the SSD distance over a window of [-15, 15] pixel displacements. Since this is not viable for the large .tif images, I searched for the optimal alignment using an image pyramid. I scaled each image down by a factor of 4 until it was no greater than 350 x 350 pixels (mostly arbitrarily chosen - around the size of the jpg images). At that size, I found the best displacement using the [-15, 15] pixel window and SSD distance metric described earlier. As the image scaled back up, I multiplied the best displacement (at 350x350 pixel size) by 4 as a coarse estimate and fine-tuned that estimate by searching for an updated best displacement using a [-1, 1] pixel window and the same SSD distance metric. The [-1, 1] window was sufficient, and increasing the window significantly increased computation time but did not significantly improve alignment. I repeated this until the image was back to its original size.
Before computing alignment, I cropped the edges because many edges did not align and actually hurt the alignment. I also cropped the final aligned images to get rid of the visual artifacts in the borders.
Output on Example Images
The results of this approach on the example images can be seen below.
icon.tif
GB Offset: [43, 16]
RB Offset: [92, 16]
harvesters.tif
GB Offset: [60, 16]
RB Offset: [124, 12]
lady.tif
GB Offset: [48, 11]
RB Offset: [112, 12]
monastery.jpg
GB Offset: [-3, 2]
RB Offset: [3, 2]
nativity.jpg
GB Offset: [3, 1]
RB Offset: [7, 1]
self_portrait.tif
GB Offset: [76, 27]
RB Offset: [172, 32]
settlers.jpg
GB Offset: [7, 0]
RB Offset: [14, -1]
three_generations.tif
GB Offset: [48, -4]
RB Offset: [112, 0]
train.tif
GB Offset: [43, 0]
RB Offset: [80, 32]
turkmen.tif
GB Offset: [48, 16]
RB Offset: [112, 27]
village.tif
GB Offset: [64, 11]
RB Offset: [139, 12]
Bells and Whistles
Better Features - Edge Detection
As stated in the spec, using SSD on the raw pixels was insufficient to align some images, namely emir.tif. This is because the raw pixels do not have the same brightness values. In order to fix the alignment, I applied the Sobel filter on each of the three images to implement edge detection. By detecting the edges, the differing brightness values are ignored and the images can be aligned more accurately.
Results of alignment without edge detection and with edge detection can be seen below. Only emir.tif and cathedral.jpg are shown. Using the raw pixels was sufficient to align the other images and implementing edge detection did not noticeably improve alignment, so their before and after images are not shown.
emir.tif
GB Offset: [48, 27]
RB Offset: [96, -208]
This image had the worst alignment. The emir image aligned using SSD on raw pixels can be seen below.
The new emir image aligned using edge detection can be seen below.
GB Offset: [48, 16]
RB Offset: [107, 32]
cathedral.jpg
The cathedral image aligned using SSD on raw pixels can be seen below.
GB Offset: [5, 2]
RB Offset: [12, 3]
The new cathedral image aligned using edge detection can be seen below.
Output on the Prokudin-Gorskii Collection
The results of my algorithm on select images from the Collection can be seen below.
Old Gates in the Church of the Assumption of the Mother of God
Old Lantern in the Church of the Assumption of the Mother of God
View of the Sawmill
