CS 194-26 Images of the Russian Empire

Overview

In this project, I took three images, each corresponding to either red, green, or blue, and stacked them into one color image. I first had to align them automatically, since the red, green, and blue slides of Prokudin-Gorskii's images don't align perfectly.

Part 1: Single Scale for Smaller Images

In order to line up my smaller images, I used the Sum of Squared Differences (SSD). I traversed over all possible displacements within the windows [-15, 15] in both the x and y direction, and after applying this displacement to the image being aligned, measured what the SSD would be between the image being aligned and the image we want to align it with. I checked over all of the SSDs and found the smallest one to use as the official displacement for alignment.

Originally I had not cropped the borders when calculating the displacement. Instead, I just cropped out the unclean borders after alignment. But, this wasn't as effective as cropping before applying my align function because the borders actually affected the SSD measurements, causing the resulting displacement to be less optimal. So, I cropped the images by 10 pixels in all directions to remove the borders before passing them into my align function.

Cathedral: G: (2, 5) R: (3, 12)

Monastery: G: (2, -3) R: (2, 3)

Tobolsk: G: (3, 3) R: (3, 6)

Part 2: Image Pyramid for Larger Images

The single scale implementation works on small images that are 300x300 pixels wide. However, when aligning higher quality images, there are more pixels to check, which makes the program run slower due to computation.

To fix this, I created an image pyramid, so I could work with multiple scales of the same image. To do this, I rescaled the image (to a 1/5 smaller scale), and checked the same [-15, 15] range in both x and y directions. I then rescaled the image back up and used this optimal displacement to update the image before checking the next scale level. This recursion allows us to update the displacement based on an already more optimal alignment than before.

With the increased size of the images, first, I tried to crop before aligning as I did before. But, I wasn't getting great results on some of them (especially melons). So, rather than cutting out a set number of pixels before aligning, I tried aligning without cropping. After finding the most optimal displacement amongst scales, I recalculated displacement after cropping the edges. This worked much better.

The alignment worked poorly on Emir because the clothes in the image are strikingly blue. The brightness of this blue compared to the dimmer red and green on this image causes the displacement to be off.