CS 194-26 Project 1

Images of the Russian Empire: Colorizing the Prokudin-Gorskii photo collection

---

Overview

---

Our goal is to align three different color channels of an image to result in a colorized image. Since the images in Prokudin-Gorskii's collection are recorded on glass plates using three different filters, naive alignment (just stacking the three channels on top of each other) does not produce optimal results.

Instead of simply stacking all three channels on top of each other, we must use some image processing techniques to figure out the optimal displacement for each channel so that our resulting image has the least amount of artifacts.

Approach

---

For small images, I would simply search over a window of displacements that was [-15, 15] by [-15, 15] to find the best displacement. The metric I ended up using to score the displacements was the normalized cross correlation, but I also tried sum of squared differences, which yielded similar results.

For larger images, I built an image pyramid by rescaling the original image by a factor of 2. I would search for an optimal displacement in this downsampled image and use this coarse displacement to help me find a finer tuned displacement in the lower levels of the pyramid. This speeds up computation when dealing with large images that have displacements that are potentially greater than [-15, 15].

Problems I ran into, and my solutions

---

The approach described above yields decent results but fails on some images. Some images, like monastery.jpg, seemed to fail because the borders were interfering with the computation of the normalized cross correlation. I fixed this issue by simply pre-cropping 10% off of each image's border so that the correlation would not be skewed by the results.

For the image pyramid, I also had to tinker with some parameters to make sure I wasn't searching too large of a section of the image. I ended up using 500 pixels as my threshold for the smallest layer of the pyramid, and recursed down until I reached 3000 pixels.

Results

---

Small images

Large images

Using the image pyramid seemed to yield decent results for all of the images except emir.tif. This is probably because the three channels have very different brightness values, which distorts the correlation. To fix the emir image, I calculated the alignment based on the green channel instead of the blue one. One other way to solve this is through feature detection, which I implemented in the bells and whistles section.

Other Images

Here are some other images I selected from the Prokudin-Gorskii photo collection.

Bells and Whistles

Edge detection

I used a Sobel filter for edge detection, which allowed me to get some good results on the emir image without needing to switch the color channel I used as a base. Here are some photos of the Sobel edge detection output on various photos: