CS196-26 Project 1

Overview

The goal of this project is to restore a few select images from the Prokudin-Gorskii image collection to their original colored versions given three versions of the image taken using RGB filters. This is done by first aligning the RGB channels and then stacking them.

Approach

The original image is first divided into 3 equal parts based on height to get the three separate RGB filtered images. Then the green and red images are independently aligned to the blue image using the blue image as the reference. The alignment is done by looping through a range of possible displacements on both the X and Y axes and then picking the (X, Y) displacement combination that minimizes an alignment metric, specifically SSD in this case. I looped through a range of [-20, 20] in both the X and Y directions. However, even after picking the alignment the final results were still not very sharp; I fixed this by cropping each side by 300 pixels before doing any sort of alignment to remove any values along the border that may be throwing off the alignment metric comparisons. This in general worked for the smaller .jpg files, but took too long for the large .tif files. In order to make the algorithm run quickly for large files, I implemented the image pyramid search procedure. After tweaking some of the parameters such as the number of pyramid layers and the resize factor for each layer, I found that having 5 layers with a resize factor of 0.5 (shrink original image by half per layer) produced clear results for most of the images.

Below are the results of the algorithm on all the provided images and a few images I selected from the image collection along with the (X, Y) offset of each R/G channel calculated in order to produce the final colored result.

Provided Images

Cathedral

R(2, 5); G(3, 12)

Monastery

R(2, -3); G(2, 3)

Tobolsk

R(3, 3); G(3, 6)

Castle

R(3, 35); G(4, 98)

Emir

R(24, 49); G(-702, 75)

Harvesters

R(16, 59); G(13, 124)

Icon

R(17, 41); G(23, 89)

Lady

R(9, 50); G(11, 112)

Melons

R(10, 82); G(13, 178)

Onion Church

R(26, 51); G(36, 108)

Self Portrait

R(29, 78); G(37, 176)

Three Generations

R(14, 53); G(11, 112)

Train

R(5, 42); G(32, 87)

Workshop

R(0, 52); G(-12, 105)

Picked Images

Camel

R(-3, 46); G(-19, 98)

Cotton Mill

R(-13, 57); G(-58, 132)

Prison

R(-13, 25); G(-21, 71)

Failures

Most of the final images were decently clear except for emir, where the red channel seemed to be misaligned. This is due to the difference in brightness between the channels, which causes the alignment algorithm to pick the wrong optimal displacement if the three channels aren’t consistent.