CS194-26: Image Manipulation and Computational Photography

Project 1: Images of the Russian Empire

Colorizing the Prokudin-Gorskii Photo Collection

Fang Shuo Deng, Spring 2020

Write-up

Brief overview

This project required us to take the digitized Prokudin-Gorskii glass plate images and, splitting it up into 3 sections (since the digitalized glass plates were stacked vertically), produce a colorized image that aligned output of the 3 exposures.

My Approach

There were a couple of components that I implemented to get this to work. The first was a metric that was used to compute the difference between 2 images. I used the Sum of Squared Distances (L2 loss) in this case, as suggested by the spec.

Next, I implemented an algorithm to align the green and red exposures to blue exposure. I first implemented a naive approach (as suggested by the spec) to some possible values from [-15, 15] inclusive. This worked with smaller images but did not work with larger images (.tif files).

Thus, I implemented an image pyramid (again suggested by the spec), by using an iterative (as opposed to recursive) approach. I first generated a list of images, each a factor of smaller than the one before it in the list.

For the smallest image, I then used my naive algorithm to find the correct displacement. I multiplied this displacement by 2 for the next image, and then did some minor tweaking (used my algorithm with a space of possible values from [-5, 5]) before repeating this process until I find the desired offset values for my actual image. I then saved this image as a .jpg file (images shown below).

Problems and how I tried to solve them

The 1 major problem I faced was the naive algorithm I implemented took too long. To solve that, I used the image pyramid suggested in the spec.

I faced some other problems structuring my functions because initially, I faced some weird bugs where the dimensions of the images I was trying to compare with the Smallest Squared Distance function was mismatched. To solve that, I had to step through my code line by line, and that was when I discovered I was accidentally comparing my images to the largest-sized blue image regardless of which smaller sized exposure I was working with. Rectifying this error allowed me to solve the problem.

Failure cases

From my observation, it seemed like the alignment of emir.tif was not successful, because the image is still rather blurry. I think this is because the images have very different values at the same pixel values across the 3 channels for the correctly aligned image. After looking through Piazza, I saw that a viable approach would have been to align the images to the Green channel (as opposed to the blue channel like what was done for the other images), and that is what I would have loved to try, given more time.

Bells and Whistles

I did not implement any bells and whistles.

Online Images (from the Prokudin Gorskii Collection)

Green Offset: (0, 3)

Red Offset: (0, 6)

Green Offset: (-4, 88)

Red Offset: (-10, 112)

Green Offset: (0, 1)

Red Offset: (0, 6)

Example Images

Green Offset: (16, 42)

Red Offset: (22, 89)

Green Offset: (-1, 1)

Red Offset: (-1, 7)

Green Offset: (7, -3)

Red Offset: (17, 107)

Green Offset: (-3, 118)

Red Offset: (7, 120)

Green Offset: (-6, 57)

Red Offset: (-17, 123)

Green Offset: (4, 83)

Red Offset: (7, 176)

Green Offset: (0, -6)

Red Offset: (1, 9)

Green Offset: (22, 52)

Red Offset: (0, 108)

Green Offset: (-2, 50)

Red Offset: (-5, 130)

Green Offset: (5, 52)

Red Offset: (7, 108)

Green Offset: (2, 3)

Red Offset: (3, 6)

Green Offset: (-7, 111)

Red Offset: (1, 107)

Green Offset: (-7, 144)

Red Offset: (-15, 118)

Green Offset: (-5, 53)

Red Offset: (-16, 69)