CS 194 - Project 1: Images of the Russian Empire:

Overview

The goal of this assignment is to take the digitized Prokudin-Gorskii glass plate images and, using image processing techniques, automatically produce a color image with as few visual artifacts as possible. In order to do this, I extracted the three color channel images, cropped off the thick borders, and aligned them so that they form a single RGB color image, and placed them on top of each other.

Code

Location

The code for this project is contained in the ipython notebook in the file titled "notebook_proj1.ipynb" and will process the named images in the "images" folder, returning the single RGB color images in the "transformed" folder.

Bells & Whistles

I implemented four different variants of this image processing method.

This is because I used two separate objective functions to minimize.

The negative of the cross correlation
The sum of squared distance

And two distinct features with which to represent the photos.

This pixel brightness
The gradient (relative contrast to nearby pixels) of pixel brightness.

Mixing and matching these, I found that in general, the sum of squared distance used on the gradient of the images resulted in the best aligned for my algorithm.

Transformed Pictures (w/ gradient and sum of squared distance)

Offsets for green and red

'cathedral.jpg': {'g_offset': {'x': 2, 'y': -11}, 'r_offset': {'x': 10, 'y': -18}}

'emir.tif': {'g_offset': {'x': 22, 'y': -99}, 'r_offset': {'x': -228, 'y': -142}}

'harvesters.tif': {'g_offset': {'x': 16, 'y': -88}, 'r_offset': {'x': 14, 'y': -172}}

'icon.tif': {'g_offset': {'x': 16, 'y': -111}, 'r_offset': {'x': 23, 'y': -211}}

'lady.tif': {'g_offset': {'x': 8, 'y': -93}, 'r_offset': {'x': 11, 'y': -180}}

'melons.tif': {'g_offset': {'x': 18, 'y': -68}, 'r_offset': {'x': 14, 'y': -124}}

'monastery.jpg': {'g_offset': {'x': 2, 'y': -19}, 'r_offset': {'x': 2, 'y': -29}}

'onion_church.tif': {'g_offset': {'x': 24, 'y': -98}, 'r_offset': {'x': 34, 'y': -193}}

'self_portrait.tif': {'g_offset': {'x': 30, 'y': -70}, 'r_offset': {'x': 38, 'y': -128}}

'three_generations.tif': {'g_offset': {'x': 18, 'y': -93}, 'r_offset': {'x': 12, 'y': -184}}

'tobolsk.jpg': {'g_offset': {'x': 3, 'y': -13}, 'r_offset': {'x': 3, 'y': -26}}

'train.tif': {'g_offset': {'x': 1, 'y': -111}, 'r_offset': {'x': 28, 'y': -216}}

'village.tif': {'g_offset': {'x': 13, 'y': -86}, 'r_offset': {'x': 84, 'y': -134}}

'workshop.tif': {'g_offset': {'x': -4, 'y': -98}, 'r_offset': {'x': -14, 'y': -193}}}

Success Pictures

Wow these are some beautiful pictures, especially considering their age!

Problem Pictures

Both cathedral.jpg and emir.tif failed to align well. I believe the picture of Emir was taken with varying brightnesses and affected the contrast between pixel values slightly, which may have caused the disalignement. Cathedral is just somewhat blurry, but not too far off perfect alignment. These could both be the result of poor border cropping, which makes it easier to find a higher sum of squared distance for misalignment where borders are still crossing over. It may have also failed because the relative contrast of pixels was actually pretty consistent in a lot of places. Photos with significant spaces dominated by one color can sometimes get a bit misaligned by my algorithm and find themselves stuck at a local minimum.