Project 1

Joanne Qi, Spring 2020

The project was to align and colorize the Prokudin-Gorskii photo collection. There are three images provided (representing the red, green, and blue plates) and the goal was to find the best offsets for the plates so that together, they form a colorized image.

Process

First, I wrote the naive implementation. Given two images A and B, as well as a range, it tries all of the offsets between (-range, range) for x and y, and returns the pair of offsets that score the best. The scoring formula I used was sum of squared differences.

To improve the result of the alignment, I also cropped 10% of the margin of the images, to eliminate some of the white and black borders.

The next step was implementing image pyramids to improve performance.

Given two images and a depth, the image pyramid approach first scales the image down to (0.5^depth) of its original size. I used 4 as the depth. The best offset for the scaled-down image is calculated with the naive implementation and a range of 15.
I keep a variable total_offset updated with the offset for the full-size image. For this value to be accurate for the full-sized images, the offsets found at each iteration had to be multiplied by 2 ** depth .

Then, scale the image up to (0.5^ (depth -1)). I use np.roll to update the image with the values in total_offset. Using the naive implementation, search a small range to potentially update total_offset. If an offset is found that works better, update total_offset. Repeat until depth is 0.

Provided Images

The alignment is not very successful for the Emir because the images have very different pixel values at the "correct" alignment. The blue color of the coat looks very different between the red, green, and blue plates. An approach that may work better for this image is edge detection.

The alignment is not very successful for the Tobolsk for similar reasons to the Emir image. The blue river looks significantly darker on the red plate.

Extra images