CS194 PROJECT ONE - Alex Wu

Overview

The project aims to colorize the Prokudin-Gorskii photo collection by aligning the three color channels and overlapping them.

Implementation

I first load the raw image and seperate them into three color channels. To have a better alignment, I decided to crop out a fix amount of border. After comparing several images, I cropped out 1/15 of the image from each side to ensure that we can achieve the best alignment.

I have used Sum of Squared Differences (SSD) as the scoring function. It has performed decently well for me. For the alignment function, I use two for loops to search through a window size of 15 * 15 pixels. I displaced the image through np.roll() function and compare each displaced image with the base image. When the two images has a lower SSD score, I store the score and the displacement vector. This is how the single scale version of the alignment function works.

To acheive a better result for multi-scale alignment, I have implemented a multiscale pyramid version alignment function. The multi-scale alignment function first rescale the images to a lower resolution, perform alignment on both images, get the displacement vectors, and then rescale the images back and use the displacement vector as the base of the alignment searches. To accomplish this, I added two parameters, the x-displacement and the y-displacement, to the alignment function. Once we get the displacement vector from a lower resolution, we pass them in as the x-dis and y-dis of the new alignment function. We will "shift" the search window accroding to this displacement vector. This allows a better performance when the image is of very high resolution. Note that the multi-scale alignment function is recursive, so that an image can be rescaled multiple times until we get the displacement vector. The rescale factor is 0.5, and the base resolution is 500 * 500.

cathedral.jpg

Displacement for Green Channel: [5, 2].

Displacement for Red Channel: [12, 3].

castle.tif

Displacement for Green Channel: [35, 3].

Displacement for Red Channel: [98, 4].

harvesters.tif

Displacement for Green Channel: [59, 16].

Displacement for Red Channel: [124, 13].

icon.tif

Displacement for Green Channel: [40, 17].

Displacement for Red Channel: [89, 23].

lady.tif

Displacement for Green Channel: [48, 9].

Displacement for Red Channel: [112, 11].

emir.tif

Displacement for Green Channel: [49, 24].

Displacement for Red Channel: [88, 44].

melons.tif

Displacement for Green Channel: [82, 10].

Displacement for Red Channel: [179, 13].

monastery.jpg

Displacement for Green Channel: [-3, 2].

Displacement for Red Channel: [3, 2].

onion_church.tif

Displacement for Green Channel: [51, 26].

Displacement for Red Channel: [108, 36].

self_portrait.tif

Displacement for Green Channel: [78, 28].

Displacement for Red Channel: [176, 36].

three_generations.tif

Displacement for Green Channel: [53, 14].

Displacement for Red Channel: [112, 11].

tobolsk.tif

Displacement for Green Channel: [3, 3].

Displacement for Red Channel: [6, 3].

train.tif

Displacement for Green Channel: [42, 5].

Displacement for Red Channel: [87, 31].

workshop.tif

Displacement for Green Channel: [52, 0].

Displacement for Red Channel: [104, -12].

Other Images:

beach_first.jpg

Displacement for Green Channel: [1, 1].

Displacement for Red Channel: [3, 3].

beach_second.jpg

Displacement for Green Channel: [1, -3].

Displacement for Red Channel: [3, -7].

house.jpg

Displacement for Green Channel: [1, 1].

Displacement for Red Channel: [4, 1].