Project 1 - Colorizing the Prokudin-Gorskii photo collection

Carolyn Duan

Project Overview

Given three exposures taken using a green, red, and blue filter, reproduce the color image using visual processing techniques with as few visual artifacts as possible, in a performant way.

Approach

For the lower-resolution images, I aligned the three parts by performing an exhaustive search over a displacement window of [-15, 15]. My metric for evaluating the best alignment was the minimnum SSD (sum of squared differences), calculated over the middle slice portion of each image (both to avoid irregularities present at the borders, and to improve performance of the processing). As was suggested in the starter code, I aligned the green channel to the blue channel, and the red channel to the blue channel.

For the high resolution images, I followed the suggestion in the spec to use an image pyramid approach to calculate the alignment between two channels, still using SSD to evaluate alignments. This process starts with the smallest resolution of the images, finds the best alignment, and then recursively works its way up to the biggest resolution (doubling in resolution each time) and recomputes the estimate for the best alignment along the way. To increase performance, the search was done over a [-10, 10] displacement window instead.

Emir did not look good when I aligned green to blue and red to blue. Looking at the original image, you can kind of see why: The values for the red channel (bottom) and blue channel (top) are quite different near his clothing, since his clothing is such a vivid blue. So, even a "correct" alignment would have a high SSD. To counteract this, I aligned blue to green and red to green instead.

Results on small example images

[cathedral.jpeg] (g,b)=(2,5) (r,b)=(3,12)

[monastery.jpeg] (g,b)=(2,-3) (r,b)=(2,3)

[tobolsk.jpeg] (g,b)=(3,3) (r,b)=(3,7)

Results on large example images

[church.tif] (b, g)=(-4, -25) (r, g)=(-8, 33)

[emir.tif] (b, g)=(-24, -48) (r, g)=(17, 58)

[harvesters.tif] (b, g)=(-17, -59) (r, g)=(-3, 64)

[icon.tif] (b, g)=(-18, -41) (r, g)=(5, 49)

[lady.tif] (b, g)=(-8, -53) (r, g)=(4, 60)

[melons.tif] (b, g)=(-9, -84) (r, g)=(3, 96)

[onion_church.tif] (b, g)=(-27, -50) (r, g)=(10, 58)

[self_portrait.tif] (b, g)=(-29, -78) (r, g)=(8, 98)

[three_generations.tif] (b, g)=(-14, -51) (r, g)=(-3, 59)

[train.tif] (b, g)=(-6, -43) (r, g)=(26, 43)

[workshop.tif] (b, g)=(1, -53) (r, g)=(-11, 52)

Results on other images from collection

[one.jpeg] (g,b)=(1,4) (r,b)=(2,8)

[two.jpeg] (g,b)=(1,3) (r,b)=(1,77)

[three.jpeg] (g,b)=(1,4) (r,b)=(2,8)