Overview

The goal of the project is to colorize black and white photos taken by Prokudin-Gorskii by overlaying the R, G, and B color channel photos. The primary challenge is the procedurally align the photos with as little human intervention as possible.

My Procedure

I aligned the photos using the least squares of the Sobel filter on the center 25% of the image. I ran a convolution over all channels with first a vertical edge detector:

1	0	-1
2	0	-2
1	0	-1

The absolute value of this image produces an image with bright highlights at all vertical edges. I then run the original image through a horizontal edge detector:

1	2	1
0	0	0
-1	-2	-1

Once again, I make sure to take the absolute value so that all the horizontal edges are highlighted and non-edges are dark. I then take the net magnitude of the vertical edge image and horizontal edge image at eat pixel and generate a new image. This is the sobel filtered image. To increase consistency and prevent precision loss, I scale all the images so that their max pixel intensity is 1.

Now begins the actual alignment search. My search does a binary search using downsampled versions of the color channels with a sobel filter applied on top (Note that the filter is applied after downsampling, otherwise sobel edges are blurred). The search starts by aligning 16x16 images or smaller and works its way up in multiples of two till max resolution. This reduces search time from n^2 comparisons to log(n) comparisons where n is the number of pixels.

Challenges

The primary challenge was memory. I used some caching on the blue channel to save time, but it ended up greatly increasing memory usage. There also were some inefficiencies with the starter code that I had to remove. What ended up being the problem though, was that I had a 32 bit python installed. I no longer had memory issues after upgrading to 64 bit, however using 32 bit did push me to keep memory usage to a minimum during most of my programming.

Image Gallery

Custom Gallery

Bells and Whistles

My implementation uses a Sobel filter for feature recgnition rather than color intensity. This allows me to use the simple least squares heuristic on the pixel values without issue and makes the alignment essentialy fault free. Unfortunately, I do not have any examples with the filter disabled.