CS194-26 Project 1 - Colorizing the Prokudin-Gorskii Photo Collection

Prangan Tooteja.

Overview

This project focused on aligning three black and white images taken by applying R, G, B filters over the camera lens. The project determines the pixel offset of the red and blue images and aligns them with respect to the green image. Initially, I used blue as my base reference, but this did not work well for some images - and was notably bad for the image of the Emir. I changed my base to green after our lecture on the color spectrum and how human eyes are most receptive to green frequencies.

There are two versions of the alignment algorithm to handle small and large images. The algorithm for small images does a simple offset calculation over a [-15, 15] pixel shift range and uses NCC to score how well images are aligned. The algorithm for large images recursively calls this function and rescales the images to a lower resolution so that the offset can be found using a pyramid search which is a sort of two-way binary search. This works well because a large range of pixel shifts can be iteratively calculated without having to look over the entire range. I used a scale-factor of 4 with a pyramid-depth of 4 to reduce the images and searched over a window of [-4, 4] pixels at each layer. The lower resolution images provide a directionality for the higher resolution images so that if the displacement is [-15, 4], we only explore the displacement space corresponding to an x-shift of +15 in the lowest resolution space.

Some images were misaligned slightly because of the dark border surrounding each image, when cropped to focus on the central part of the image, there is much better alignment.

Initially my algorithm aligned images to the Blue filter. This failed to align the Emir image because of the large difference between the red image and blue image pixel values. I adjusted the algorithm to align to the Green pixel values to resolve this issue as explained above.

Standard Implementation

Offsets

Bells And Whistles

For my bells and whistles addition I added is to use Histogram Equalization to match the contrast in each image. This led to better color veracity and more natural looking images. Instead of using the exact method described in lecture - I came across Adaptive histogram equalization which better handles images where different parts have significantly different pixel values. This produced more vivid colors and more detailed images although in some instances this made the image look unnatural. The default parameter looked good for photos of objects and nature but made faces look unnatural, because it increased the contrast by too much, to counteract this, I decreased the clip_rate value to decrease the contrast gain.

As shown in the images below, the bells and whistles addition improves image detail and makes the colors appear to be more realistic