CS194-26 Project 1

Overview

The goal of this project was to produce color images by combining sets of three Prokudin-Gorskii glass plate images, each measuring the brightness of a different color of light. The main challenge was to automatically align the images using image processing techniques, reducing artifacts like blurring and color-bleeding along edges. My solution involved minimizing the the sum of squared differences (SSD) over pairs of offset differential/gradient matrices.

Approach

First, the program splits the given image into thirds in order to separate the red, green, and blue channels (the original Prokudin-Gorskii images, photographed through colored filters).

Red	Green	Blue

Next, it computes a "gradient" matrix by calculating the change in brightness between neighboring pixels. These differences are calculated across rows as well as down columns. The absolute value of these differentials are then added together to get the approximate gradient magnitude at each pixel. The resulting gradient images (example shown above) contain brighter pixels where pixel brightness is changing quickly in the original image, and darker pixels where the pixel brightness is not changing much. Therefore, edges stand out for all three color channels, helping greatly with alignment.

I then computed the right/left and up/down shifts needed to bring these gradient images into alignment. Alignment was achieved by minimizing the SSD between pixels in the blue gradient image and a shifted version of the red or green gradient image. The SSD metric was computed only over the inner pixels of the images, with 10% of the pixels cropped from each edge.

The search for the optimal offsets was done recursively, using the image pyramid technique (shown above). The images were downscaled by a factor of two on each recursive step, down to a minimum of 10x10 pixels. The offset from the previous step is multiplied by 2 then added to the offset found in the current step. The search window itself is 5x5 pixels, centered around the offset found in the previous recursive step. The SSD is computed for each of the 25 offsets, and the offset with the minimum SSD is chosen.

Finally, with the row and column offsets chosen for both the red and green channel, those two images are "rolled" by the offset amounts, then layered onto the blue channel to create the final image.

Gradient Alignment Method	Simple Alignment Method

Red shift: (107, 40), Green shift: (49, 24)	Red shift: (-443, -785), Green shift: (48, 21)

Results

Emir

Red shift: (107, 40), Green shift: (49, 24)

Train

Red shift: (85, 29), Green shift: (41, 1)

Church

Red shift: (58, -14), Green shift: (25, 4)

Harvesters

Red shift: (123, 4), Green shift: (61, -7)

Icon

Red shift: (89, 23), Green shift: (39, 16)

Lady

My program didn't do a great job aligning this one. This may be because some of the edges aren't well defined in the source images, especially the red channel. The hair and skin are overexposed and blend in with the background in some places.