CS 194-26 Intro to Computer Vision and Computational Photography

Project #1: Colorizing the Prokudin-Gorskii Photo Collection

By Alek Petuskey

Overview

In this project, the goal is to separate, colorize, and align glass plate images from the Prokudin-Gorskii collection. As seen in the example below each of these photos contain three sperate color channels Red, Green, Blue color channels are stack on top of each other vertically. Once these color channels are separated they can be aligned and combined to make a single RGB color image.

drawing drawing

Approach

Results Exhaustive Search

I initially started with a simple process to split up and align some of the lower resolution images from the collection. This process consisted of aligning the blue and red channels to match the green channel's position using an exhaustive search window (-15,15). While iterating through this window, I shifted the other image by the current iteration amount using np.roll and calculated the normalized cross-correlation (NCC), which is the dot product between two normalized vectors: (image1./||image1|| and image2./||image2||). If the NCC of that iteration was a maximum I set the corresponding shifting valuer used in np.roll to be the values I would use to shift the other color to match up with green I then repeated this process on green and the other color.

Blue Layer:[5,-2] | Red Layer:[-4,1] Blue Layer:[13,-2] | Red Layer:[-4,1] Blue Layer:[5,-1] | Red Layer:[-4,1]
drawing drawing drawing

Results Image Pyramid

While the simple aligning implementation worked for smaller images it was to slow to handle the higher resolution tif files. To improve speed, I implemented an image pyramid to represent the inputs in a multi-scale manner and found the optimal displacements iteratively. This process was much more efficient and still gave strong results in image quality.

Blue Layer:[47,-24] | Red Layer:[-41,17] Blue Layer:[57,-17] | Red Layer:[-50,5]
drawing drawing
Blue Layer:[63,-3] | Red Layer:[-33,0] Blue Layer:[45,0] | Red Layer:[-45,10]
drawing drawing
Blue Layer:[35,-17] | Red Layer:[-32,-3] Blue Layer:[41,-8] | Red Layer:[-38,3]
drawing drawing
Blue Layer:[15,-10] | Red Layer:[-2,3] Blue Layer:[45,-27] | Red Layer:[-40,10]
drawing drawing
Blue Layer:[18,-30] | Red Layer:[0,8] Blue Layer:[43,-13] | Red Layer:[-38,2]
drawing drawing
Blue Layer:[55,-5] | Red Layer:[-53,27]
drawing

My Prokudin-Gorskii Collection Picks

Blue Layer:[49,-19] | Red Layer:[-35,17] Blue Layer:[75,-17] | Red Layer:[-51,-6]
drawing drawing

Bells and Whistles

Edge detection:

Canny edge detection is a technique to extract useful structural information from different vision objects and dramatically reduce the amount of data to be processed. I used OpenCV's built in Canny function.

drawing

Automatic Color Balancing:

In a well balanced photo, the brightest color should be white and the darkest black. Thus, we can remove the color cast from an image by scaling the histograms of each of the R, G, and B channels so that they span the complete 0-255 scale.

drawing