Project 1

Colorizing the Prokudin-Gorskii Photo Collection

Michael Wang

Overview

This project attempts to automatically crop, align, and overlay the three photo plates within the Prokudin-Gorskii Photo Collection. Since each plate was essentially the same photo taken through a blue, green, and red filter respectively, our goal to create a color composite would be to overlay them and form a proper RGB image by replicating the way the human eye sees color.

Technical Approach

In the library, the three plates that comprise a color image are stacked vertically atop each other. The first step was to manually crop off any borders on the image. Then, the stacked image was divided into thirds to isolate each colored plate as their own image. The next step came from overlaying them properly so that the details on each plate aligned with each other. To automatically find the correct anchor points to align each photo with, this project fixes a base plate (in this case, green), and overlays other plates at varying displacements to find the best displacement where details match identically. To do so, this project uses Sum of Squared Differences (SSD) to compare brightness values at different displacements. However, simply comparing brightness values, while effective for most images, failed in others. This showed rather strikingly in the emir.tif file, where attempting to use SSD on the brightness values for each color plate resulted in a severely misaligned image. This is due to the unequal brightness values in each plate. Points that were supposed to match on each plate do not have similar brightness values, throwing off the SSD score. This will be remedied by using a different metric later in the project. Since using an exhaustive search becomes prohibitavely slow for larger .tif files, an image pyramid was implemented to reduce the range of searches. This project uses 5 layers, resulting in a marked speedup as lower resolution images pinpoint a general area to align the image at, and lower levels help refine the search area with a much smaller window.

Image Results

Using the blue plate resulted in crisp pictures for the most part, but utilizing a different base plate (e.g. red) helped add more clarity to pictures like workshop.

Other Images

Bells & Whistles

Due to the inconsistent brightness values across channels in emir.tif, this projects implements edge detection to remedy this issue. Rather than comparing brightness across the values, edge detections allows us to extract the salient features of an image that are present regardless of color channels, and align the image through that.