Overview

In early 20th century, a Russian man named Sergei Mikhailovich Prokudin-Gorskii envisioned a plan to document the vast Russian Empire using new advancements in color photography. He traveled across the country and took thousands of pictures of people, buildings, landscapes... For each scene, he recorded three exposures onto a glass plane using a red, a green, and a blue filter. These valuable glass plates were later purchased by the Library of Congress. The LoC digitized the negatives and made them public on-line.

In this project, I utilize image processing techniques to automatically produce color images from the digitized Prokudin-Gorskii glass plate images. The program extracts the three color channel images from a glass plate image and aligns them to form a single RGB color image.

Approach

The original glass plate image is divided vertically into three equal parts, corresponding to B, G, R channels. To align two parts, search within a window of possible displacements and choose the displacement with the best score using some metric described below.  Because the edges of the three channels cannot be aligned anyway and create noise, I disregard 5% area of each edge when computing losses.

Metrics to measure the quality of alignment:

  -  L2 loss: sum of squared differences between two images

  -  NCC loss: the negative of the dot product between two normalized images

For low resolution images (~300*300 pixels), I search in range [-15, 15] pixels along each axis. Using two metrics renders identical result for every image. For high resolution images (~3000*3000 pixels),  an exhaustive search over a reasonable range of displacement is too slow. Therefore, I use a coarse-to-fine image pyramid to search for the correct displacement. The image is downscaled by a factor of 2 at a coarser level. The image is first aligned at the coarsest level (smallest scale), then at a finer level, a search in range [-1, 1] pixel along each axis is sufficient to find the best adjustment to the previous estimate of the correct displacement. While using two metrics achieves similar results for most images, NCC loss performs obviously better on some images such as emir.tif and self_portrait.tif.

                                     self_portrait.tif using L2 loss                                                                                                 self_portrait.tif using NCC loss

Results on Low-Resolution Images

Results on High-Resolution Images (use NCC loss)

Extra Results

For some images, balancing between the contrasted and uncontrasted versions seems to create more realistic results. Some examples are showed below.