Background
Sergei Mikhailovich Prokudin-Gorskii was a photographer in the early 20th century Russian empire. He traveled across Russia from 1909 to 1915, taking multiple rapid photos of scenery, using red, green, and blue filters. The glass plate negatives of these photos eventually ended up in the US Library of Congress, who then digitized and uploaded the content. The photos then found their way to me through CS194-26. I now aim to reconstruct the photos in their original color, using only the negatives below and some good ol' CS.
Reconstruction
To reconstruct the photos, I needed to implement two different approaches, based on two main factors: file size and time efficiency. In both approaches, the original negative is first split into 3 parts, which will give us a red, green, and blue image. The first approach, a Single-Scale approach, is meant for smaller images. In this approach, we start off by overlaying the green image onto the blue, and then performing an exhaustive search of pixel shifts, using the Sum of Squared Differences to find where we have a minimal SSD and therefore a maximal overlap. To ensure its efficiency, we only perform this computation across the middle third of each picture, both length and width wise. The process is then repeated for the red negative, except this time the red is overlayed across our now newly shifted green.
For larger images however, this process becomes much too time-consuming, so we must implement the Image Pyramid approach. This approach resizes the image, in my case by a factor of 1/10 of its height and width. It performs this process of downsizing until the image is sufficiently small enough. From there, it performs the exhaustive search, going from the coarsest image to the finest, aka going down the pyramid. At each step, it performs a small version of the Single-Scale approach, until it gets closer and closer to the correct offset. This method of going from coarse-to-fine offers much better efficiency, allowing me to work with larger files such as pngs and tifs.
Issues
Along the process of reconstruction, I faced several problems. The first was that my original approach involved aligning green over blue, and then red over blue as well. This resulted in pictures that were ever so slightly off. After experimenting with various different overlaying orders, what seemed to work best across both big and small images for me was the Green over Blue, Red over Green approach. Another issue was that of time. To process all input images under a minute, I needed to ensure that the search window was small, but that it still had enough information to produce a proper alignment. This led to several iterations of trial and error, as well as heavy experimentation into what features were best suited to align an image. Choosing a window that was too small or too distant from the desired features resulted in incoherent images. You can see some of my failures, or perhaps one could say artistic alternatives, below.
Extra
Although the color bars on the borders certainly add flavor, there is some merit in removing them to get better cropped pictures. I implemented a cropping function that takes the top 1/10th and the bottom 1/10th of the image, as well as the left 1/10th and the right 1/10th of the image. It then goes through those sections to see if the average of any of the color channels is above or below a certain threshold. By only taking the 1/10th of each side, we can ensure that we aren't losing much information in the main content of the image, and this means we can set a looser threshold to remove as much of the color borders as possible. The resulting images are displayed below. Hover to see the original image.
