Sergei Mikhailovich Prokudin-Gorskii was a brilliant photographer who spent his years experimenting with color photography. He traveled across the Russian Empire and took pictures of everything imaginable. As a man living in a time period when color photography was just coming to fruition, he photographed by recording three exposures of every scene, using a red, green, and blue filter for each one respectively. He envisioned that at some point in the future, technology would advance to combine the colored glass plate images he generated into color photographs. This project looks into bringing his dream into reality - colorizing Prokudin-Gorskii's images with his RGB glass-plate negatives!
In this project, I begin colorizing each image from the Prokudin-Gorskii collection by initially separating it into its three constituent color channels. Naturally, these do not align perfectly when overlaid with one another leads to distorted images such as this one:
A simple technique to correct these alignment errors is to select one of the color channels as a reference channel, and align the other two against this selected reference by performing an exhaustive search over a window of displacements (selecting green as my reference channel here seemed to work best). The size of the window of displacements is a hyperparameter that should be tuned for each specific use case, and I found searching over an interval of [-32, 32] along both the x and y axes gave good results across all images while still processing them at a very reasonable rate (~15-20 sec per image).
For each displacement in the window, I measured its "goodness of fit" with two different metrics: the sum-of-squared differences (ssd) and the normalized cross-correlation (ncc). Empirically, I failed to observe any noticeable difference between the generated images when alternating between the two, and both aligned images like the one below incredibly well!
However, I had two main issues to work through still:
Here are the results of running the vanilla pyramid alignment algorithm on all the rest of the example images:
The computed optimal displacements for the red and blue channels for each image are annotated below:
Here are the colorized images for a few additional images I handpicked from the Prokudin-Gorskii collection:
I implemented an algorithm known as histogram equalization that automatically applies contrasting to the images in the collection. This algorithm works by creating a histogram of pixel values across the image, then scaling these pixel values to stretch out their range, working to populate underpopulated intensity values. Increasing the prevalence of these underpopulated intensities enhances contrast in areas of the image that were previously not present. The results of running histogram equalization on two of the Prokudin-Gorskii images can be seen here:
Additionally, I implemented a white balancing algorithm named gray world in each of the provided images as well. The theory behind gray world assumes that the average reflectance across a scene is achromatic. This is to say that the mean pixel value across each of the three color channels should be roughly equivalent. To this end, I computed this average for each channel and scaled each pixel value across the channels by a gain factor selected to make the means equivalent. The results for a couple of selected images are displayed below:
For all images, I tried combining the two filters and applying both white balancing and auto-contrasting, leading to even more drastic differences in image appearance. Quite exciting indeed! Here are a couple select results that seemed to look particularly good after application of the combined filters:
Overall, this project was a wonderful introductory learning experience into the field of computational photography. I learned quite a bit about how the different types of ways images can be represented by a computer, as well as a good amount about various color spaces. I was able to get some first-hand experience implementing and experimenting with image filters - something I intend to spend more time playing around with in the future. There certainly were roadblocks and many failures in terms of image alignment initially, but by the time of completion, my algorithm worked efficiently and didn't fail to align any of the images.