In this project, I wrote algorithms to colorize images from the digitized Prokudin-Gorskii glass plate photo collection. Prokudin-Gorskii traveled across the Russian Empire, capturing color photographs by recording exposures of his surroundings onto a glass plate using a red, green, and blue filter. As a result, each source image contains three distinct negatives stacked on top of each other, with each negative corresponding to the intensity of a singular RGB channel. However, while the negatives are all shot from approximately the same location and angle, the digitized scans introduce positional misalignments. The goal of this project is thus a problem of image alignment: given three separate image intensity maps in the R, G, and B color channels, align them such that the resulting image is appropriately colorized.
Alignment Method: Exhaustive Search
For smaller images (less than 500x500 pixels in size), I implemented an exhaustive algorithm to search for optimal pixel offsets when aligning the three color channels. The algorithm
takes in a base image intensity map and another image intensity map to align. It searches for the optimal horizontal and vertical offset within the range of [-20, 20] for the misaligned
image intensity map, producing the offset that minimizes the SSD (Sum of Squared Differences) distance between intensity maps. Since I realized that the image borders are rather noisy,
I opted to remove the borders from SSD distance calculation altogether. I crop out 20% of the image from each border when calculating SSD distance, resulting in only the middle 60%
of the image being 'judged' on alignment. The next step was to pick a color channel as the base that I'd align other channels on. After some experimenting, I found out that aligning on
the green channel produced the best results. I thus first align the blue channel with the green channel with the appropriate offsets produced by my exhaustive algorithm, then align the red channel
with the green, and finally stitch all three channels together to produce the final colorized image. Some results of this approach are shown below. The offsets are provided as tuples in the format
(x, y).
|
|
|
Alignment Method: Pyramid Search
For larger image files (such as the .tif images provided), the exhaustive approach takes too much time. As the bulk of the computation is done when calculating the alignment offsets, it is ideal
to first reduce the size of the image intensity map to find the offset and then translate it to some corresponding offset in the original image intesntiy map. To do this, I implemented an image
pyramid search. The image pyramid stores five different layers of images. Starting from layer 0 (the original image intensity map), the boundaries of each layer are further downscaled by a factor
of two (layer 4 stores the image at 1/256 the original resolution). At the highest level, or the layer with the lowest quality, I perform the exhaustive search to find the optimal offset at that level.
For each subsequent level, I then scale the previous level's offset by the same factor the previous layer was resized by and search for an optimal offset within [-2, +2] the scaled offset. This
eventually reaches the base case of level 0, providing an efficient means of finding the optimal offset for the original image intensity map. The rest of the algorithm follows the exhaustive approach
pretty similarly. The results of this approach can be seen below.
|
|
|
|
|
|
|
|
|
|
|
Additional Images
|
|
|