Project 1: Images of the Russian Empire
By Sriharsha Guduguntla
Project 1 entails taking digitized RGB glass plate images taken by Mikhailovich Prokudin-Gorskii and aligning them to reveal beautiful full color images. For this project, I took two main approaches to aligning the plates. I tested my algorithms on all the example images provided in the project spec, but I also tried it on 3 more images from the collection that are shown below.
Exhaustive Search (Approach 1): I used a window of
[-15,
15] and tried
different
displacements of two images (R + G,
B + G) and
minimized the Sum of Squared Differences
(SSD) which is defined as
sum(sum((img1-img2)^2)) to get the optimal alignment horizontal (X) & vertical
offsets (Y).
- Aligned the Red (R)
plate with
the Green (G) plate using
np.roll - Aligned the Blue (B)
plate with the Green (G) plate using
np.roll - Stacked the new aligned Red (R) and Blue (B) plates with the original Green (G) plate to produce a full color picture
Pyramid Search (Approach 2): Instead of exhaustively searching the
full resolution image as I did in the previous approach, I decided to rescale the given image
down to either 25% of its size or 6.25% of its size both of which I
determined empirically for
each image. Then, starting with the smallest image, I recursively performed a search
using a window of [-15, 15] to get the optimal alignment horizontal (X) &
vertical offsets (Y). In each recursive step, I rescaled the image to twice its previous size
and started performing the search starting from the previously accumulated offsets scaled by 2,
only stopping when it reaches the original full size image.
In other words, this algorithm goes from a coarser image to a finer image (top of the pyramid to
the bottom) while finetuning the alignment offsets as it continues.
Challenges & Problems: I faced a number of interesting caveats while developing the algorithms above. I noticed that it was difficult to generalize certain parameters of the algorithm across every image without adding an extra layer of complexity. For example, when rescaled the images all the way down to 6.25% of their original size, this did not always produce a well-aligned result, but 25% of the original size worked perfectly. This is likely due to the fact that the search space is severely limited for some images when they are downscaled too much. In terms of how well the algorithms did, the emir and lady images are not as well aligned as some of the other images likely because of unique discrepancies in the image properties. The emir image also had different brightness values for each plate which could have caused issues with the alignment as well. Another issue I faced was speed because some of the larger tiff images took a long time to complete with pyramid search and I had to make sure I was avoiding doing any extra or unnecessary computation to ensure that the program is as optimal as possible. I also tried aligning different plates to each other and after enough playing around with the images, I noticed that aligning the red and blue plates to the green plate was for the most part, producing the best alignments.