For this project, I took the image negatives that Sergei produced and alined them to create beautiful images of the Russian empire.

The very first steps were to import the image, and split it in three segments. The first block was the blue scale image (B). Then came green (G). Last was red (R). Once the images were imported, I began work on alinement.

0-Alignment

To aline the images, I started by seeing what the images looked like before they were alined. This I called 0-Alinement. As you can see in the image table at the bottom of this page, those images are extremely blurry and very hard to decipher. So much for viewing the Russian Empire.

Scoring Mechanism

The next approach I took was to aline the images using a scoring mechanism. This meant that I would shift the G or R to match the B frame (B was my reference frame). Then each alinement would be assigned a score. The alinement with the best score was the image. For these alinements, I attempted shifting the image between [-15, 15] in the (X,Y) direction. This meant that just to aline G with B, I was comparing 600 images (all shifted versions of G) with B. This technique was pretty good with smaller images, but with larger images, there were two problems. The [-15,15] was too small for images with heights and widths in the thousands. These would require larger ranges and the larger the range, the more comparisons. The second issue was that the large images were taking too much time. One image took me around 480 seconds! So for larger images, I decided to use a different approach.

Before talking about the approach for larger it is important to mention that there were two algorisms that I used to score the alinement for images. The first was Sum of Squared Differences.

sum((image1-image2)^2)

The second approach was normalized cross-correlation. This provided me with better data and so I used this in the future as well. Normalized Cross-correlation is the dot product of the images divided by their norm.

image1/||image1|| (dot) image2/||image2||

For the approach above, I removed 25 pixels for smaller images. This removed the black border.

While this approach was good with smaller images, it had several issues with larger images, which were mentioned above. The next approach I took was using an image pyramid.

Image Pyramids

For this approach, I took each image compressed to to half the size, till the size was less than 200 pixels.

Then I used the NCC scoring mechanism to match the image with its reference. Once this was done, I scaled up the image and doubled the value of the alinement vector. This gave me really good results.

The only issue I had with the result was that the borders were giving me problems. To remove them, 4% from the side of the images. This led to most of the images being alined.

At this point, the only image that was not alined was Cathedral and Emir. Cathedral was not working because a lot of the border was still present after the 4% cropping. This was fixed when I applied 8% cropping. Emir was not working because it has a different color channel.

Fixing Emir

To fix the Emir, I applied a strong contrast to R. If pixel intensity was less than 0.5 then it was assigned a 0 and if greater or equal, it was assigned a 1. This made the red image to the black figure of the Emir, with a few highlights on the ground. This also aligned well with the Green and Blue Channel.

Future Improvements

In the future, I would like to implement other scoring mechanisms. I would also like to automate border cropping and contrasting.

0-Alignment

Normalized Cross-Correlation

Pyramid w/ Cropping