~~~ Images of the Russian Empire ~~~

~~~~~~~~~ George Lee ~~~~~~~~~~~

~~~~Project 1 CS194-26 Fall 2018~~~~

In this rendition of project 1, I wrote code to align the various RGB color channels that are found within a color image. For all of the images, both the red and the green channels were aligned with respect to the blue channel. I used the Sum of Squared Differences (SSD) to calculate the relative similarity between the alignments of two channels. To find the best alignment for a respective channel, I initially iterated over a range of [-20, 20] pixels in both the x and y direction.

This initial method, however, was quickly realized to be highly time intensive in terms of the raw number of iterations that needed to be done to fully align an image. As image sizes increase, the computational time to calculate the SSD will also linearly increase. One speedup that I implemented were image pyramids. The concept of an image pyramid is to fully align an image on a downscaled version of itself. Due to the smaller scale, computation time can be decreased. Any alignment offsets that are produced from a downscaled image can be directly translated to the original size image.

One extra step that I took in my implementation was to use edge detection as a means to help align the images. I initially experimented with Canny edge detection alone, using the built-int opencv function, cv2.Canny, but realized that some results were not optimal. In instances where a channel was too saturated, the edge detection algorithm failed to provide distinct lines that could be matched. An example of this is shown below. The two images below represent different channels.

To fix this, I first preprocessed the image by using histogram equalization using cv.equalizeHist, before applying the Canny edge detection step. The results using the same channels is shown below.

As seen now, the two channels look much more similar to each other. Performing this optimization helped significantly when aligning the image for Emir. The blue robe that Emir wears appears to significantly impact the ways the other channels are aligned, if they were to be aligned solely by color. Below is a comparison between three different alignments of Emir (from left to right: unaligned, aligned by color only, aligned with edge detection)

Below are two images that I chose personally to align. The first image is a river, which was easily aligned using the current algorithm.

The next image was not able to be aligned using the optimized implementation with edge detection, but was able to be aligned with histogram equalization alone. This is likely to be due to the high amount of edges that are present within the image. Below are the images (from left to right: unaligned, aligned with edge detection, aligned with color)

Below are the images provided by the project specifications

Unaligned Aligned
r: x 3 y 12 g: x 2 y 5
r: x 107 y 41 g: x 49 y 24
r: x 124 y 13 g: x 59 y 16
r: x 89 y 23 g: x 40 y 17
r: x 110 y 11 g: x 49 y 8
r: x 2 y 3 g: x 2 y -3
r: x 0 y 8 g: x 1 y 3
r: x 175 y 36 g: x 78 y 29
r: x -1 y 14 g: x 0 y 7
r: x 111 y 11 g: x 53 y 13
r: x 87 y 31 g: x 42 y 4
r: x 116 y 27 g: x 56 y 21
r: x 138 y 22 g: x 65 y 11