CS194-26 Proj 1: Images of the Russian Empire

Brian Aronowitz: 3032201719

First pass

I started by implementing basic SDD/Normalized cross-correlation between the jpg images: comparing the R,G,B values over an exhaustive [-15, 15] window of shifts of the G and B images. The R channel is never shifted. I added an additional optimization, to only search the center third of the image, in an effort to ignore the black borders of the images. Below are some results from this implementation.

Second pass

Unfortunately, the naive searching algorithm, even with the center-third optimization, results in terrible looking alignments for larger images. This is because the images need to be moved relatively far to get ideal alignments, which given the naive implementation, would be extremely computationally inefficient. I implemented an image pyramid to deal with this. It is dynamically sized, reducing the images by a constant factor until it reaches a 300 pixel threshold that I set. On average, about seven levels are created for the larger images. On each level of the pyramid, the alignment routine returns the computed ideal offsets to the pyramid level above it. This offset is then applied to the images, and then the images on the current level are computed. It continues to compute these offsets iteratively, storing all the offsets in a sum, which is at the end applied to the original images. Below are some results of this image pyramid optimization.

As show below, the algorithm fails in some cases. For Emir the image pyramid, which uses SSD, simply breaks, and gives a horrifiying ugly result. The dense patterns make the center searching algorithm freak out and misalign the image.

Bell and whistle 1/ Emir Alignment solution

To deal with this, I implemented edge detection on the R,G,B channels, with the reasoning that pure white pixels, surrounded by blackness, would allow for more precise alignment of the images. The edge detection algorithm is an implementation of Sobel Edge detection. It works by first convolving a 2D, normalized gradient-detection matrix over the whole image, to detect gradients in the x direction.

It then convolves the 90 degree rotated version of the first matrix over the image to detect gradients in the y direction. The resulting two images containing the gradients are squared, added, then their square root is taken. This resulting image (which holds the image gradient information) is multiplied against the original image to extract it's edges.

The end