CS294-26 Project 1

Alvin Wan

I used both the single-scale and multi-scale approaches for images below. Run a naive search within a small delta window, then upscale the image and shifts accordingly. Continue iteratively. The project also includes explorations for various color alterations: auto-contrast, auto-white-balance, different bases for alignment. It includes some border experimentation, but in the end, naive border detection trumped the rest (in roi--efficiency per unit time--over a convolutional approach).

Problems: I found many issues with both emir and village. So, I cut off margins more aggressively (lower threhsolds, bigger margin with naive cut) The colors in others looked pale, so I pursued more color alterations below.

Bells and Whistles: (1) autoconstrast (2) auto white balance, (3) different basis for colors (no visible improvement) and (4) different border algorithms (no visible improvement)

Failures: Algorithm did not match up village and emir correctly. Looking at the images, the different in darkness is just too different. I suspect an edge detector or gradient-based method would be more effective. I personally was more interested in exploring color though.

Single-Scale, Multi-scale Image Pyramid Results

Below, we include the shifts for each image and the time taken to run the algorithm on that image (in seconds). Below, I ahd to run Emir and Village with a smaller delta so that it did not attempt to shift the channels off of the image.


name, dx1, dy1, t1, dx2, dy2, t2
Cathedral, 1, 0, 0.69, -2, 5, 0.68
Emir, -17, 136, 12.3, 290, 186, 11.5, 90.95
Harvesters, -3, 59, 100.65, 3, -105, 100.83
Icon, -5, 47, 110.17, -23, 300, 104.92
Lady, -10, 37, 101.93, 6, -50, 104.3
Monastery, 1, -1, 0.72, -2, 10, 0.89
Nativity, 0, -3, 0.72, -1, 4, 0.78
Self_Portrait, 7, 91, 104.88, 2, -44, 105.52
Settlers, -1, 1, 0.73, 0, 0, 0.89
Three Generations, 0, 51, 99.41, -5,-46, 99.47
Train, -3720, -3132, 98.30, 7, -97, 95.80
Turkmen, 6, 53, 116.65, -4, -50, 104.87
Village, -7, -1, 123, 13.47, -67, 183, 11.44

More samples from the gallery.

    
00090r,-212, -190, 0.78, -215, -65,0.75
01790r,-207, -190,0.73,-207, -179, 0.73
01043r,-1,12,0.04,-1,26,0.05

Auto contrast

The images are more rich but some colors are blown out too much. Left is original, and right is autocontrast.

Auto white balance

As opposed to autocontrast, autowhitebalance makes the images more rich, without blowing up colors unnecessarily. Left is original and right is auto white balance.

It's worth comparing autocontrast with autowhitebalance, too. Left is autocontrast and right is autowhitebalance. Notice the soil near the bottom of the image; the former leaves a strange red hue but the latter does not, rendering a more realistic brown.

Bases for Color

Funnily enough, no big difference. Here, I change the frame that serves as a "base". The other color channels are then aligned with the channel of cohice. The images run use channels blue, green, red from left to right.

Borders

The results were much more expensive to compute but in many cases were not clearly better. I probably have a bug. (code is in the submission of course) Anyways, some of the results above may look funky because the borders are all different. I admittedly was playing with borders as I was working on these comparisons. At least the color alterations are still valid comparisons. For some larger images, cutting off the border was more vital to successful image alignment (makes sense, as there are more pixels with 0 value that could be aligned).