Intro

The goal of this assignment is to take the digitized Prokudin-Gorskii glass plate images and, using image processing techniques, automatically produce a color image with as few visual artifacts as possible.

Methods

Naive

We implemented both single-scale alignment and multi-scale alignment. In the single-scale alignment, we use the naive approach of searching over a window of possible displacement. The metric we use for determining the quality of alignment is SSD (sum of squared difference). During the search, we maintain the best displacement found, and at the end of the algorithm, we return the displacement found.
We then implemented multi-scale alignment for large images using image pyramid where the coarsest image is kept at a resolution about 64 x 64. Starting from the coarsest image, we update our estimate of displacement along the way.

Bells & Whistles

We first realize that the current approach does not work for images like 'emir' where the artifacts on the rough borders of the image may disrupt the calculation of SSD, and the pixel brightness across three channels are not uniform. Hence instead of aligning to the red channel, we use green channel as the anchor. We then decide to do automatic cropping with edge detection first before estimating the displacement. Specifically, we use the Canny Edge Detector to find the edges in all three channels. Then to align three channels, we find the maximum cropping area across three channels and apply the same cropping. This enhances the quality of lots of tested images but may have had negative impact on the image like 'cathedral' due to unknown reasons.
We also try automatic contrasting using OpenCV where we adjust the color histogram automatically to increase contrast. This proves to be a useful techniques.

Results

See below for results and displacement for each image, and the last two images are the extra images.

File	Naive	Bells & Whistles
monastery.jpg	blue: (-1, -9) green: (-1, -6)	blue: (0, 3) red: (0, 6)
melons.tiff	blue: (-7, -176) green: (-3, -96)	blue: (-8, -82) red: (3, 96)
three_generations.tiff	blue: (-7, -108) green: (0, -57)	blue: (-11, -54) red: (-1, 58)
onion_church.tiff	blue: (0, -108) green: (-10, -57)	blue: (-25, -52) red: (10, 57)
lady.tiff	blue: (17, -123) green: (10, -44)	blue: (1, -55) red: (7, 59)
workshop.tiff	blue: (16, -69) green: (11, -51)	blue: (1, -53) red: (-12, 52)
self_portrait.tiff	blue: (5, -130) green: (-7, -98)	blue: (-28, -78) red: (8, 98)
tobolsk.jpg	blue: (-3, -6) green: (-1, -4)	blue: (-2, -3) red: (1, 4)
village.tif	blue: (14, -281) green: (7, -103)	blue: (-12, -66) red: (11, 72)
cathedral.tif	blue: (1, -7) green: (0, -7)	blue: (-2, -5) red: (1, 1)
emir.jpg	blue: (-17, -107) green: (-9, -116)	blue: (-23, -49) red: (17, 57)
train.tif	blue: (-1, -107) green: (-1, -111)	blue: (1, -43) red: (26, 43)
icon.tif	blue: (-22, -89) green: (-5, -48)	blue: (-16, -42) red: (5, 48)
harvesters.tif	blue: (-7, -120) green: (3, -65)	blue: (-16, -60) red: (-3, 65)
00176u.tif	blue: (-30, -156) green: (-9, -82)	blue: (-20, -75) red: (10, 82)
01512u.tif	blue: (-25, -251) green: (-13, -77)	blue: (-13, -63) red: (13, 77)