Introduction

The Prokudin-Gorskii image collection from the Library of Congress is a series of glass plate negative photographs taken by Sergei Mikhailovich Prokudin-Gorskii. For each scene, three photos are taken, each representing a color channel corresponding to red, green or blue. We can overlay the three color channels and recreate the colored image. However, the channels are not perfectly aligned and the goal of this project is to perform automatic alignment of the channels and remove the artifacts caused by overlaying.

The following algorithms are implemented for this task:

Exhaustive search to find proper alignment: In a window of (-15, 15), the algoritm tries each alignment and selects the best one based on the metric used.
Pyramid search: Recursively scale down the image by a factor of 0.5 until the image is small enough. Then we perform an exhaustive search on the smallest image. We then use that alignment information on bigger images and shrink the search window to make finer adjustments.
Edge detection: Perform edge detection before calculating the metric (SSD) can improve the performance of the algorithm significantly
Automatic cropping: For rows (columns) near the border of the image, we calculate the average gradient between each row (column) of pixels and the next row (column) of pixels. For any row (column), if there is a channel with mean gradient higher than the threshold, we mark it as a candidate. The row (column) closest to the center is selected as the new border

Results on example images

Offsets

Image Name	G to B alignment	R to B alignment
castle.tif	(36, 4)	(95, 4)
cathedral.jpg	(5, 1)	(12, 3)
emir.tif	(53, 27)	(107, 40)
harvesters.tif	(55, 10)	(117, 11)
icon.tif	(38, 16)	(90, 22)
lady.tif	(56, 10)	(120, 13)
melons.tif	(79, 9)	(177, 13)
monastery.jpg	(-3, 2)	(3, 2)
onion_church.tif	(52, 30)	(107, 34)
self_portrait.tif	(81, 27)	(175, 37)
three_generations.tif	(58, 14)	(115, 12)
tobolsk.jpg	(3, 2)	(6, 3)
train.tif	(40, 8)	(84, 28)
workshop.tif	(50, -2)	(101, -12)

Other examples from Prokudin-Gorskii collection

Extra Credit

Border Cropping

As described in the introduction, border cropping is performed by calculating average gradient of rows and cropping those that exceeds a given threshold. Effects are shown below with a few examples.