Project 1

Introduction

(Introduction heavily adapted from the project description)

The goal of this assignment is to take the digitized Prokudin-Gorskii glass plate images and, using image processing techniques, automatically produce a color image with as few visual artifacts as possible. In order to do this, we will need to extract the three color channel images, place them on top of each other, and align them so that they form a single RGB color image.

Details

There were several things I considered for this process.

I implemented both sum of squared differences (SSD) and normalized cross-correlation (NCC) as alignment algorithms. I found greater success using NCC for the smaller dimension photos, so I used that going on. All the images used the blue channel as a reference to align the red and green channels to.

My original implementation searched over a 31x31 window to find the optimal alignment; however, for the larger photos, 31x31 was not enough. It would be too intensive to exhaustively search, so I implemented a pyramid seaarch to more effciently traverse the images. This involved searching 31x31 windows recursively while decreasing the image size at every level of the pyramid. I stopped the pyramid when the largest dimension was less than 100. Given the search window, the algorithm will not be searching over the entire image. However, if the optimal shift involved shifting over 30% of the image off the screen, the resulting colored image would not look aesthetic anyway.

Because the photos are scans of camera plates, most of the photos had black bars along the edges. These bars caused issues when finding the optimal alignment, so I automatically cropped the photos as part of the preprocessing procedure. In some scenarios, I also cropped into the image, but for the sake of demonstration I think I left enough of the photo.

The offsets I list do not take into account the cropping I did. It should not affect the offset values, but in case it does, I wanted to point that out.

Small Images

Cathedral

Original Offset (G: [5, 2]; R:[12, 3])
unaligned_cathedral.jpg aligned_cathedral.jpg

Monastery

Original Offset (G: [-3, 2]; R:[3, 2])
unaligned_monastery.jpg aligned_monastery.jpg

Large Images

Emir

Original Offset (G: [49, 24]; R:[0, -869])
unaligned_emir.jpg aligned_emir.jpg

My algorithm failed to align this image. Looking at the original plate scans, the Emir's outfit is overwhelmingly pure blue. My algorithm tries to align the red and green channels to the blue channel and a large portion of the image is blue. It will seem like the images are never "aligned" because the blue channel has different brightness magnitudes.

Harvesters

Original Offset (G: [59, 16]; R:[124, 13])
unaligned_harvesters.jpg aligned_harvesters.jpg

Icon

Original Offset (G: [40, 17]; R:[89, 23])
unaligned_icon.jpg aligned_icon.jpg

Lady

Original Offset (G: [47, 9]; R:[113, 11])
unaligned_lady.jpg aligned_lady.jpg

Melons

Original Offset (G: [82, 10]; R:[179, 13])
unaligned_melons.jpg aligned_melons.jpg

Onion Church

Original Offset (G: [51, 26]; R:[108, 36])
unaligned_onion_church.jpg aligned_onion_church.jpg

Self Portrait

Original Offset (G: [78, 28]; R:[176, 36])
unaligned_self_portrait.jpg aligned_self_portrait.jpg

Three Generations

Original Offset (G: [53, 14]; R:[112, 11])
unaligned_three_generations.jpg aligned_three_generations.jpg

Train

Original Offset (G: [42, 5]; R:[87, 31])
unaligned_train.jpg aligned_train.jpg

Village

Original Offset (G: [65, 12]; R:[137, 22])
unaligned_village.jpg aligned_village.jpg

Workshop

Original Offset (G: [52, 0]; R:[104, -12])
unaligned_workshop.jpg aligned_workshop.jpg

Extra Photos

Boris-Gleb Monastery

Original Offset (G: [42, 11]; R:[98, -5])
unaligned_Boris-Gleb%20Monastery.jpg aligned_Boris-Gleb%20Monastery.jpg

Church of Saint George

Original Offset (G: [73, 22]; R:[154, 35])
unaligned_Church%20of%20Saint%20George.jpg aligned_Church%20of%20Saint%20George.jpg

City of Rzhev

Original Offset (G: [61, 32]; R:[129, 68])
unaligned_City%20of%20Rzhev.jpg aligned_City%20of%20Rzhev.jpg

Clock

Original Offset (G: [52, 24]; R:[110, 18])
unaligned_Clock.jpg aligned_Clock.jpg