cs194-26-abs: proj1

1: Overview

In this project, we are tasked with reconstructing RGB images from unaligned scans of the individual red, green, and blue components. We must minimize visual artifacts by writing a xy-translation-based algorithm optimized for high-resolution images.

2: Approach

My initial move was to reconstruct low-resolution images (cathedral.jpg, monastery.jpg, tobolsk.jpg) using a naive scan over my chosen offset range (a 31 x 31 window), testing both sum of squared differences (SSD) and normalized cross-correlation (NCC) as my similarity algorithms. I discovered that NCC performed better, so I chose NCC for the rest of the project. I also discovered that the image borders created interference in similarity measurements, so I performed the similarity tests over a rectangle with max size 120 x 80 centered on the x and y axes.

The 31 x 31 window was ineffective on the high-resolution images, but the naive scan had excessive runtime when this window size was increased, so I implemented a Gaussian pyramid-based search. The search repeatedly halves the scale of the image until both dimensions are under 500x500, then searches each scaled image from smallest to largest, trying offsets over a 31 x 31 window centered around the previous scale's best offset. I discovered that algorithm performed best on the provided batch of images when finding the relative offsets of the red and blue images from the green images.

The resulting algorithm takes approximately 15 seconds to produce a final image of about 3700 x 3200.

3: Images

Alignment offsets are show relative to the green images

3.1: Low-Resolution Sample Images

cathedral.jpg

Original Aligned [R: (7, 1), B: (-5, -2)]

monastery.jpg

Original Aligned [R: (6, 1), B: (3, -2)]

tobolsk.jpg

Original Aligned [R: (4, 1), B: (-3, -3)]

3.2: High-Resolution Sample Images

emir.tif

Original Aligned [R: (58, 17), B: (-48, -23)]

This image had trouble when using blue as the reference for aligning red and green, so the algorithm was changed to use green as the reference.

harvesters.tif

Original Aligned [R: (64, 0), B: (-59, -19)]

icon.tif

Original Aligned [R: (48, 5), B: (-41, -18)]

lady.tif

Original Aligned [R: (59, 4), B: (-51, -7)]

melons.tif

Original Aligned [R: (97, 3), B: (-83, -8)]

This image had trouble when using blue as the reference for aligning red and green, so the algorithm was changed to use green as the reference.

onion_church.tif

Original Aligned [R: (58, 11), B: (-50, -26)]

self_portrait.tif

Original Aligned [R: (98, 8), B: (-78, -29)]

three_generations.tif

Original Aligned [R: (59, -5), B: (-49, -15)]

train.tif

Original Aligned [R: (43, 26), B: (-42, -6)]

village.tif

Original Aligned [R: (72, 10), B: (-64, -13)]

workshop.tif

Original Aligned [R: (53, 12), B: (-53, 0)]

3.3: High-Resolution Self-Selected Images

church.tif

Original Aligned [R: (44, 0), B: (-40, -8)]

cross.tif

Original Aligned [R: (28, -6), B: (-18, -8)]

novaia_ladoga.tif

Original Aligned [R: (22, 6), B: (-12, -21)]

peasant_girls.tif

Original Aligned [R: (26, 9), B: (22, -19)]

steamboat.tif

Original Aligned [R: (127, 12), B: (-17, -17)]