Overview

The heart of the image stacking algorithm is in computing a cross-correlation between the base image (the blue plate) and the overlay image (green and red plates). My algorithm begins by cropping around 10% off every edge to reduce the effect that edge noise will affect my measurements. Then, a simple cross-correlation is taken between the base and overlay multiple times, where the overlay is displaced one pixel for every computation. The displacement vector which returns the highest cross-correlation is used to decide the final offset for the overlay images. After displacements are calculated for both overlays, the three plates are stacked to produce an RGB output.

In the image pyramid modification, instead of taking the unmodified plates, I start with downscaled images at 1/32 resolution to find a rough displacement with the above algorithm. Then, I increase the resolution by a factor of two, and run another alignment pass, using the displacement calculated as the stating offset, rather than (0, 0). Because the starting offset is already roughly correct, I can scan a smaller range of pixels nearby, refining the measurement. The plates are then scaled up by 2 again and the process is repeated until we return to the original scale. Using this technique, I save a substantial amount of compute effort, decreasing the run-time of very large images to an execution of less than 10 seconds in most cases.

There was one image where I was unable to obtain a good match - emir.tif. This was due to inconsistency in the light properties (brightness, contrast, etc) across the various plates that a naive cross-correlation can't correctly compute.

No extra credit (bells & whistles) was implemented for this project.

Results

castle.tif: The G plate was shifted by (3, 34), and the R plate was shifted by (4, 98)

cathedral.jpg: The G plate was shifted by (-272, -473), and the R plate was shifted by (-271, -466)

harvesters.tif: The G plate was shifted by (17, 59), and the R plate was shifted by (13, 123)

icon.tif: The G plate was shifted by (17, 41), and the R plate was shifted by (23, 89)

lady.tif: The G plate was shifted by (8, 52), and the R plate was shifted by (11, 113)

melons.tif: The G plate was shifted by (10, 81), and the R plate was shifted by (13, 178)

monastery.jpg: The G plate was shifted by (-271, -481), and the R plate was shifted by (-271, -475)

onion_church.tif: The G plate was shifted by (27, 51), and the R plate was shifted by (36, 108)

self_portrait.tif: The G plate was shifted by (29, 78), and the R plate was shifted by (37, 176)

three_generations.tif: The G plate was shifted by (13, 52), and the R plate was shifted by (11, 111)

tobolsk.jpg: The G plate was shifted by (-275, -475), and the R plate was shifted by (-275, -472)

train.tif: The G plate was shifted by (6, 42), and the R plate was shifted by (32, 87)

workshop.tif: The G plate was shifted by (0, 53), and the R plate was shifted by (-12, 105)

Examples

First self-selected example: The G plate was shifted by (13, 56), and the R plate was shifted by (24, 107)

Second self-selected example: The G plate was shifted by (5, 24), and the R plate was shifted by (7, 110)

Third self-selected example: The G plate was shifted by (6, 60), and the R plate was shifted by (4, 126)

Failures

Unfortunately, my program could not align emir.tif properly due to the unique layout of the plates. G and B had similar lighting features which allowed them to align properly, but the R plate differed substantially in lighting, causing a significant gap between its position and a "good" plate.