Overview

Given three black-and-white images representing the red, green, and blue color channels, how can we align/combine them to produce a high quality color image?

My Approach

My solution aligns the red and blue channels to the green channel. As preprocessing, it crops out the outer 1/6 of the image, as we mainly care about the center of the image and the nasty borders on the sides can really screw things up. To align two channels, it tries to find a displacement that maximizes some image similarity metric. I used normalized cross-correlation (NCC), which is essentially a dot product between the two normalized images.

To find a good displacement, I used an image pyramid, a recursive solution. I halve the resolution and find a good displacement for these smaller images. The equivalent displacement is used as a starting point for the larger images, and then I try all displacements in [-2, 2]. (Ideally, [-1, 1] is enough to cover all possible displacements, but a slightly larger range allows the algorithm to correct itself a bit.) I stop recursing when the width or height of the image is at most 30 pixels, in which I just assume the best displacement is no displacement. The image pyramid is much faster than exhaustive search. Since it builds off of low-res/blurred images, the image pyramid works with larger features within the image, and so I imagine it would be less affected by noise compared to exhaustive search.