Image Colourizer

Jesse Gao, CS194-26-afi



Overview

The goal of this project is to colorize black and white negatives from Prokudin-Gorskii' photo collection. Prokudin-Gorskii took three separate photos of each scene through colored filters, with the intention of capturing color in an age of black and white cameras. This project puts those photos together in an attempt to recreate a colored picture from each set of three negatives using various alignment techniques.

Naive Implementation

The simplest implementation of colorization simply tried out all possible horizontal and vertical displacements for a pair of images within a certain range and picked the displacement with the smallest SSD (squared summed difference) of all the pixels. The displacement was then used to "roll" the "red" and "green" image lined up with the "blue" one, and they are stacked to form an RGB picture.

Pyramid Implementation

For much larger images, exhaustively searching through each displacement pixel by pixel was just not feasible and heats up my computer really fast, so a pyramid implementation was built to search through the displacements better. Essentially, instead of making really small displacements each time it makes wider strides. It downsamples the image by .5 and multiplies the displacement by two, so that each displacement is doubled as you go down the layers. There are a maximum of 5 layers, so the strides go up to 32 at a time

The pictures are shown below. Overall, the alignment has mixed results because the algorithm attempts to optimize for the alignment that shows the least pixel by pixel difference between black and white pictures of varying brightness. This means that very saturated areas such as green trees and blue skys will be drawn towards each other. I tried cropping out the border of each image, and while it made processing faster, it changed very little in terms of alignment quality. Using sobel edge detection somehow made alignments worse.



cathedral G: (-10, 3), R: (-10, 9)
emir G: (-150, -22), R: (-550, 123)
lady G: (-530, 129), R: (-370, 133)
village G: (-490, 39), R: (-490, 294)
harvesters G: (-510, 83), R: (-630, 165)
monastery G: (-10, -3), R: (-10, 3)
nativity G: (-10, 3), R: (-10, 7)
self_portrait G: (-550, 105), R: (-550, 217)
settlers G: (-10, 7), R: (-10, 9)
three_generations G: (-410, 65), R: (-630, 71)
train G: (-490, 102), R: (-590, 334)
turkmen G: (-630, -11), R: (-630, 72)
icon G: (-320, 32), R: (-610, 109)
prisoners G: (-10, 2), R: (-10, 7)