CS 194-26 Project 1: Colorizing the Prokudin-Gorskii Photo Collection

By Eric Li

Project Overview

The goal of this project was to create a colorized image given a .jpg or .tif file from the Prokudin-Gorskii Photo Collection. We were to do this by extracting the RGB color channels from each raw image and applying our algorithm to colorize the photo.

Low-Resolution Approach

For low-resolution photos (hundreds of pixels), we can apply a naive brute-force approach. First, I cropped each color channel border by 35 pixels. Then, for each non-blue color channel, I iterated over all possible (x, y) offsets with in the range -15 < x < 15 and -15 < y < 15 and chose the one which minimized the sum of squared differences (SSD) between itself and the blue channel. I then stacked all three color channels to generate a colorized image.

High-Resolution Approach

For high-resolution photos (thousands of pixels), we want to change our approach to be more efficient. The method I chose was to implement an image pyramid recursively. An image pyramid works by first searching for the best offset for the same image at a lower scale (resolution), then propagating that offset upwards recursively reducing the offset search range at for itself at a higher scale. I chose a scaling factor of .25 with a maximum depth of 5 or when the images were scaled down to a 400 pixel count or less. I cropped each channel border by 800 pixels before processing and kept SSD as my cost function.

Bells & Whistles: better features with Sobel filter

To combat variations in brightness, I applied first-order derivative edge detection to my channels through Sobel edge detection using the skimages filters. This feature improvement helped greatly with colorizing and refining images, especially the Emir image, as seen in the before and after below.

before Sobel edge detection after Sobel edge detection (observe Emir no longer has color shadows)

before Sobel edge detection after Sobel edge detection (harder to see, but observe the table leg edges no longer have color shadows)

Challenges

It was difficult finding the right scaling factor for the pyramid base case. I went though a lot of manual trial and error when searching for the one I thought worked best. I also noticed in the output, there were mismatched borders at times and color flares. I resolved this by cropping the edges of the photos.