CS 194-26 Project 1

Jason Zhang (zhang.j@berkeley.edu)

Project Spec: https://inst.eecs.berkeley.edu/~cs194-26/fa18/hw/proj1/

Overview

The goal of this assignment is to take the digitized Prokudin-Gorskii glass plate images and, using image processing techniques, automatically produce a color image with as few visual artifacts as possible. In order to do this, you will need to extract the three color channel images, place them on top of each other, and align them so that they form a single RGB color image.

Alignment

I began by exhaustively searching over possible displacements and scoring each possible displacement using normalized cross-correlation (NCC).

For larger , this quickly became computationally infeasible, so I implemented an image pyramid to check over a smaller window at a variety of scales. This significantly improved runtime.

For better matching, I tried the following edge detection methods:

 begin{bmatrix} 1 & -1 end{bmatrix}
 begin{bmatrix} 1 & 0 & -1  2 & 0 & -2  1 & 0 & -1 end{bmatrix}

The Sobel filter and Canny Edge Detector both performed reasonably well.

Below is a visualization of the different levels of the image pyramid. From left to right: resized image, Canny edge detector applied to R, G, and B channels respectively. The edges in the R and G channels were then compared to the edges with the B channel using NCC. I didn't tune the thresholds properly so the edges don't great the higher resolutions layers.

alt text 
alt text 
alt text 
alt text 
alt text 
alt text 

Cropping

To crop the image to remove the borders, I first applied a Sobel filter to the gray-scale image to get horizontal and vertical edge detectors.

alt text 

Notice how the borders were picked up relatively strongly by the edge detectors. I averaged the vertical edge detector along the horizontal dimension and the horizontal edge detector along the vertical dimension. The indices with large means thus correspond to the borders of the picture.

Here are some visualizations. The blue lines on the margins are the mean values from the edge detectors (after smoothing and normalizing). The red lines are the thresholds to crop.

alt text 

Failure Modes

This method assumes the borders are axis-aligned and produce a sharp gradient.

For the most part, the borders were axis-aligned but the gray-scale versions of the images did not always have sharp edge. For example, the cathedral's yellow bar has a very similar luminance to the blue sky.

alt text 

Perhaps this method would have worked better if applied on each channel individually rather than on the grayscale image.

Recoloring

I began by playing around with the temperature of the color palette by simply rescaling the channels. I think there was some improvement since indoor pictures should have warmer light and the outdoor picture should have cooler lighting. Overall, the effect with negligible and in many cases actually looked worse.

alt text 
alt text 

Then, I tried a bunch of techniques with varying success:

alt text 
alt text 
alt text 
alt text 
alt text 
alt text 
alt text 
alt text 
alt text 
alt text 
alt text 
alt text 
alt text 
alt text 
alt text 

Offset

image disp_r disp_g
cathedral (12, -3) (-5, -2)
emir(-107, -40) (-49, -24)
harvesters(-124, -14) (-60, -18)
icon(-90, -23) (-39, -16)
lady(-120, -13) (-56, -10)
monastery(-3, -2) (3, -2)
nativity(-8, 0) (-3, -1)
self_portrait(-175, -37) (-77, -29)
settlers (-14, 1) (-7, 0)
three_generations(-111, -8) (-58, -17)
train(-85, -29) (-43, -5)
turkmen(-118, -29) (-57, -22)
village(-137, -21) (-65, -11)
chalice (-4, -2) (-1, -1)
poles (-6, -3) (-2, -2)