CS 194-26 Project 1 - Pinxiu Gong

Overview

This project tries to recreate color images using Sergei Mikhailovich Prokudin-Gorskii's glass plates. The plates do not align with each other perfectly, and their quality might have been affected by exposure time and erosion. This project aims to counteract the loss of information in the glass plates.

Approach

Basic Approach: I first crop off 0.1 of each edge of the picture to avoid black/white borders. Then, i use two nested for loops to test the best shift offset in the range of [-15, 15]. THe best shift offset should produce an adjusted image that has the lowest score with the reference image (in most cases the b graph). I offer three algorithms for user to choose, which will be used by the scoring function.

Algorithm 1: SSD on pixel value.

Algorithm 2: SSD on the difference between the pixel vaue and the average value of all the 8 adjacent pixels.

Algorithm 3: Additional to Algorithm 2, scale the difference value for each pixel based on the min and max difference value within the graph.

Advance Approach: For each big image, I recursively scale it down by 1/2, 1/4, 1/8... until the size of the image is not greater than 500 * 500. Then I find the best shift values, x and y, and multiply the values by 2, return to the caller function, and continue to align the image with a higher precision. I repeat until reaching the original picture. In order to speed up, each time before I enter the for loop and call the scoring function, I first select a maximum 500 * 500 window of the image.

Extra Credit: I implemented automatic cropping with Canny library. For each of the four sides of the image, I search within the 0.1 of the border, and crop off along the lines where Canny returns more than 1/3 of each of the line's total length number of True value. In other words, I crop off along the borders that are of 1/3 length as the corresponding width or height of the image.

I also implemented automatic contrasting using skimage.exposure library.

I implemented white balance by modifying the histogram and creating a new transformation to the image. I first clear to zero all the bins with a frequency less than 0.05% of the total number of pixels. Then I adjust the corresponding cdf so that it becomes almost linear. Then I generate the new image using cdf as the transformation lookup table.

I generated all the 6 possible color mappings (g, r, b), (g, b, r), (b, r, g), (b, g, r), (r, g, b), (r, b, g) with the same order as the glass plates in source images.

I also used a better feature. The difference value of pixels that I used in Algorithm 2 and 3 is a different feature from RGB similarity.

Results

Algorithm 1

cathedral.jpg
G = [2, 5] R = [3, 12]

monastery.jpg
G = [2, -3] R = [2, 3]

nativity.jpg
G = [1, 3] R = [0, 7]

settlers.jpg
G = [0, 7] R = [-1, 14]

emir.jpg
G = [23, 48] R = [-5, 58]

This is off because the intensities of different colors vary a lot, which throws SSD off.

harvesters.jpg
G = [17, 58] R = [14, 122]

icon.jpg
G = [17, 40] R = [22, 89]

lady.jpg
G = [7, 52] R = [7, 114]

self_portrait.jpg
G = [28, 77] R = [36, 174]

three_generations.jpg
G = [13 48] R = [11 109]

train.jpg
G = [5, 41] R = [31, 85]

turkmen.jpg
G = [18, 54] R = [27, 116]

village.jpg
G = [12, 64] R = [22, 137]

Algorithm 2 (Only show images with different result from Algorithm 1)

emir.jpg
G = [23, 48] R = [41, 107]

harvesters.jpg
G = [18, 59] R = [15, 123]

icon.jpg
G = [17, 40] R = [23, 89]

nativity.jpg
G = [1, 3] R = [0, 8]

lady.jpg
G = [2, 51] R = [0, 112]

three_generations.jpg
G = [13, 49] R = [11, 101]

The contrast of b image is different from the other two (above the middle man's head), which throws the difference scoring function off.

train.jpg
G = [5, 42] R = [31, 86]

turkmen.jpg
G = [22, 56] R = [29, 117]

village.jpg
G = [12, 65] R = [22, 138]

Algorithm 3 has the same result as Algorithm 2.

Self-selected Images

01484a.tif

01893a.tif

Images improved by cropping, using Algorithm 2

Images improved by contrasting, using Algorithm 2

Images improved by white balance, using Algorithm 2

Images improved by color mapping, using Algorithm 2

Plate order: R, G, B