Project 1: Images of the Russian Empire
cathedral.jpg
G: [2,5]
R: [3,12]
monastery.jpg
G: [2,-3]
R: [2,3]
tobolsk.jpg
G: [3,3]
R: [3,7]
emir.tif
G: [24,49]
R: [41,106]
harvesters.tif
G: [17,59]
R: [14,123]
icon.tif
G: [18,41]
R: [23,90]
Zhi Chen cs194-26-afa
Overview
Around 1909, Sergei Mikhailovich Prokudin-Gorskii formulated a plan to document the Russian Empire in color photography systematically. He recorded three exposures of every scene onto a glass plate using a red, a green, and a blue filter. However, he had no way of combining them into a single color image. So the goal of this project is to take the digitized Prokudin-Gorskii glass plate images and automatically produce a color image using image processing techniques.
Exhaustively Match
Preprocessing three plates by cropping all borders and only using the pixels in the middle of the images.
Fixing blue plate at bottom, the naive approach is to exhaustively move red and green plates pixel by pixel on blue plate and find which moving vector will work best.
To define which moving vector is best, we need to define the loss function that calculate the degree of fitness. Here I just use SSD Distance.
Pyramid Match
When facing large images, exhaustively match consumed too much time. In order to make the algorithm more efficient, I applied pyramid technique.
I compressed image layer by layer with a compressing factor (default = 2) until the total number of image pixels is less than 10000. Using exhaustively match at the last level. Rescale image to the next larger layer and adjust the moving vector by searching around the previous result. Repeat rescale until it arrives the original layer.
Bells & Whistles
Results
Even though the pyramid match works in most of images. Some images still cannot be matched in a satisfactory way. Reasons may be various. For example, in emir.tif, the red plate has higher brightness than the other two plates.
In this case, I tried to apply Canny Edge Detection technique in order to increase the effectiveness of the algorithm.
One important part in Canny Edge Detection should be the upper_threshold and lower_threshold that need to be set.
However, exhaustively match can only treat small images, because exhaustively match is really time- consuming!
Source from internet: https://iipimage.
sourceforge.io/documentation/images/
If a pixel gradient is higher than the upper threshold, the pixel is accepted as an edge
If a pixel gradient value is below the lower threshold, then it is rejected.
low lower_threshold
Emir.tif and village.tif have huge differences on brightness between three plates provided. So pyramid match algorithm did not achieve a good performance. Here, we applied Canny Edge Detection and find it can improve the performance a lot.
Normal Match
Normal Match
Canny Detection
Canny Detection
Results
lady.tif
G: [8,51]
R: [11,111]
melons.tif
G: [9,81]
R: [12,179]
onion_church.tif
G: [27,50]
R: [37,108]
self_portrait.tif
G: [29,78]
R: [37,175]
three_generation.tif
G: [14,50]
R: [12,110]
train.tif
G: [6,42]
R: [32,85]
village.tif
G: [13,65]
R: [22,137]
workshop.tif
G: [-1,53]
R: [-12,105]
G: [24,48]
R: [-3,-221]
G: [23,49]
R: [41,106]
G: [13,65]
R: [22,137]
G: [12,64]
R: [-135,0]