William Choe Frank CS194-26 Proj1
Images of the Russian Empire:
Colorizing the Prokudin-Gorskii photo collection
Overview
The goal of this assignment was to take the digitized Prokudin-Gorskii
glass plate images and produce a color image. The images can be partitioned
into 2 groups: low resolution .jpg images and high resolution .tif images.
Approach
The main problem with colorizing the glass plates was that they are not
exactly aligned. So in order to colorize these images which is done by
"stacking" the apropriate color channels on top of eachother, we need to
first align each color channel.
Low Resolution Approach
For the lower resolution photos, a brute force method is sufficient. I took
the given pixel shift range of [-15,15], applied that to both axes, and
tested all combinations of the range and axes (x and y) with a metric. The
metric I used was the Sum of Squared Differences distance. For a given displacement
of red or green, I calculated the SSD with respect to the blue channel, and kept track
or which displacement had the lowest SSD distance which in theory gives the displacement that has the most
similar color intensity to the blue channel.
This alone is not enough though. The edges of the plates often don't
have accurate information (due to wear / fade of the glass before it was
digitized, etc.). These edges mess with the algorithm so before I scored each
displacement, I cropped 10% from each edge from the image I was alligning and
the blue channel.
Low Resolution Photos
Displacement Values
cathedral R[12, 3], G[5, 2]
monastery R[3, 2], G[-3, 2]
nativity R[7, 0], G[3, 1]
settlers R[14, -1], G[7,0]
High Resolution Approach
The brute force method mentioned above is no longer sufficient once you get to
larger images because the optimal displacement can be larger than the [-15,15] range,
and it is very suboptimal to just scale that range with the size of our image. Our
search space because very large very fast. To work around this issue, I implemented
a recursive algorithm technique called the image pyramid.
The main idea behind the image pyramid is that suppose we are given an image, im, where length = width = x.
Also suppose the optimal displacement for this image for our equal in size blue channel is [a,b].
If we rescale im by 2 (doubling its size) so now that length = width = 2*x, the displacement [2*a,2*b]
is very close to the optimal displacement.
The implications of this is that we can recursively decrease the size of the image we are trying to align, im,
and the blue channel, b. Once we reach a certain size, I chose 150 pixels,
we can apply the low resolution method on this now small image. Once we find
the best displacement, we propagate those x,y values up the recursive call stack,
and then at the next level, we apply the previous displacement (multiplied by a factor of 2),
then apply the low resoltion method, then propagate the total displacement up the
recursive call stack, and this will eventually created the desired displacement
over a significantly smaller search space.
Note: When I mention "low resolution method" here, the window I used for the image pyramid
was [-4,4] instead of [-15,15]. Also cropping still takes place in this method before computing metrics
and the metric used is still the SSD distance.
High Resolution Photos
Displacenet Values
emir R[57, 17], B[-49, -24]
harvesters R[124, 14], G[60, 17]
icon R[90,23], G[41, 17]
lady R[120, 11], G[52, 9]
self_portrait R[176, 37], G[79, 29]
three_generations R[112, 11], G[53, 14]
train R[87, 32], G[42, 6]
turkmen R[116, 28], G[56, 21]
village R[128, 22], G[64, 12]
Note on emir: I alligned the red and blue channels with green because the image
wouldn't align when aligning with the blue channel.
Additional Imgages
Displacement Values
cross R[28, -2], G[8, 4]
landscape R[30, 3], G[9, 4]
house R[60, 5], G[26, 6]
References
Image Pyramid: https://en.wikipedia.org/wiki/Pyramid
HTML Reference: https://inst.eecs.berkeley.edu/~cs194-26/fa17/hw/proj1/