Colorizing the Prokudin-Gorskii
Photo Collection
Nadia Hyder
OVERVIEW
In this project, I produced color images from the Prokudin-Gorskii glass plate images by dividing the glass plate
images into their three color channel images: B, G, and R. These channels are
then aligned and stitched together to create a single RGB color image with as
few visual artifacts as possible. Here is an example of what my code
accomplishes:
blue channel (input) |
green channel (input) |
red channel (input) |
colored image (output) |
METHODOLOGY
INITIAL APPROACH
My initial logic for aligning these 3 channels was to:
1.
Find the best displacement
between the green and blue channel over a [-15,15] window using some image
matching metric (I used sum of square distances, as the computation was faster
than L2 norm) and align the green
channel relative to the blue channel
a.
the best displacement is defined
as that which gives the minimum possible sum of square distances between pixel
values between the two channels
2.
Find the best displacement
between the red channel and the aligned green channel over a [-15,15] window
and align the red channel relative to the aligned green channel.
However, this exhaustive search only worked well for smaller
images (.jpg files) but was neither efficient nor thorough enough for larger
images.
IMPROVEMENTS
1.
To (somewhat) ignore the image
borders, I manually cropped the images to account for only the center
85% of the image.
2.
To more efficiently and
accurately calculate the appropriate displacement, I implemented an image
pyramid.
I sequentially calculated the appropriate displacement over a [-15, 15] window
over multiple scales (scaled by a factor of 2)– scaling from the coarsest image
to the actual image size, aligning the actual image using the pixel
displacement scaled to the appropriate factor relating the smaller image and
the actual image. This significantly
reduced runtime and produced good alignments, even for the Emir of Bukhara.
3.
(Bells and
Whistles) To align well regardless of
color channels and RGB similarity, I aligned the images using canny edge
detection (reference: https://scikit-image.org/docs/dev/auto_examples/edges/plot_canny.html).
Comparing the edges rather than pixel values allows for more accurate
alignment, and the images matched do not need to have the same brightness
values. Though the difference is just noticeable, alignment using edge
detection worked better than alignment using raw pixel values.
The
difference after applying canny edge detection is just noticeable. Comparing
the appearance of the eyes in these two pictures eyes reveals edge detection
worked better– there is no redness in the eyes in the image on the right, the
eyes are well defined because the image is better aligned.
Emir
of Bukhara using SSD on pixel values |
Emir
of Bukhara using SSD on edges |
OUTCOMES & OFFSETS
|
stitching without
alignment |
alignment using SSD on raw
pixel values |
alignment using SSD on
edges |
monastery.jpg |
|
Green dx: 2, dy:
-2 Red dx: 2, dy: 4 |
Green dx: 2, dy:
-4 Red dx: 4, dy: 2 |
tobolsk.jpg |
|
Green dx: 2, dy:
2 Red dx: 2, dy: 6 |
Green dx: 2, dy:
2 Red dx: 2, dy: 6 |
cathedral.jpg |
|
Green dx:2, dy: 4 Red dx: 2, dy: 10 |
Green dx:2, dy: 4 Red dx: 2, dy: 10 |
workshop.tif |
|
Green dx: 0, dy:
52 Red dx: -10, dy:
104 |
Green dx: -2, dy:
52 Red dx: -12, dy:
104 |
emir.tif |
|
Green dx: 24, dy:
48 Red dx: 40, dy:
48 |
Green dx: 24, dy:
40 Red dx: 40, dy:
108 |
three_generations.tif |
|
Green dx: 14, dy:
54 Red dx: 12, dy:
112 |
Green dx: 18, dy:
56 Red dx: 12, dy:
116 |
castle.tif |
|
Green dx: 2, dy:
34 Red dx: 2, dy: 98 |
Green dx: 4, dy:
36 Red dx: 4, dy:
100 |
melons.tif |
|
Green dx: 10, dy:
82 Red dx: 14, dy:
178 |
Green dx: 10, dy:
80 Red dx: 12, dy:
176 |
onion_church.tif |
|
Green dx: 26, dy:
52 Red dx: 36, dy:
108 |
Green dx: 28, dy:
52 Red dx: 36, dy:
108 |
train.tif |
|
Green dx: 4, dy:
42 Red dx: 30, dy:
86 |
Green dx: 8, dy:
40 Red dx: 34, dy:
82 |
icon.tif |
|
Green dx: 16, dy:
40 Red dx: 22, dy:
88 |
Green dx: 16, dy:
42 Red dx: 22, dy:
90 |
self_portrait.tif |
|
Green dx: 28, dy:
78 Red dx: 36, dy:
176 |
Green dx: 30, dy:
80 Red dx: 38, dy:
178 |
harvesters.tif |
|
Green dx: 16, dy:
60 Red dx: 14, dy:
124 |
Green dx: 16, dy:
60 Red dx: 14, dy:
124 |
lady.tif |
|
Green dx: 8, dy:
48 Red dx: 12, dy:
110 |
Green dx: 8, dy:
56 Red dx: 12, dy:
118 |
ADDITIONAL IMAGES
Aligned using canny edge detection
Loparev cardboard factory Green dx: 2, dy:24 Red dx: -6, dy: 92 |
Exit from the yard of the Church of Saint George Green dx: 24, dy: 72 Red dx:38, dy: 154 |
Fishing settlement Green dx: -32, dy: 8 Red dx: -70, dy: 30 |
On the island of Capri Green dx: -16, dy: 32 Red dx: -24, dy: 78 |
Detail of the Por-Porog
waterfall Green dx: -10, dy: 34 Red dx: -18, dy: 122 |
-