Sergei Mikhailovich Prokudin-Gorskii took photos all over Russia with red, green, and blue filters. At the time, there was no way to print color photographs, yet he awaited the day when his collection of photos would be shown in full color. The glass plate negatives were bought by the Library of Congress, and now the negatives and restored images are available digitally. The goal of this project is to take the glass plate negatives and automatically produce a color image.
To align the three color channels, I performed a search over a window of displacements, finding the displacement that yielded the most similarity between channels. To do this, I used the L2 norm, or sum of squared differences (SSD) as a scoring metric. In the search over displacements, I simply calculated the score at each displacement and used the displacement with the lowest score (most similar).
For large images, it would take too long to search over a significant enough range of displacements. Thus, the images need to be processed with an image pyramid. The image pyramid represents the given image in multiple scales, and the image is processed and displacement updated at each step. The processing starts at the coarsest scale, the top of the pyramid, and searches over the largest displacement there. Then, for all subsequent steps of the pyramid, we simply need to fine tune the large displacement produced in the first step.
For bells and whistles, I implemented edge detection to try to improve the aligning procedure. I used the Scharr transform from skimage.filters to detect edges, then fed the detected edge images into the previous aligning procedure. For most images, this had little to no effect. However, it yielded a noticeably better result for emir.tif, and a noticeably worse result for onion_church.tif and napoleon.tif. The result for sunset.tif is very different in terms of the offset, but the image does not necessarily look better or worse, just different.
| File, offsets |
Aligned Image |
New Offsets (Edge Detection) |
Bells and Whistles |
|
cathedral.jpg
Offsets:
Green: (2, 5)
Red: (3, 12)
|
 |
Green: (2, 5)
Red: (3, 12)
|
 |
|
monastery.jpg
Offsets:
Green: (2, -3)
Red: (2, 3)
|
 |
Green: (2, -3)
Red: (2, 3)
|
 |
|
tobolsk.jpg
Offsets:
Green: (3, 3)
Red: (3, 6)
|
 |
Green: (2, 3)
Red: (3, 6)
|
 |
|
emir.tif
Offsets:
Green: (24, 49)
Red: (58, 102)
|
 |
Green: (25, 49)
Red: (41, 106)
|
 |
|
harvesters.tif
Offsets:
Green: (18, 59)
Red: (15, 124)
|
 |
Green: (18, 59)
Red: (13, 123)
|
 |
|
icon.tif
Offsets:
Green: (18, 41)
Red: (23, 89)
|
 |
Green: (18, 41)
Red: (22, 92)
|
 |
|
lady.tif
Offsets:
Green: (10, 54)
Red: (12, 116)
|
 |
Green: (12, 55)
Red: (15, 118)
|
 |
|
melons.tif
Offsets:
Green: (12, 81)
Red: (14, 177)
|
 |
Green: (10, 79)
Red: (13, 177)
|
 |
|
onion_church.tif
Offsets:
Green: (27, 50)
Red: (37, 106)
|
 |
Green: (-49, 51)
Red: (-35, 106)
|
 |
|
self_portrait.tif
Offsets:
Green: (30, 77)
Red: (36, 177)
|
 |
Green: (29, 77)
Red: (36, 176)
|
 |
|
three_generations.tif
Offsets:
Green: (15, 53)
Red: (11, 113)
|
 |
Green: (14, 54)
Red: (10, 111)
|
 |
|
train.tif
Offsets:
Green: (7, 43)
Red: (34, 87)
|
 |
Green: (7, 42)
Red: (32, 86)
|
 |
|
village.tif
Offsets:
Green: (14, 65)
Red: (28, 137)
|
 |
Green: (14, 64)
Red: (23, 137)
|
 |
|
workshop.tif
Offsets:
Green: (1, 51)
Red: (-11, 104)
|
 |
Green: (0, 53)
Red: (-13, 105)
|
 |
|
stream.tif
Offsets:
Green: (1, 40)
Red: (9, 96)
|
 |
Green: (-2, 42)
Red: (10, 97)
|
 |
|
sunset.tif
Offsets:
Green: (7, 42)
Red: (-77, -47)
|
 |
Green: (-199, 39)
Red: (-203, -127) |
 |
|
napoleon.tif
Offsets:
Green: (7, 63)
Red: (0, 132)
|
 |
Green: (3, 5)
Red: (0, 84)
|
 |