The goal of the project was to colorize the Prokudin-Gorskii photo collection, a series of black and white photos taken by the namesake photographer in the 20th century. The black and white photos contained 3 images of the same scene, each taken with a different physical filter: red, green and blue. However, these pictures don't align perfectly due to them being taken at different times, so one must first align the grayscale images and then stack them in RGB order to produce the colorized image.
To colorize the images, I first divided the images into their respective color channels, and then cropped 10% of the images on all four sides to remove the extra whitespace and black edges from the glass plate images. For the smaller jpg images, it was possible to just search through a displacement window of (-15,15) on the four sides of the image and find the shift with the highest normalized-cross-correlation (NCC), a measure of similarity between two vectors. The window of (-15,15) was found through experimenting and NCC was chosen because it was the first similarity measure I tried, and it worked pretty well. For the larger tif files, it was unreasonable to search through windows of hundreds of pixels long/wide so a faster algorithm was needed. I used a recursive image pyramid to solve this issue. This scales down the image by 0.5x until it's a reasonable size (a maximum size of 200x200 pixels was my threshold). Then, we can search for the best displacement in this coarse image. I settled on the (-15,15) window again after experimentation. AFter that, the algorithm recursively searches a very narrow window of (-2,2) of a finer image using the best displacement from the previous coarser image, until it aligns the full-sized image. Lastly, I cropped 5% from the edges of the pictures to crop out any noise from the alignment process.
| Image | Offset | |
|---|---|---|
| cathedral.jpg | |
Green Displacement: [5, 2] Red Displacement: [11, 3] |
| emir.tif* | |
Green Displacement: [49, 24] Red Displacement: [95, -249] |
| harvesters.tif | |
Green Displacement: [59, 16] Red Displacement: [123, 13] |
| icon.tif | |
Green Displacement: [41, 17] Red Displacement: [89, 23] |
| lady.tif | |
Green Displacement: [51, 9] Red Displacement: [112, 11] |
| melons.tif | |
Green Displacement: [81, 10] Red Displacement: [178, 13] |
| monastery.jpg | |
Green Displacement: [-3, 2] Red Displacement: [3, 2] |
| onion_church.tif | |
Green Displacement: [51, 26] Red Displacement: [108, 36] |
| self_portrait.tif | |
Green Displacement: [78, 29] Red Displacement: [176, 37] |
| three_generations.tif | |
Green Displacement: [53, 14] Red Displacement: [112, 11] |
| tobolsk.jpg | |
Green Displacement: [3, 3] Red Displacement: [6, 3] |
| train.tif | |
Green Displacement: [42, 5] Red Displacement: [87, 32] |
| village.tif | |
Green Displacement: [64, 12] Red Displacement: [137, 22] |
| workshop.tif | |
Green Displacement: [53, 0] Red Displacement: [105, -12] |
The aligning algorithm did not work as well on the Emir image because the different color filtered images were taken at different brightnesses. The red channel is visibly unaligned. This was fixed by running a sobel filter, an edge detecting filter through the different color channels first, before attempting to align them. The improved results can be seen below:
| Old Picture | New Picture | Old Red Offset | New Red Offset |
|---|---|---|---|
|
 |
Red Displacement: [95, -249] | Red Displacement: [107, 40] |
I also chose 3 other images from the Prokudin-Gorskii to colorize as well. The results of the colorization of the .tif files can be seen below.
| Image | Offset | |
|---|---|---|
| bird.tif |  |
Green Displacement [28, -2] Red Displacement: [67, 0] |
| dog.tif |  |
Green Displacement: [61, 7] Red Displacement: [138, 7] |
| fish.tif |  |
Green Displacement: [47, 21] Red Displacement: [109, 34] |