CS194 Project 1
Colorizing the Prokudin-Gorskii photo collection
Xuxin Cheng
Goal
Provided with a 3-channel image, produce a color image as output. In order to achieve this goal, we need to extract the three color channel images, place them on top of each other, and align them so that they form a single RGB color image.
Methods
Small images
For small images, we implement a simple version in which we exhaustively search over a window of possible displacements, score each one using some image matching metric, and take the displacement with the best score. There are 2 simple methods to score each displacement.
- SSD(Sum of Squared Differences): Sum of squared difference of two flattened image vectors. We want the displacement corresponding to minimum score.
- NCC(Normalized Cross-Correlation): The dot product between two normalized image vectors. We want the displacement corresponding to maximum score.
In this project I use SSD as my scoring metric since the two method are similar in terms of performance. And I first cropped 10% of the edges of the images before aligning.
Large images
For high resolution images, simply applying a exhaustive search will be too much computationally expensive and time costing. I implemented a faster search procedure called an image pyramid. An image pyramid represents the image at multiple scales and the processing is done sequentially starting from the coarsest scale (smallest image) and going down the pyramid, utilizing the information from last level.
I used 16, 8, 4, 2, until 1 as the scales of the pyramid. The coarsest displacement search range is [-16,16]. In higher resolution images, it is [-4, 4] rather than [-2, 2] to get higher accuracy without much loss in time.
For emir.jpg, the output is very weird. It seems that the red channel was incorrectly moved too much. One possible reason is that the cloth is mostly blue, so the value in blue channel is big while small in other two channels. As a result the correct alignment may has high scores. After implementing Canny Edge Detection using opencv library(threshold1=50, threshold2=150), we get the correct alignment.
Results
Small images
| Image name | Original Image | Aligned Image |
|---|---|---|
| Cathedral Green (5, 2) Red (12, 3) |  |  |
| Monastery Green (-3, 2) Red (3, 2) |  |  |
| Tobolsk Green (3, 3) Red (6, 3) |  |  |
Large images
| Image name | Original Image | Aligned Image |
|---|---|---|
| Workshop Green (52, 0) Red (105, 12) |  |  |
| Emir Green (49, 24) Red (95, -249) |  |  |
| Emir(edge detection) Green (49, 23) Red (107, 40) | |  |
| Three_generations Green (52, 14) Red (112, 11) |  |  |
| Melons Green (81, 10) Red (178, 13) |  |  |
| Onion_church Green (51, 26) Red(108, 36) |  |  |
| Train Green (42, 5) Red(87, 32) |  |  |
| Village Green (64, 12) Red(137, 22) |  |  |
| Harvesters Green (59, 16) Red (124, 13) |  |  |
| Self_portrait Green (78, 29) Red (176, 37) |  |  |
| Icon Green (41, 17) Red (89, 23) |  |  |
| lady Green (51, 9) Red (112, 11) |  |  |
Other photos from the collection
| Image name | Original Image | Aligned Image |
|---|---|---|
| House Green (26, 9) Red (63, 10) |  |  |
| Monument Green (22, 21) Red (60, 29) |  |  |