The goal of this project is to colorize the Prokudin-Gorskii images, digitized by the LoC. Given a grayscale image that contains the B, R, G color channels, stacked in that order, we want to output an aligned RGB image, and the lateral translations that align the R and G color channels to the B channel.
The color channels are aligned using an intensity-based image registration algorithm. Images are aligned by splitting the input image evenly along its height into the B, R, G channels. For each possible offset in some given range, the offset is scored by a L2/ SSD based metric. The offset with the minimum score is optimal because it minimizes the difference between pixel intensities. To handle larger images, this brute force offset search is applied to each level of an image pyramid, beginning at the coarsest level. Previously determined offsets are used to inform the next level of search in the pyramid. The SSD metric is unable to handle cases where the brightness between color channels is different.
Artifacts and defects in the color channels images may adversely affect the intensity-based SSD metric. Low pass filtering the images makes alignment more robust to these artifacts; this is implemented in the code using a Gaussian filter. The SSD metric also does not account for differences between color channel intensities, but this issue can be resolved by instead aligning the binary edge images for each color channel, as this accounts for relative brightness across color channels. Using the edge technique still results in some misaligned images (see: melons, self-portrait, textiles). This could likely be fixed by tuning the sensitivity of the edge detection, which I have not done. :) Finally, 15% of the edge pixels are cropped to remove borders.
Below are the results from the naïve and extra credit implementation. The listed translation values are for the extra credit implementation.
| File | Naïve | Bells & Whistles |
|---|---|---|
| cathedral.jpg G: [5 2], R: [12 3] |
 |
 |
| emir.tif G: [49 23], R: [107 40] |
 |
 |
| harvesters.tif G: [60 18], R: [123 14] |
 |
 |
| icon.tif G: [38 16], R: [90 22] |
 |
 |
| lady.tif G: [56 10], R: [120 13] |
 |
 |
| melons.tif G: [81 10], R: [177 13] |
 |
 |
| monastery.jpg G: [-3 2] R: [3 2] |
 |
 |
| onion_church.tif G: [52 24], R: [107 34] |
 |
 |
| self_portrait.tif G: [78 29], R: [175 37] |
 |
 |
| three_generations.tif G: [57 16], R: [115 10] |
 |
 |
| tobolsk.jpg G:[3 3], R: [7 3] |
 |
 |
| train.tif G: [40 8], R: [85 34] |
 |
 |
| village.tif G: [65 11], R: [138 22] |
 |
 |
| workshop.tif G: [53 -1], R: [105 -12] |
 |
 |
| bashkir.tif G: [59 -6], R: [130 -21] |
 |
 |
| estate.tif G: [72 13], R: [146 23] |
 |
 |
| icon2.tif G: [49 14], R: [113 19] |
 |
 |