Overview of the Project 1¶
1. Overview of Project 1¶
The goal is to take the digitized Prokudin-Gorskii glass plate images and automatically produce a color image with as few visual artifacts as possible.
2. Approach¶
- Use Blue channel as reference, align green and red channels to the blue channel.
- For small images(<500$\times$500 pixels, e.g., jpg files), exhaustively search over a window of possible displacements [-20,20] pixels for green and red channel respectively, and score each alignment with normalized cross-correlation within the overlapping area. Find the max score as well as its correspondent displacement.
- For large images(>500$\times$500 pixels, e.g., tif files), rescale each channel. For each level, down-scale each channel by a factor of 2. For typical tif files in this Prokudin-Gorskii dataset which is ~3000$\times$3000 pixels, we need to down scale 4 times (i.e., 4 levels, result in a $2^4$ smaller figure). Sarting from the top level (the smallest images), align three channels with the first method and find the displacement (dx1,dy1). Then go to next level where we search [-3,+3] around (2dx, 2dy) to find the best alignment, where we get (dx2, dy2). And the rest can be done in the same manner until we get the displacement in the original level.
- Use the displacement to realign three channels and stack them together. Crop out the region where three channels are not overlapping. Show and save the image.
- Auto-cropping is also done. It is shown in section 4. ## 3. Results
self_portrait.tif¶
Original size: 3251 3810
Wall time: 20.3 s
| Level\Displacement | Green (x,y) | Red (x,y) |
|---|---|---|
| Level 4 | (0, -5) | (0, -11) |
| Level 3 | (0, -10) | (0, -22) |
| Level 2 | (0, -19) | (-1, -45) |
| Level 1 | (0, -38) | (-1, -90) |
| Level 0 (Original scale) | (-1, -77) | (-2, -180) |
tobolsk.jpg¶
Original size: 341 396
Wall time: 2.22 s
| Level\Displacement | Green (x,y) | Red (x,y) |
|---|---|---|
| Level 0 (Original scale) | (2, -3) | (3, -6) |
three_generations.tif¶
Original size: 3209 3714
Wall time: 19.9 s
| Level\Displacement | Green (x,y) | Red (x,y) |
|---|---|---|
| Level 4 | (0, -3) | (0, -7) |
| Level 3 | (1, -7) | (1, -14) |
| Level 2 | (1, -13) | (2, -29) |
| Level 1 | (2, -26) | (4, -57) |
| Level 0 (Original scale) | (5, -52) | (7, -114) |
melons.tif¶
Original size: 3241 3770
Wall time: 19.3 s
| Level\Displacement | Green (x,y) | Red (x,y) |
|---|---|---|
| Level 4 | (0, -5) | (0, -11) |
| Level 3 | (0, -10) | (1, -23) |
| Level 2 | (1, -21) | (2, -45) |
| Level 1 | (2, -42) | (4, -90) |
| Level 0 (Original scale) | (3, -83) | (8, -180) |
icon.tif¶
Original size: 3244 3741
Wall time: 23.6 s
| Level\Displacement | Green (x,y) | Red (x,y) |
|---|---|---|
| Level 4 | (1, -3) | (1, -6) |
| Level 3 | (2, -5) | (3, -11) |
| Level 2 | (4, -11) | (5, -23) |
| Level 1 | (8, -21) | (11, -45) |
| Level 0 (Original scale) | (16, -42) | (22, -91) |
emir.tif¶
Original size: 3209 3702
Wall time: 19.7 s
| Level\Displacement | Green (x,y) | Red (x,y) |
|---|---|---|
| Level 4 | (0, -2) | (1, -7) |
| Level 3 | (1, -5) | (2, -15) |
| Level 2 | (2, -10) | (4, -29) |
| Level 1 | (4, -20) | (8, -58) |
| Level 0 (Original scale) | (8, -41) | (17, -117) |
onion_church.tif¶
Original size: 3215 3781
Wall time: 27.5 s
| Level\Displacement | Green (x,y) | Red (x,y) |
|---|---|---|
| Level 4 | (0, -5) | (2, -7) |
| Level 3 | (2, -7) | (4, -14) |
| Level 2 | (6, -13) | (9, -27) |
| Level 1 | (11, -26) | (17, -54) |
| Level 0 (Original scale) | (22, -53) | (35, -108) |
lady.tif¶
Original size: 3212 3761
Wall time: 21.6 s
| Level\Displacement | Green (x,y) | Red (x,y) |
|---|---|---|
| Level 4 | (0, -5) | (-1, -8) |
| Level 3 | (-1, -11) | (-2, -16) |
| Level 2 | (-2, -21) | (-4, -31) |
| Level 1 | (-4, -43) | (-8, -63) |
| Level 0 (Original scale) | (-7, -85) | (-16, -125) |
train.tif¶
Original size: 3238 3741
Wall time: 21.2 s
| Level\Displacement | Green (x,y) | Red (x,y) |
|---|---|---|
| Level 4 | (0, -3) | (0, -9) |
| Level 3 | (0, -5) | (0, -18) |
| Level 2 | (-1, -10) | (0, -37) |
| Level 1 | (-1, -21) | (0, -73) |
| Level 0 (Original scale) | (-2, -42) | (-1, -146) |
workshop.tif¶
Original size: 3209 3741
Wall time: 19.4 s
| Level\Displacement | Green (x,y) | Red (x,y) |
|---|---|---|
| Level 4 | (0, -4) | (-1, -6) |
| Level 3 | (-1, -7) | (-2, -13) |
| Level 2 | (-1, -14) | (-4, -26) |
| Level 1 | (-2, -27) | (-7, -51) |
| Level 0 (Original scale) | (-4, -54) | (-14, -102) |
village.tif¶
Original size: 3270 3819
Wall time: 21.4 s
| Level\Displacement | Green (x,y) | Red (x,y) |
|---|---|---|
| Level 4 | (0, -10) | (-1, -10) |
| Level 3 | (-1, -19) | (-2, -21) |
| Level 2 | (-2, -38) | (-4, -42) |
| Level 1 | (-3, -77) | (-7, -83) |
| Level 0 (Original scale) | (-7, -154) | (-14, -166) |
monastery.jpg¶
Original size: 341 391
Wall time: 4.07 s
| Level\Displacement | Green (x,y) | Red (x,y) |
|---|---|---|
| Level 0 (Original scale) | (0, 6) | (1, -9) |
harvesters.tif¶
Original size: 3218 3683
Wall time: 22.2 s
| Level\Displacement | Green (x,y) | Red (x,y) |
|---|---|---|
| Level 4 | (0, -8) | (0, -10) |
| Level 3 | (0, -16) | (0, -20) |
| Level 2 | (-1, -32) | (0, -39) |
| Level 1 | (-1, -64) | (0, -78) |
| Level 0 (Original scale) | (-3, -130) | (0, -156) |
cathedral.jpg¶
Original size: 341 390
Wall time: 6.82 s
| Level\Displacement | Green (x,y) | Red (x,y) |
|---|---|---|
| Level 0 (Original scale) | (-1, -12) | (-1, -12) |
4. Bells & Whistles: Autocropping¶
The edge was autofound to be (left,right,top,bottom)=(178,3558,0,3054).
The logic is find out all the blank lines (criterion: more than 90% of the pixels in this line are with values smaller than thresh ~0.1). Then find out the max indexes on the left side and top side, the min indexes on the right side and bottom side of those blank lines correspondently. '... side' is defined by a range of indexes. After finding out those indexes of edges, crop the image.
It works well.
5. Discussion¶
There are several figures with less than ideal alignment. The possible reason is that there are black edges in each channel, and normalized cross-correlation is dominently high when the black edges are aligned between each channel, instead of aligning the real content of figure. Using Sum of Squared Differences (SSD) might be better in this content.