This project extracts the three color channel images, place them on top of each other, and align them so that they form a single RGB color image.
For the baseline naive implementation, I first crop away the margins that are filled with noises and exhaustively search over a window of possible displacements (of size [-15,15] pixels). Then, I score each one using the Sum of Squared Differences(SSD) image matching metric and take the displacement with the best score. This works well for smaller jpeg images but for larger tif images, this method becomes too computationally expensive. Thus, I implemented the image pyramid search procedure by representing the image at multiple scales (scaled by a factor of 2).
These are the results generated from main.py. Each picture is attached with the information of the horizontal shift of the green channel (g_x), the vertical shift of the green channel (g_y), the horizontal shift of the red channel (r_x) and the vertical shift of the red channel (r_y) in pixels.
Baseline Exhaustive Search
Cathedral.jpg (g_x, g_y) = (5, 2), (r_x, r_y) = (12, 3) | Monastery.jpg (g_x, g_y) = (-3, 2), (r_x, r_y) = (3, 2) |
---|---|
Nativity.jpg (g_x, g_y) = (3, 1), (r_x, r_y) = (8, 0) | Settlers.jpg (g_x, g_y) = (7, 0), (r_x, r_y) = (14, -1) |
---|---|
Pyramid Search
Harvesters.tif (g_x, g_y) = (59, 16), (r_x, r_y) = (123, 13) | Icon.tif (g_x, g_y) = (41, 17), (r_x, r_y) = (89, 23) |
---|---|
Lady.tif (g_x, g_y) = (51, 9), (r_x, r_y) = (111, 12) | Self_portrait.tif (g_x, g_y) = (78, 29), (r_x, r_y) = (176, 37) |
---|---|
Three_generations.tif (g_x, g_y) = (53, 14), (r_x, r_y) = (112, 11) | Train.tif (g_x, g_y) = (42, 5), (r_x, r_y) = (86, 32) |
---|---|
Turkmen.tif (g_x, g_y) = (56, 21), (r_x, r_y) = (116, 28) | Village.tif (g_x, g_y) = (64, 12), (r_x, r_y) = (137, 22) |
---|---|
I used an edge detection technique called Sobel filter to further improve the alignment since Pyramid Search alone is insufficient to align emir.tif well.
Emir.tif
Pyramid Search (g_x, g_y) = (49, 24), (r_x, r_y) = (78, -712) | Sobel (g_x, g_y) = (49, 24), (r_x, r_y) = (105, 41) |
---|---|
Before/After Sobel
Harvesters.tif (g_x, g_y) = (59, 16), (r_x, r_y) = (123, 13) | Harvesters.tif (g_x, g_y) = (56, 11), (r_x, r_y) = (118, 10) |
---|---|
Icon.tif (g_x, g_y) = (41, 17), (r_x, r_y) = (89, 23) | Icon.tif (g_x, g_y) = (39, 16), (r_x, r_y) = (89, 23) |
---|---|
Lady.tif (g_x, g_y) = (51, 9), (r_x, r_y) = (111, 12) | Lady.tif (g_x, g_y) = (57, 9), (r_x, r_y) = (121, 13) |
---|---|
Self_portrait.tif (g_x, g_y) = (78, 29), (r_x, r_y) = (176, 37) | Self_portrait.tif (g_x, g_y) = (74, 25), (r_x, r_y) = (175, 37) |
---|---|
Three_generations.tif (g_x, g_y) = (53, 14), (r_x, r_y) = (112, 11) | Three_generations.tif (g_x, g_y) = (58, 16), (r_x, r_y) = (115, 11) |
---|---|
Train.tif (g_x, g_y) = (42, 5), (r_x, r_y) = (86, 32) | Train.tif (g_x, g_y) = (40, -1), (r_x, r_y) = (85, 28) |
---|---|
Turkmen.tif (g_x, g_y) = (56, 21), (r_x, r_y) = (116, 28) | Turkmen.tif (g_x, g_y) = (58, 21), (r_x, r_y) = (118, 29) |
---|---|
Village.tif (g_x, g_y) = (64, 12), (r_x, r_y) = (137, 22) | Village.tif (g_x, g_y) = (64, 10), (r_x, r_y) = (137, 22) |
---|---|
I then added auto-contrast.
Before/After Sobel + Autocontrast
Cathedral.jpg (g_x, g_y) = (5, 2), (r_x, r_y) = (12, 3) | Cathedral.jpg (g_x, g_y) = (5, 2), (r_x, r_y) = (12, 3) |
---|---|
Monastery.jpg (g_x, g_y) = (-3, 2), (r_x, r_y) = (3, 2) | Monastery.jpg (g_x, g_y) = (-3, 2), (r_x, r_y) = (3, 2) |
---|---|
Nativity.jpg (g_x, g_y) = (3, 1), (r_x, r_y) = (8, 0) | Nativity.jpg (g_x, g_y) = (3, 1), (r_x, r_y) = (8, 0) |
---|---|
Settlers.jpg (g_x, g_y) = (7, 0), (r_x, r_y) = (14, -1) | Settlers.jpg (g_x, g_y) = (7, 0), (r_x, r_y) = (14, -1) |
---|---|
Emir.tif (g_x, g_y) = (49, 24), (r_x, r_y) = (78, -712) | Emir.tif (g_x, g_y) = (49, 24), (r_x, r_y) = (105, 41) |
---|---|
Harvesters.tif (g_x, g_y) = (59, 16), (r_x, r_y) = (123, 13) | Harvesters.tif (g_x, g_y) = (56, 11), (r_x, r_y) = (118, 10) |
---|---|
Icon.tif (g_x, g_y) = (41, 17), (r_x, r_y) = (89, 23) | Icon.tif (g_x, g_y) = (39, 16), (r_x, r_y) = (89, 23) |
---|---|
Lady.tif (g_x, g_y) = (51, 9), (r_x, r_y) = (111, 12) | Lady.tif (g_x, g_y) = (57, 9), (r_x, r_y) = (121, 13) |
---|---|
Self_portrait.tif (g_x, g_y) = (78, 29), (r_x, r_y) = (176, 37) | Self_portrait.tif (g_x, g_y) = (74, 25), (r_x, r_y) = (175, 37) |
---|---|
Three_generations.tif (g_x, g_y) = (53, 14), (r_x, r_y) = (112, 11) | Three_generations.tif (g_x, g_y) = (58, 16), (r_x, r_y) = (115, 11) |
---|---|
Train.tif (g_x, g_y) = (42, 5), (r_x, r_y) = (86, 32) | Train.tif (g_x, g_y) = (40, -1), (r_x, r_y) = (85, 28) |
---|---|
Turkmen.tif (g_x, g_y) = (56, 21), (r_x, r_y) = (116, 28) | Turkmen.tif (g_x, g_y) = (58, 21), (r_x, r_y) = (118, 29) |
---|---|
Village.tif (g_x, g_y) = (64, 12), (r_x, r_y) = (137, 22) | Village.tif (g_x, g_y) = (64, 10), (r_x, r_y) = (137, 22) |
---|---|