This project involved taking greyscale pictures taken by Prokudin-Gorskii and colorizing them. Each image has 3 associated pictures taken using a red, green, and blue filter. Combining these three channels would produce a fully colorized photo. However, the images taken were not all exactly from the same location. Thus, the challenge is to align the pictures and then stack the pixel values which would in turn produce the color photo. To align the images, I used the sum of squared distances metric, SSD. SSD is sum(sum((image1-image2)^2)). By minimizing this value between two images you can find the best alignment. Choosing one color channel, you could align the other two color channels to the base and then stack all 3. I chose to use the green color channel as my base because blue did not work for emir.tif but green did. To find the best alignment, I cropped each photo to exclude the border and then performed an exhasutive search in a +- 15 pixel range for both the x and y directions. However, when increasing the size from jpg to tif, the window needed to be significantly expanded making the exhaustive search far too expensive. To fix this, I implemented the pyramid search algorithm. This algorithm would rescale the image to half of its size until the dimensions on both x and y were below 500 pixels. From here I could perform the 15 pixel window search and return double the calculated offset. It is a recursive algorithm so after the base case is reached, I align the pictures at the next level using double of the calculated offset. I repeat this until the original call where you return the running total of the offset calculated so far. The trick in this algorithm is that now you can do the 15 pixel window search at every level making it far faster and efficient.
One problem I encountered was that when using the blue channel as the base channel, emir.tif did not align properly. By switching it to green, it worked perfectly. Also, I noticed that after the first layer of the pyramid the new offset for alignment was never greater than 5. So to improve runtime I change the window from 15 pixels to 5 pixels after the base case has returned. This made the algorithm faster without sacrificing any accuracy. In addition, I cropped each photo by 10% on each side before any computation so as to avoid the border throwing off the SSD calculation.
Blue Filter X Offset: -2
Blue Filter Y Offset: -5
Red Filter X Offset: 1
Red Filter Y Offset: 7
Blue Filter X Offset: -2
Blue Filter Y Offset: -3
Red Filter X Offset: 1
Red Filter Y Offset: 6
Blue Filter X Offset: -3
Blue Filter Y Offset: -3
Red Filter X Offset: 1
Red Filter Y Offset: 4
Blue Filter X Offset: -4
Blue Filter Y Offset: -25
Red Filter X Offset: -8
Red Filter Y Offset: 33
Blue Filter X Offset: -24
Blue Filter Y Offset: -49
Red Filter X Offset: 17
Red Filter Y Offset: 57
Blue Filter X Offset: -16
Blue Filter Y Offset: -59
Red Filter X Offset: -3
Red Filter Y Offset: 65
Blue Filter X Offset: -17
Blue Filter Y Offset: -41
Red Filter X Offset: 5
Red Filter Y Offset: 48
Blue Filter X Offset: -9
Blue Filter Y Offset: -51
Red Filter X Offset: 3
Red Filter Y Offset: 61
Blue Filter X Offset: -10
Blue Filter Y Offset: -81
Red Filter X Offset: 3
Red Filter Y Offset: 96
Blue Filter X Offset: -26
Blue Filter Y Offset: -51
Red Filter X Offset: 10
Red Filter Y Offset: 57
Blue Filter X Offset: -29
Blue Filter Y Offset: -78
Red Filter X Offset: 8
Red Filter Y Offset: 98
Blue Filter X Offset: -14
Blue Filter Y Offset: -53
Red Filter X Offset: -3
Red Filter Y Offset: 58
Blue Filter X Offset: -5
Blue Filter Y Offset: -42
Red Filter X Offset: 27
Red Filter Y Offset: 43
Blue Filter X Offset: -14
Blue Filter Y Offset: -46
Red Filter X Offset: -5
Red Filter Y Offset: 60
Blue Filter X Offset: 22
Blue Filter Y Offset: -30
Red Filter X Offset: -23
Red Filter Y Offset: 100
Blue Filter X Offset: -25
Blue Filter Y Offset: -69
Red Filter X Offset: 10
Red Filter Y Offset: 78