This project entailed colorizing the Produkin-Gorskii photo collection. Produkin-Gorskii was extremely ahead of his time and he knew that color photos were the wave of the future. He took each photo 3 times in sequence and covered it with red, green, and blue plates. We used the black and white negatives of the photos Produkin-Gorskii took and used image processing techniques to extract the 3 photos, align them, and stack them on top of each other. The hardest part of this project was figuring out the optimal alignment of the photos while managing speed and efficiency. This was a great project to intro us to image processing techniques.
For smaller images, I ran the algorithm of minimizing sum of squared differences. I used blue as the bottom image, and aligned the red and green image to the blue image. I cut 8% on each side of the photo to ignore the randomness of the borders and iterated through a window of [-15, 15] displacement in both the x and y direction. I used the sum of the square difference in my align function to minimize the pixel differences and determine how much to shift up/down and left/right the green and red images to best align with the blue image. After shifting the green and red images, I stacked these three images on top of each other to return the optimally colored picture. Below are the results which include monastery, nativity, settlers, and cathedral.
R [2, 3] G [1, -3]
R [1, 7] G [1, 3]
R [-1, 14] G [0, 7]
R [-1, 9] G [-1, 2]
For larger tif images, I used the image pyramid technique to scale the images down to a practical size and then ran the SSD algorithm as described above when aligning the red and green image. For these images, I again cropped off 8% of the image on all sides to ignore randomness of the borders and ran the image pyrmaid scaling process. I iteratively scaled the image down (1/2) until it reached a size where the height or the width is less than 100 pixels. I then ran the SSD algorithm in the [-15, 15] displacement in both the x and y direction on these pixels to determine how much to shift left/right and up/down these scaled down red and green images. The scaled down red/green images were compared to the scaled down blue image when finding the optimal displacements. Then when I was going back up the pyramid, I made sure to update the optimal displacement by testing the displacement +/- 5 units. At each step up the pyramid we also multiply the optimal displacement by a factor of 2 to adjust for the bigger images. This approach was way faster than doing the naive approach described above since we're not testing every possible combination, but rather iteratively testing our displacements through a smaller window.
R [17, 101] G [9, 45]
R [9, 123] G [11, 58]
R [22, 89] G [16, 41]
R [-16, 106] G [-5, 47]
R [-2, 172] G [-1, 76]
R [8, 111] G [6, 52]
R [5, 113] G [7, 55]
R [-15, 135] G [-7, 62]
R [3, 90] G [-2, 42]
Below is the algorithm run on new images found from last semester. This includes melons, onion_church and bridge tif images!
R [9, 178] G [4, 81]
R [35, 108] G [23, 52]
R [4, 68] G [-7, 12]
The challenges with these images where the ones that had multiple color changes that were concentrated in a tight area. For example this included both the emir file and the self-portrait file, and that caused the alignment to be a little bit off as well as some blurriness.