I enjoyed writing the code for this project a lot. At first it seems complicated how we're going to align the channels. As I refer back to lectures, I realize that each pixel is just a number and therefore we just have to manipulate these numbers. Then it feels much better to get my hands on the project.
The first challenge was to understand the dimensions of the image array. I didn't know it was a 2D matrix so I spent a lot of time wondering why a single np.sum() will error out
The second challenge is to crop the image. I wasn't careful enough about reading the project spec and I missed the part where we should crop the image. I kept getting a weird fuzzy output no matter what I tried. Later on I realized it's because the borders are very noisy and impacted the result.
Later on, I struggled with implementing the pyramid. I discovered that I could use a built-in function to shrink the image. That saved my life. Meanwhile, I didn't know when to stop the recursion. I went to Office Hours and got some help on that. Finally, I had a hard time figuring out what to do with the offsets from the last recursive call. I realized that we needed to multiply them by 2 and that was bugging me for a long time.
Overall, it was a fun project. I did have stressful times where I didn't know what to do. I'm happy that I re-read the project spec or went to office hours to seek for help.
I split the tricolor plane into the components and then aligned red and green to the blue channel. I cropped out the channels so that only the middle 40% is used to compute offsets. I used a simple for loop to search for the best offset within a 15x15 window. I used the SSD method where L2 Loss is the metric. The best offset minimizes L2 loss. As I mentioned before, I didn't crop the image originally so the outputs were messy
In order to run the program on larger images whose displacement can be very big, I implemented a pyramid speedup optimization. I called the single-layer ssd function recursively (usually 5 times for big images and 2 times for smaller ones). On each level, the image is rescaled by half and I'd look for the best offset over a 15x15 window with the lowest SSD. On the next level (with an image twice as big as the previous level), I'd scale the estimates by two and search over a 15x15 window over the new image. Repeat the process when we go back to the image of the original size. Return the final estimate.
These are the outputs of the required images
The offsets are in the order of X, Y. (horizontal, vertical).
Channel G: 2, 34
Channel R: 5, 98
Channel G: 2, 5
Channel R: 3, 12
Channel G: 24, 48
Channel R: -197, 235
This one didn't work quite well because the exposures of the three channels are different. The whitest white and the darkest black are not consistent across the channels
Channel G: 17, 59
Channel R: 14, 123
Channel G: 18, 41
Channel R: 23, 90
Channel G: 8, 51
Channel R: 11, 111
Channel G: 9, 81
Channel R: 12, 179
Channel G: 2, -3
Channel R: 2, 3
Channel G: 27, 50
Channel R: 37, 108
Channel G: 29, 78
Channel R: 37, 175
Channel G: 14, 50
Channel R: 12, 110
Channel G: 3, 3
Channel R: 3, 7
Channel G: 6, 42
Channel R: 32, 85
Channel G: -1, 53
Channel R: -12, 105
The three images I picked were all big tif images before alignment. (around 30MB) To save space, the outputs are all jpg.
Channel G: -5, 49
Channel R: -23, 96
Channel G: 10, -22
Channel R: 12, -33
Channel G: -2, 24
Channel R: -2, 113
Emir.jpg didn't work well because the channels have different brightness. The displacement SSD is really skewed.