CS194-26 Project 1
Alex Zhang
Overview
This project was about combining black and white images taken of each color channel individually to recreate a color photo. However, since the photos of each channel is taken separately, they may need to be realigned before combining the three into a single color photo. The approach I used to facilitate the alignment initially was to use the normalized cross correlation on the inner 2/3rds of the image with a scan window of [-20,20]. We only use the inner region of the photo due to the noise at the borders of the image (discoloration, distortion, etc). This worked pretty well for the small JPEGs but was not efficient for the larger TIF files. Also, since the TIFs were so large the displacement could easily be outside of that range. Thus, I implemented the image pyramid to operate on the TIFs efficiently as well, that kept halving the image sizes until around 500px.
This method performed well on almost every image (including the original smaller JPGs, providing the same answer as the naive alignment function) except Emir. For him, I used a basic sobel filter on each channel to perform edge detection and also align by edge instead as a bell & whistle feature which actually ended up working really well.
Results on provided images
Note: coordinates are (x,y), with the origin starting at the top left and y growing downwards and x rightwards. This convention may be different from what other students have, since I deliberately flipped it on purpose to make it more intuitive for myself.
cathedral
green-blue offset: (2, 5)
red-blue offset: (3, 12)
emir
This is basically the only the photo that visibly failed to align acceptably. I believe this is due to the vibrantly colored robes that he is wearing. If you look at the original image of each channel, the robe looks completely different in each channel due to how it reflects each wavelength most likely. Since it almost looks black in the bottom most frame, it must reflect very little red light and thus throws off the alignment. The other frames are much brighter so it probably reflects a lot more blue and green light.
green-blue offset: (24, 49)
red-blue offset: (44, 69)
harvesters
green-blue offset: (17, 59)
red-blue offset: (14, 124)
icon
green-blue offset: (17, 41)
red-blue offset: (23, 89)
lady
green-blue offset: (9, 55)
red-blue offset: (12, 117)
monastery
green-blue offset: (2, -3)
red-blue offset: (2, 3)
nativity
green-blue offset: (1, 3)
red-blue offset: (0, 8)
self_portrait
green-blue offset: (29, 78)
red-blue offset: (37, 176)
settlers
green-blue offset: (0, 7)
red-blue offset: (-1, 15)
three_generations
green-blue offset: (14, 52)
red-blue offset: (11, 111)
train
green-blue offset: (6, 43)
red-blue offset: (32, 87)
turkmen
green-blue offset: (21, 56)
red-blue offset: (28, 116)
village
green-blue offset: (12, 65)
red-blue offset: (22, 137)
Results on additional images of my choosing
Bells and Whistles
Using edge detection
Since Emir was the only image out of all of the ones I tested that was visibly misaligned, using edge detection only had significant results on him, so I only showed his improved image below:
On the left is the emir aligned without a sobel filter (same as above), and on the right is emir aligned by using the sobel filtered version of the rgb channels to perform NCC. You can see the sobel filter actually does a really good job of improving the alignment by a lot more since the sobel filter finds the edges of the photos and makes NCC align based off edges instead of RGB similarity. Alignment offsets for the right photo:
green-blue offset: (24, 49)
red-blue offset: (40, 107)
