Wheeler (Taken with pinhole camera)
Sharpened (sigma: 10, alpha: 0.6)
The main idea of unsharp masking is that image blurring takes away details from an image, but if we add those details back to the image scaled by a certain factor, the details would be enchanced and we get a "sharper" image. The unsharp mask filter is therefore defined as below:
In my implementation, I used a standard Gaussian filter with sigma set to 10 for my smoothing filter. And I obtained the details by subtracting the smoothed image from the original image. Lastly, I add back the details by scaled by a factor of alpha to the original image.
The some results of using this technique to sharpen images:
This part of the assignment is to use the method described in the SIGGRAPH 2006 paper by Oliva, Torralba, and Schyns, to create hybrid images that appear different to human eyes at different distances. The blending method is based on the fact that high frequency tends to dominate perception when it is available, but can't be seen at a distance. Therefore only the low frequency part of the signal can be seen from a distance. To create such hybrid images, I low-pass filter an image using a Gaussian filter to obtain the low frequency image, then high-pass filter another image using the impulse filter minus the gaussian filter to obtain the high frequency image. The two images added together results in a hybrid image that is perceived differently depending on distance.
The following hybrid is not so satisfactory. Up close, the dog seems to have the man's thick eyebrows, and at a distance, the man still seems to have the dog's wrinkles and folds. This is probably because the man's eyebrows are very thick, but don't aline with any features on the dog's face. And the wrinkles and folds of the dog are too many and too sharp as details to be ignored even from a distance.
From the 2D Fourier transforms, we can see that after applying a low-pass filter to the smile image, there are less sharp edges. After applying a high-pass filter to the frown image, it is clear that while the edges remain, most of the other pixels are lost. In other words, only the details are kept after going through the high-pass filter.
In this part, we implement Gaussian and Laplacian stacks and use them to analyze images that contain structure in multiple resolution, including the hybrid images created in the last part. To create a Gaussian stack, I start with the original image as the first level and apply the Gaussian filter to the current level image to obtain the next level, until I reach the desired level. To create a Laplacian stack, each level is obtained by subtracting the next level in the Gaussian stack from the corresponding level of the Gaussian stack. The last level of the Laplacian stack is set to be the same as the last level of the Gaussian stack to preserve the lowest frequencies. To verify the correctness of the stacks, I check that if I add up all the levels in the Laplacian stack, I get back the original image. Observing the results, we can see that with each level deeper stacks, we see less details of the image.
In this part of the project, we try to blend two images into one by using multi-resolution blending. The idea is that using the gaussian and laplacian stacks we implemented in the previous parts, we compute a gentle seam between the two images seperately at each band of image frequencies, resulting in a much smoother seam.
In this part of the project, we explore gradient-domain processing, specifically how it can be used to fuse and blend images together. The idea behind gradient-domain processing is to create an image by solving for specified pixel intensities and gradients. The specific method we focus on is called Poisson Blending, which makes use of the fact that people usually care more about gradients in an image than the overall intensity. So we can set up the problem as finding values for the target pixels that maximally preserve the gradient of the source region without changing any of the background pixels. Then we can use Ordinary Least Squares to solve the problem.
As a warmup, we implement gradient-domain processing to reconstruct a given image. The problem can be described with 3 objectives (v is the target image, s is the source image):
To use poisson blending to seamlessly blend together two images, we follow these steps:
Using poisson blending, we can hardly notice any seam at all in the blended image. But this is largely due to the fact that the backgrounds of both images are smooth. The color of the blended region was not changed much either, because the background colors of the two images were also similar. Therefore, the blended result looks very natural. On the other hand, it is clear that if we naively blend the images together by copying the source image region to the target region, we will see very noticeable seams, even if the backgrounds of the two images are similar.
Because the background of the targer region is darker than the source region, the outcome blended region becomes dark and unnatural.
Mixed gradient blending has the same steps as Poisson blending, but use the gradient in source or target with the larger magnitude as the guide, rather than the source gradient:
Here "d_ij" is the value of the gradient from the source or the target image with larger magnitude, i.e. if abs(s_i-s_j) > abs(t_i-t_j), then d_ij = s_i-s_j; else d_ij = t_i-t_j. Show at least one result of blending using mixed gradients. One possibility is to blend a picture of writing on a plain background onto another image.
In this example, clearly, the most seamless blending is from poisson blending. However, multi-resolution blending preserves the original color of the rose the best. Mixed-gradient blending does not work so well in this case because the background is showing too much.
In this example, although multi-resolution blending and poisson blending gave similar results, it appears that multi-resolution blending is most natural. If you look closely, in the result of the poisson blending, the colors of Hermione's eyes are different. Interestingly, mixed gradient blending produced a result where most of Hermione facial features were preserved, but we could still see some traces of Ron on her face.
Generally, when it is important to preserve the color of the source image, or acheive natural colors in the blended area, multi-resolution blending would be a better choice. But because of the potential color difference, multi-resolution blending may result in a gradual but noticeable seam. Poisson blending produces less noticeable seams and thus would be a good choice if color and intensity doesn't matter that much. If we want to keep prominent features on both the source image and the target image, mixed gradient blending is a better choice. However, no matter what blending method we choose, it is important to keep the background texture similar to avoid blurriness or weird colors around the blended region.