Using our finite difference filters dx = [1, -1] and dy = [[1, -1]] (column), I convolved our original image with these filters to find the partial derivates in x and y.
Gradient magnitude Calculation:
With both the partial derivatives dx and dy, I first squared each and then summed then together (all element wise). Finally, I took the element wise square root, in order to get the gradient magnitude image.
Lastly, I used a threshold to binarize the magnitude to achieve an edge image (any value above or below the threshold gets clipped to a value).
Dx
Dy
Gradient magnitude
Binarized gradient magnitude
1.2: Derivative of Gaussian (DoG) Filter
I first created a 2D gaussian filter by using cv2.getGaussianKernel() to get 2 1D gaussian kernels, and taking the outer product of these to get a 2D kernel.
Blurred Camerman
Next, I convolved my orignal image with this 2D gaussian kernel to get a blurred image. I then repeated the process in part 1.1 to obtain the below plots.
Magnitude (2 Convolutions)
Binarized magnitude (2 Convolutions)
What's Different?
Compared to part 1.1, our binarized magnitude image has much smoother edges. In addition, it seems as though we've filtered a lot more noise (less random particles). Our edges are smoother, thicker, and more defined using the DoG filter, and edges seem more connected (less breaks in the edges).
Single Convolution:
To avoid having to convolve with the image twice, I instead first convolved my 2D gaussian with dx and dy, and then convolved the resulting filters with my image. This reduces the process to only 1 convolution with original image. As we can see, we achieve the same results as the above that uses 2 convolutions!
Magnitude (1 Convolution)
Binarized magnitude (1 Convolution)
Resulting DoG filters visualized:
Gaussian convolved with dx
Gaussian convolved with dy
2.1: Image "Sharpening"
Using the gaussian blurring technique, we blur and image by convolving it with a gaussian. We can then subtract this blurred version from our original image to obtain only the high frequencies. Finally, we can add this high frequency image back to our original to achieve a sharpened version.
Original Taj
Blurred Taj
Sharpened Taj
Original Corgi
Blurred Corgi
Sharpened Corgi
Original Spiderman
Blurred Spiderman
Sharpened Spiderman
The sharpened images look a fair bit darker than the original images. The sharpened images are not nearly as clear as the original image but still contain more detail than the blurred image.
2.2: Hybrid Images
In order to acheive a "hybrid image", we blended a high frequency version of one image with a low frequency version of another.
The low frequncy image was acheived by convolving with a gaussian. The high frequency was achieved by blurring it with a gaussian to get the low frequencies, and then subtracting this blurred image with the original.
Derek
Nutmeg
Derek Nutmeg Hybrid
Shang
Mulan
Shang Mulan Hybrid
Washington
Lincoln
Washington Lincoln Hybrid
Fourier Plots for Washington-Lincoln:
Washington Fourier
Lincoln Fourier
Low-pass Washington Fourier
High-pass Lincoln Fourier
Washington and Lincon Fourier
Failure Case:
Shangen (friend)
Simu Liu
Shangen Liu Hybrid
Here, I tried to merge an image of my friend Shangen with a picture of simu liu. In the hybrid image, I mainly see Shangen and it is very hard to tell any details from Simu Liu (it just looks like a blurrier image of Shangen).
In this example, the lower frequency image (Shangen) dominates the higher frequency one so I cannot tell the features of Simu.
Bells and Whistles:
I implemented color images to see if it would enhance the hybrid effect for either of the two images (I describe how I was able to accomplish using color in the bells and whistles for part 2.4).
With color, it seems better to use color for the lower frequency image since the higher frequncy image does not seem to really retain that much color after stacking. What this does is it makes it easier to see the lower frequncy image, but the higher frequency image becomes clearer as well so the hybrid effect is overall enhanced.
2.3: Gaussian and Laplacian Stacks
I made a gaussian stack by repeatedly convolving the original image with a gaussian, effectivley repeatedly blurring an image. I created a laplacian stack by taking a difference between consecutive levels of the gaussian stack.
Gaussian Stack for Apple:
Laplacian Stack for Apple:
Gaussian Stack for Orange:
Laplacian Stack for Orange:
The last level of the laplacian stack uses the last level of the Gaussian Stack as it contains the lowest frequencies.
2.4: Multiresolution Blending (a.k.a. the oraple!)
Using the gaussian and lapacian stacks from 2.3, I created laplacian stacks for two images. I also created a mask image and created a laplacian stack for this. Finally, I combined the levels of the lapcian stacks for the mask along with the two images and summed them all together to get a blended image.
Blending of the orange and apple:
Left column is mask applied to laplacian stack for the apple, middle column is the mask applied to laplacian stack for the apple, right column is the sum of the the 2 columns.
The bottom row is the sum of all the prevous rows together to get the final blended image.
Bells and Whistles:
Instead of merging only gray scale images, I implemented color images as well to enhance the blending effect. With color, it is much easier to see a cleaner blend as with gray scale images, it is sometimes hard to discern how the images were merged and the transition might seem sharper. This was achieved by importing images as 3D arrays with 3 channels, and then applying convolutions and operations to all three channels rather than one. Finally, I concatenated the results of the three channels back into one image to produce a colored image.
The coolest thing that I learned about this project is that frequencies are embedded in images. Originally, I only thought of frequencies in terms of signals and waves, but it was so cool being able to visually see the seperation of an images frequencies into layers (as see in the gaussian and lapacian stacks). In addition, seeing the application of blurring to masks applied to blending images (part 2.4) was super interesting to implement and play around with.
