CS 194-26 Proj 2: Fun with Filters and Frequencies!

Part 1.1: Finite Difference Operator

To calculate the partial derivative in x and y, I convolved the image with finite difference operators Dx and Dy

For Dx, it basically gives us the results of img[y, x] - img[y, x + 1] and similarly for Dy, we get img[y, x] - img[y + 1, x] which finds differences between adjacent pixels which can approximate rate of change. To get the gradient magnitude I took sqrt((dx/dt)^2 + (dy/dt)^2). To find the real edges without noise, I binarized the gradient magnitude by choosing an appropriate threshold.

Part 1.2: Derivative of Gaussian (DoG) Filter

First, I applied a Gaussian filter on the cameraman image before applying the same Dx, Dy operations.

These gradient images now have less noise than the ones that were not first filtered with a Gaussian. Also, the edges were now slightly blurred, which leads to them showing up as thicker. The edges also stick out more consistently.

Now, to speed things up, I can apply the Dx and Dy filters on the gaussian kernel itself (which is a much shorter operation given the sizes of kernels compared to the entire image). Then, I can use the new kernels on the image, saving us a convolution operation on the entire image.

Part 2.1: Image "Sharpening"

Image sharpening was implemented using the unsharp mask technique. This involves getting high frequencies of the image by subtracting a blurred version from the original image. Then we can add some scaled version of the high frequencies (scaled by alpha) to the original image to increase the higher frequencies in the original image and "sharpen" it.

We can also apply this to another image.

In this final example, I will take an image, apply a Gaussian blur to it, and then try to sharpen the blurred image.

Part 2.2: Hybrid Images

Hybrid images involve being able to see one thing when very close up to the image and another when far away. Our eyes are best at picking up high frequency signals when close, and low frequency signals from far. To take advantage of this, we can blend two images together by taking the low frequency signal from one of them (the one we want to see from far away) and the high frequency signal from the other (the one we want to see close up). To get the low frequency signal, we can apply a Gaussian blur to the image. To get the high frequency signal, we can subtract the blurred image from the original image.

In this example, I implemented the Bells and Whistles of using color. Here, both Derek and Nutmeg were kept in color since that seems to give the best results. In general, it would be reasonable to remove the color for the higher frequency image, since it doesn't show up that clearly even when kept in. Here are all the other combinations of color.

It is hard to see the color on Nutmeg, but if you focus on the right side arm in the photo, you can see some difference.

Here are some other examples that I thought looked pretty cool:

Fourier Analysis

I also did a frequency analysis of this process for the Messi + Maradona photos: Final Good Example: Here are some examples that did not work too well: The biggest problem with this image was that I couldn't get the close up version to work properly. I think the problem was due to a lot of key features of the face not lining up properly. From far, however, it looks fine. The problem with this one as well was that the animals didn't quite line up and their features are quite different. In order to make the lion visible close up, I also had to blur the elephant quite a lot.

Part 2.3: Gaussian and Laplacian Stacks

To get a Gaussian stack, I took the original image and apply a Gaussian filter on it. Then, we can continue applying Gaussian filters over and over again on them. This is basically a stack of images with decreasing frequency ranges. To get a Laplacian stack, We take each consecutive image from the Gaussian stack and subtract the two. This gives us mid-band frequency images. The final image would be the same as the lowest frequency image in the Gaussian stack. If we add all the images in the Laplacian stack, we will get back the original image.

Gaussian Stack for Apple

Laplacian Stack for Apple

Gaussian Stack for Orange

Laplacian Stack for Orange

Part 2.4: Multiresolution Blending (a.k.a. the oraple!)

To blend these images, we can take the a mask image that represents the boundary and create a Gaussian stack using them. We also take the Laplacian stacks for each of the images and sum up mask * lapacian image 1 + (1 - mask) * laplacian image 2. This gives us a merged version at each frequency band. Then, I can sum up all the merged frequency bands.

Gaussian Mask

Apple Masked

Orange Masked

Oraple Laplacian Hybrid

Final Summed Hybrid Image

Batman + Joker

Golden Gate Bridge + Lightning

Gaussian Mask

Golden Gate Masked

Lightning Masked

Golden Gate + Lightning Laplacian Hybrid

Final Image

Golden Gate Bridge + Northern Lights

Final Image

Earth from Earth

Final Image

Bells and Whistles

I discuss the Bells and Whistles within each section directly. I used color for everything by processing the red, green, and blue channels separately. For the hybrid images, I found that using colors for both high and low frequency images looked the best. I think the coolest part/most important thing I learned from this project was the impact frequency has on images and human vision. The way hybrid images allow us to see different things up close versus far away was pretty incredible since it just uses differing high and low frequency blends.