Project 2: Fun with Filters and Frequencies!

CS194-26 Intro to Computer Vision and Computational Photography | Jingyi Zhou

Part 1: Fun with Filters

Part 1.1: Finite Difference Operator

When we convolve a finite difference operator with an image, all it does is taking the difference of the adjacent pixels. Intuitively, this gives us the partial derivative across the according axis. The gradient points in the direction of most rapid increase/decrease in intensity, which is orthogonal to the direction of the edges. Therefore the edge strength is given by the gradient magnitude, calculated by np.sqrt(np.square(partial_x) + np.square(partial_y)). *I used row(1, 0, -1) and col(1, 0, -1) as the finite difference operators instead of (1, -1); a threshold of 0.3 is applied to binarize the magnitude image.

fdo dx

fdo dy

fdo gradient magnitude

fdo edge (binarized grad mag with threshold = 0.3)

Part 1.2: Derivative of Gaussian (DoG) Filter

Notice in the previous edge picture the lines are thin and there are still un-eliminated noise. We can get rid of such noise first by passing the original image through a low-pass filter and repeat the procedure from the previous part:

gaussian + derivative dx

gaussian + derivative dy

gaussian + derivative gradient magnitude

gaussian + derivative edge (binarized grad mag with threshold = 0.08)

As you can see, the edge picture this time is a lot clearer and stronger, as the reduced noise allowed me to binarize with a threshold of as low as 0.08

dog dx

dog dy

dog gradient magnitude

dog edge (binarized grad mag with threshold = 0.08)

Notice that the results are not exactly the same as before; that is because I used the "same" mode when computing the convolutions.

Part 2: Fun with Frequencies

Part 2.1: Image "Sharpening"

In order to sharpen a image, we add some high frequencies to it. First I took the naive and computationally costly approach by finding the image's gaussian, obtain it's laplacian by subtracting the gaussian from the original image, and add a factor of the laplacian back to the image:

original image

taj sharpened with multiple convolutions

I then combined this into a single convolution operation which is called the unsharp mask filter: unsharp_mask = (1 + alpha) * unit_impulse(G_2D.shape) - alpha * G_2D

taj sharpened with the unsharp mask

Then I picked my own image, blurred it, and tried to sharpen it again:

original image

cr7 blurred

cr7 sharpened

The result is quite unsatisfactory; in the end, adding high frequencies can only full human eyes to an extent. The sharpening scheme we developed can never give the blurred image the information it never had.

Part 2.2: Hybrid Images

Hybrid images are static images that change in interpretation as a function of the viewing distance. The basic idea is that high frequency tends to dominate perception when it is available, but, at a distance, only the low frequency (smooth) part of the signal can be seen. By blending the high frequency portion of one image with the low-frequency portion of another, we get a hybrid image that leads to different interpretations at different distances.

My SWEEEET cat Isabel

A FIERCE snow leopard

the log magnitude of the Fourier transform of Isa

the log magnitude of the Fourier transform of the leopard

isa hf

leo lf

isa hf fft

leo lf fft

hybrid

hybrid fft

Bells & Whistles

I tried to use color to enhance the effect for the following images by coloring both the high and the low frequency component:

Me

My cat

hybrid

sunny day

cloudy day

To my chagrin, this hybrid is a failure, no matter at what frequencies I choose to use color.

Multi-resolution Blending and the Oraple journey

The idea of multi-resolution blending is simple: high frequency contents should be blended with low feathering widths to prevent ghosting, while lower frequencies have to be blended with high feathering widths to prevent sharp edges.

Part 2.3: Gaussian and Laplacian Stacks

Part 2.4: Multiresolution Blending (a.k.a. the oraple!)

Oraple

Clounny

Annisa

Conclusion: Cool Stuff I've Learned

Just seeing how the theories in class turn into live images excites me greatly -- I cannot imagine what Salvador Dali was thinking when he manually created the hybrid painting that we saw in class, but he was indeed a genius.