CS294-26 Project 2: Fun with Filters and Frequencies

Romil Bhardwaj

Part 1

1.1 - Finite Difference Operator

The partial derivative in x and y computed as the convolution of the finite difference operator with an input image. Here's the input and the x derivative and y derivative:

After computing the partial derivatives, the magnitude of the gradient can be computed as L2 distance of the x and y derivatives (ie., sqrt(dx^2 + dy^2)).

To extract edges from this gradient magnitude image, we must threshold this image. After some fine-tuning, I used a threshold of 0.28 which gave me the following result:

1.2 - Derivative of Gaussian (DoG) Filter

To remove the noise from these gradients and get more robust edges, we can use Derivative of Gaussian (DoG) filters. Gaussian filtering blurs the image, removing noise, and then we can use the finite difference operator to find edges. We observe that the edges are thicker and stronger. Here's the result:

In order to reduce convolution operations, we can combine the two operations in a single derivative of gaussian filter. Here's a visualization of the X and Y DoG filters:

On convolving with these filters, we get the same result as with separate convolutions.

1.3 - Image Straightening

To straighten an image, we can compute the angle between the x and y derivatives of the images at each pixel using np.arctan2(dy/dx). This will yield the orientations at each pixel, which can then be histogrammed. Since right angles are considered to be "straight",  we try different orientations which maximize the number of elements in the -90, +90, -180, +180 buckets. This is done by computing the histogram with np.hist(), and then comparing the histogram vector with an ideal vector (which has density 1 at -90, +90, -180 and +180)  by taking their dot product. This gives us a score which we can use to rank different rotations. Here are the results:

Failure case - this running track at MLK Jr. School in Berkeley has curving lines on the track and a foggy horizon which makes it hard for our algorithm to find edges to straighten.

Part 2

2.1 - Image Sharpening

An image can be sharpened by boosting the higher frequencies. To get the higher frequencies, we can subtract a blurred image from the original image. This can be combined into a single unsharp filter using a weighted combination of unit impulse function and gaussian filter. The results below show the effect of the unsharp filter on sharpening an image and what happens if you blur an image and try to sharpen it again (spoiler alert: blurring is lossy, thus resharpening doesn't help and the image loses detail).

2.2 - Hybrid Images

A hybrid image is a combination of two images which is interpreted differently by humans at different viewing distances. This effect is achieved by combining high frequencies from image with lower frequencies from another image. At closer distances, higher frequencies are prominent, while at larger distances lower frequencies dominate our interpretation of the image. Here are some hybrid images:

Derek Nutmeg

Luke Vader

Failure case - Obiwan Kenobi and Palpatine. Poor alignment and Palpatine's wrinkles dominate the higher frequencies.

"Hello There." "General Palpatine!"

FFT Analysis for Luke Vader

Darth Vader Image

Raw Input Darth Vader Image LPF Filtered Darth Vader Image - Higher frequencies are removed and lower frequencies are allowed

Luke Skywalker Image

Raw Input Luke Skywalker Image HPF Filtered Luke Skywalker Image - Higher frequencies are allowed and lower frequencies are removed

Final Hybrid Image FFT

2.3 - Laplacian and Gaussian Stacks

I implemented Gaussian stacking by repeatedly blurring an image with a kernel of size 15 and sigma = 2. Here are the gaussian and Laplacian stacks for Salvador Dali's "Lincoln in Dalivision" and Luke Vader image from the previous part. As get deeper into the gaussian stack, the original low frequency image emerges more clearly. In the laplacian stack, we can see the higher frequency image clearly in the first few images (eg. You can see Luke skywalker more clearly at depth=1).

Lincoln - Gaussian Stack

Lincoln - Laplacian Stack

LukeVader - Gaussian Stack

LukeVader - Laplacian Stack

2.3 - Multiresolution Blending

Multiresolution blending seamlessly blends two images by using laplacian and gaussian stacks to create a smooth transition with a mask. Here are the results:

Oraple




Lost travolta




Seasons