Fun with Filters and Frequencies!

by Nir Levin

Overview

In this project, I manipulated images using convolutions and frequencies. An important concept that I used here a lot is the fact that gaussian convolution acts as a low pass filter. We can also subtract the convoluted image from the original image to get a result that acts as a high pass filter.

Part 1: Fun with Filters

Finite Difference Operator

In vector calculus, the gradient is the direction of greatest change, which is the vector comprised of partial derivatives in the X and Y directions. The magnitude of this vector is largest when there's a large change in pixel values, so it's a good way of finding edges.

Derivative of Gaussian (DoG) Filter

The main difference between this edge image and the previous one is that this one is smoother and less jagged. The previous image seems like it's made of a combination of vertical and horizontal lines.

Part 2: Fun with Frequencies!

Part 2.1: Image "Sharpening"

It's possible to combine all the below operations into one gaussian which we can convolve with the original image to get the sharpened image. This gaussian is equal to ((1 + alpha) * unitImpulse - alpha * gauss), where alpha is the multiple of details we're adding and gauss is a simple unaltered gaussian.

Below I attempt to blur an already sharp image, then resharpen it using the technique above. It's not perfect but the result is about halfway between the original and the blurred image.

Hybrid Images

Here I wanted to create an image that looks like one thing when viewed up close and another when viewed from afar. In order to do this, I took advantage of our brain's innate ability to focus on the fine, and not broad, details up close but focus on the broad, and not fine, details from far away. This is accomplished by passing one image through a low-pass filter and the other one through a high-pass filter, and then averaging them.

Fourier Analysis of Above Process

Other Hybrid Images

Gaussian and Laplacian Stacks for Multiresolution Blending

I made the Gaussian stack of the Oraple by first convolving the apple and orange images with five separate times, each with a gaussian of different standard deviation. The sigma value is doubled for each level of the stack. I used sigma levels of 2, 4, 8, 16, and 32. Each level of the Laplacian stack below is the difference between two consecutive levels of the Gaussian stack.

In order to combine half of the apple and the other half of the orange, I used a step function mask. I created a Gaussian stack of this mask as well so that there's a more gradual rise from the dark left side to the bright right side of the mask. I then did element-wise multiplication of the mask and the orange for each level of the stack. I did the same for the apple and (1 - mask) in order to capture opposite sides of each fruit. For each level, I then added the masked apple and orange to get the rightmost column.

With the mask convolutions, I found it more effective to start with a mask that's too large, and then do a "valid" (as opposed to "same") style convolution with a large gaussian in order to get the most gentle gradient. When I used the "same" style convolution for the mask, there were very noticeable boundary artifacts.

These results below are calculated by adding up all the images in the laplacian stack.

More multiresolution bending

The most interesting thing I learned in this assignment is how a Gaussian blur is just convolving the image with a matrix based off of a gaussian curve. I've used cv2.gaussianBlur a lot before but it's always been a black box for me.