fun with filters & frequencies

Nadia Hyder

PART ONE

1.1: FINITE DIFFERENCE OPERATOR

A picture containing outdoor, person, object, photo

Description automatically generated

 

For this part of the assignment, I worked with the cameraman image to create its edge image. I convolved the image with finite difference operators [[1,-1][ and [[1],[-1]], found their gradient magnitude image, and binarized it to create an edge image.

 

Partial derivative in x

Partial derivative in y

Gradient magnitude

Edge image

A picture containing diagram

Description automatically generated

A picture containing diagram

Description automatically generated

Diagram

Description automatically generated

Diagram, schematic

Description automatically generated

 

 

 

1.2 DERIVATIVE OF GAUSSIAN FILTER

The results from 1.1 were noisy, so I convolved the image with a gaussian filter to blur the image, then convolved with the finite difference operators. I then binarized the resulting gradient magnitude image to create a less noisy edge image. Because convolution is commutative, we can achieve the same results using a single convolution instead of two by creating derivative of gaussian filters. This gave a less noisy edge image.

A close up of a logo

Description automatically generated 

 

 

 

1.3 IMAGE STRAIGHTENING

I implemented an image straightening algorithm which relies on a statistical preference for vertical and horizontal edges in most images. In my implementation, I tested rotations from -10 to 10 degrees and computed the gradient angle of the edges in the image (after cropping out the outer edges). The straightened image is the rotation with the greatest proportion of horizontal and vertical edges (sum of # of -180, -90, 0, 90 degree edges).

 

Original image

Histogram of angles

Straightened image (rotated by -3 degrees)

Histogram of angles

A large building with many windows

Description automatically generated

Chart, histogram

Description automatically generated

A large building with many windows

Description automatically generated

Chart, histogram

Description automatically generated

 

 

 

Here are the results on additional images of my choosing:

 

 

Original image

 

Histogram of angles

 

Straightened image

 

Histogram of angles

 

A large clock tower on a green field

Description automatically generated

Chart, histogram

Description automatically generated

A large building

Description automatically generated

rotated 7 degrees

Chart, histogram

Description automatically generated

The roof of a building

Description automatically generated

Chart, histogram

Description automatically generated

A house with trees in the background

Description automatically generated

rotated 10 degrees

Chart, histogram

Description automatically generated

 

 

Failure case

The straightening algorithm failed on the following image because it has many edges, but in this case horizontal and vertical edges cannot be used to determine whether the image is straight.

 

A close up of a tall building

Description automatically generated

 

Chart, histogram

Description automatically generated

A view of a city

Description automatically generated

rotated 10 degrees

Chart, histogram

Description automatically generated

 

 

 

 

PART TWO

2.1 IMAGE SHARPENING

I implemented image sharpening by applying a Gaussian filter to the image, then creating an “unsharp mask” by subtracting the blurred image from the original, then adding the mask multiplied by some weight to the original image.

Here are some images before and after image sharpening:

 

A large building

Description automatically generated

A large building

Description automatically generated

Diagram

Description automatically generated

Diagram

Description automatically generated

 

I then tried blurring a high-res image and sharpening it. While the sharpened version is not as sharp as the original high-res image, it is considerably sharper than the blurred image.

 

High-res image

Blurred image

Blurred then sharpened image

Shape

Description automatically generated

A close up of a sign

Description automatically generated

A close up of a sign

Description automatically generated

 

 

 

 

 

2.2 HYBRID IMAGES

To create image hybrids, I aligned the two images on the eyes, applied a low pass filter (Gaussian) to the first image, and a high pass filter to the second image. I then averaged the two images to create the final result.

 

Image 1

image 2

hybrid

A person posing for the camera

Description automatically generated

A cat sitting on top of a wooden door

Description automatically generated

A cat that is looking at the camera

Description automatically generated

 

 

Here is another hybrid I tested and its FFT analysis:

A close up of a cat looking at the camera

Description automatically generated

A dog looking at the camera

Description automatically generated

A dog looking at the camera

Description automatically generated

 

 

 

 

Image 1

Image 2

High frequency

Low frequency

Hybrid

A picture containing background pattern

Description automatically generated

Background pattern

Description automatically generated

A picture containing background pattern

Description automatically generated

A picture containing chart

Description automatically generated

A picture containing background pattern

Description automatically generated

 

 

 

2.3 GAUSSIAN AND LAPLACIAN STACKS

To create a Gaussian stack, I continuously convolved the input image with a Gaussian filter at every level of the stack. To create the Laplacian stack, I found the differences between adjacent levels of the gaussian stack.

 

Gaussian stack

 

A picture containing table, light, green, colorful

Description automatically generated

A blurry image of a building

Description automatically generated

A blurry photo of a building

Description automatically generated

A blurry image of a building

Description automatically generated

A blurry image of a building

Description automatically generated

A picture containing street, blurry

Description automatically generated

 

 

Laplacian stack

 

A close up of a brick building

Description automatically generated

A picture containing building, street, photo, brick

Description automatically generated

A picture containing building, street, circuit, sign

Description automatically generated

A picture containing building, street, sign, circuit

Description automatically generated

A circuit board

Description automatically generated

A picture containing street, blurry

Description automatically generated

 

 

 

2.4 MULTIRESOLUTION BLENDING

I implemented multiresolution blending by combining Laplacian layers and a gaussian layer. I first created a gaussian stack for a binary mask, and gaussian and Laplacian stacks for the input images. We blend the images with lower frequencies increasingly blurred, creating a stack given by the equation described in the following paper:

 

 

 

The blended image is the sum of all the layers.

 

 

A picture containing sitting, indoor, orange, table

Description automatically generated

A picture containing sitting, indoor, orange, table

Description automatically generated

A picture containing shape

Description automatically generated

A picture containing elephant, sitting, cat, close

Description automatically generated

A picture containing outdoor, person, riding, slope

Description automatically generated

A person standing on a beach near a body of water

Description automatically generated

A picture containing shape

Description automatically generated

A person standing near snow covered mountains in the background

Description automatically generated

A close up of a toy

Description automatically generated

A car driving down a country road

Description automatically generated

A close up of a logo

Description automatically generated

A person driving a car

Description automatically generated

 

 

 

Laplacian stacks of Mario Kart + Tesla (irregular mask image)

 

A picture containing diagram

Description automatically generated

A picture containing outdoor, water

Description automatically generated

A picture containing outdoor, water, group

Description automatically generated

A picture containing outdoor, water, large, group

Description automatically generated

A close up of a road

Description automatically generated

A person driving a car going down the road

Description automatically generated

 

 

 

 

 

 

TAKEAWAYS

While this project offered several important insights, the biggest one is probably how powerful and versatile the Gaussian high-pass filter is, and how pivotal convolution is to image manipulation. I have always wondered how the iPhone performs automatic image straightening, and I now have the tools in my toolbelt to perform it myself! This project has been very rewarding. After this project I have gained a lot of insight about how to perform edge detection, alignment, image sharpening, and image filtering. At the core of it all is the gaussian filter!