Project 2: Fun with Filters and Frequencies!

Justin Chen

Overview

The primary goal of this project is to explore the different behaviors and associated methods of various filters and image processing techniques that allow us to edge detect, sharpen, and blend images. Many of these methods involve the convolution of an image with a gaussian filter of varying parameters in order to exploit its use as a low-pass filter, allowing us to manipulate the many different frequencies within an image to create the desired effect.

Part 1: Fun with Filters

Part 1.1: Finite Difference Operator

In order to compute the gradient magnitude of an image, first we need to compute the derivatives of the image with respect to both the "x" and "y" directions. To do this, we use the differentiation operators D_x = [1, -1] and D_y = [1, -1].T and convolve them each with the original image (converted to grayscale). Now that we have the derivatives of the image in both directions, we can compute the gradient magnitudes by simply applying the magnitude formula: (D_X^2 + D_Y^2) ^ 0.5 where D_X and D_Y are our derived image in both the "x" and "y" directions respectively.

Edge binarization parameterized by threshold value = 0.2

FiniteDx
Cameraman x-derivative
FiniteDy
Cameraman y-derivative
FiniteGrad
Camermaman gradient magnitudes
FinitEdge
Cameraman binarized edges

Part 1.2: Derivative of Gaussian (DoG) Filter

In this part, we work through the same process as earlier, except this time applying a gaussian filter first to blur the image and remove some of the higher frequencies. As a result, the detected edges are a lot less noisy and stronger, giving us a better edge detector.

All images were generated with a gaussian filter of size 3 and sigma 1.

GaussianBlurred
Gaussian blurred
GaussianDx
Gaussian blurred x-derivative
GaussianDY
Gaussian blurred y-derivative
FiniteEdge
Gaussian blurred gradient magnitudes

By deriving the gaussian filters first, we can see that the resulting detected edges are essentially identical to that of the prior two-step method. Both methods are taken with threshold value = 0.1.

GaussianEdge
Two-step DoG filter
GaussianV2
Single-step DoG filter

Part 2: Fun with Frequencies!

Part 2.1: Image "Sharpening"

To sharpen images for this section, we apply an unsharp mask filter to the original image. To do this, we take the original image and apply a gaussian blur like in previous sections, then subtract the blurred image from the original in order to obtain just the high frequencies. Then, all that is left is just to add a multiple (parameterized by alpha) of the high frequencies onto the original image, resulting in a sharpened image.

All images were generated with a gaussian filter of size 3 and sigma 1.

Taj
Taj Original
TajBlurred
Taj with gaussian blur
TajHigh
Taj high frequencies
TajSharpened
Taj sharpened with alpha=2
Mountains
Mountains Original
MountainsBlurred
Mountains with gaussian blur
MountainsHigh
Mountains high frequencies
MountainsSharpened
Mountains sharpened with alpha=5

Part 2.2: Hybrid Images

To create a hybrid image, first the images are aligned with the provided starter code. Then, we take the low frequencies of the first image by convolving it with a gaussian kernel, while we get the higher frequencies of the second image by subtracting the low frequencies of the second image (similarly convolving the second image with a gaussian kernel) from the original (aligned) image. Adding these two together outputs the final hybrid image

Derek – Nutmeg
Derek
DerekPicture Original
Nutmeg
Nutmeg Original
DerekNutmeg
Hybrid DerekPicture and Nutmeg with sigma1=4, sigma=10
Car – Airplane

For a failure case, we can use this car and airplane example. This might be because the two objects are simply too different, resulting in a less than optimal hybrid blend.

Car
Car Original
Airplane
Airplane Original
CarAirplane
Hybrid car and airplane with sigma1=4, sigma=12
Federer – Nadal
Federer
Federer Original
Nadal
Nadal Original
Fedal
Hybrid Federer and Nadal with sigma1=1, sigma=8
FedererFreq
Federer frequencies
FedererFilteredFreq
Federer filtered frequencies
NadalFreq
Nadal frequencies
NadalFilteredFreq
Nadal filtered frequencies
FedalFreq
Hybrid Federer and Nadal frequencies

Bells and Whistles

Comparing the three different variations, it seems like coloring the low-frequency component produced the best result. This could just be due to alignment or due to the compability of the two images at a deeper level, as the results in general aren't all that great.

Parameters: sigma1=1, sigma2=8

FedalColor1
Hybrid Federer (Color) and Nadal
FedalColor2
Hybrid Federer and Nadal (Color)
FedalBoth
Hybrid Federer and Nadal (Both Color)

Part 2.3: Gaussian and Laplacian Stacks

For this section, the construction of the Gaussian and Laplacian Stacks involve convolutions with a gaussian kernel in a manner much like the previous parts. First, for the Gaussian Stack, a gaussian kernel is recursively convolved with the image, essentially layering blur actions on top of each other. Then, for the Laplacian Stack, each level is computed by taking the difference between successive Gaussian Stack layers (element [i] - element [i + 1]). In generaly, the depth (or iterations) was set to 5.

Dali Stacks

Parameters: size=3, sigma=1

Dali
Dali Original
DaliG1
Dali Gaussian Stack Layer 1
DaliG2
Dali Gaussian Stack Layer 2
DaliG3
Dali Gaussian Stack Layer 3
DaliG4
Dali Gaussian Stack Layer 4
DaliG5
Dali Gaussian Stack Layer 5
DaliL1
Dali Gaussian Stack Layer 1
DaliL2
Dali Gaussian Stack Layer 2
DaliL3
Dali Gaussian Stack Layer 3
DaliL4
Dali Gaussian Stack Layer 4
DaliL5
Dali Gaussian Stack Layer 5
Federer Stacks

Parameters: size=5, sigma=2

Federer
Federer Original
DaliG1
Federer Gaussian Stack Layer 1
DaliG2
Federer Gaussian Stack Layer 2
DaliG3
Federer Gaussian Stack Layer 3
DaliG4
Federer Gaussian Stack Layer 4
DaliG5
Federer Gaussian Stack Layer 5
DaliL1
Federer Gaussian Stack Layer 1
DaliL2
Federer Gaussian Stack Layer 2
DaliL3
Federer Gaussian Stack Layer 3
DaliL4
Federer Gaussian Stack Layer 4
DaliL5
Federer Gaussian Stack Layer 5
Apple Stacks

Parameters: size=5, sigma=2

Apple
Apple Original
AppleG1
Apple Gaussian Stack Layer 1
AppleG2
Apple Gaussian Stack Layer 2
AppleG3
Apple Gaussian Stack Layer 3
AppleG4
Apple Gaussian Stack Layer 4
AppleG5
Apple Gaussian Stack Layer 5
AppleL1
Apple Gaussian Stack Layer 1
AppleL2
Apple Gaussian Stack Layer 2
AppleL3
Apple Gaussian Stack Layer 3
AppleL4
Apple Gaussian Stack Layer 4
AppleL5
Apple Gaussian Stack Layer 5
Orange Stacks

Parameters: size=5, sigma=2

Orange
Orange Original
OrangeG1
Orange Gaussian Stack Layer 1
OrangeG2
Orange Gaussian Stack Layer 2
OrangeG3
Orange Gaussian Stack Layer 3
OrangeG4
Orange Gaussian Stack Layer 4
OrangeG5
Orange Gaussian Stack Layer 5
OrangeL1
Orange Gaussian Stack Layer 1
OrangeL2
Orange Gaussian Stack Layer 2
OrangeL3
Orange Gaussian Stack Layer 3
OrangeL4
Orange Gaussian Stack Layer 4
OrangeL5
Orange Gaussian Stack Layer 5

Part 2.4: Multiresolution Blending

Multiresolution blending in this part builds upon the stacks created in the last part. Gaussian and Laplacian Stacks are constructed for both imageA (GA, LA) and imageB (GB, LB) and a Gaussian Stack (GR) is also constructed for the mask denoting the area to focus blending. Then, at each layer the algorithm (GR * LA) + ((1 - GR) * LB) is applied to blend at each level. Once this new blended stack is created, the output image is just the layers summed up in addition to the last layer of the Gaussian Stacks of imageA and imageB respectively.

Tennis Ball / Softball

Parameters: size=15, sigma=4

Apple
Apple
Orange
Orange
Oraple
Blended Apple / Orange (Vertical Mask)
Tennis Ball / Softball

Parameters: size=3, sigma=1

Tennis
Tennis Ball
Softball
Softball
TennisSoftball
Blended Tennis Ball / Softball (Vertical Mask)
Shark / Dolphin

Parameters: size=15, sigma=4

Shark
Shark
Dolphin
Dolphin
TennisSoftball
Blended Shark / Dolphin (Vertical Mask)
Cat / Dog

I used an irregular mask here, specifically a box mask centered around the faces/head areas of the cat and dog.

Parameters: size=5, sigma=2

Cat
Cat
Dog
Dog
DogCat
Blended Cat / Dog (Irregular Box Mask)
Mcgregor / Tiger

For this one, to get the tiger to align correctly with Mcgregor's tattoo I had to resize the tiger image by padding it appropriately to match up. For the mask, I created a box centered around the tiger and the aligned region so that most of the blending would be focused in that area.

Parameters: size=21, sigma=23

Mcgregor
Mcgregor
Tiger
Tiger
McgregorTiger
Blended Mcgregor / Tiger (Irregular Box Mask)

Mcgregor Laplacian Stack

Mcgregor1
Mcgregor2
Mcgregor3
Mcgregor4
Mcgregor5

Tiger Laplacian Stack

Tiger1
Tiger2
Tiger3
Tiger4
Tiger5

Blended Laplacian Stack

Blended1
Blended2
Blended3
Blended4
Blended5

Bells and Whistles

As seen above, all of the blended images were done with color. For me personally, I learned a lot about how gaussian filters and kernels can be used to process images! I had always known that it acts like a low-pass filter for images from previous classes, but expanding upon that in application and using them to sharpen, blend, and create stacks is very interesting!