CS 194-26 Project 2 - Fun with Filters and Frequencies!

Jerry Zhang

Overview

In this project, I explored image manipulations using different filters and frequencies. Starting first with using the finite difference operator to find edge images, then working on using Gaussian filters, this project utilized a variety of filtering techniques to edit and enhance images. Then working with frequencies, I again used Gaussian filters to sharpen and blur images, produced hybrid images, and created an Oraple.

Part 1: Fun with Filters

Part 1.1: Finite Difference Operator

In this section, I started with the original cameraman.png image and found the partial derivatives in x and y by convolving with the finite difference operators D_x = [1, -1] and D_y = [[1], [-1]].

To compute the gradient magnitude image, I added the element-wise square of each partial derivative, then took the element-wise square root of that sum.

I used a threshold of 0.2 to binarize the gradient magnitude image and turn it into the edge image. I found that this threshold suppressed noise while maintaining the real edges in the image.

"cameraman.png" Cameraman Partial Derivative in x Cameraman Partial Derivative in y Cameraman Gradient Magnitude Image Cameraman Edge Image

Part 1.2: Derivative of Gaussian (DoG) Filter

In this part, I used the Gaussian filter with kernel size 6 and sigma 2 to blur the original image, then using the procedure from part 1.1 to find the edge image. In this part I used a threshold of 0.065 to obtain the edge image from the gradient magnitude image.

What are the differences between the image from this part and the last part? There is less noise after blurring, and the edges that are detected end up being more pronounced and thicker.

Cameraman Blurred Cameraman Blurred Partial Derivative in x Cameraman Blurred Partial Derivative in y Cameraman Gradient Magnitude Image Cameraman Edge Image

The same thing can be obtained with a single convolution rather than two because convolution is associative and commutative. We can create a derivative of Gaussian filters and then use those DoG filters on the image, reducing the number of convolutions we do per image after obtaining the filters.

Convolve Gaussian with D_x Convolve Gaussian with D_y Image Convolved with Gaussian Convolved with D_x Image Convolved with Gaussian Convolved with D_y Gradient Magnitude Image Cameraman Edge Image

This result is the same as the previous part.

Part 2: Fun with Frequencies!

Part 2.1: Image "Sharpening"

To "sharpen" an image, we use the Gaussian (low-pass) filter to get the low frequencies, then subtract it from the original image to get the high frequencies of the image. We then scale and add the high frequencies to the image, resulting in the effect that the image looks sharper. For these sharpened images I added in alpha = 4 times the high frequencies.

"taj.jpg" Blurred taj.jpg taj.jpg high frequencies Sharpened taj.jpg Sharpened taj.jpg using unsharp mask filter

Using the unsharp mask filter, with just a single convolution operation, results in the same "sharpened" image as going through the steps described above.

"newyork.jpg" Sharpened newyork.jpg
"losangeles.jpg" Sharpened losangeles.jpg

For the Yellowstone image, the progression from the original image to the blurred image, then to the extracted high frequencies, and finally to the final image is shown below.

"yellowstone.jpg" Blurred yellowstone.jpg yellowstone.jpg high frequencies Sharpened yellowstone.jpg

I also tried to blur an image and resharpen it, to see if sharpening undoes the effects of the blur. From comparing the below images, the original and the sharpened image are definitely different (particularly in the numbers on the Campanile, and details in the fur and grass). Blurring removes the high frequencies, and sharpening utilizes the remaining high frequencies rather than the highest frequencies removed by the blur effect.

"llamas.jpg" Blurred llamas.jpg Sharpened llamas.jpg after blur

Part 2.2: Hybrid Images

Using concepts of human perception to keep or remove low/high frequencies in an image allows for the concept of hybrid images, where an image looks like one thing from close-up while something else appears when further away.

To get a low-frequency image, convolve the image with a Gaussian. For high frequency, convolve with a Gaussian and subtract that result from the original image. High frequencies tend to dominate when they can be seen, but they are more difficult to see at a distance.

For most of the below images, I aligned them on their eyes. Note: Hover above image to zoom, helps to see the Hybrid effect

Image Pair 1: Derek + Nutmeg

Derek Nutmeg Hybrid

Image Pair 2: Professor DeNero + Ron (from Harry Potter)

Prof. DeNero Ron Hybrid

Image Pair 3: Professor Efros + Louis XIV

Prof. Efros Louis XIV Hybrid

Fourier Analysis: log magnitudes of the Fourier transform are shown below.

efros.jpg input aligned efros.jpg high pass filter LouisXIV.jpg input aligned LouisXIV.jpg low pass filter Combined

Image Pair 4: Mr. Incredible + Hulk (Failure case). Although from "far away" the image looks fine (where only Mr. Incredible appears), from close up, the Hulk's body alignment and size differs too drastically from that of Mr. Incredible, so the image of Mr. Incredible is still very obvious.

Mr. Incredible Hulk Hybrid

Part 2.3: Gaussian and Laplacian Stacks

In this part I implemented Gaussian and Laplacian Stacks, which are pyramids but without performing any downsampling. Using the stacks, I was able to recreate the outcomes of Figure 3.42 to create an Oraple.

Apple Orange Combined

Part 2.4: Multiresolution Blending

This part builds upon the previous part to implement multiresolution blending, using any kind of mask.

Bells and Whistles:

I used color for all the images here in order to enhance the blending effect.

Sand + Mountains
Mountains Original Sand Original
Mask
Output
Louis XIV (Sun King) + Sun; Irregular Mask
Louis XIV Original Sun Original
Mask

Laplacian Pyramid Blending Details

Output

Most Interesting Thing

The coolest thing I learned from this assignment was how to implement hybrid images. I found it super interesting how low frequency and high frequencies can be manipulated to show two seemingly different images to people who view things at different distances.

The most important thing I learned from this assignment was how to use Gaussian and Laplacian stacks to generate images that are filtered in different ways.

Acknowledgements

Project description and some starter code provided by CS 194-26 course staff

Images are not my own, sourced from different locations on the internet: