Fun with Filters and Frequencies!

By Cameron Hui, chui3@berkeley.edu

Overview

This project focuses on image blurring, sharpening, blending, and playing with frequencies in an image. First we apply filters to images to blur and sharpen images. Then we blend images by creating hybrid images as well as blending images.

Part 1: Fun With Filters!

1.1 Finite Difference Operator

This part of the project finds the edges in an image by convolving images with the finite difference operators Dx and Dy.

harvesters
Partial derivative in x
harvesters
Partial derivative in y
harvesters
Gradient magnitude

To calculate a gradient magnitude, I calculated the square root of the sum of the squares of the image convolved by the two finite difference operators. Squaring the two convolutions removes negative values to get the magnitude of each convolution, and then the square root of the sum normalizes the values.

harvesters
Edge image

1.2 Derivative of Gaussian Filter

This part of the project clarifies the edges in the image by blurring the image using a Gaussian filter before finding the edges.

harvesters
Image convolved with Gaussian
harvesters
Partial derivative in x on Gaussian
harvesters
Partial derivative in y on Gaussian
harvesters
Gradient magnitude on Gaussian
harvesters
Edge image on Gaussian
What differences do you see?

The main difference is that the lines are clearer and there is less noise around the image. The lines are clearer as well as thicker, which makes edges easier to determine. The edges of the buildings in the background are also harder to see, and some of the silhouette lines like the legs and coat are a little less clearly defined as edges. This might be because I used a higher threshold to reduce the noise from the ground, which meant lost information from the background.

Verify that you get the same result as before using a single convolution by convolving the Gaussian with the finite difference operators before applying the Gaussian to the image.
Convolved Gaussian with Dx and Dy filter, respectively (sized up for easier viewing)
harvesters
Gradient magnitude on pre-convolved Gaussian
harvesters
Edge image on pre-convolved Gaussian

Part 2: Fun with Frequencies!

2.1: Image "Sharpening"

This part of the project sharpens images using the unsharp mask filter. The unsharp mask filter works by subtracting the low frequencies from an image to get the high frequencies only (aka details), and then adds them back on top of the original image. This can be done with a single convolution by applying the Laplacian of Gaussian filter on the image to get only the high frequencies.

Low frequencies of image are generated by applying a Gaussian filter (low pass filter) on the image
High frequencies of image are isolated by subtracting the lower frequencies from the original image
The image is sharpened by applying the Laplacian of Gaussian filter on the image
Before and After using alpha = 0.5
For evaluation, pick a sharp image, blur it and then try to sharpen it again. Compare the original and the sharpened image and report your observations.

For the sake of keeping my image sizes lower, I used cropped images to run the resharpening on.

For the painting image (a digitally produced image that I made with Vermillion, a VR painting app), there is resharpening of the image, but there are still details that are somewhat lost such as the canvas-like texture of the picture. As I increased the alpha value, the image got darker overall, but the edges didn't really come back into focus as much as it did originally. For the baby picture, there is also some resharpening of the image, but the original picture was somewhat noisy so the edges didn't refocus as easily.

Once I used alpha=10, the image is approximately the same brightness as the original image, but still loses a lot of detail.

Original Baby Image
Resharpened images at alpha=1, 2, and 5, respectively
Before and After using alpha = 10
Original Painting Image
Resharpened images at alpha=1, 2, and 5, respectively
Before and After using alpha = 10

2.2: Hybrid Images

This part of the project creates hybrid images - from up close you'll see one image, and from far away you'll see a different image. This is essentially achieved by combining a low pass filtered image with a high pass filtered image, so that you'll see the details of the high pass image up close while you'll see the low pass image far away.

Sample image of Derek and Nutmeg
Hybrid of my chinchilla and a squirrel (squirrel image from this website)
Hybrid of my chinchilla and a baby photo of me.
Hybrid fail of two of my cousins - I believe this failed because the proportions of the faces are very different and aligning the eyes don't fit the faces together well.
Fourier Transform Breakdown on Baby/Chinchilla image
Input images
Lowpass and Highpass
Hybrid
Bells and Whistles - Hybrid on Color Images
Both Gray
Color on Highpass Only vs Color on Lowpass Only
Both Color

I found that coloring the high pass image and keeping the low pass image grayscale gives the best effect on the combined image overall (at least when looking close up/far away). If I could somehow reduce the colors so that the highpass colors were more transparent I would do that so that the colors blended better.

2.3: Gaussian and Laplacian Stacks

This section of the project implements the Gaussian and Laplacian stacks needed to blend two images together.

Gaussian Apples
Laplacian Apples
Gaussian Oranges
Laplacian Oranges
Recreating Laplacian Pyramid Table Showing Levels 0, 2, and 4

2.4: Multiresolution Blending (a.k.a. the oraple!)

This section blends images together using a Gaussian stack to blur splines across the image.

Oraple

On my own images

Two Taiyaki Showing Levels 0, 2, and 4
Two Pizzas Using Circular Mask Showing Levels 0, 2, and 4

Conclusion

In conclusion, the biggest thing I learned during this project was how to take advantage of high and low frequencies in an image to retrieve details, colors, and more to perform sharpening as well as other image manipulation effects. Once I figured out how layer masks could be blurred at different Laplacian levels to blend two images together uniquely, I thought about how I could apply this knowledge to when I do digital art (which does have masking features, but has a much more hands-on method of drawing masks rather than prodedurally generating them). This was super fun and I think I understand all of the content!