Fun with Filters and Frequencies!

Tyler Zhu

Overview

Our goal this project is to experiment with different filters and frequencies and apply them to photos. Notably, we will be "sharpening" images, creating hybrid images, and applying Laplacian pyramids introduced by Burt and Adelson in 1983 to create a "blur" effect and do multi-resolution blending.

Part 1.1: Finite Difference Operator

In this section, we explore different filters on the cameraman picture below.

A camerman.

We can compute the partial derivative in x and y by convolving the appropriate filters, which look like the following.

Dx of the camerman.

Dy of the camerman.

Combining these gives us the gradient magnitude by using the formula given in class below.

Gradient formula from class (Source: Alyosha's slides).

Getting the magnitude of the gradient also lets us binarize it to get an edge map. Both of these are shown below.

Gradient magnitude obtained from the formula.

Edge map obtained from binarizing at 0.7 (out of 3).

Part 1.2: Derivative of Gaussian (DoG) Filter

For this section, we created a gaussian filter and convolved it first before all of our other filters.

One interesting heuristic I came up with to estimate a good sigma for a kernel size so that the Gaussian doesn't level out is for sigma to be 1/3 of the kernel size. This is because a 2d gaussian will have sqrt(2)*sigma as its standard deviation, and if we want 95% of the values to be captured, we really want 2 standard deviations to be in the kernel. This means our kernel should be ~ 1.4*sigma * 2 = 3 sigma, which is Alyosha's suggestion for us.

Gaussian Filter with kernel size 9 and sigma of 3.

A blurred version of the original picture is here.

We can visualize our convolved gaussian with Dx and Dy below.

Derivative of Gaussian plotted.

Then we can look at the image convolved with the one resulting kernel for each case.

DoG of Dx visualized.

DoG of Dy visualized.

The end result we get is that our resulting edge map is sharper as a result of smoothing out the noise in the original image.

Edge map from low-pass filter, visualized.

Part 2.1: Image "Sharpening"

We showcase our result on the provided image, with some intermediate results. Qualitatively I think alpha = 3 looks the best.

Taj picture with 9 equally spaced sharpenings from 0 to 4.

Here's another picture I chose to try and sharpen. It's been a while since this photo was taken, and camera's weren't very good back then so it's nice to see some of my old features more prominently for once! Unfortunately the picture is so grainy sharpening just enhanced the graininess. So much for that!

Taj picture with 9 equally spaced sharpenings from 0 to 4.

Part 2.2: Hybrid Images

For this part, I found it took a lot of trial and error to find a good split of kernel size and sigma for the high pass and low pass filters to work, especially on my own photos when they didn't match up. For reference points, Lily's advice to pick the pupils of the eyes worked beautifully, better than my original idea of picking the nose and the chin. I think this leads to more photorealistic transforming.

Here are some of the results I got. First the original one.

The original hybrid picture from the paper.

It looks pretty good. Derek is mostly blurred out when you look at his cat, while he comes into focus much more clearly in the distance.

Seeing that there is a pet/owner theme, I tried it on three sets of pet/owners; my friends Chris and Amanda, and myself.

Chris and his dog Fiona.

Amanda and her dog Duoduo.

Me and my dog Snowy.

For those curious, here are the original pictures. Chris and Fiona turned out the best, and I have to attribute this to Fiona having the most distinct features. Duoduo and Snowy were too uniformly white to be very distinctive (i.e. my failures).

Chris.

Fiona.

Amanda.
Duoduo.

Tyler.

Snowy.

I did my frequency analysis on Chris and Fiona since that one turned out the best. I zoomed in on the frequency plots around the center for general clarity and to see it better.

Chris.

Fiona.

These are pictures after passing them through the low and high pass filters.

Chris.

Fiona.

Hybrid picture analysis.

As expected, the frequencies in both the low and high pass images are primarily low and high. However, in Chris' image there are still large frequencies corresponding to vertical and horizontal edges which I'm not sure where they come from. If you look close enough, you can see that the center of the frequency plot for Fiona has been damped significantly by the high-pass filter.

Part 2.3 & 2.4: Multiresolution blending.

I blended first in grayscale, which gave me the following orapple.

Grayscale orapple.

Here is the laplace stack visual.

Laplace visual.

I tried to make it happen for color by applying to each color channel, but there were mediocre results (blue background).

Color orapple.

Finally, I also tried it on a nectarine and kiwi.

Color orapple.