CS 194-26: Project 2

Fun with Filters and Frequencies!

Imaani Choudhuri
Part 1.1

In this part of the project, the partial derivatives with respect to X and Y are estimated using convolution of the cameraman image with the finite difference operators: [1, -1] and its transpose vector. To construct the gradient, a vector of partial derivatives, we can simply create a vector with the two partial derivatives found earlier as elements. To compute the magnitude of this gradient, we can compute it the same way we would compute the magnitude of any vector: by squaring each element, taking the sum, and then taking the square root. Since the elements of the gradient vector are matrices, we conduct these actions on the matrices: element-wise square, matrix addition, and element-wise square root to get our gradient magnitude matrix, which we then binarize to get edges.

hover to enlarge

Partial with respect to X

Partial with respect to Y

Gradient magnitude

Edge image

Part 1.2

Now, we convolve the image with a 2d Gaussian before doing the same actions as 1.1. We can see that the partials have much more defined and continuous edges than 1.1, and the resulting gradient magnitude image has smooth edges with less noise. By binarizing, we find thicker edges with less noise. This is due to the smoothing effect of Gaussian blur - by averaging each pixel with its surrounding pixels, we reduce the small, abrupt changes in value and preserve only the major shifts.

Partial with respect to X

Partial with respect to Y

Gradient magnitude

Edge image

To reduce computation but get the same result, we can convolve the Gaussian with the finite difference operators first, and convolve that result with the original image. These operations are equivalent to what we did just before due to the commutativity and associativity of convolution. The resulting edge images are the same.

Derivative of Gaussian wrt X

Derivative of Gaussian wrt Y

Convolve DoG wrt X with original

Convolve DoG wrt Y with original

Gradient magnitude using partials from applying DoG

Binarized gradient magnitude using (original * (Gaussian * finite difference))

Binarized gradient magnitude using ((Gaussian * original) * finite difference)

Part 1.3

In this part, we computationally determine the best angle to straighten an image. By selecting the rotation which maximizes angles of 0, 90, and 180 in the gradient, we have the straightenened image. This rotation was selected by maximizing the sum of the +/- 0, 90, and 180 degree bins from a histogram of angles in the gradient (above a certain magnitude).

Original

Straightened (-4 degrees)

Original histogram

Straightened histogram

Original

Straightened (-19 degrees)

Original histogram

Straightened histogram

For this image, it's debatable if the image straightening succeeded. The horizon is somewhat obscured due to the clouds and irregular rock positions, but it is much closer to being straight. There may not be a perfect straightened image for this since it depends on where you consider the horizon.

Original

Straightened (-26 degrees)

Original histogram

Straightened histogram

For this image, we can see the algorithm failed. We can see from the histogram how there aren't many gradient angles in the +/- 0, 90, 180 degrees to begin with, and that there are four evenly spaced peaks in the histogram compared to the previous images, which had odd numbers of peaks. That means there's no rotation where the peaks of the histogram coincide with the gradient angles we want.

Original

Straightened (-49 degrees)

Original histogram

Straightened histogram

Part 2.1

In this part, we derive the unsharp mask filter by creating a convolution equivalent to adding a high pass filter to our image. As a result, this filter adds more high frequencies to the image.

Original

Sharpened

Here, we blur the cat image and sharpen it again using our filter. We can see that our filter has enhanced the edges on the blurry photo, but it's still lost a lot of detail compared to the original. For example, the items in the window are harder to make out and some stripes on the cat are gone even after re-sharpening.

Original

Blurred

Re-sharpened

Part 2.2

In this part, we create 'hybrid images.' Up close, the high frequencies dominate our interpretation of the image, while farther away, the lower frequencies form out interpretation of the image. These images are formed by an average of a high-pass filter on the first image and a low-pass filter on the second, after rotating and aligning.

Nutmeg

Derek

Nutmeg + Derek

For the hybrid image above, Fourier analysis was performed by computing the log magnitudes of each image in the Fourier domain.

Fourier on Nutmeg source image

Fourier on Derek source image

Fourier on Nutmeg, after highpass filter

Fourier on Derek, after lowpass filter

Fourier on hybrid image

Go Won pouting

Go Won smiling

Hybrid pout + smile

Wolf

Olivia

Olivia + Wolf

I consider the pizza cake to be a failure - despite the supposed similarity of the shapes, the intensity of the cake overwhelms the pizza and the shape of the cake is too distinguishable. Even with more blurring, the cake is too identifiable up close, despite the high frequency of the pizza also being visible.

Pizza

Cake

Pizza + Cake?

Part 2.3

In this part, I implemented a Gaussian and Laplacian stack. The Gaussian stack has a Gaussian with increasing sigma applied to the image as you descend the stack. The Laplacian stack at index i is the difference between the Gaussian images on the stack at index i and i+1. The final image on the Laplacian stack is the last image on the Gaussian stack. In the example below, we see that descending deeper Gaussian stack increasingly appears to be Lincoln, while the Laplacian shows the woman and her surroundings more sharply defined.

Original

Sigma = 1

Sigma = 2

Sigma = 4

Sigma = 16

Here I apply the Gaussian and Laplacian stacks to one of my hybrid images. We can see that traversing deeper into the Gaussian stack increasingly shows the low-frequency smiling image, while the Laplacian stack starts with the high-frequency pouting image, which becomes more obscured the deeper you go.

Original

Sigma = 1

Sigma = 2

Sigma = 4

Sigma = 16

Part 2.4

Finally, we look at multiresolution blending: blending two images together across a seam in a smooth way. This blend is achieved by applying a mask onto each level of the Laplacian stacks of the source images, and recombining the resulting Laplacian to the result image.

Orange

Apple

Oraple

Oraple

The following examples use irregular masks to join images together.

Pineapple

Go Won

Mask

Result

Onion

Go Won

Mask

Result

Coolest thing I learned

I think Laplacian stacks and their applications were super cool. I had no idea that an image could be deconstructed and (mostly) re-constructed that way, and using them to split together images and create the blended images in 2.4 felt more like magic than math.

thanks for checking out my submission! :D