Project 2: Fun with Filters and Frequencies!¶
Richard Liu (3033944112)
Part 1.1: Finite Difference Operator¶
For the first part, we took the gradient of the cameraman image by convolving with the $D_x$ and $D_y$ vectors, $[[-1, 1]]$ and $[[-1], [1]]$ respectively.
The actual countours are a bit hard to see, so we apply a binarization function with a threshold of $0.1$ to clearly see the edges.
Part 1.2: Derivative of Gaussian (DoG) Filter¶
Even with the binarization, there's still a decent amount of noise and the partial derivative edges aren't super clear-cut and thick. To address this, we smooth the image by convolving with a 2D Gaussian kernel before taking the derivative, resulting in a DoG image that eliminates a large portion of the noise.
Part 1.3: Image Straightening¶
One useful application of the gradient and DoG filter is the ability to align a tilted photo. We can simply iterate through a variety of possible "rotation angle", create a histogram with the angles found within the picture, and take the photo with the largest number of vertical (0 or 180 degrees) and horizontal (90 and 270 degrees) angles. Here's the result of running this algorithm on a building facade
Facade before:
Facade after:
Leaning tower of Pisa (before, optimal histogram, after)
Keyboard (before, optimal histogram, after)
Orbit (before, optimal histogram, after -> this is a failure case because the algorithm ends up optimizing for the shadows (which are correctly aligned) than the tilted orbit (which is 5 degrees off).
Part 2.1: Image "Sharpening"¶
Using Gaussians, we know that we can essentially apply a low-pass filter to an image to isolate the low frequencies. Using this technique, we can also isolated the high frequences by subtracting the low-passed image from the original image. If we then add these high frequencies to the original image, we can make the image look sharper.
Taj before sharpening:
Tag after sharpening with $\alpha=3$
Here's the low passed image below:
If we try to sharpen the blurred image with $\alpha=3$, we get something similar to but not exactly equal to the original image, as we cannot recover the information lost from the Gaussian smoothing on the first blur.
Here we sharpen a picture of Matt Damon.
Part 2.2: Hybrid Images¶
Now that we can generate low- and high-frequency images, we can make hybrid images by combining the low frequencies of one image to the high frequencies of another.
Here we merge Derek and his car Nutmeg.
Here are the starting, intermediate, and final images in Fourier space.
Here's another example with Matt Damon and Leonardo DiCaprio.
Here's a failure case where I tried to combine Gon and Luffy. Hybrid images with sharp contours (like anime characters) seem to be difficult.
Bells & Whistles¶
I started out with grayscale, but transitioned to color by applying the low- and high-pass to all three color channels rather than just the single grayscale channel. It works better to use color for both the low and high frequency components rather than one color and one grayscale.
Part 2.3: Gaussian and Laplacian Stacks¶
Gaussian stack for painting:
And the Laplacians here (scaled from $[-1, 1]$ to $[0, 1]$ to prevent negative values):
We can also apply this to our Matt/Leo hybrid photo.
Part 2.4: Multiresolution Blending¶
Here, we blend together an apple and an orange through the algorithm described in the P. J Burt and E. H. Adelson paper. We create Laplacian stacks for the apple and the orange, a Gaussian stack for the stepwise vertical mask, and combine them together for a smooth transition.
Here we apply the same technique to two animated characters.
Here, I use a custom mask to blend my face into Rich Brian's album cover.
Bells & Whistles¶
For color, I simply applied the greyscale process to each of the three channels. I learned so much from this assignment! The most interesting I learned was how even seemingly simple things like a low-pass filter can create such photorealistic hybrid images.