CS194 - Fun with Frequencies and Gradients!

Nathan Yuchi(CS194-26-acl)

Warmup

To create a sharpening effect, we want to compute the high pass filter of an image. To do so, we can put a picture through a Gaussian filter to obtain a blurred photo. To get a Gaussian filter we must calculate a kernel and then apply a 2D convolution to the image to perform a Gaussian blur. Then we can subtract the blurred image from the original image to get the details, or the high pass filter. Next we add the details to the original image, creating a sharpened look.

Original

"Sharpened"

Hybrid Images

To create hybrid images we use Gaussian blurs and the unmask masking technique from the previous question. We blur both images using convolution with a kernel we created, creating two Gaussian blurred images. We are done with the image we want to be low-pass filtered. Next we use the unmask masking technique and subtract the blurred second image from the original image to create a high-pass filtered image. We simply add the two resulting images to create our hybrid image.

We can see changes in frequency in the frequency analysis. We use a Fourier transform on the images and display the results. As we can see, blurring reduces the high frequencies and sharpening with the unmask masking technique increases the high frequencies.

Derek and Nutmeg

Derek and Nutmeg, In Color

RDJ as Iron Man

RDJ as Iron Man, In Color

Frequency Analysis

Failure example

I would consider this a failure because of how prominent the Canadian flag even after being blurred. The reason that it is still so visible is because the image doesn't have much detail to blur away. After applying the Gaussian blur, the main structure of the side bars and the leaf are still present so the Canadian flag ends up competing with the USA flag in high frequency space.

Bells and Whistles: Color

Using color for hybrid images benefits the low frequency image more. Because the high frequency image contains the thin edges of an image, the color isn't that noticeable. Color is much more visible in the low-frequency image and helps the low frequency image stand up when the high frequency image is filtered away.

Gaussian and Laplacian Stacks

The Gaussian stack is created by repeatedly using a Gaussian blur on an image. When a Gaussian blur is applied, the image is stored on the stack. The LaPlacian pyramid is formed by looking at the layers of the Gaussian stack. The pyramid stores the difference between a Gaussian stack layer and the layer above it. The last layer of the LaPlacian pyramid is equal to the last layer of the Gaussian stack.

By using Gaussian stacks and LaPlacian pyramids we can see the progression of high frequency to low frequency. As we go deeper into the stacks, the low frequencies become more prominent while the high frequences fade away.

Note that in the LaPlacian pyramid images below, each layer has beeen interpolated between 0.0 and 1.0 in order to see the differences. In reality, the differences are very small and the images from the LaPlacian pyramid look almost black.

Iron Man, Gaussian on top, LaPlacian on bottom

Lincoln in Dalivision, Gaussian on top, LaPlacian on bottom

Einstein and Marilyn, Gaussian on top, LaPlacian on bottom

Multiresolution Blending

To implement multiresolution blending we need 3 images. 2 of the images are the images that we want to blend and the last image is a mask. We use the methods from the previous problem and generate LaPlacian pyramids on the images that we want to blend and generate a Gaussian stack of the mask. Then, we generate the layers of the blended image by using the formula layer = mask[i] * pyramidA[i] + (1 - mask[i]) * pyramidB[i]

Oraple

Potato Potato, Tomato Tomato

Dinosaur!

Dinosaur!

City to be Attacked

Mask

Bells and Whistles: Color

To perform the multiresolution blending on a color image, we compute the Gaussian stack and LaPlacian pyramid for each color channel of the image. We then blend together the layers of each channel using the techniques described above. By stacking the results of each channel together, we get the blended color image.

Gradient Domain Fusion

The purpose of gradient domain fusion is to blend and object from a source image onto another image seamlessly. We want to match the maintain the color or intensity of the target image and use the gradient of the source image. This problem can be modelled as a least squares problem and we use the following equation:

Toy Problem

We start with the warmup toy problem where we reconstruct an image using the gradient of an image with the top left pixel value as a constraint for intensity. We create a system of linear equations and fill in a sparse matrix A. The first height * width rows of A represent the horizontal gradients and the next height * width rows represent the vertical gradients. Then we fill in another sparse matrix b with the gradient values from the image. Using a sparse least squares solver we can solve for v in Av = b. v gives us the reconstructed image.

Original

Reconstructed from Gradient