Fun with Frequencies and Gradients!

CS 194-26: Image Manipulation and Computational Photography

Casey Takahashi

Part 1: Frequency Domain

Part 1.1: Warmup

Image Sharpening: To sharpen the image, I computed the laplacian of the original image by subtracting the gaussing kernel from a unit impulse, and convolving the image with the laplacian kernel. I found an alpha of 0.7 gave the best result. The larger the alpha, the more edges are added back into the image.


Part 1.2: Hybrid Images

To create hybrid images, the first step is to align the images on 2 points. Using a low pass filter to extract the low frequencies from one image, and a high pass filter to extract the high frequencies from the second image, you can then add the two together to form a hybrid image.

More Hybrid Photos




Bells and Whistles

Used color!


Part 1.3: Gaussian and Laplacian Stacks

Applied a low pass and high pass filter over hybrid images for multiple layers to separate the frequency bands.

For this part, I used sigma = 1 with 5 layers for the stacks.

Derek-Cat Hybrid

Marilyn Monroe and Einstein Hybrid

Part 1.4: Multiresolution Blending

For Multiresolution blending, I separated the mask into different frequency layers by applying a gaussian filter one right after the other. In addition, I collected the laplacian stack for both images of interest. Layer by layer, I multiplied the mask by the image, and added the other image to the product. Lastly, I summed all the layers to reconstruct the blended image.



Bells and Whistles

Used color!


Part 2: Gradient Domain Fusion

Part 2 aims to seamlessly blend an object or texture from a source image into a target image. We can achieve this through Poisson Blending. This approach focuses on matching gradients rather than pixel values because the human eye is more attuned to change in brightness rather than actual pixel brightness.


Part 2.1 Toy Problem

Objectives:
1. minimize (v(x+1, y)-v(x, y) - (s(x+1,y)-s(x, y)))2
2. minimize (v(x, y+1)-v(x, y) - (s(x, y+1)-s(x, y)))2
3. minimize (v(1,1 )-s(1, 1))2


Part 2.2 Poisson Blending

Poisson blending works by calculating the gradients of the optimal image. We do this by taking the target pixel values as they are for all pixels outside the mask, and getting the in the source image for everything in the mask. We construct a matrix to compute the gradients of an image (taking the 4 neighbors into account - up, down, left, right). We can solve the equation Ax=b by isolating x and solving for x. Now that we have the output, we can reshape the pixels to form our output image!



Failure: This blend didn't work as well as I had hoped. I suspect that it's because (1) the color of the background of the images don't match that well and (2) I was pasting the girl on a surface that had a lot of details rather than an even background. You can tell that the entire image was faded a lot to get the gradients to match, so a lot of detail and vibrance was lost.


Comparison: Laplacian Stack vs Poisson Blending

You can see the poisson blending does a much better job of blending the two images. There's a slight yellow hue around the scuba diver when using the laplacian stack, which comes from trying to blend the yellow patches on the suit with the surrounding pixels. However, in poisson blending, the focus is on gradients rather than pixel values, so you don't get the yellow hue.


What I learned

I learned a lot in this project, especially how to process color images (do each color individually, then stack the three colors). In addition, it was really interesting learning how reconstruction and poisson blending works, and combining my knowledge of gradients and matrix calculations to images.