Overview

In this project, we worked with frequencies and gradients in order to enhance and combine images. We used filters to blur and sharpen images, as well as to separate higher and lower frequencies from each other to create images that look different at different distances. We also used laplacian stacks to blur images together, and finally implemented poisson blending to use gradients to blur images together.

Section I: Frequency Domain

Part 1.1: Warmup

Here are the results for sharpening an image using the unsharp mask filter.

Part 1.2: Hybrid Images

We applied a gaussian filter as a low pass to one image, and a gaussian minus an impulse to another image as a high pass, then combined the two images to form a picture that looks different from a distance than it does up close.

An analysis of the fourier space shows relatively what we expected. The first image, of the fft of an image after gaussian blur, is dark on the outside, as all the high frequencies have been removed. The second has a lot of high frequencies after the high-pass filter. The combined image has a mix of both, as do the original images.

Here are some more results.

The last image did not work as intended because the gradients of the sharp image didn't match the blurred image enough. I realized that in order to get this to work, the two images have to be somewhat similar in composition (ex: eyes, nose, and mouth in relatively similar positions). Otherwise, both images become incomprehensible.

Part 1.3: Gaussian and Laplacian Stacks

We formed Gaussian and Laplacian Stacks by continuously applying a gaussian filter to an image (gaussian stack) and by subtracting the gaussian stack at level i from the gaussian at level i-1 (laplacian stack). The last image in the laplacian stack is the same as the last layer of the gaussian stack and is not subtracted from anything, in order to make sure we keep all the information necessary to recreate the image.

Part 1.4: Multiresolution Blending

Here, we perform blending at multiple resolutions using laplacian stacks, interpolated with a mask that also has a gaussian stack applied to it. Below are the additional images I used, and the results.

Here are the results of the laplacian stack for the watermelon canteloupe image.

Section II: Gradient Domain Fushion

Overview

The purpose of this section is to implement Poisson blending, which is when you blend using the x and y gradients of an image instead of the color values of the image. We did this by imposing constraints upon the pixel values we want to find, and solving a least squares optimization problem to obtain the best values for those pixels. The constraints are as follows:

We will discuss these constraints in greater detail in the following parts.

Part 2.1: Toy Problem

First, we solve a simple version of this larger problem by only considering the first set of constraints out of the two listed above. These are simply to constrain the gradients of the image inside a boundary to be the same as the gradients of the source image. For the toy problem, we consider the whole thing to be the source image, and simply obtain the same image back again. This was just to check our understanding of the concepts.
I also used this to practice setting up and using a sparse matrix, and using the sparse matrix to solve for the image.

Part 2.2: Poisson Blending

Next, we implemented the full technique by adding in equations to the A matrix to fulfill the second set of constraints. These apply to pixels on the boundary, and try to make sure that the difference between the selected pixel and the target image is very similar to the gradient of the original source image.

Using these, we were able to blend much more interesting images.

This one didn't work as well, because the gradient of the source image turned dark at the borders and I didn't do a good enough job of cutting it off.

I tried blending using the laplacian stack method and the poisson blending method on the image below. They didn't work that well on their own, because in the source image, my hair was too close to the borders of the image, and I didn't do a good enough job in cutting it off in the mask.

However, by the color of the image, we can clearly tell that the poisson result would have worked better for this case.

In fact, we can extrapolate from this that the poisson works better if the gradient of the images match up already and the color is different, and the laplacian method works better if the colors can be merged separately, but the gradients are too different.