CS194-26 Project 3: Fun with Frequencies and Gradients
Joey Barreto
1.1: Warmup
In this section, the high frequency components of an image are added back to the image to sharpen its features--the parameter alpha controls how much is added back.

alpha = 0.0

alpha = 0.5

alpha = 0.9

1.2: Hybrid Images
In this part filtering is used to combine the high frequency components of one image with the low components of another. The two images will be distinguishable at different viewing distances due to detail loss and gain of the human visual system.

Old and young

Recession of the Muir glacier since 1941

Moon and golf ball

Woman and leopard

Fourier transforms of the original images, the filtered images, and the final Fourier transform of the hybrid image from the leopard-woman are displayed below. Overall I think color does help the low frequency component become more recognizable at a distance.

FFT of woman

FFT of leopard

Low-pass image of woman

High-pass image of leopard

FFT of hybrid

1.3: Gaussian and Laplacian Stacks
For this section my Gaussian filter's width scales like 2*i + 5 pixels, where i is the iteration counter. I felt this helped blur the low frequencies more strongly than a constant width. For each example below, the first row, going right, are the successive Gaussian filtered images, and below them are the Laplacians, with the high frequency bands on the left and low on the right. Each Laplacian is (above left) - (above image) in the table.

Gaussian and Laplacian stacks of a checkerboard plane.
Gaussian and Laplacian stacks of an Escher drawing
Gaussian and Laplacian stacks for the leopard-woman
1.4: Multiresolution Blending
Here Gaussian and Laplacian stacks are used to blend images.
Gaussian and Laplacian stacks for the blend of the modern-day beach with the WW2 scene from D-Day.
2.1: The Toy Problem
In the second half of the project, we explore using gradients instead of frequencies to blend images. We desire to cut a region from one image and paste it onto another, while preserving gradients on the boundary and throughout the cutout (at the expense of the actual intensity values) to minimize the artifical look of the procedure. The below equation expresses our constraints:
Gradients between pixels in the interior of the cutout in the final image should be similar to those of the corresponding pixels of the source image (first sum), and likewise for those on the boundary, but since the bordering pixels of the final image are known, this fixes one term in the second summation. All these constraints can be formulated as a least-squares problem, where I included four constraints for each pixel (2 x gradients and 2 y gradients). Each constraint becomes a row in a matrix (A), and the desired gradient values populate a vector (b). Solving Ax = b for x gives an array that can be reshaped into an image where the gradients have been preserved as best as least-squares can get. The constraints can be encoded in different ways (like with a 3x3 Laplacian), but as long as all vertical and horizontal nearest-neighbor gradients are covered, the result will be the same. In the toy problem there is no cutout to be solved for--we just encode the gradients of an image, set the value of one pixel in the output, and let the solver find an image that satisfies the gradient constraints... which naturally will be the original image!

Original

Reconstructed

2.2: Poisson Blending
In Poisson blending, we want to solve for the new values of a cutout region that preserves gradients as explained above. In the below example a ship from the left painting is transplanted into right. The pure copy/paste method is shown for comparison, and the procedure is replicated but with the multiresolution blending from earlier as well. Poisson blending does a much better job of making the ship appear more sunlit because it is allowed to change the intensity values of the cutout--the multires blending cannot change its interior too much. The result also looks good because the ship was already lit by a sun on the right in the original image.

Source image

Target image

Copy and paste

Poisson blending

Laplacian stack blending

The following two examples show how Poisson blending can fail--if the boundaries colors cause the interior intensities to change too much. It's not too bad in the freeway sign image, but the Shrekpea did not fare well.

Another Poisson example

The failed Shrekpea

I think Poisson blending will work well for cutouts with similar original background colors, and multires blending for image pairs that were chosen because they had similar lighting conditions, angle of view, scale, etc. Overall the more suited the image pairs are to merging--ie the more care someone puts into finding and aliging two images--the better the results will be.