Project 3: Frequencies and Gradients

Jonathan Chu

Background

Part one of this project explores image sharpening, hybrid images, creating Gaussian and Laplacian stacks of structured images, and using these stacks to spline images via multi-resolution blending across different frequency bands.

Part two of this project focuses on gradient domain processing, specifically Poisson blending, to combine parts of images with others smoothly.

Part 1.1: Image Sharpening

To sharpen images we first applied a Gaussian filter over the original image in order to get a blurred, low frequency view. Next we subtracted this low frequency image from the original to obtain the details of the image. This essentially is a high-pass filter. Finally, we added the high-pass filter details back to the original image to sharpen the details.

Waterfall

Lowpass Waterfall

Highpass Waterfall

Sharpened Waterfall

Part 1.2: Hybrid Images

Hybrid images are images that appear to look like one thing up close, but something else from far away.The approach to create these hybrid images is to align one image that has gone through a high pass filter with one image that has gone through a low pass filter. The high pass filtered image will appear up close while the low pass filtered image will appear from far away.

Mr. Derek

Nutmeg

Mr. Nutmeg

Vin Diesel the wizard:

Vin Diesel

Gandalf

Vin Diesel x Gandalf

FFT of Vin Diesel x Gandalf

Vin Diesel FFT

Gandalf FFT

High Pass Diesal FFT

Low Pass Gandalf FFT

Hybrid FFT

Part 1.3: Gaussian and Laplacian Stacks

Gaussian and Laplacian stacks can help one discover the structure of an image at each resolution. Gaussian stacks are created by repeatedly applying a Gaussian filter over the source image (i.e. blurring repeatedly). Laplacian stacks are similar to Gaussian stacks, but they repeatedly apply a high pass filter instead.

Here are the Gaussian and Laplacian stacks for Dali

Gaussian First level

Gaussian Second level

Gaussian Third level

Gaussian Fourth level

Gaussian Fifth level

Laplacian First Level

Laplacian Second Level

Laplacian Third Level

Laplacian Fourth Level

Laplacian Fifth Level

Part 1.4: Multiresolutional Blending

Multiresolution blending computes a gentle seam between the two images seperately at each band of image frequencies, resulting in a much smoother seam. The algorithm involves convolving different levels (multiple resolutions) of the Laplacian stack for each image with either the corresponding levels of the Gaussian stack for a binary image mask (that determines which pixels come from which image) or its complement (convolve with GR or 1-GR). Using the Gaussian stack with the binary mask allows the seam to be smooth.

Apple

Orange

Appleorange

Horse

Lion

Horse Lion

Jack Black

Emma Watson

Jack x Emma

Part 2.1: Toy Problem

We want to reconstruct an image, given a single pixel from the original image and the gradients along the x and y axis between each pixel. We can accomplish this by setting up a system of linear equations, Ax = b. Our x vector, with dimensions (height * width, 1), is the vector of the unknown pixels needed to reconstruct the image. A is a sparse matrix with a width equal to the number of pixels in our image and height equal to number of linear equations we are solving. Each linear equation matches with a squared error we are trying to minimize as defined by the equation in the project spec.

Original

Reconstructed

Part 2.2: Poisson Blending

Given a source region S (image cut-out) and target region t, the goal is to seamlessly blend the boundaries of the source image region with the location in the target image. The problem is then to set v to the same gradients as the background image, aka minimize the difference between the x and y gradients. This can be done by solving the least squares problem stated in lecture for each rgb color channel and then reconstructing the image from the channels we solve for.

Penguin

Hikers

Hikers with Penguin

Penguin Chick

Hikers

Hikers with Penguin Chick

Emma

Jack

Emma Wart

Multiresolutional vs Poisson Blending and Problems Faced

I found that Poisson Blending in general was easier to use when you wanted to specify boundaries and borders of the source image. However, my images also showcase a problem I faced. Because I did not have enough memory on my computer to download matlab and use the provided image crop, I had to make do with writing a makeshift image crop in python. This prevented me from getting the exact masks I wanted and ultimately led to some failures.

Multiresolutional blend

Poisson blend