Project #3: Fun with Frequencies and Gradients!

Part 1: Overview

In the first part of the project, we perform various types of image blending and hybridization while working in the frequency domain. We sharpen images, create frequency-hybrid images, and use Gaussian and Laplacian stacks to perform blends.

Part 1.1: Warmup

For this part, I sharpened a blurry image by passing the image through a high-pass filter and adding the result back (scaled by a factor) to the original image. I obtained the high-pass filtered image by subtracting the Gaussian filtered image from the original image.

Original Image

Sharpened Image

The best Gaussian filter was σ = 3, and the best factor to scale the high-passed filtered image by was α = 0.7.

Part 1.2: Hybrid Images

To create hybrid images, I took the low-frequency component of one image and the high-frequency component of another and added them together (aligning and resizing as appropriate).

Fourier Spectra

FFT of Panda

FFT of Raccoon

FFT of Filtered Panda (Low Pass)

FFT of Filtered Raccoon (High Pass)

FFT of Hybrid

Failure Case: I tried to create an owl and fox hybrid, but because their different head structures, the top half of the fox's head in the resulting image is transparent over a black background.

Bells and Whistles

Using color on the high-frequency component works better, because the high-frequency component is less prominent and adding color brings it out more when looking at the image up close.

Part 1.3: Gaussian and Laplacian Stacks

I implemented the Gaussian stack by applying the Gaussian filter with σ = 1, 2, 4, 8, 16 and the Laplacian stack by taking the difference between consecutive layers of the Gaussian stack.

Salvador Dali's painting of Lincoln and Gala

Mona Lisa

Cristina and Meredith Hybrid

Part 1.4: Multiresolution Blending

Here, I created Laplacian stacks for each of the two input images and blended each layer with a Gaussian filtered mask. The masks I used were horizontal and vertical step functions.

Apple + Orange

Grape + Grapefruit

Avocado + Pear

Irregular Mask: Hand + Sky

I created an irregular mask for the hand and performed the same multiresolution blending.

Bells and Whistles

Apple + Orange

Avocado + Pear

Part 2: Overview

In Part 2 of the project, we would like to create seamless blends with irregular masks. Instead of working in the frequency domain, we switch to the gradient domain. We explore image reconstruction with using the gradients of the original image. We then perform a gradient-based blend known as the Poisson blend.

Part 2.1: Toy Problem

The goal of this part was to reconstruct the original image using only the gradients in the x- and y-directions and providing exactly one of pixel values from the original image. This can be done with a set of linear equations representing the gradient constraints and using least squares to solve for the optimal image.

Original Image

Reconstructed Image

To speed up computation, I used a sparse matrix for the linear constraints. The loss from the least squares operation was 2.61659036521e-05, and the l2 distance between the original and reconstructed images was 3.17018500686e-4.

Part 2.2: Poisson Blending

In this part, we wanted to seamlessly blend an object from one image into another. I created a sparse matrix to minimize the squared differences between the source and target gradients and cast the following blending constraint problem as a linear least squares instance:

We perform this procedure for each of the three color channels independently.

Penguin in the Snow

Source

Target

Directly Copied Penguin

Blended Penguin

Mini Shark Attack

Source

Target

Directly Copied Shark

Blended Shark

Mickey's Cousin

Source

Target

Directly Copied Mouse

Blended Mouse

Failure Case: Baby Alien Cow on the Moooo-n

This blend turned out poorly, because the green background of the cow did not closely match the gray background of the moon. As a result, the cow has a pinkish hue instead of a white color as in the original image.

Source

Target

Directly Copied Cow

Blended Cow

A Comparison of Laplacian Stack Blending and Poisson Blending

Let's revisit the hand in the sky hybrid created with Laplacian stack blending. The two methods have unique downsides here. Poisson blending creates a nice seamless blend, but the resulting hand is transparent. Laplacian stack blending, on the other hand, retains the original colors of the hand but we can clearly see the seams of the blend and the background of the source image. If we are willing to lose the original colors of the source image for perfect seamless blending, then Poisson blending would be a better choice.

Source

Mask

Target

Direct Copy

Laplacian Pyramid Blending

Poisson Blending

Reflection

Some things I learned while working on this project include: how to build and index (sparse) matrices efficiently, how to use basic Photoshop tools, and how to beautifically blend photos with something so simple as linear least squares!