Part 1: Frequency Domain

1.1 Warm-Up: Sharpen Using Unsharpen

1.2 Hybrid Images

Hybrid images are static images that change in interpretation as a function of the viewing distance. The basic idea is that high frequency (sharp edges) tends to dominate a person's perception of an image when it is available, but, at a distance, only the low frequency (smoothed) part of the signal can be seen. By blending the high frequency portion of one image with the low frequency portion of another, we get a hybrid image that leads to different interpretations at different distances.

To create these images, we take two photos and then filter out the low frequencies in one photo and the high frequencies in the other photo. We then align and layer the two filtered images on top of one another; if the images are chosen and aligned correctly, we end up with a nice perspective change when we view the image first from up close and then from far away.

Derek // Nutmeg

Smiling Dog // My Sister

My Sister // Me - FAILURE

I believe this one failed because the hybrid image doesn't reveal multiple interpretations even when viewing it from different distances (apparently we look more alike than I thought). Once the faces are lined up like this, the stacked high and low frequency images don't change the interpretation of hybrid image, and it just remains a sort-of average of our faces. In addition, the sigma values chosen for this particular blend may have also contributed to this failed hybrid image as well, since the low and high frequency images look rather the same. We could possibly improve the hybrid image if we chose different sigmas that emphasized the opposite ends of the frequency spectrum more.

Fourier Transformations: Smiling Dog // My Sister

We can analyze and illustrate this process through frequency analysis, by showing the Fourier Transformations (FFT) of the different types of images: the two input images, the filtered (lowpass, highpass) images, and the hybrid image.

1.3 Gaussian and Laplacian Stacks

With hybrid images, either focusing on the low frequencies versus the high frequencies lets us perceive two different images. We can also use filtered stacks, which are similar to pyramids but without the downsampling, to analyze these hybrid images. Gaussian filters smooth the image, whereas Laplacian images bring out the sharper edges. After using Gaussian and Laplacian stacks to create successive layers of the Gaussian and Laplacian filter on an image, we can analyze the images at their different levels of frequencies. We can more clearly see how the lower and higher frequencies come in and out of focus as the stacks progress in their levels.

Lincoln // Gala

Smiling Dog // My Sister

1.4 Multiresolution Blending

Multiresolution blending uses Laplacian and Gaussian stacks to blend two images seamlessly along a mask. To build up a full image, we sum up the result of multiplying different levels of resolutions of a Gaussian filtered mask with different levels of resolutions of a Laplacian filtered image. The resulting image spline (the place where the two images connect) is a smooth seam that joins the two images together by gently distorting them along the mask's edge. Multiresolution blending computes this gentle seam between the two images seperately at each band of image frequencies, resulting in a much smoother, feathered seam than naive cut and paste.

Apple // Orange

Inspired Movie Scenes

Kimi no Na wa

And the world has somehow shifted

We must be swift as the coursing river

Bells and Whistles

For some bells and whistles, I implemented the Laplacian stack blending in RGB color. The color images are shown above since they create better visual effects :-) Below are the black and white versions.

Part 2: Gradient Domain Fusion

2.1 Toy Problem

Before we begin using gradient domain fusion to blend multiple images, this toy problem gets us started on gradient domain processing by allowing us to practice working with solving a least squares problem in a matrix. In this toy example, we set up our sparse matrix of constraints, as explained in depth on the project page, which lets us recompute the pixels to reconstruct the original image. In a summary, we create a sparse matrix A of constraints, a vector b of known values, and solve for x in Ax = b.