CS 194-26: Image Manipulation and Computational Photography, Fall 2018
Project 3: Fun with Frequencies and Gradients!
Bernie Wang, cs194-26-afg
Overview
In this project, we explore different image blending techniques that involve filtering and manipulating the frequency domain as well as blending in the gradient domain.
Part 1: Frequency Domain
1.1 Warmup
We'll first start off with "sharpening" an image by smoothing it with a Gaussian kernel, then substracting that from the original image to get the high frequency details. This is basically applying a high-pass filter. Then was add back the details to the original image with a scaling factor in order to enhance the details.
Details (Image - Smoothed Image)
Source Image
Sharpened Image (Image + alpha x Details, where alpha = .5)
1.2 Hybrid Images
In this section we will create hybrid images by taking the low frequencies of one image and the high frequencies in the other, then averaging them. This way, from up close you can see the high frequency image, and from far away you can see the low frequency image.
Image 1
Image 2
Hybrid Image (sigma1 = 5, sigma2 = 10)
Image 1
Image 2
Hybrid Image (sigma1 = 7, sigma2 = 5)
2DDFT of Naruto
2DDFT of low-passed Naruto. Notice how low frequencies are stronger and high frequencies are attenuated.
2DDFT of Sasuke
2DDFT of high-passed Sasuke. Notice how the high frequencies are stronger.
2DDFT of Naruto-Sasuke hybrid
Failure
Here's an example of a failed hybrid image. Even though the eys and mouth of both images align, the proportions of each of their heads differ significantly. The result is a hybrid image where the source images do not overlap well.
Image
Image 2
Hybrid Image (sigma1 = 7, sigma2 = 5)
1.3 Gaussian and Laplacian Stacks
To create Gaussian stacks, we successively filter images with a 2D gaussian, each getting more and more blured. To create Laplacian stacks, we take the difference between adjacent layers of the Gaussian stacks, getting the finer details and borders.
Image
Layer 1 of Gaussian Stack
Layer 2 of Gaussian Stack
Layer 3 of Gaussian Stack
Layer 4 of Gaussian Stack
Layer 5 of Gaussian Stack
Layer 6 of Gaussian Stack
Image
Layer 1 of Laplacian Stack
Layer 2 of Laplacian Stack
Layer 3 of Laplacian Stack
Layer 4 of Laplacian Stack
Layer 5 of Laplacian Stack
Layer 6 of Laplacian Stack
Image
Layer 1 of Gaussian Stack
Layer 2 of Gaussian Stack
Layer 3 of Gaussian Stack
Layer 4 of Gaussian Stack
Layer 5 of Gaussian Stack
Layer 6 of Gaussian Stack
Image
Layer 1 of Laplacian Stack
Layer 2 of Laplacian Stack
Layer 3 of Laplacian Stack
Layer 4 of Laplacian Stack
Layer 5 of Laplacian Stack
Layer 6 of Laplacian Stack
1.4 Multiresolution Blending (With Color Bells & Whistles)
Taking advantage of Gaussian and Laplacian stacks, we can create Laplacian stacks for two images and a Gaussian stack for a mask. At each layer we blend the two images with the mask, and then add together all the layers to create a frequency-blended image that is blended on all frequency spectrum.
Image 1
Image 2
Combined Images (not blended)
Combined images, blended
Image 1
Image 2
Combined images (not blended)
Combined images, blended
Image 1
Image 2
Combined images (not blended)
Combined images, blended
Part 2: Gradient Domain Fusion
In this section we explore blending images using gradient domain fusion. In Laplacian blending, we blended images based on their frequency response, but boundaries still appear in across multiple bands. Another perspective of image blending is gradients, which represent local changes. Since the human eye responds more to intensity changes than to the intensities themselves, blending using gradients offers a good alternative.
If we have a masked source image s that we want to blend into target image t, we want to transform s to v such that the following is satisfied:
2.1 Toy Problem
We'll first start off with a simpler problem. Blending a source image s into a blank image t of the same size. The shape of the mask should be the same size as both s and t.
Toy image
Image from solving least-squares problem
Error (toy image - generated image)
SSE = 1.243056973618237e-06
2.2 Poisson Blending
We'll first start off with a simpler problem. Blending a source image s into a blank image t of the same size. The shape of the mask should be the same size as both s and t.
Source Image
Target Image
Overlayed Image
Blended Image
Source Image
Target Image
Overlayed Image
Blended Image
Failure Case
Here, the source image is a picture of my friend (I conventiently had a picture of her), and the target image is the Mona Lisa. Since the Mona Lisa is an oil painting, due to aging, it's gradient is much higher than that of the picture of my friend. Because the least-squares problem focuses on matching the gradient of the source image and the gradient on the boundary between the source and target, the complexion of my friend's face in the blended image does not blend well with that of the Mona Lisa.
Source Image
Target Image
Overlayed Image
Blended Image
Poisson vs. Laplacian
In my opinion, Poisson blending works better for the red panda/Van Gogh fusion than Laplacian blending. You can see that Poisson blending altered the color of the red panda, especially around its snout, since gradient domain fusion cares more about gradients than intensities. Multiresolution blending preserves color, but boundaries between images look sharper. The red panda's face is less blended into Van Gogh for Laplacian blending than Poisson blending.
Source Image
Target Image
Poisson Blended Image