CS194-26

Project 3: Fun with Frequencies and Gradients!

Sharabesh Ramesh

Overview

This project fundamentally focused on developing the pixel manipulation techniques in order to merge images seamlessly. In the first part of the project we used Gaussian and Laplacian filters to create static images which changed with viewing distance, as well as blended images. In part 2 we used a least squares solution in order to minimize gradients between images thus ensuring that image gradients between a target and source matched as closely as possible. Overall this `Poission blending` created far more effective blends than the corresponding method from part 1.

Part 1.1: Warmup

Here we sought to sharpen an image by using Gaussian filters. Using the form

im  + alpha * (im - gaussian)
we show variants on levels of sharpness using the base image. Note that the alpha parameter hear specifies the degree of sharpening. We include an extreme example in order to show the effect the excessive alpha values can have on an image.
original image unsharpened
alpha = 1,5,9
Note the additional detail that we can observe as opposed to the unsharpened squirrel.

Part 1.2 Hybrid Images

We further generated hybrid images which are static images that change as a function of viewing distance. In order to present one image up close we ran a high pass filter over one and a low pass filter over the other in order to present the low_passed value up close. For the high pass filter we used the

im - gaussian_blur(im)
and for the low pass filter we used the
gaussian_blur(im)
The first image shows a blending between Derek and his cat. We also show a blending of color images where we attempt color for both high and low intensities. It appears that overall effect was much better when color was applied to both intensity streams as shown in the Bill Clinton images.
Derek
Nutmeg
Derek and his cat merged
Here it is in color:

The Log pass Filters:
Log Transform of Derek
Log Transform of Nutmeg
Log transform of the hybrid

And one more image of the merge of Chancellor Dirks and Carol Christ. Note that this merge was a failure case. The high frequencies overwhelmingly dominate especially with the presence of Dirks' glasses.

Here is an image merging JFK and Bill Clinton in Color. Note that under each image is the log transformation for that image. And the final image in Black and White:

Part 1.3: Gaussian and Laplacian Stacks

We first implement a Gaussian and Laplacian stack for a Salvador Dali image. Note how there is a progressive blending across the Gaussian stack and we tend to lose definition and sharp lines across the laplacian stack. For sigma values we used

2^n for for n in {1,levels of the pyramid}
thus creating a greater blend at each stage. The laplacian as described was used to take the difference of of the two gaussians above it. Note that we did not use downscaling here, thus the gaussians and laplacians produced were Laplacian and Gaussian Stacks rather than pyramids. The following is another Gaussian and Laplacian stack of Salvador Dali's Lincoln painting:
Untransformed Dali Painting
Transformed Dali Image

Part 1.4: MultiResolution Blending

To complete this step we first generated our gaussian and laplacia pyramids for each of the two images we sought to spline. We added the last level of the gaussian onto the end of the laplacian stack and then applied a mask to each level of the laplacian stack before merging them into the final image. For the Oraple blend with a regular mask, we simply used a half matrix of ones while in the irregular shape (we applied the mask generated from part 2.2 to try and blend a penguin into the background of an image. We used a kernel size of 70 as well as a sigma of 5 in order to generate these images.

Orange to be Blended
Apple to be Blended
Oraple Blend
Bill Clinton to be Splined
JFK to be Splined
A 50-50 Spline of Bill Clinton and JFK
Blend Image 1
Blend Image 2
Mask for the Blended Image
Final Blended Image with Irregular Mask

Part 2: Overview

The second part of the project was largely centered around using gradients rather than the absolute intensities to think about how we percieve images. We begin with the observation that we observe gradients far more than absolute pixel values and we can use this fact to essentially control our perception of images by maximizing or minimizing gradients between fields. We approach two examples: one of a small toy and the other of larger more complex objects to evaluate how we can blend objects effectively into eachother with minimal disturbance.

Part 2.1: Gradient-Domain Fusion

In this problem we sought to reconstruct the Toy image using gradient domain processing. In order to complete this task we minimized the Least sqaure error of

    {
        v(x+1,y) - v(x,y) - (s(x+1,y) - s(x,y))^2,
        ( v(x,y+1)-v(x,y) - (s(x,y+1)-s(x,y)) )^2,
        (v(1,1)-s(1,1))^2
    }
    
where the last condition specifies pinning a single point on the target and destination image so as to fix a solution overall. To compute this argmin, we construct an A matrix and a B vector where the A matrix represents the x and y-gradients of the original image and the b vector represents the original pixel intensities. We then solve the least squres form given by (Av-b)^2 for v which becomes our outcome image.
Original Im
Reconstruction

Part 2.2: Gradient-Domain Fusion

This problem used a modification of the algorithm in the previous part. We sought to minimize the blending constraint given by
The following shows the image generated using this approach by copying a penguin into a scene with skiers. Note how this approach compares to the approach from part 1.4. The penguin in this is far more accurately blended into the background than is the penguin from the 1.4. Fundamentally this approach relied on creating a large A matrix to represent the Mask as well as a b matrix to represent the pixel intensities. The A-matrix represented vertical and horizontal gradients between fields and basically computed the pixel values that would minimize the gradient for a given mask. These values were taken directly and put back into the original image. Masks were generated using Nikhil's code on piazza here. The first image is the penguin merged with the hikers. Note how the background is entirely removed and the penguin becomes slightly darker to match the background of the image.

Blend Image 1
Blend Image 2
Mask for the Blended Image
Final Blended Image
And another Similar Image with Poission Blending. Here we see how the penguin is again merged into the background of the seal.
Blend Image 1
Blend Image 2
Mask for the Blended Image
Final Blended Image
The following image was not as effective in being blended into the background most likely due to the strong differences between the source and target backgrounds. There is a strong green background and thus remnants of the original mask boundaries still remain in the output.
Blend Image 1
Blend Image 2
Final Blended Image. Note that there is still the remnant of the mask where the image was pasted in
The following shows a failed example of blending when we tried to blend one face into another using a mask over half of the first face. Likely because the gradients were sharply different and there wasn't necessarily a very uniform background to blend into, this approach was far worse than our multiresolution blending approach in part 1.4. We do observe however, how the skin tones are more even and tend to blend together more effectively. Because of the vast difference in the specific facial features (note the mouth line and forehead lines) the dividing line between the two images are more evident.
Clinton's Face to Blend
JFK's Face to Blend
Final Blended Image. Compared to Multi-Resoltion Blending in part 1.4 there is a substantial difference.
This shows another example of an effective blend into the background of an image. Note the darkening of the Brocolli in response to the changed environment.
Penguin to be merged into
Brocolli to Add to Image
Final Blended Image. Note the subtle blends into the background. The edges are entirely removed and the brocolli was darkened.