CS194-26
Project 3: Fun with Frequencies and Gradients!
Sharabesh Ramesh
Overview
This project fundamentally focused on developing the
pixel manipulation techniques in order to merge images seamlessly.
In the first part of the project we used Gaussian and Laplacian filters
to create static images which changed with viewing distance, as well
as blended images. In part 2 we used a least squares solution
in order to minimize gradients between images thus ensuring that
image gradients between a target and source matched as closely
as possible. Overall this `Poission blending` created far more
effective blends than the corresponding method from part 1.
Part 1.1: Warmup
Here we sought to sharpen an image by using Gaussian filters.
Using the form
im + alpha * (im - gaussian)
we
show variants on levels of sharpness using the base image. Note that the
alpha parameter hear specifies the degree of sharpening. We include an extreme
example in order to show the effect the excessive alpha values can have on
an image.
original image unsharpened
|
alpha = 1,5,9
|
Note the additional detail that we can observe as opposed to the unsharpened squirrel.
Part 1.2 Hybrid Images
We further generated hybrid images which are static images that change as a function of viewing distance.
In order to present one image up close we ran a high pass filter over one
and a low pass filter over the other in order to present the low_passed value up close.
For the high pass filter we used the
im - gaussian_blur(im)
and
for the low pass filter we used the
gaussian_blur(im)
The first
image shows a blending between Derek and his cat. We also show a blending
of color images where we attempt color for both high and low intensities.
It appears that overall effect was much better when color was applied to
both intensity streams as
shown in the Bill Clinton images.
Derek
|
Nutmeg
|
Derek and his cat merged
|
Here it is in color:
The Log pass Filters:
Log Transform of Derek
|
Log Transform of Nutmeg
|
Log transform of the hybrid
|
And one more image of the merge of Chancellor Dirks and Carol Christ.
Note that this merge was a failure case. The high frequencies
overwhelmingly dominate especially with the presence of Dirks' glasses.
Here is an image merging JFK and Bill Clinton in Color. Note that
under each image is the log transformation for that image.
And the final image in Black and White:
Part 1.3: Gaussian and Laplacian Stacks
We first implement a Gaussian and Laplacian stack for a Salvador Dali image.
Note how there is a progressive blending across the Gaussian stack and
we tend to lose definition and sharp lines across the laplacian stack.
For sigma values we used
2^n for for n in {1,levels of the pyramid}
thus creating a greater blend at each stage. The laplacian as described
was used to take the difference of of the two gaussians above it. Note
that we did not use downscaling here, thus the gaussians and laplacians produced
were Laplacian and Gaussian Stacks rather than pyramids. The following is another
Gaussian and Laplacian stack of Salvador Dali's Lincoln painting:
Untransformed Dali Painting
|
Transformed Dali Image
|
Part 1.4: MultiResolution Blending
To complete this step we first generated our gaussian and laplacia
pyramids for each of the two images we sought to spline. We added the
last level of the gaussian onto the end of the laplacian stack and then applied
a mask to each level of the laplacian stack before merging them into the final
image. For the Oraple blend with a regular mask, we simply used a half matrix of ones
while in the irregular shape (we applied the mask generated from part 2.2 to
try and blend a penguin into the background of an image. We used a kernel size of 70
as well as a sigma of 5 in order to generate these images.
Orange to be Blended
|
Apple to be Blended
|
Oraple Blend
|
Bill Clinton to be Splined
|
JFK to be Splined
|
A 50-50 Spline of Bill Clinton and JFK
|
Blend Image 1
|
Blend Image 2
|
Mask for the Blended Image
|
Final Blended Image with Irregular Mask
|
Part 2: Overview
The second part of the project was largely centered around using gradients
rather than the absolute intensities to think about how we percieve images.
We begin with the observation that we observe gradients far more than
absolute pixel values and we can use this fact to essentially control
our perception of images by maximizing or minimizing gradients between
fields. We approach two examples: one of a small toy and the other
of larger more complex objects to evaluate how we can blend objects
effectively into eachother with minimal disturbance.
Part 2.1: Gradient-Domain Fusion
In this problem we sought to reconstruct the Toy
image using gradient domain processing. In order
to complete this task we minimized the Least sqaure error
of
{
v(x+1,y) - v(x,y) - (s(x+1,y) - s(x,y))^2,
( v(x,y+1)-v(x,y) - (s(x,y+1)-s(x,y)) )^2,
(v(1,1)-s(1,1))^2
}
where the last condition specifies pinning a single
point on the target and destination image so as to fix
a solution overall. To compute this argmin, we construct
an A matrix and a B vector where the A matrix represents
the x and y-gradients of the original image and the b vector represents
the original pixel intensities. We then solve the least squres form
given by (Av-b)^2 for v which becomes our outcome image.
Original Im
|
Reconstruction
|
Part 2.2: Gradient-Domain Fusion
This problem used a modification of the algorithm in
the previous part.
We sought to minimize the blending constraint given by
The following shows the image generated using this approach
by copying a penguin into a scene with skiers. Note how this
approach compares to the approach from part 1.4. The penguin
in this is far more accurately blended into the background
than is the penguin from the 1.4. Fundamentally this approach
relied on creating a large A matrix to represent the Mask as well
as a b matrix to represent the pixel intensities. The A-matrix
represented vertical and horizontal gradients between fields
and basically computed the pixel values that would minimize the
gradient for a given mask. These values were taken directly
and put back into the original image. Masks were generated using
Nikhil's code on piazza here.
The first image is the penguin merged with the hikers. Note how the background is
entirely removed and the penguin becomes slightly darker to match the background
of the image.
Blend Image 1
|
Blend Image 2
|
Mask for the Blended Image
|
Final Blended Image
|
And another Similar Image with Poission Blending. Here we see how the penguin
is again merged into the background of the seal.
Blend Image 1
|
Blend Image 2
|
Mask for the Blended Image
|
Final Blended Image
|
The following image was not as effective in being blended into the background
most likely due to the strong differences between the source and target
backgrounds. There is a strong green background and thus remnants of the original
mask boundaries still remain in the output.
Blend Image 1
|
Blend Image 2
|
Final Blended Image. Note that
there is still the remnant of the mask where the image was pasted in
|
The following shows a failed example of blending when we tried to blend one face into another
using a mask over half of the first face. Likely because the gradients were
sharply different and there wasn't necessarily a very uniform background
to blend into, this approach was far worse than our multiresolution blending approach
in part 1.4. We do observe however, how the skin tones are more even and tend
to blend together more effectively. Because of the vast difference in the specific
facial features (note the mouth line and forehead lines) the dividing line between
the two images are more evident.
Clinton's Face to Blend
|
JFK's Face to Blend
|
Final Blended Image. Compared to Multi-Resoltion
Blending in part 1.4 there is a substantial difference.
|
This shows another example of an effective blend into the background of an image.
Note the darkening of the Brocolli in response to the changed environment.
Penguin to be merged into
|
Brocolli to Add to Image
|
Final Blended Image. Note the subtle
blends into the background. The edges are entirely removed and
the brocolli was darkened.
|