Christine Zhou, cs194-26-act
In this part, I "sharpened" an image using an unsharp mask. I took an image and convolved it with a 2D Gaussian that was created using some sigma value, and this resulted in a blurred image. Then, I subtracted this blurred image from the original to get "details" from the image. Then, I added the "details" back to the original image to get a "sharpened" original image. The original and the corresponding sharpened image is displayed below:
The sigma value used when creating the Gaussian is 2.
In this part, I combined the low frequencies of one image with the high frequencies of another image in order to get the resulting image. The resulting image, when looked at from a distance, looks like the low frequency image. However, when up close, it looks like the high frequency image.
To get our image, I used similar techniques to what I did in part 1.1 and blurred the image that I wanted to see at a distance using convolution with a 2D Gaussian and sharpened the image that I wanted to see up close using convolution with a 2D Gaussian. Then, I averaged the pixels together to get a mix of the low and high frequency values.
Examples of the original images and the blended image are shown below.
Here is the corresponding 2D Fourier transform of the above image:
Here is another image of me and a fish.
Below is an example of images that weren't blended very well together...
In this part, I implemented a Gaussian and Laplacian stack on a given image, which means that the image is manipulated in some way at each level. To create the first level of the Gaussian stack, we convolved the image with a 2D Gaussian generated with some sigma value. To get the next levels of the Gaussian stack, I would convolve the previous level's Gaussian stack image with the same 2D Gaussian.
To create the first level of the Laplacian stack, I got the original image and subtracted the Gaussian from the first level of the Gaussian stack. To get the next levels of the Laplacian stack, I would get the corresponding level of the Gaussian stack and subtract the previous level of the Gaussian stack from it. For the last level of the Laplacian pyramid, I would use the image of the last level of the Gaussian stack.
For the following images, I chose a stack of height 5.
The Gaussian stack levels are shown for the painting, Gala Contemplating the Mediterranean Sea which at Twenty Meters Becomes the Portrait of Abraham Lincoln-Homage to Rothko. It is then followed by the Laplacian stack levels.
The Gaussian stack levels are shown for the hybrid image of me and the fish, followed by the Laplacian stack levels.
In this part, I used the Gaussian and Laplacian stacks from part 1.3 along with a mask to create multiresolution images. Multiresolution images are smoothly blended along the mask, since the images are blended together at each level of the given stacks. Each level of the stack represents a different frequency of the image.
To get the images, I calculated the Gaussian and Laplacian stacks for both images and the mask. After this calculation, for each level of the stack I multiplied the Gaussian of the mask by the Laplacian of the first image and added it to 1 - the Gaussian of the mask multiplied by the Laplacian of the second image. This combined each image with the mask at each frequency, and adding them together combined each masked image with each other. Then, all these masked images were summed together to get the resulting seamlessly blended image.
The original image of an apple and orange are shown below, followed by the combination of the two, called an "orapple".
Another example I did with a regular mask is a blending of my twin sister and I. Original images are shown below, following by the multiresolution images of us.
Here's an example of a multiresolution image with an irregular mask:
In this part, I implemented Poisson blending, which helps us blend a source image into a target image. When directly copying the pixels, the seam is usually very defined, so I chose to blend the source into the target by changing the intensity and color of the source image pixels to align more closely with the target image pixels, while preserving the original gradients of the source image.
As a sanity check, I reconstructed the toy example, as shown below:
To do this, I set up a system of equations to be minimized. We wanted the
gradients of the source image to be as close as possible to what was displayed
in the output image, so we wanted to minimize the difference between
(output(i, j) - output(i, j + 1)) - (source(i, j) - source(i, j + 1)) and
(output(i, j) - output(i + 1, j)) - (source(i, j) - source(i + 1, j)), which
represent the gradients in the x and y directions. We also wanted the top left
corner of both images to be as close as possible, so we wanted to minimize the
difference between (output(1, 1) - source(1, 1). This totaled in
height * width + (height - 1) * width number of equations, with each pixel
being an unknown resulting in height * width number of unknowns. As a result,
we created a (height * width + (height - 1) * width, height * width) matrix
A that contained all the coefficients of the equations and a
(height * width + (height - 1) * width, 1) vector b that contained all the
values. We then used least squares to solve for each pixel value, and
constructed our image from that.
In this part, we used the same tactic that we did in part 2.2 but took into account the source image, the target image, and whether the pixel was inside the region that we wanted blended (whether it was inside the mask or not).
We wanted to minimize the change in gradient from the source image to the new
output image, so we again wanted to use least squares to solve for the pixel
values. However, in this example, we wanted to look at the neighbors of each
pixel (called the "4-neighbors") to compute the gradients. In addition, we took
into account whether the neighbor was inside the region of pixels that should be
from the source or not. If the neighbor was in the region that was in the
source, then we would want to preserve the gradient of the source image.
However, if the neighbor was not in the region that was in the source, then we
would want blend the source and the target together to remove the seam. As a
result, the gradient that we wanted to preserve was t(j) - s(i), which
represents both the target and source being taken into account.
I used a matrix of size (number of pixels * 4, number of pixels), since each
pixel had four equations representing looking at all the neighbors. We wanted to
solve for each pixel in the region, so the number of unknowns was the number of
pixels itself. The vector was of size (number of pixels * 4, 1).
Below shows an example of combining Ootori-sama from Spirited Away into a stock photo of a sun. The first image is directly copying the pixels from Ootori-sama into the sun, while the second image is blended using Poisson blending.
Here's another example of Poisson blending:
Here's an example of a failure case of Poisson blending, where the boundaries are still visible:
In part 1.4, I blended myself with the CalHacks bear using Laplacian blending. The output is shown below:
However, I decided to blend the same two images using Poisson blending instead. The output is shown below:
The approach that worked the best for this was the Poisson blending, since it does a better job of preserving the gradients of the original image. The Laplacian blending does not preserve the original image's gradients as well, and that can be seen in the corresponding output where the top seam of the bear is not blended very well into the shadow of my face. On the other hand, the image created with Poisson blending preserves the gradients of the original image to blend the bear into my face very seamlessly.
It seems that the Laplacian blending is good when we aren't trying to blend an image into another image. If preserving the original image's gradients are not important, then it is okay to use something like Laplacian blending where it ensures that the seam is smooth along the mask's edges but does not care about the gradients. Poisson blending is good when we are interested in combining two images together but preserving the gradients of the original image.