Part 1: Frequency Domain
Part 1.1: Warmup
The unsharp masking technique is done by taking a blurred version (here created with a gaussian filter) of the original image and adding a multiple of the difference between the original and blurred version back to the original to "sharpen" the image.
The equation is sharpened = original + α(original - blurred)
Part 1.2: Hybrid Images
In this section we explore the idea of hybrid images. Hybrid images are created by using a high pass filter on an image, a low pass filter on another image, and outputting the average of the two. The high pass filter makes it so that the first image can then only be seen close up, as high frequencies dominate perception when available. The low pass filter blurs the second image so that when close up you mainly notice the first image, but when far away, you can see the low frequencies of the second image. Thus, what you perceive from the hybrid image depends on the distance from which you view it at.
Fourier Analysis - But it was me, Dio!
Other Results
This last result can be considered a failure because both nearby and far away the second image dominates for the most part (besides the face being visible close up for image 1). The images are of different quality, and the bodies are of different length, which makes it hard to align properly. Most importantly, the second image has Obama wearing a black suit, which makes it near impossible to see image 1's different hand position due to suit color.
With regard to using color, it seems like using color for the low pass filtered image is better because the high pass filtered image is mostly edges, so it does not support color well, while the low pass filtered image allows the color to be seen both close and far, but the close up color does not work as well if the two original images are of different color.
Part 1.3: Gaussian and Laplacian Stacks
In this section, we create Gaussian stacks by blurring images repeatedly along each layer of the stack, and then take the differences between the layers to create the Laplacian stack. On top of the difference images, we add the final image of the Gaussian stack to the Laplacian stack, which allows us to recreate the original image by summing the images of the Laplacian stack. What should be noticed in the following images is that the key features seen when seeing the image from a farther distance are most notably in the last layers of the stacks, while the features seen when close up are featured in the first layers of the stack. For example, the lady in Dali and Joseph Joestar can be seen more clearly in the first Laplacian layer, while Mona Lisa's "smile" and Dio Brando can be seen more clearly in the second to last Laplacian layer.
The Laplacian stack images are normalized to gray for better visibility and display
Lincoln in Dalivision
Mona Lisa
Slave Market with the Disappearing Bust of Voltaire
Stack of Hybrid Image - But it was me, Dio!
Part 1.4: Multiresolution Blending
The goal of this section is to blend two images together seamlessly. To do so, we apply Gaussian stacks of a filter to Laplacian stacks of the two images, and by summing them back up together, we take advantage of Laplacian stacks adding up to recreate the original image while making the seam less noticeable through the Gaussian stacks of the filter.
Oraple
Gandore or Dumbledalf
Gandore or Dumbledalf - Laplacian Stack
Images are normalized to gray for better visibility
Irregular Mask
The "Disappearance" of Haruhi Suzumiya
Phelps' Dip in Venice
Part 2: Gradient Domain Fusion
In this part of the project, we focus on gradient domain processing through the technique poisson blending. When posed with the problem of blending a source object into a target background, one of the main concerns is making sure there is a seamless transition between the two images while maintaining consistency within each image. We can formulate this as the following least squares problem:
The first part of the equation asks that we minimize the difference in gradient of the pixels that we fill into the source area from the original source, as minimizing gradient allows us to change pixel values while allowing the blended in area to be perceived as the same as before. The second part of the equation asks that we minimize the gradient at the edges between the source area and the target image to provide a seamless transition. Solving this optimization problem gets us v, which is a vector containing the pixel values that will be filled into the area of the target image where we want the source to be.
Part 2.1 Toy Problem
As seen above, applying gradient domain processing with the target and source to be the same returns the same image.
Part 2.2 Poisson Blending
Kiki Plays Quidditch
This was created using the technique above. One key thing when creating the matrix for the least square problem is to only have a row there if it fully expresses the optimization equation, because if you don't have both the positive and negative term to represent a gradient between two pixels, having just one encourages that pixel towards 0 to minimize the objective function. Notice that in the blended version, while Kiki is slightly different colored, it is a minor change when compared to the dark blue areas around Kiki blending into the green background, which is obvious when comparing to the direct copy.
Macron's Revolution
Cooking with Chef Ramsay
Chef Ramsay Yelling at Anime
Kiki's Quidditch 2
Most failures in using poisson blending are relative. For example, the last two examples can be considered failures because the sources images are overly distorted compared to the original source. This is due to the difference pixel values between the edges of the source and target being too much, lead to the gradient being "carried over" to within the source image more than it should in order to keep the gradient between source and target smooth, while keeping the gradient within the source relatively consistent. In Chef Ramsay Yelling at Anime, that leads to the source being too dark, while in Kiki's Quidditch 2, it leads to the source's face looking like it is being directly shined upon by the sun.
Comparing Multiresolution Blending to Poisson Blending
The "Disappearance" of Haruhi Suzumiya
Phelps' Dip in Venice
It appears that multiresolution blending is better at keeping the majority of the pixel values of the source image the same because it just smoothes the edges. On the otherhand, poisson blending may alter the source image's pixel values, but does a better job of blending the color and texture so that there is a smooth transition between the target and the source.
In both of these cases, it appears that poisson blending works better than multiresolution blending because the result depends more on a smooth texture between the source and target more so than maintaining the same colors within the source. This is showcased in the Phelps/Venice example, as we can see that grayscale multiresolution blending works better than its colored counterpart because the water color being blended seamlessly is less important in grayscale, but poisson blending works better in the colored version because it can blend the water together.
Reflection
The most important thing I learned from this project is that in computational photography there is no right answer. While there is getting the algorithm right, there are tradeoffs that need to be subjectively decided upon, like parameter tuning, type of algorithm, and photos used. By devoting more time and thinking openly and creatively, it is more likely to get results that you and your audience can enjoy.