In this project, we explore different ways of analyzing an image, namely from the frequency and gradient domains, through image-blending.
Original
Sharpened
In this part, two images were taken, aligned, and then averaged together after applying a low-pass filter for one image (essentially blurring it), and a high-pass filter for the other (keeping high frequencies). The last row of images shows the images of John Denero, Paul Hilfinger, John filtered, Paul filtered, and their combination, respectively, in frequency space.
John
Paul
John filtered
Paul filtered
Combined
In this part, a Gaussian and Laplacian stack were created for two images. A Gaussian stack is a stack of images created by repeated applying a Gaussian filter over the previous image in the stack. A Laplacian stack is created by taking a Gaussian stack and taking the difference between each adjacent image in the stack. the last Gaussian is kept a the bottom of the Laplacian stack.
The idea in this part is to create a seamless blending to combine two pieces of two images. The algorithm is simple:
In this toy problem, we recreate an image by constructing and solving a system of equations using information about it's x- and y-gradients and an intensity constraint.
We can construct a matrix system, Ax = b, as follows:
where e is the equation counter, coord_to_index is a function that maps a row and column to the corresponding flattened index.
Original
Reconstructed
Finally, the best part of the project, gradient-domain blending. In this part, a source image is blended into a target image in a mask area. Inside the mask, we want the gradient of output image to be as close as possible to the source image. In addition, we impose the constraint that the pixels on the edge (and everywhere else) has to be the same as the target image. This can be expressed as a system of linear equations, which we can solve. The following equation is minimized using least-squares linear regression:
Shown below are multiple examples of gradient-domain blending. Included are the source, target, naive copy-and-pasted, and gradient-domain blended images.
As we can see from "Moderate Candidate", gradient-domain blending works much better than multiresolution blending. This is because when swapping faces, one of the most important aspects for a seamless blend is a consistent skin color, something that gradient-domain blending does really well with. Instead of transitioning colors, it completely changes the color of the source image to match the target. Instances like this, when we want the color of the source to directly match the target, is where gradient-domain blending will work much better than multiresolution.
Hilary
Trump
Naive
Multiresolution
Gradient-Domain
So when does gradient-domain blending not work so well? Looking at "Dreams of Oski", we see that perhaps simply performing the naive copy-and-paste would have yielded better results. This is a bad situation for using gradient-domain blending because this time, we do not want the colors to bleed over from the target image to the source. It is because of this, due to the sky background, that we end up with a ghost-like image of Oski. In addition, there is a large contrast between Oski's original background and the target background.
Naive
Gradient-Domain