Part 1: Frequencies

Part 1.1: Warmup

My pictures are the sharpest, simply the greatest. There are non quite as sharp as mine. I only use the best, most cutting edge methods to sharpen my images. You're wondering, what do I do to sharpen my images? My techniques are the best, the greatest. I use the unsharp mask method to create the sharpest images.

Translation: We can sharpen images using the unsharp mask method. The unsharp mask is created by applying a gaussian filter to an image and then subtracting the result of filtering from the original image. Since a gaussian filter acts like a low pass filter, we are left with just the high frequencies, or edges. We then add these edges back to "sharpen" the image.

Part 1.2: Hybrid Images

Bells and Whistles: Color

The best images, in my honest opinion are hybrid images. Hybrid images are so advanced that they are two images in one. You can not find that anywhere else. I've read the science behind it and it's the most difficult, complicated science. There are fast pixels (high frequency) and slow pixels (slow frequency). When you look at it close up, you can see the slow pixels but when you look at it from far away you can see the fast pixels. It's the most advanced images I know.

Translation: We can create hybrid images by leveraging the limits of human perception. There are high frequencies and low frequencies in images. High frequencies correspond to edges and sharpness, while low frequencies represent the non edges and blurriness. From further away, we are unable to perceive frequencies and low frequencies begin to seem like high frequencies. Thus, if we layer a high frequency image with a low frequency background, we can create a hybrid image that changes based on the distance which we perceive it at.

Part 1.3: Gaussian and Laplacian Stacks

My favorite painting is Lincoln in Dalivision, by Dali. There is no one who loves this painting more than me. The absolute best way to see this painting, in my honest opinion, is through Gaussian and Laplacian stacks. These stacks are the most advanced in stack technology. They let you see the fast pixels and slow pixels by splitting the image into different layers.

Translation: Gaussian stacks are created by repeatedly applying a gaussian filter to an image and storing the results in a stack. Similarily, laplacian stacks are created taking the difference of the image and the gaussian stack. These are useful in multi-resolution blending as we can blend various frequencies separately rather than blending the entire image at once. It also helps us visualize the various frequencies that exist within an image.

Part 1.4: Multi Resolution Blending

Bells and Whistles: Color

Mutli resolution blending is the best way to blend two images. Simply the best. It uses the absolutely most advanced technology. It uses Laplacian stacks and Gaussian stacks, the most advanced forms of stack algorithms and an image spline to get the best seam between images. My favorite way also uses a Gaussian mask to lightly distort the seam. It is the greatest way to combine images.

Translation: Multi-resolution blending utilizes gaussian and laplacian stacks to blend across different frequencies in images. Instead of blending the entire image at once, we can split the image into its basis frequencies and customize the blend at each level of the stack. This results in smoother and more realistic blending.

Part 2: Gradients

When we combined Lips with Trump, we saw some of the shortcomings of multi-resolution blending. Essentially the difference in skin tone created a noticeable gradient in the blended region, which detracts from the effect of blending. A different way to approach this problem is inspired by the following insight: people often care much more about the gradient of an image than the overall intensity. So we can set up the problem as finding values for the target pixels that maximally preserve the gradient of the source region without changing any of the background pixels. It's possible that ignoring intensities may distort colors, but the benefit of optimizing over the gradient domain is a more seamless image. Part 2 explores gradient domain processing; we will focus on "Poisson blending".

Part 2.1: Toy Problem

Before attempting Poisson blending, we will construct a toy problem: compute the x and y gradients from an image s, then use all the gradients, plus one pixel intensity, to reconstruct an image v. In other words, we want to solve the following optimization problem:

Equation 1 attempts to minimize the squared error between the x gradients of the reconstructed and source images. Equation 2 attempts to minimize the squared error between the y gradients of the reconstructed and source images. Since equation 1 and equation 2 can be solved by adding any constant value to v, we use equation 3 to match the top left pixel of the reconstructed image and the source image.

Part 2.2: Poisson Blending

The first step is to select the source and target regions. I select the boundaries of a region in a source image and specify a location in the target image where it should be blended. Then, I transform the source image so that the indices of pixels in the source and target regions match. To do this I used Jun-Yan's provided starter code.

The second step is to solve the blending constraints as defined poisson blending. This is similar to the toy problem, where we want to match the gradients (top, bottom, left, right) between the reconstructed image and the source region. A key difference in the optimization constraints is the existence of pixels outside the mask range. To handle these pixels, we use the corresponding target image pixel to minimize the difference between gradients of the reconstructed and source image. I used sparse matrices to optimize runtime and a least squares solver to solve the optimization problem.

After solving the optimization problem, the last step is to simply copy these reconstructed pixels into the target input location. If done correctly, this should result in a smoothly blended image. For displaying the results, the first row contains the target, the source, and the mask in that order. All combinations include a side by side comparison of direct copying with poisson blending. The last combination of Trump and Lips includes the multi-resolution blending as well.