Fun with Frequencies and Gradients
Utkarsh Singhal
Part 1: Frequency Domain
Part 1.1: Warmup
Unsharp mask is quite straightfoward, and is calculated by subtracting the blurred image from the original. While this method doesn't actually improve resolution of the image, it can create the illusion of sharp edges by enhancing local changes. For example, hair details are much more notieable in the unsharpened image below.
Part 1.2: Hybrid images
In order to create fourier hybrid of two images im1 and im2, I low pass im1, high pass im2, and add the resultants together. The process and the results are shown below
Derek and Nutmeg
Snoop-Dog
Cheetah and puppy
(Failure) Giraffe and horse
Frequency domain for giraffe and horse
Horse and giraffe composite fails because there is no proper alignment due to structural differences between giraffe and horse skeletons. This causes unpleasant mis-alignments in the image
Part 1.3: Gaussian and Laplacian stacks
Displayed below are gaussian and laplacian stacks for two images: The Dali lincoln art, and a replica of Mona Lisa made using lego pieces
These stacks are especially useful in understanding how fourier hybrid images work. The laplacian stacks, for example, decompose information into different frequency bands. In the following image, one can see that the cat is contained almost entirely in high-frequency bands, while Derek is in low-frequency bands
Part 1.4: Multi-resolution blending
Displayed below are two images blended using irregular masks, and two images blended using conventional masks
Oraple
Corapple (orange in the core of an apple) [Irregular mask]
Bat-Yann [Irregular mask]
(Failure) Skylines
The image size makes it difficult to see, but two buildings on the image boundaries blur into each other, making the Skylines blend a failure.
Gradient Domain Editing
As we saw in the class, gradient domain is an incredibly powerful image editing tool. This project revolves around using gradient domain techniques to do image blending, tone mapping, and multi-channel de-speckling. The general method is to pose the problem as a poisson equation, and then to use least squares to solve for pixel values. Using different gradient values in the poisson equation gives drastically different results.
Part 2.1: Toy Problem
The goal of this part is to reconstruct an image enitrely using its gradients. In my formulation of the least squares matrix, my boundary conditions are just zeros, and the gradients are equal to gradients from the target image. The reconstruction is identical to the original image.
Part 2.2: Gradient Domain Fusion
Blending in the gradient domain is much more adaptable compared to laplacian blending when the images being blended have different background color and textures. Since the gradients are from one image, and the boundary is from another image, the method automatically adjusts the color of the source path to match its surroundings.
The last blend is a failure mode. Neighborhood of the rope is not handled well due to improper cropping, and the significant disparity between colors in the source vs. target image causes noticeable amount of desaturation in the overall image as the pasted 'patch' ends up much more saturated than the boundaries. This is an artifact of the normalization process I use before saving images, so it can be corrected, but the color disparity will still cause problems.
Here is a comparison between poisson and laplace blending for the skyline images:
Poisson blending performs much better by equalizing the sky colors, and also blending the building structures much better.
(Bells and Whistles) Gradient Mixing
Gradient mixing in the context of this project refers to combining gradients from both the source image and the target image while formulating the poisson equation. If the maximum of two gradients is chosen, the resulting image retains textures from target areas where there is no texture in the source area. This is particularly helpful when blending objects with plain background into target images with rich structure. In the following example, gradient mixing preserves the target image texture in areas where the source image doesn't have the penguin.
(Bells and Whistles) Tone Mapping
The most common method for converting a color image to grayscale is to simply retain the luminosity values and discard other information. While it suffices in general usage, one of the biggest weaknesses faced by this method is that it cannot differentiate between different colors that are equally bright. This problem of mapping color shades to grayscale values is an example of tone mapping, and in this part of the project I approach tone mapping using gradient domain techniques. In particular, I mix gradients from hue and saturation gradients by taking their max. For the boundary conditions, I just use 0.
(Bells and Whistles) De-speckling Images from Russian Empire
In Project 1, we had to reconstruct photographs taken in early 1900s. One of the most noticeable artifacts in these images were bright colored speckles that were (probably) caused by dust or damage to the glass plates. Fortunately, each channel corresponds to a different glass plate, the speckles are contained to only one channel at a time. This means we can mix gradients from other two channels to fix those spots. To be precise, I use the other 2 channels to get a grayscale image, whose gradients I mixed with the original image by taking the minimum. It gives surprisingly good results in some cases, but in other cases, it can lead to de-saturated images (for example, in lady with lambda=2, the blush on her cheeks reduces a lot).
