Fun with Frequencies and Gradients!

By Jason Xie

Part 1.1 Warmup: Sharpen an image

Sharpening an image can be done in three steps:
  • Smooth the image using the gaussian
  • Subtract the smoothed image from the original to get the details
  • Add the details back to the orignal image

    • Original Picture

    Blurry Picture

    Detailed Picture

    Sharpened Picture

    Problems Encountered:

    The biggest problem that I encountered during this was dealing with RGB values of the image. When I first started I would get very strange tints in the detailed picture. I solved this by switching over to LAB color style before doing any transformation, then switching back to RGB to save the image.


    Part 1.2 Hybrid Images

    Hybrid images can be created by taking a low-pass filter to one image, a high-pass filter to another, and combining them together. The low-pass image should be visible when the image is far away, and the high-pass image is more visible when the image is close up. Here are some I made:

    Nutmeg (High-pass)

    Derek (Low-pass)

    Nutmeg-Derek

    Teddy Bear (High-pass)

    Emma (Low-pass)

    Teddy-Emma

    Teddy Bear (High-pass)

    Dwarf (Low-pass)

    Teddy-Dwarf



    Frequency Analysis:

    Below are the fft frequencies for the Teddy-Emma hybrid

    Problems Encountered:

    The primary problem I encountered here was finding images that would go well with each other, and then trying to align them. This is most obvious in my combination of Derek and Nutmeg since the cat's facial structure doesn't align very well with Derek's face.


    Part 1.3 Gaussian and Laplacian Stacks

    Gaussian and Laplacian stacks involve continually applying the Guassian and Laplacian filter to an image. This shows interesting results on the hybrid images and on Dali's mixed painting because different aspects of the picture will stand out after a few iterations. In the Gaussian stack, we can see that Lincoln becomes more and more visible but the lady disappears. In the Laplacian stack, Lincoln starts to fade and the lady becomes a little more clear. The same holds for the hybrid images.

    Dali Original

    Gauss Stack 1

    Gauss Stack 2

    Gauss Stack 3

    Gauss Stack 4

    Gauss Stack 5

    Dali Original

    Laplac Stack 1

    Laplac Stack 2

    Laplac Stack 3

    Laplac Stack 4

    Laplac Stack 5

    Hybrid Original

    Gauss Stack 1

    Gauss Stack 2

    Gauss Stack 3

    Gauss Stack 4

    Gauss Stack 5

    Hybrid Original

    Laplac Stack 1

    Laplac Stack 2

    Laplac Stack 3

    Laplac Stack 4

    Laplac Stack 5


    Problems Encountered:

    The primary problem encountered here was figuring out how to display the lapalcian correctly. Because I wasn't using the LAB color system, I realized I had to normalize the values that came out before saving the image to disk. I also had to clip the image to -1, 1 because of some errors.


    Part 1.4 Multiresolution Blending

    Multiresolution Blending uses some mix of Guassian and Laplacian stacks as well as a transition gradient mask in order to blend two images along a seem. I programmed mine to connect two images horizontally across a vertical seem. Here are some images below:

    Apple (Left)

    Orange (Right)

    Oraple

    Emma (Left)

    Teddy (Right)

    Emma-Teddy

    Sky (Left)

    Night (Right)

    Sky Night


    Problems Encountered

    I originally made Lapalcian stacks by continually calling the Laplacian operator onto the image, but that resulted in blended images that were very harsh because the detail kept being enhanced each layer. This was solved by taking the Laplacian of different Gaussian smoothed layers. Here is an example of an overly detailed blended image vs the fixed blend image:

    Part 2.1 Toy Gradient Domain Fushion

    The toy problem involved turning an image into two components, a sparse matrix A and a vector b, to solve the least squared problem, which is equivalent to solving Ax=b for x. The components are calculated by finding the gradient along the x and the y seperately, noting pairs (x,y), (x+1, y) or (x, y+1) into rows for A, and the difference in the pixel value of pairs into a rows for b. Finally, in the toy example, the vector x that we solve for at the end should give us back our original image.

    Note: A has dimensions (2*m*n+1, m*n), x has (m*n, 1), b has (2*m*n+1). And we convert x back to a (m, n) matrix to save the image

    Original

    Gradient Calculated


    Problems Encountered

    The biggest problem I encountered was figuring out a way to solve Ax=b for very large dimensions, since (2*m*n+1, m*n) was very large and hard to calculate quickly. Luckily I saw a piazza post recommending to use scipy.sparse.linalg.lsqr which worked very well for me, and ran the toy example in about two minutes.


    Part 2.2 Poisson Blending Gradient Domain Fushion

    This was an extension of the toy version to incoporate two pictures. It involved taking a selection of one picture and pasting it onto another, but making it so that the colors blend with one another. It also involves creating an equation for Ax=b. However, this time the rows of A are dependent on the Poisson neighbors, with each interior pixel as 4 and the neighbors as -1 inside the mask. Also based on the mask, we can calulate rows of b based on the source image, these together will blend the hues of the source image to match the target image.

    Source

    Target

    Penguin Hiking

    Source

    Target

    Penguin God

    Source

    Target

    Fruit Cat


    Problems Encountered:

    Also, I had difficulty using the provided MATLAB, so my masks were generated by a very simple rectangular crop function. But, this made the images not seem too well integrated because the mask covered a lot of the source's background instead of taking the background of the target. This can be seen in the second image where the penguin in the sky still has flecks of snow instead of pure sky.

    Also there was a problem when the images don't have similar backgrounds as seen in the Fruit Cat picture, because of the spotted background in the apple picture that doesn't go away with the gradients.