CS194-26: Project 2

By: Calvin Chen

Part 1.1

For this part of the project, we analyzed different edges of a photo of a cameraman, and used techniques from class to construct edge detectors using convolutions in the X and Y directions.

Photo of cameraman

In order to determine which edges existed, we convolved the image with filter in the X-direction ([1, -1]), and in the Y-direction ([[1], [-1]]). Afterwards, the magnitudes for different edges were determined using the formula: magnitude = np.sqrt(d_x 2 + dy 2). This way, a threshold could be introduced for the magnitudes to determine which edges really existed in the camera and which parts were just noise.

Gradients in X

Gradients in Y

Gradients in both directions

Part 1.2

For this part, I looked at reducing more of the noise in the image by adding some Gaussian convolutions to the image itself. This way, it would be able to smooth out the noise in the image, and prominently bring out more of the edges in the image better. Afterwards, the same process was applied from part 1.1, with the application of convolutions in the X and Y directions.

Gradient Image after applying 8x8 Gaussian Filter with sigma 2

In order to prove the associativity of convolutions, the Gaussian filter was applied to the derivative filters first, and the result was then applied to the image itself.

Although the image doesn't render the same coloring, the edges are still really well detected with the same set of convolutions, with all the excess noise removed.

Part 1.3

For this section, I explored the idea of using edge detectors to automatically straighten images. By calculating direction of the gradient for each pixel, I could determine which pictures were straightened by returning the rotated images that maximized the number of horizontal and vertical edges within the image. This process was conducted like the following:

  1. For each possible angle rotation (between -10 and 10 degrees), rotate the image by that much
  2. Crop off around 0.05 in the horizontal and vertical directions to remove any borders/edges created from rotations.
  3. Calculate the magnitude of the gradient like before for each pixel (magnitude = np.sqrt(d_y 2 + d_x 2)).
  4. Filter out pixels under a certain magnitude and plot a histogram for all the remaining angled directions of edges (angle = arctan(d_y/d_y)).
  5. Select the rotation that maximized the number of edges in the vertical and horizontal directions (0, 90, 180, 270).

However, this process wasn't always successful. In some cases, since there's a lot of extraneous information and edges that weren't originally vertically or horizontally aligned to begin with, the algorithm isn't as able to determine how exactly to orient the image. It was much better than the original, but not completely able to determine orientation.

Part 2.1

For this section, I used Gaussian filters to sharpen images. By applying a low-pass filter onto the images, and then subtracting the original image by this low-pass filter, a high pass filter could be introduced for the image. Then, adding the high-pass filter multiplied by an alpha value (in this case 2), I was able to construct visually sharper images.

Original Taj Mahal

Sharpened Taj

Skyline

Sharpened Skyline

Part 2.2

For this section, I worked on constructing hybrid images. This could be done by ensuring high frequencies were all aligned with one image, and low frequencies were aligned with a different one. That way, when looking at the image close up, you would be able to see much more of the high frequencies than the low ones, and from further away, you the lower ones would show up more prominently. I used a 20x20 Gaussian filter with varying sigmas to convolve the images together. This was done by separately convolving both with their respective sigma values, subtracting the convolution from one of the original images to construct the high pass filter, and then adding the result to the result of convolving the other image and clipping the range to 0 to 1.

Additionally, I provided a fourier analysis for the first set of images to showcase how the fourier domain changed through different filterings.

Please select 2 points in each image for alignment.

Justin Bieber

Hybrid

Fourier analysis

Other Examples

Please select 2 points in each image for alignment.

Girl 1

Girl 2

Hybrid

Please select 2 points in each image for alignment.

Nutmeg

Derek

Hybrid

Please select 2 points in each image for alignment.

Bagel

Puppy

Hybrid

Part 2.3

For this part of the project, I implemented Gaussian and Laplacian stacks. In order to create these stacks, I continuously applied a 40x40 Gaussian filter with sigma value 5 for 5 iterations onto the image to make the Gaussian stack, and I took the difference between the new image and the previous to make the Laplacian stack (and appended the last Gaussian convolution to make both stacks length of 6. The images are shown below:

Lincoln

Lincoln Gaussian Stack

Laplacian Lincoln Stack

Other Examples

Miley Cyrus and Justin Bieber

Gaussian Stack

Laplacian Stack

The above Laplacians look quite creepy, but was still able to distinguish the two separate photos in the image.

Part 2.4

For this section, I blended two images seamlessly. A bad seam can be seen like below:

Now, in order to create a better seam, I used the following steps and techniques from the above sections to create a more seamless transition between the images.

  1. Build the Laplacian stacks LA and LB for images A and B respectively.
  2. Construct a Gaussian stack GM for the mask (M) used to transition between the two images.
  3. Create a combined stack with multiple layers by summing weighted levels of LA and LB by GM (like the following):

$LS_{level} = LA_{level} * GM_{level} + LB_{level} * (1 - GM_{level})$

  1. Combined the different layers together and rescale them to fit into the 0 to 1 pixel range.

This way, we can slowly transition between the Laplacian stacks of the two images using the Gaussian mask filter, allowing for a more gradual transition between the two images.

Bells and Whistles: Additionally, I rendered the images in color so that I would be better able to see the transition effects on the hybrid images.

Oraple

Oreo

Manhole

Manholeo (manhole and oreo)

Gray T-Shirt

Mask

UC Berkeley T-Shirt

Plotting Gaussian and Laplacian stacks to make UC Berkeley T-Shirt

UC Berkeley Logo Laplacian

T-Shirt Laplacian

Mask Gaussian

Merged images