CS 194-26 Final Projects

Albert Chen

Seam Carving

For this project I followed the algorithm detailed in Seam Carving for Content-Aware Image Resizing by Shai Avidan and Ariel Shamir. Below is the original image of one case where this worked particularly well.

The first step was to figure out a suitable energy function that determines how "removeable" a certain pixel is. For this I used a laplacian operator as the energy function. The reasoning behind this is that our eyes are quick to notice large gradients, so these pixels should not be removed if possible. After this I followed the dynamic programming algorithm outlined in the paper to the cumulative energy for each seam.

After calculating the optimal seam, I remove it from the image and repeat the process until the desired dimension is reached. To change both the x and y dimensions I remove vertical seams until the desired x dimension is reached then I transpose the image and repeat the process to reach the desirec y dimension. The first 50 seams for this image are shown below.

The original image was 384x683 and the resized image is 184x483. Compared to the scaled version, it is clear that this method preserves more detail. Rather than uniformly scaling the entire image, regions that are unimportant are removed first. This results in the boats being the same size and closer together. Another disadvantage of the scaled image is that it does not preserve the ratios of the boats, and they appear compressed.

seam-carved	scaled

Success Cases

This method works very well for images that are sparse and images that do not contain too many straight lines.

original	seam-carved

Failure Cases

For the following images, this method resulted noticeable artifacting. These images tend to be very dense or contain long, straight lines that cannot be cut cleanly with a seam. In the Lego image, there are many vertical lines that make it very noticeable when seams cut diagonally across them. A similar effect occurs in the skyline image. The algorithm is biased toward removing the pixels from the empty spaces between the buildings. However, since there are so many empty spaces at the base of the buildings, the algorithm removes pixels more uniformly at the bottom of the buildings. This resultsin the the buildings warping and looking like noodles due to the uneven removal of pixels. The same thing occurs with the picture of Obama. Pixels are disproportionally removed form his body and the surrounding background, so his head looks too big.

original	seam-carved

Fake Miniatures

For this part I started by picking a point on the image where I wanted to center my mask. Then I created rectangular mask centered on that point. Then I iteratively blurred the image and dilated the mask - creating a list of gaussian-blurred images at increasing sigmas and their corresponding masks. These would be used to simulate the depth of field effect where objects appear blurrier the farther they are from the plane of focus.

I nested the masks inside each other and feathered the edges to create a smooth transition between different levels of blur. This produces an image that looks close to a miniature but still has some issues.

The rectangular mask is noticeable against the irregular shapes in the foreground. The front of the houses should all be on roughly the same plane, so they should all be in focus. However, the rectangular mask cannot cover the triangular tops of the houses. The trees in the foreground are also half in focus and half blurred due to the mask cutting them in half. The solution to this is to implement one of the bells and whistles: irregular masks. To do this, I manually selected a mask that would fit the image.

I then used the exact same method as before with a rectangular mask centered on the center of mass of my irregular mask. Then I used the irregular mask to paste in the pixels from the source image that correspond to my desired depth of field. This produces a much more convincing effect and eliminates the issue where some objects are partially blurred.

Finally, to further enhance the effect, I converted the images to hsv format so that I could easily boost saturation, contrast, and intensity to better simulate the indoor lighting to make the image look even more like a miniature.

Additional Examples

original

examples taken by me

original