Final Project

Daniel Zheng - cs194-26-acj

Seam Carving

Input pictures

All the images that I used for seam carving were taken by me. For each image, I did seam carving until the image was resized by 50% of either its height or its width. Below are the inputs and outputs. The input is the top image and the output is the bottom image.

Successful images

Beach

Bridge

Fire

Firework

Lake

Tokyo Skytree

Failed (ugly images)

Palace of Fine Arts is less important than a patch of grass

Peacock oh no

Implementation

The energy function that I used for my seam carving algorithm is the dual-gradient energy function. That is, the energy of a pixel (x, y) is determined by summing up the squared RGB components of the differences between the pixels (x-1, y) and (x+1, y) and the pixels (x, y-1) and (x, y+1), so E = [(R((x-1, y)) - R((x+1, y)))^2 + (G((x-1, y)) - G((x+1, y)))^2 + (B((x-1, y)) - B((x+1, y)))^2] + [(R((x, y-1)) - R((x, y+1)))^2 + (G((x, y-1)) - G((x, y+1)))^2 + (B((x, y-1)) - B((x, y+1)))^2]
Put simply, a pixel has high energy if the neighboring pixels change drastically in intensity, such as near an edge or a corner. A pixel has low energy if the neighboring pixels are similar in intensity, such as in a uniform area like a patch of sky.

To find the lowest-importance seam, I use a dynamic programming algorithm. I create a matrix of energy values so far and initialize the first row with the energy values of the first row of pixels in the image. Each entry in the subsequent rows (i, j) is equal to the energy of the pixel at (i, j) plus the minimum energy value of the neighboring pixels above it: dp(i, j) = E(i, j) + min(dp(i-1, j-1), dp(i-1, j), dp(i-1, j+1))
After I finish building the matrix, I start from the bottom of the matrix and find the entry with the lowest energy value. Then I work my way backwards and find the next entry neighboring my current entry that has the lowest energy value, and I continue doing this until I am back at the top of the matrix. The indices of the lowest energy values in each row are the pixels in the seam that I am going to remove. I repeat this algorithm until I have removed enough seams.

Conclusion

Seam carving seemed like black magic to me, but I realized that it was actually very simple. By using an energy function that essentially detected edges and corners, the seams with the lowest energy were those parts of the image that had the fewest edges - namely uniform patches of space. Removing these seams therefore tends to preserve the important parts of the image and removes the unimportant parts.

Vertigo Shot

Results

I assembled the images into animated gifs so that it would be easier to see the dolly zoom effect.

Bench

Fountain

Overview and process

The dolly zoom effect is produced by taking a picture of an object, then stepping backwards and zooming in at the same time and taking another picture, and repeating the process as many times as desired. The result is that while the object in the center of the frame seems to stay the same size, the background ends up changing due to the changing FOV from a zoom lens.

The camera that I used was a Nikon D3400, with a zoom lens attachment ranging from 18mm to 55mm focal length. The location I chose was a fancy building located on 2029 Durant Ave. One difficulty that I encountered was that I did not have a tripod, so I could not make the images transition smoothly, but I still did my best to line up the images according to the grid on the camera.