Project 4: Seam Carving

By Ross Teixeira

My seam carving journey began in my first semester of Freshmen year at Cal. I visited an event by Hackers @ Berkeley where students were demoing projects they had worked on, either as part of a course assignment or just for fun. I saw many cool projects there (and met some great friends), but I stopped dead in my tracks when I came across one student's seam carving algorithm, which I assume was completed for CS194 (if it existed back then). I was completely in awe. I came into Berkeley with zero coding experience, and until that moment, I had not seen a concrete example of something exciting that I personally wanted to build and would one day be capable of building. It was so freaking cool, something I had always wanted the power to do! I begged the student to send me his webpage so I could come back to it, which he did. It's still a bookmark on my browser to this day.

Ever since then, I have been waiting to implement the algorithm for myself. This project is the fulfillment of a four-year journey, that evolved me from young, malleable and full of curiosity to a veteran wielder of code, still as curious as ever. This is a way of paying homage to just how far I've come, and the images adorning this page are my testament to a bright and exciting future.

Anyway, now the implementation details! My seam carving algorithm first calculates an energy function on each pixel of the image, which corresponds to how different it is from its neighbors. I implemented my energy function as:

  • for each pixel, the sum of square-differences between each opposite pair of adjacent pixels (left - right, above - below, and diagonals).

Given these energies, I then run a dynamic programming algorithm in order to find the path through the image, spanning one pixel per row, that has the overall lowest sum of energies. This becomes a "vertical seam" through the image, that I can remove! By targeting seams with minimum energy, we are able to shrink the image without disturbing the overall quality of the image. This works for horizontal seams as well, just by rotating the image 90 degrees!

To make the process a bit faster, as I continually shrink the images, I recalculate the energy of a smaller subset of the image, which is still guaranteed to factor in the missing pixels.

Now, let's get the show rolling!

Très Bien

These are some of my nicest examples.

Symmetrical House

An image I took this summer of a remarkably symmetrical house. The vertical image shows some diagonal artifacts if you look carefully, but otherwise it's very cool! The horizontal did an excellent job of preserving the symmetry.

kljklj

Original

kljklj

Vertical Seams

kljklj

Horizontal Seams

Rainforest

http://forest.ambient-mixer.com/tropical-rain-forest

kljklj

Original

kljklj

Vertical Seams

kljklj

Horizontal Seams

Pagoda

http://www.inetours.com/Pages/SFNbrhds/Japanese_Tea_Garden.html

kljklj

Original

kljklj

Vertical Seams

kljklj

Horizontal Seams

Windows XP

Besides the interesting diamond in the sky, this one turned out cool!

kljklj

Original

kljklj

Vertical Seams

kljklj

Horizontal Seams

Calling All Dawns

I found it very interesting that the sun in the center was perceived to be a "lesser evil" to cut than going through the dark line streaking across the image.

https://www.wikiwand.com/en/Calling_All_Dawns

kljklj

Original

kljklj

Vertical Seams

kljklj

Horizontal Seams

Cabinet

Believe it or not, this cabinet is a real sculpture! http://www.dezeen.com/2013/03/14/good-vibrations-distorted-cabinet-ferruccio-laviani-fratelli-boffi/

kljklj

Original

kljklj

Vertical Seams

kljklj

Horizontal Seams

Hurricane Matthew

The text and general hurricane presence is nicely preserved in this spooky image.

http://newscult.com/hurricane-matthew-forms-spooky-scary-skeleton-haiti/

kljklj

Original

kljklj

Vertical Seams

kljklj

Horizontal Seams

Peter Molyneux

The vertical seam excellently removed the wall on the left side of the image. The vertical seam took an obvious jab at the questionable intellect of Peter Molyneux.

kljklj

Original

kljklj

Vertical Seams

kljklj

Horizontal Seams

Kevin Butler

Butler didn't survive this cutting without a few battle scars, and neither did most of the room. But God forbid we mess with those perfect pancakes!

kljklj

Original

kljklj

Vertical Seams

kljklj

Horizontal Seams

As we increase the multiplicative factor on the high frequencies, the increase the strength of our sharpening filter. The edges of the image become increasingly highlighted, until they form halos around our outlines.

Bells and Whistles

Canny Edge Detector: Energy Function

By using a Canny edge detector for my energy function, I was able to produce better results for some of my images.

Kevin Butler

This time, Butler and the room are much better preserved, while the moose was eliminated entirely from the image. Pancakes swear they had nothing to do with it.

kljklj

Original

kljklj

Canny Edge Detector

House

The Canny filter broke the symmetry of the house, in favor of preserving reality.

kljklj

Original

kljklj

Canny Edge Detector

This project is complete, and so am I. Or is it? There's a lot more I would like to implement and play around with in this project. Overall, I thoroughly enjoyed this project and I am glad to have put the issue of Seam Carving to rest, for now. I look forward to the next challenge life brings!

Diagonal Resizing

I implemented the ability to shrink a pixel both vertically and horizontally at the same time. To do this, I found the minimum seams in both directions, and applied the one with the least overall energy.

kljklj

Rainforest

490 x 320 -> 404 x 306

kljklj

Cabinet

780 x 438 -> 609 x 309

House

The Canny filter broke the symmetry of the house, in favor of preserving reality.

kljklj

Original

kljklj

Canny Edge Detector

Object Removal

I implemented the ability to select an object to remove from the image. To do this, you create a mask of your image with the intended portion to remove in white and the rest in black. I change the energies around the white portion to -infinity, and shrink the image until the object has disappeared.

It seems to have removed a lot more than the football player, but he is indeed missing!

kljklj

Original

kljklj

Mask

kljklj

Removed

This project is complete, and so am I. Or is it? There's a lot more I would like to implement and play around with in this project. Overall, I thoroughly enjoyed this project and I am glad to have put the issue of Seam Carving to rest, for now. I look forward to the next challenge life brings!