Seam Carving

For the first part of this project we will explore seam carving. Which is a context-aware technique of resizing an image such as to remove the "least interesting" information possible. For example, in this picture I took of the Antelope Canyon, carving out the vertical seams removes most of the dark area on the peripheries.

antelope antelope

We do this by first precomputing an energy function for each individual pixel in the image. In previous courses I've used the dual gradient energy function but this time I will try using the norm of X and Y Sobel convolutions on the image which gives an approximation of the gradient.

Then we use a DP algorithm with the following subproblem definition: M[r,c] = the minimum energy vertical path ending at (r, c). Then the subproblem recurrence is: M[r,c] = E[r,c] + min(M[r-1,c-1], M[r-1,c], M[r-1,c+1]) where E is the energy function.

Everything must be vectorized in order for this algorithm to be fast enough, so we compute whole rows at once. Also, an improvement that could be made is to avoid recomputation of the entire image after a seam is removed, only recomputing locally around the removed seam.

Horizontal seams are removed using the same method, just rotating the image beforehands and after seam removal.

Here are the results.

sunset sunset
The Santa Monica sunset

We can go further and the interesting details will still be preserved.

sunset
20% original height

At some point I realized that my algorithm had an issue where it was producing massive fissures that split up objects very poorly. I realized that it was because I allowed seams to wrap around the image to go to the other side. The result looked like this:

tres tres
Huge fissures

I fixed this by disallowing the wrap around, which prevented the fissures from forming.

tres tres
Fissures gone but still some artifacts

Here's one that I thought would be interesting since it features a bunch of books laid out horizontally.

books tres
It looks like the books did get thinner but some disappeared into thin air

This one is interesting because the rocks apparently have enough details that the path energies are higher but arguably the sunrise is much more interesting.

arch tres tres
Rocks or sunrise???

Mt. Fuji has a ton of detail around the leaves that hides some of the imperfections of the method. Also interesting is that the reflection still sort of holds after the carving.

fuji fuji
Mt. Fuji

One for funsies.

mao mao
Mao

Something that doesn't work is when you try to scale down too much on an image that's expected to have a very particular shape or form.

cake cake
Cake becomes semi-cake (because of the honeydew balls)

Vertigo

In the second part I explored reproducing the Dolly effect by capturing pictures of an object at different optical zooms. I used a canon DSLR.

Which has a 55-250mm zoom lens. At 250mm, the field of view is the smallest, which means that the smallest amount of the background will be included in the shot. At 55mm, we have a much wider field of view, which includes more objects from the background. I tried to keep the object centered and positioned in the same way as best as I could so that the effect of the background morphing into the frame would be most pronounced.

A challenge that I encountered was that I had to choose objects at eye level, or the perspective of the background would become too prominent over the shifting field of view. So I tried shooting at this bottle while moving away linearly.

bottle
Bottle
bottle bottle bottle bottle bottle bottle bottle
Bottle

The most difficult challenge of this part was using a reference point to keep the objects in the same position for each frame. I tried to position all objects just within the circular marker in my viewfinder, and it worked a lot better in this shot.

Also I realized that the effect would work best if the object were more distanced from the backgorund and if I had more room to move backwards and use higher zooms, so I moved outside and looked for something better.

ticket
Ticket Office
ticket_office ticket_office ticket_office ticket_office ticket_office
Ticket Office