Final Project Seam Carving and Vertigo
Seam Carving
For the first part of this project we will explore seam carving. Which is a context-aware technique of resizing an image such as to remove the "least interesting" information possible. For example, in this picture I took of the Antelope Canyon, carving out the vertical seams removes most of the dark area on the peripheries.
We do this by first precomputing an energy function for each individual pixel in the image. In previous courses I've used the dual gradient energy function but this time I will try using the norm of X and Y Sobel convolutions on the image which gives an approximation of the gradient.
Then we use a DP algorithm with the following subproblem definition: M[r,c] = the minimum energy vertical path ending at (r, c). Then the subproblem recurrence is: M[r,c] = E[r,c] + min(M[r-1,c-1], M[r-1,c], M[r-1,c+1]) where E is the energy function.
Everything must be vectorized in order for this algorithm to be fast enough, so we compute whole rows at once. Also, an improvement that could be made is to avoid recomputation of the entire image after a seam is removed, only recomputing locally around the removed seam.
Horizontal seams are removed using the same method, just rotating the image beforehands and after seam removal.
Here are the results.
We can go further and the interesting details will still be preserved.
At some point I realized that my algorithm had an issue where it was producing massive fissures that split up objects very poorly. I realized that it was because I allowed seams to wrap around the image to go to the other side. The result looked like this:
I fixed this by disallowing the wrap around, which prevented the fissures from forming.
Here's one that I thought would be interesting since it features a bunch of books laid out horizontally.
This one is interesting because the rocks apparently have enough details that the path energies are higher but arguably the sunrise is much more interesting.
Mt. Fuji has a ton of detail around the leaves that hides some of the imperfections of the method. Also interesting is that the reflection still sort of holds after the carving.
One for funsies.
Something that doesn't work is when you try to scale down too much on an image that's expected to have a very particular shape or form.
Vertigo
In the second part I explored reproducing the Dolly effect by capturing pictures of an object at different optical zooms. I used a canon DSLR.
Which has a 55-250mm zoom lens. At 250mm, the field of view is the smallest, which means that the smallest amount of the background will be included in the shot. At 55mm, we have a much wider field of view, which includes more objects from the background. I tried to keep the object centered and positioned in the same way as best as I could so that the effect of the background morphing into the frame would be most pronounced.
A challenge that I encountered was that I had to choose objects at eye level, or the perspective of the background would become too prominent over the shifting field of view. So I tried shooting at this bottle while moving away linearly.
The most difficult challenge of this part was using a reference point to keep the objects in the same position for each frame. I tried to position all objects just within the circular marker in my viewfinder, and it worked a lot better in this shot.
Also I realized that the effect would work best if the object were more distanced from the backgorund and if I had more room to move backwards and use higher zooms, so I moved outside and looked for something better.