CS194-26 Project 7 - Alex Zhang
Vertigo Shot
Overview
I borrowed a Canon DSLR camera from a friend to complete this project. I chose to shoot indoors in the VLSB library and in a cafe since it was quite windy that day and I noticed that the inconsistent lighting conditions from shadows outside affected the quality of the shots. For each sequence, I used the grid built into the viewfinder of the camera to try and keep the subject in the same place at the same size for each shot while varying the distance between me and the subject, as well as the zoom of the lens. I encountered some difficulty in getting the camera to actually focus on the subject for some reason, and also it was not always possible to keep the subject at the exact same position due to human error. Keeping the camera steady between shots was also challenging, I ended up having to use a table to steady the camera.
Water Cup
GIF of sequence (Bells and Whistles)
Lip Balm
GIF of sequence (Bells and Whistles)
Pusheen Pencil Case
This was a bit ofo a failure since I didn't get the focus quite right in most of the shots, and also didn't keep the case centered and the same size exactly in each shot.
GIF of sequence (Bells and Whistles)
Bells and Whistles: Varying subject instead of camera
For each shot, I would move the camera back and zoom in. I tried playing with moving the subject and keeping the camera instead for the lip balm sequence, and this is the result:
GIF of sequence (Bells and Whistles)
You can see that it has a slightly different effect as the background appears to zoom in a lot more compared to the other case where I moved the camera instead (rightfully so, since the lip balm is physically getting closer to the pole in the background).
Seam Carving
Overview
We follow the procedure laid out in the paper quite faithfully. As an energy function, we use the sobel filter from the skimage filter library that incorporates both the vertical and horizontal sobel filter into the result (by taking the square root of the sums). We then use the energy function to calculate the cost of each path (seam) in the image by using the following dynamic programming recurrence: M(i, j) = e(i, j)+min(M(iā1, j ā1),M(iā1, j),M(iā1, j +1)). We find the seam that costs the least and delete it. We repeat this process once for however many pixels we want to remove from the image horizontally. To shrink the image vertically, we just rotated the image by 90 degrees initially, ran the horizontal algorithm, and rotated it back at the end.
Example Seam
We see the line highlighted in red is a good seam because most of things in the path have very little differences. It took the path through the cloud that didn't have a shadow, which shows that it's trying to minimize the cost of path actively.
Results
All the following images were taken from Google images.
Arashiyama Bamboo Grove
Original
Shrunk Horizontally by 200px
Togetsukyou Bridge
Original
Shrunk Horizontally by 200px
Grand Canyon
Original
Shrunk Horizontally by 200px
Kiyomizudera
Original
Shrunk Horizontally by 200px
Fire
Original
Shrunk Horizontally by 200px
Monument
Original
Shrunk Horizontally by 200px
Christmas Tree
Original
Shrunk Horizontally by 200px
Empire State Building
Original
Shrunk Horizontally by 200px
Failure Cases
Most of the failure cases are due to what we focus on as a subject of the photo having texture that is too similar to itself, making it a prime candidate as a low cost seam. The above photos all have subjects that have a varied texture.
Shinzo Abe
We can see that his shoulder was significantly cut off on the right. This is probably due to his jacket being a uniform color.
Original
Shrunk Horizontally by 200px
Tokyo Tower
The bottom left of the tower has become jagged.
Original
Shrunk Horizontally by 200px
Kinkaku Temple
The right face of the temple was collapsed in a strange manner.
Original
Shrunk Horizontally by 200px
Fushimi Inari
The poles on the right got really thin, and some people got cut in half.
Original
Shrunk Horizontally by 200px
What I learned
I learned that a simple algorithm akin to basically finding the shortest path can produce very powerful results that are much more interesting than just simple cropping of a photo. It's almost a little scary how effective it is on certain photos, like the Arashiyama bamboo grove one. Ultimately however, the algorithm is not aware of the actual contents of the image, and sometimes removes the subject of the photo because to the algoritihm, it is the lowest cost seam. So it's important to pick vivid subjects that have a big variation in their texture.