To accomplish depth refocussing, I average shifted the light field camera images by multiple alpha values. Doing this allowed me to focus on different parts of the image. I used alpha values from -3 to 2 to create 5 images which focus on different parts of the chess board (from front to back). I then combined my images together to get the following gif.
To adjust the aperture of the given images, I simply incorporated fewer pictures. I used an aperture radius from 1 to 8 to create 7 new images with different apertures. The aperture radius determines how many images are used to create the resulting image, with a smaller aperture radius corresponding to fewer images used. Finally, I combined my aperture images together to get the following gif.
Overall, I really enjoyed this project. Light field cameras seemed very abstract to me when we were learning about them in class, so it was really nice to work with some data from a light field camera and learn how to recover a variety of images from it. I am happy with the way both my depth refocussing and aperture adjusting gifs turned out, but I do wish I used more pictures for depth refocussing to create a smoother gif.
The goal of this project was to use eularian video magnification to uncover temporal image variations that are difficult for humans to visualize. An example of this is to detect the pulse of a man from a simple video of him staring at the camera. To accomplish this, I started by breaking down an input video into an array of frames. Then for each frame I constructed a Laplacian Pyramid, which was used to find regions of quick intensity change. Next, I was able to apply a bandpass filter to each level of the pyramid. This allowed me to pull out the frequencies I wanted to amplify. For the "face" and "baby" videos provided, I used the frequency band from .83 to 1, which is the same range used in the linked paper. After I had filltered out the frequencies I needed, I was able to amplify the corresponding pixel's signals to uncover temporal image variations. Finally, to create the magnified video, I collapsed the pyramid together to recover each frame and combined the frames back together into a video.
Looking at my results, my magnified videos are definitely not perfect. I believe this is partly because I was not able to thouroughly test variables such as my amplification factor and the number of levels to my pyramid. I ended up settling on three levels to my pyramid and an amplification factor of 25 for the "face" video and 20 for the "baby" video. I also believe my code needs some tweaking to properly match the "face" video example given in the spec, but I unfortunately ran out of time. However, as you can see below, my results do amplify the pulse of the man in the "face" video and the pulse of the baby in the "baby" video (though the amplifcation is a bit more subtle than I would have liked).
For Bells and Whistles I tried to amplify low-amplitude motion on my own video of a tree (the leaves of the tree are moving slightly due to the wind). I followed a similiar process as above, but this time I used the frequency band from 2 to 5 and an amplifcation factor of 2. I've included the original video and the magnified video below.
Overall, I found this project to be interesting but also quite challenging. I would have liked to have a bit more time to play around with some of my parameters and tweak my algorthim to improve my results, but I was not able to do so. I found it very interesting that through linear algebra, I was able to extract temporal frequencies and amplify them, without the need for machine learning or artificial intelligence, which I did not realize was possible.
Finally, I want to thank the Professor Efros and the course staff for a great semester. I learned a lot from this class without it being too stressful, which is much appreciated during these crazy times! :)