Final Project: Abel Yagubyan

Project 1 - Lightfield Camera : Overview

This project aims to attempt depth refocusing and aperture with images that are captured with lightfield camers, where these cameras capture the origin and direction of the light ray. Now, we will go ahead and analyze the dataset from the Stanford Light Field Archive, where there are 289 rectified images taken by such cameras, where each location is different.

Depth Refocusing

What I went ahead and did was I chose the center-equivalent image of the grid by using the co-ordinates of x=8,y=8, where I shifted all of the images by the depth parameter accordingly, then lastly average them. It's easy to notice that increasing the depth allows the image to move towards the front (and vice-versa), and we can see an example of the chess image from depth values from -3 to 4 (from left to right, top to bottom):









And here are example GIFs of the depth refocusing from -3 to 4 for the chess images and jelly beans images:



Aperture Adjustment

Similarly to the depth refocusing section, a greater aperture implies that a larger amount of light rays enter (and vice-versa), so the implementation is quite similar to changing the focusing depth by iterating the aperture values from 0 to 8 (non-inclusive) with depth of 0, which generates images with different aperture yet the same focusing depth. To correctly filter images to a certain radius, I use the square root of the sum of squares of x and y cordinates to see if it is less than the aperture radius, and if so, we accept the value, otherwise we do not. We can see an example of the toy image from aperture values from 0 to 7 (from left to right, top to bottom):









And here are example GIFs of the aperture adjusting from 0 to 7 for the chess images and lego images:



Bells & Whistles

For the bells and whistles, I went ahead and collected my own data by capturing images using my phone and implementing refocusing/aperture adjustment. As seen below, the results are absolutely terrible. There's a very good reason for this, and that's because I'm a terrible photographer and can't achieve similar results to the Stanford dataset, hence the images are blurry and not useful at all (I must've rotated the phone slightly too when taking the images, since the angle is not perfect).



Summary

This was a really cool project! I really enjoyed the outcomes and it was a completely new understanding for me when it comes to lightfield cameras.

Project 2 - A Neural Algorithm of Artistic Style

Within this project, I implemented an Artistic Style Neural Algorithm inspired from the paper mentioned in the course, where we essentially attempt to use a Deep Neural network to create a model that draws things as beautifully as possible on prior paintings. With my current model, here is my setup for the Algorithm (Where I initially rescale images, setup the Content and Style losses as mentioned on the paper, replace maxpool layers with average pool layers, use conv_1/2/3/4/5 for style representation and conv_5 for content representation, then normalize the data, and begin to correctly train it with the help of the VGG19 model. To be more precise, we first calculate the Gram matrix for filter response correlations, then combine the content and style for total loss as a metric.



And here are some cool outputs! The order of styles go from starry night to the scream to the scream again and then to the picasso painting from top to bottom.(Pls ignore my face which looks demonic since it obviously doesn't work well with faces lol):