Amol's CS194-26 FINAL PROJECT

Amol's CS194-26 FINAL PROJECT Part 1: POOR MAN'S AR

Amol Pant

Setup

To begin with, I take a video of my box and mark all the points on it for the first frame

Image

Propogating Keypoints to other Images in the Video

I personally chose an off the shelf tracker called CSRT for opencv since i prioritize accuracy and not so much performance.
I setup tracker for each point. I also defined corrosponding 3d coordinate for each as shown below.

Image

Calibrating the Camera

After propogating the points, I also keep track of an array of matrices for each frame which tells us the transformation from 3-d to 2-d using least squares.

Image
https://drive.google.com/file/d/14oE_9lqDe4Yp_leRdh5DBuRxQ8CPIo-5/view?usp=sharing

Projecting a cube in the Scene

Now I can use my transformation matrix for each frame and project a cube onto a set coordinate for each frame using opencv draw functions.
Here is the result

Image
https://drive.google.com/file/d/1VZgbZvxZSFXwks-z-tgL38wfDrgUvwMq/view?usp=sharing

Amol's CS194-26 FINAL PROJECT Part 2: EULERIAN VIDEO MAGNIFICATION

I first begin with create a laplacian pyramid for each frame of the video and then stacking all of the pyramids to make a video pyramid.

Image

For color amplification, I found that the butterworth filter from python with an order of 2 worked best for this. The one problem I faced was that the order caused delay later on in the video.
For motion, I found that performing fft and then removing the indices worked best, discarding the first layer of the video since it was too noisy. Here are ther results.

Image
https://youtu.be/s4lbASp7tto
https://youtu.be/zJ9-2haKGCs

Bells and whistles:
I plotted the entire process for face.mp4's 250x250th pixel and how it was transformed from one domain to the next and filtered.

Image	Meaning

	filtered frequency
	basic pixel tracking
	bandpassed filter in time domain
	butter pass in frequency domain
	butter pass in time domain