Tony Lian's Final Project
Project 1: Lightfield Camera
Part 1: Depth Refocusing (30 pts)
Refocusing effect is achieved by changing the offset to average the image with. Taking the offset as C * (u, v), where u, v is the offset of the carema for image capture, with different C, allows different refocusing effects.
Refocusing with a different C:
(Chess)
(Bulldozer)
Part 2: Aperture Adjustment (20 pts)
Through adjusting what range of image to average, we can achieve an effect of adjusting aperature. Since the image is on a 17 x 17 grid, we have 8 different choices for averaging images. The focus is the lego ground and the men close to us.
Adjusting aperature:
(Lego)
Part 3: Summary (0 pts)
This is a very cool project which gives me more understanding about the lightfield.
Bells and Whistles: Interactive Refocusing (up to 5 pts)
The browser needs to support HTML5 for interactive refocusing (for example, use Chrome/Safari). Click on the image to refocus to where you click.
The Chess images:
If it doesn't load, you could view the video with source frames and drag the control to "focus" interactively.
Project 2: Poor Man's Augmented Reality
Part 1: Keypoints with known 3D world coords
Labeled points ((0,0,0) is the lower left point):
(Labeled points)
(Axis from the projection matrix)
Part 2: Propogating Keypoints to other Images in the Video
I use a corner detector and pass the keypoint to the consecutive frame. If there is no match in the next frame, keep the current keypoint position. I use a moving average to stabilize the points.
(Note: red is from past frame, blue is current frame. They don't differ a lot.)
Part 3: Calibrating the Camera
I recalibrate the points for every frame.
(Intermediate frame.)
(Another intermediate frame.)
Deliverable
The box AR with the corresponding input images:
Bells and Whistles
Placing an arbitrary mesh onto the scene. This is non-trivial. I loaded a model of effiel tower and put its triangular mesh into the scene. This requires much more adjustments than the box. I apply a linear adjustment to the position of the object at the end of video and smoothing of the keypoints.
The model:
You can see the height of the tower varies. This is because the tip is too far away from where I put the points. Even a slight shift of points leads to a large variation. This is more pronounced when I accelerate the video to create this GIF due to upload size limitations (you can see that the input video also "jumps" because I don't have a tripod to control the movement of camera).