Lightfield Camera

CS194-26 Proj 4 (Project Web Page) : Depth Refocusing and Aperture Adjustment with Light Field Data

Project Overview

In this project we implemented depth refocusing and aperture adjustment. Several images were taken from translated points. The images were just taken over a regularly spaced grid with the individual offsets and index numbers provided. Then just using shifting and averaging, we can produce some cool effects.

For reference, here are the original images in their center position.

Chess
Bracelet

Code Usage

      
      usage: main.py [-h] [-c C_SCALE] [-d DATA_DIR] [-r R_LIM] [{get_aperture}]

      positional arguments: {get_aperture}
      optional arguments:
        -h, --help            show this help message and exit
        -c C_SCALE, --c_scale C_SCALE
        -d DATA_DIR, --data_dir DATA_DIR
        -r R_LIM, --r_lim R_LIM
      
      The output is 'output/{data directory}_{C}_{R limit}.png'
    

Part 1: Depth Refocusing

The point of this part is to set a reference point and then shift all other images to this reference point. The other images which are focused on the other parts of the image will vary significantyl when it is shfted so the focus will be the reference image with the surrounding area being blurred.

Here are the steps I took

  1. Parse all the jpg files in the data directory and extract the (x,y) grid position as well as the (u,v) absolute offset
  2. Used the center image at 8,8 For each image and the correseponding u',v', get the difference to the reference offset. (u - u', v - v')
  3. Shift each image by C * (u - u', v - v')
  4. Average all the images togeher
C=0.05
C=0.25
C=0.5
C=0.9

via GIPHY

Aperture Adjustment

The next part of this project is to do aperture adjustment, wherein we repeat the exact same steps as last time except we only use the images that are within some value R from the source (i.e. u - u' and v - v' are both less than R absolutely). Then we can do some interest aperture adjustments at a given point

C=0.33; R=1
C=0.33; R=10
C=0.33; R=25
C=0.33; R=40
C=0.33; R=40

Bracelet

C=0.33; R=1
C=0.33; R=1
C=0.33; R=1
C=0.33; R=1

via GIPHY

Summary

This project was pretty cool and it was neat to discover various ways to change focus using an array. While somewhat impractical, it gave me a better appreciation of Ren Ng's work to use just one camera instead of the whole array.