Light Painting

CS 194-26: Image Manipulation & Computational Photography // Final Project

Amanda Bui (-abd), Emily Tsai (-aal)

Final Report

Overview

Light painting is a technique, in which a long exposure photograph is taken to capture a moving light source. Photographer Stephen Orlando used this technique to capture motion through time and create light trails in his "Motion Exposure" series. Many of his images are of musicians, sliding their bows along the strings of their instruments such as the violin or the cello. The goal of this project was to mimic these effects and track the movement of objects in a video.

Musician playing the viola by Stephen Orlando

Algorithm

Detection

To track the objects in our videos, we used color detection. For every object we wanted to track, we wrapped it in a bright green colored tape. The following algorithm describes how to track a single object.

For each frame in the video:

  1. Create a mask to isolate the green colored object.
  2. Using the mask, find the contour the object, which is just a continuous set of points that outlines the object.
  3. Average the set of points in each contour. The averaged point is the "coordinate" of the object.
  4. Add the averaged point to the list of points garnered in total.
  5. Draw a line between adjacent points in the list.

Tracking the end of a pencil, wrapped in green tape.
The mask corresponding to the image on the left.
Tracking the movement of a single object.

Creating the "light" colors

We used a range of 500 colors on the RGB scale to draw the lines between adjacent points. Each line started with the same color. For each subsequent frame following the first, the color of the line between the new collected point and the last point from the previous frame was determined by incrementing the previous color by 1. If there were more than 500 points, the colors would loop back to the beginning.

Shifting Points

To capture the "passing of time" similar to the effect in Orlando's images, especially if the subect stayed in one area, we needed to shift the points.

Given a frame:

  1. For every point garnered so far, shift it by the given amounts for the x and y directions.
  2. Add the new point found from this frame without shifting it to the list of shifted points.
  3. Draw a line between all of the adjacent points in the list of shifted points.

Shifting the points of the object.

Tracking Multiple Objects

Tracking multiple objects in a video is very similar to tracking a single object. However, there are multiple contours, using the mask, since there are multiple objects in the frame that contain the green tape/color. This means we don't have a single list of points collected from the frames. Instead, we have multiple lists, a group of points for each object that we want to track. This requires us to find the group that each of the new points from the current frame belongs to. The following steps describe how to group the points.

Given a frame, for each group of points:

  1. Find the distance between the point from the previous frame in this point group and each new point found from the contours of the current frame.
  2. Keep the new point with the minimum distance from the point in the previous frame in this group.
  3. If the distance is less than a certain threshold, add the new point to this group.
  4. If the new point was already assigned to another group, assign the new point to the group that is closer.

Midpoints

We took a video of our friend playing the violin. We placed the green tape on the two ends of her bow. Combining the steps from the other sections, we get the image on the left below.

Tracking two objects.
Musician playing violin by Stephen Orlando.

To better capture the effect created by Orlando, we needed more points in between the ends of the bow. We initially thought we would put more green tape along the length of the bow. However, this made assigning groups to the new points more difficult since the green objects were so close together. We found it easier to just find the midpoints, uniformly distributed, between the two ends.

Violin Light Painting.

Darkening the frames

As a last step, we darkened each frame by weighing it by an alpha value of 0.2 so that the "lights" appeared to glow brighter.

Violin light painting with darkened frames.

Challenges

Losing Points

If we moved too fast in the video, the frames would be blurry and this made it more difficult for the green color to be detected, i.e. only one end of the violin bow would "move" because only one point instead of two was detected. This would mess up the colors of the lines in the video because the lines of each point group would no longer be synchronized.

Using Color Detection

Because we used color detection, our program was very sensitive to the colors in our videos. We had to make sure that we didn't have any other green objects in our frames.