Software Image Realignment Stabilization
Image Stabilization is a reference to a host of technologies designed to compensate for camera shake and eliminate blur. The technology usually refers to a mechanism in the camera where for every movement of the camera itself, an opposite motion is induced in a camera element, such as the lens or the sensor, to compensate for the motion and maintain the light path on a consistent location on the sensor.
While almost all IS technology today is implemented in some similar fashion to this, it's also possible to achieve the very same effect through software, in a technique we'll call Image Realignment Stabilization. The principle is the exact same - while a lens or sensor can shift in real-time, we can similarly take our image, divide it up into many smaller 'time slices', and "shift" these slices back into alignment. One big difference is that while lens and sensor-shift elements shift in continuous time, we are shifting in finite segments - for example dividing up our 1 second exposure into eight 1/8 second exposures to play around with.
The first example here is a 2 second exposure, taken at a 75mm focal length, hand held. As you might expect, there is a significant amount of blur caused by camera shake.
As we've talked about before, if we use a shorter shutter speed, the amount of blur will generally decrease. Here is the same scene, taken at 1/8 second, all other parameters staying the same.
While we can barely make it out, perhaps judging by the faintly visible sign on the door, we can see that this image is significantly less blurry than the full 2-second one. However, the image is of course much darker, since with only 1/8 second of light instead of the full 2 seconds, we collect far less photons, and thus have a darker image.
If we brighten the image in post-processing, it's a bit clearer to see that this image is indeed less blurry than the original.
As we can see, even though the underexposed image looked completely dark and unusable, hidden underneath all that shadow there is still plenty of light information - roughly 1/16th of the light information that is needed, in fact (since these 1/8 second images are 1/16th the length of the full 2 second image). In fact, if we took 16 pictures at 1/8s, and combined them all together, we'd get something very similar to the full 2s image!
However, this is the resulting image from simply stacking each of the images on top of one another. Since each of our images exist as their own, discrete time slices, we can rearrange them to align with one another:
This image uses the exact pieces as the one above, but by rearranging them we offset the blur caused by the camera shaking (and thus being in a slightly different place for each of the 1/8s time slices). This is much like the effect obtained through lens-shift or sensor-shift methods, although this is a bit better, even - the lens/sensor shifts in real-time according to information from a gyroscope - thus it's liable to lag behind the actual camera shake. Our alignment is based on analysis of the final output images - thus we're guaranteed to get the most stable image possible. Another interesting feature of this method is the ability to scale it almost infinitely - with lens or sensor shift systems, manufacturers claim a "3-4 stop" improvement - equivalent to 8 or 16 times shorter shutter speed. On the other hand, we can divide our time slices as finely as we want (in theory) - instead of 1 second we could do one thousand images at 1/1000s, achieving a 10-stop improvement. Obviously this would be tedious to realign all those time slices manually, but we could easily write software to do this automatically, based on contrast detection, and with a fast enough processor this could even be done in camera!
Here are the summarized results:
 |
 |
 |
 |
 |
Note the stark difference between the partial 1/8s raised exposure and the partial 1/8s stacked exposure, with realignment. For one, the raised exposure is much noisier than the stacked exposure, and this is a result of multiplying the image (and magnifying all the color and brightness variations along with it). As we'll see next week when we discuss noise, by adding images with a random pattern of noise, the noise adds to itself and negates itself in equal amounts, and will tend to 'even out' in the long run. Notice that the unaligned stacked image exhibits more noise - this is due to fixed-pattern noise, rather than random noise. Another important thing to notice is that the stacked image has less blur than the raised image - as we discussed probability before (and experimented with last week's multiple shot method), the amount of blur is not static, but is a random occurrence that will change from shot to shot. If we take only one image, as I did here, there's a chance it will be sharper than the stacked image (which averages out all the blur, if any), but there's also a chance that the single partial image will be even blurrier.
As you can see, the final images aren't all equal to each other. Part of this is due to my image stacking being incomplete - this is actual a stack of 10 images, not the full 16 images required (I was rushed and didn't get a series of 16 pictures in, sorry). But aside from that, you'll notice that the color is a bit off, and even the stacked exposure has some grain/noise. This is mostly the result of our software methods - these were out-of-camera JPGs, which digitized the information to only 24 bits, and on top of that introduced things like JPG artifacts, so our experiment is less than ideal. If this were done in-camera, with the RAW data, however (and if before converting to digital, even better), then our results could very easily match a stabilized 2s exposure.
The resources used for this are provided below. See the Week 4 Assignment here.
Resources
Full 2 second exposure
Partial 1/8 second exposure
Partial 1/8 second raised to full exposure
Partial 1/8 second stacked to full exposure
Partial 1/8 second stacked to full exposure
Photoshop file of stacked exposure (layers) 61MB