The script combines multiple images taken from the same location but with different shutter speed into one HDR image.
Slight misalignment in the images will cause the algorithm to perform much worse since it assumes a fixed point in different images would have the same real-world irradiance. We use cv2.createAlignMTB to automatically align the images, independent of exposure. We assume that the shift is less than 3 pixels (max_bits=2).
To figure out the radiance, we follow Debevec and Malik 1997. Assuming an image is taken with shutter speed t, we solve for a exposure response function g such that a pixel value z conforms to that g(z) = log(e) + log(t), where e is the real-world radiance at that pixel. [1]
Since cameras are usually more lossy at extreme exposures, we use a trapezoid weight function to penalize the pixels where it's too bright or too dark during the sampling process. We sample 500 points from each image.
Below is the estimated response curve for the image set chapel. The term log(t) is not included for viewing purposes. The colored dots are the sample points in images with different shutter speed, and the black line is the estimated radiance response curve.
We apply the algorithm on each of the three channels. We also tried to experiment on other color space such as HSV or L*a*b*. However, due to the compression of JPEG, the hue channel in HSV usually have noticeable blocky artifacts when the environment is dark. This causes the result to have some over-saturated spots even if we intensely penalize the dark areas. We do not use L*a*b* since it only have 100 lightness values compared to the usual 256.
Below are some results with HSV color space. Note the over-saturated artifacts.
To show the image with a high-dynamic-range, we follow Durand 2002. We use a bilateral filter to filter out the low frequencies, and offset and scale the low frequencies. Then we put back the high frequencies and the colors. [2] We use a bilateral filter with diameter of 3 and sigma color and sigma space of both 64 pixels.
Below the are the estimated relative radiance map of the image and the final result after tone mapping.
Remarks: since we are using Debevec and Malik 1997 on each of the three channels. It's difficult to get color balance and saturation right, and the image usually looks under-saturated if taken in a bright environment and over-saturated if taken in a dark environment. An improvement could be manually choose a mid-exposed image as a reference and readjust color according to the reference image.
| Radiance Map | Result |
|---|---|
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
By putting multiple cameras on a grid perpendicular to the imaging direction, we can postprocess parameter such as depth of field or aperture with the excess data.
To mimic the change depth of field, we average all image each with a shift of c * ([u, v] - [u_bar, v_bar]), where u, v are the real-world coordinates of the camera, u_bar, v_bar are the average of all camera coordinates, and c is a parameter controlling the depth of the focus plane.
To mimic the change of aperture, we average all images that (u - u_bar)^2 + (v - v_bar)^2 <= r^2, where u_bar, v_bar are the desired focus depth, and r is a parameter controlling the size of the aperture.
| changing depth of field | changing aperture |
|---|---|
 |
 |
 |
 |
 |
 |