Human eyes can perceive detail from scenes with very large range of illuminations. Cameras and displays however do not have sugh a large dynamic range. In order to reproduce the rich detail in both well lit scenes and very dim scenes that our eyes can see in a photograph, we can take multiple exposure images and combine them to create a single high dynamic range photograph.
To create a HDR image, we have to first recover the radiance map. Radiance map records the absolute value of lightness/radiance at each pixel, which by nature is very high dynamic range. Therefore, the second part involves conversion from high dynamic range to low dynamic range, i.e. tone mapping. To find the radiance, we recover the inverse of the function mapping exposure to pixel value. For the second, we explored local and global tone mapping to create a displayable image.
As described in this paper, to recover the radiance map, we take multiple images at different exposures at the same time. Since the timespan is short, we can assume that each pixel in different images corresponds to the same light source. The pixel value in an image Zij (i is the pixel in image j), is the result of a function of unknown scene radiance and a known exposure: Zij = f(Ei Δtj ), where Ei is the unknown radiance. We didn't solve for f, but rather for g=ln(f-1). This g function maps pixel values (0:255) to the log of the exposure values: g(Zij = ln(Ei) + ln(tj ). As such, we have four variables: two knowns (Zij, ln(tj ) ), and two unknowns (g and ln(Ei ) ). However, the scene is static so we know that Ei remains constant throughout the image sequence.
Each exposure only gives us trustworthy information about certain pixels (i.e. the well exposed pixels for that image). For dark pixels the relative contribution of noise is high and for bright pixels the sensor may have been saturated. To make our estimates of Ei more accurate we need to weight the contribution of each pixel. In this project we use the triangle function that peaks at Z=127.5, and is zero at Z=0 and Z=255.
An ddditional consideration we took into account was keeping g smooth by adding a penalty accodring to the magnitude of the second derivative: g''(x) = g(x-1) + g(x+1) - 2×g(x).
Once we set up the matrix, we can use least square solver to solve for g. And once we have g, recover the absolute radiance map is easy. We have ln(Ei) = g(Zij)-ln(Δ tj) and we exponentiate this value to find the radiance at all pixel locations.
Only recovering the absolute radiance is useless since our display is limited by the range it can display. Therefore, we need to apply a mapping that maps the high dynamic range values to the range of the display in a meaningful and representative way, i.e. tone-mapping.
The most straightforward tone-mapping is to a linear scaling such that the highest radiance value is mapped to the maximum display value and the lowest radiance value is mapped to the minimum display value. However this usually results in an image that is too dark
To allow more range to represent darker pixels, we can take a log of the radiance values such that the difference between high values and low values are greatly reduced, following by linear scaling we can have pretty good results.
If we want more control over, we can employ a more adaptive method described here. Using the the following operator:
where Imin and Imax are the minimum and maximum luminance of the scene, and Dmax and Dmin are the minimum and maximum luminance of the visualization devices, and τ = α(Imax - Imin), α ≥ 0. By varying α, we can control scene illumination.
All methods mentioned above are global tone mapping operators that directly maps radiance values to pixel values. To achieve a more context-aware tone-mapping, we can use local tone-mapping described here. The steps are the following
Here are the local tone-mapping results