TienDaiIFHDR
Project Title
A Versatile Image Fusion Method
Introduction
High Dynamic Range and Exposure Fusion
Dynamic range of a scene is defined as the ratio of the highest to the lowest luminance. The real world scenes often have a very wide range of luminance, sometimes exceeding 10 orders of magnitude. Fig. 1 shows a HDR scene with a dynamic range of about 167, 470:1. To reproduce these scenes presents a challenge for conventional digital capture and display devices, which suffer a limited dynamic range of only 2 orders of magnitude.
The most common solution to address this problem is to take a sequence of low dynamic range ( LDR ) images of the same scene under different exposure intervals to capture all the radiance information and then render the captured stack to display. There are generally two pipelines. One way is to firstly estimate the camera response function from the image sequence to recover the true radiance of the original scene ( recorded as a 32 bit float radiance map ) [1, 2], and then tone map the created radiance map for display on LDR reproduction media ( usually 8 bit per channel ) [3, 4, 5]. Although this way gives very satisfying result, it's computationally expensive and time consuming. The other way is to fuse the captured images directly without the intermediate step of creating radiance map [6, 7], which is usually referred as "Exposure Fusion ( EF )" [7]. EF produces HDR-like images, which are comparable to those tone-mapped results, at a much lower computational cost. Due to its effectiveness and computational efficiency, EF is adopted by most of HDR applications on mobile platform, which has limited computational power [8, 9].
All-in-focus Imaging
In fact, EF essentially solves the problem of merging multiple images, and consequently could be easily extended to deal with other imaging and photography challenges except for HDR. The most direct application is to fuse multiple focus image stack ( Fig. 2 ) to produce an all-in-focus image [9].
The size of a camera's aperture provides a trade-off between the depth of field ( DoF ) and the amount of light that is captured by an image with a given exposure time. For an image to be sharp across a large range of depths in the scene, a small aperture is required. However, decreasing the aperture size is not always feasible. On the one hand, most low-end cameras, like cellphone cameras, have a fixed aperture size. On the other hand, small apertures require slower shutter speeds, which can result in image blur due to handshake and motion of objects in the scene. EF successfully address this problem to render all pixels in focus. It's also worthy to mention that EF could also combine flash/ no flash image pair taken under low light condition to fight with the artifacts caused by flash light [7].
Project Content
In this project, we would study EF from following aspects:
1) Analyze and implement the algorithm to create HDR image.
2) Extend the algorithm to all-in-focus imaging.
3) Propose ways to accelerate the algorithm.
Methods
EF computes the desired image by keeping only the "best" parts in the multi-exposure image stack. The final image is obtained by collapsing the stack using weighted blending, guided by simple quality measures, namely contrast, saturation and well-exposuredness. The process is done in a multi-resolution fashion in order to avoid undesirable artifacts. It is assumed that the images are perfectly aligned, possibly using a registration algorithm [10]. We would firstly go through the original algorithm of exposure fusion and then describe how to extend it to create all-in-focus image.
Weighting Map
In the multiple exposure image stack, over-exposed and under-exposed regions are flat and colorless, which should receive less weight during fusion. While areas under good exposure contain bright colors and details and they should be preserved with more weighting. The algorithm uses the following measures to decide the weighting for each of the pixels in the image stack.
Contrast ( C )
Under- and over-exposed regions are relatively more "flat" or "uniform" without much fluctuation of intensity, or less contrast. Besides, texture and edges are visually important elements. As a result, pixels of high contrast should be assigned large weighting. The algorithm applies a Laplacian filter to the grayscale version of each image following [11], and take the absolute value of the filter response as a simple indicator C for contrast. Fig.3 shows the contrast maps calculated from the image stack in Fig. 1.
Results
test
Conclusions
test
References
[1] P. E. Debevec and J. Malik, “Recovering high dynamic range radiance maps from photographs”, Proc. ACM SIGGRAPH’97, pp. 369 – 378, 1997.
[2] T. Mitsunaga, S. K. Nayar, Radiometric self calibration,“Proceedings of the Computer Vision and Pattern Recognition, vol.1, 1999, pp.374–380.
[3] G. Ward, A contrast-based scalefactor for luminance display, in: Graphics Gems IV, Academic Press, 1994, pp. 415–421.
[4] F. Durand and J. Dorsey, “Fast bilateral filtering for the display of high-dynamic-range images”, ACM Trans. Graph. (special issue SIGGRAPH 2002) 21, 3, 257-266, 2002.
[5] Q. Tian, J. Duan, M. Chen and T. Peng, "Segmentation Based Tone-mapping for High Dynamic Range Images", Advances Concepts for Intelligent Vision Systems, pp.360-371, 2011.
[6] A. Goshtasby. Fusion of multi-exposure images. Image and Vision Computing, 23:611–618, 2005.
[7] Mertens, T. and Kautz, J. and Van Reeth, F. “Exposure fusion”, Computer Graphics and Applications, 2007. PG'07. 15th Pacific Conference on, 382--390, 2007.
[8] Natasha Gelfand, Andrew Adams, Sung Hee Park, and Kari Pulli, “Multiexposure imaging on mobile devices,” in Proc. of the ACM Multimedia, 2010.
[9] Vaquero, D. and Gelfand, N. and Tico, M. and Pulli, K. and Turk, M., “Generalized Autofocus”, Applications of Computer Vision (WACV), 2011 IEEE Workshop on, pp. 511--518, 2011.
[10] G. Ward. Fast, robust image registration for compositing high dynamic range photographcs from hand-held exposures. Journal of Graphics Tools: JGT, 8(2):17–30, 2003.
[11] J. M. Ogden, E. H. Adelson, J. R. Bergen, and P. J. Burt. Pyramid-based computer graphics. RCA Engineer, 30(5), 1985.
Acknowledgement
We would like to sincerely thank various authors for making their data available on the Internet for experiments. Images used in this project courtesy of corresponding author(s).
Appendix I - Code and Data
test
Appendix II - Work Partition
test