Google HDR+ Image Processing Pipeline

Introduction

HDR Overview High dynamic range, or HDR is a image processing technique which allows photographers to create vibrant images with color contrasts which are more representative of what is seen by the human eye. Traditional HDR methods utilize a technique called exposure bracketing where images are taken at different exposures (one under exposed, one at normal exposure, and the last over exposed) and the best parts of each are used to produced the final image. Typically, under exposed images will capture the bright areas of a scene the best, while over exposed images will capture the darker parts of the scene the best. Specifically, parts of each exposure are chosen such that features of the bright and dimmer areas of the photo are visible and clear.

HDR+ Pipeline While the exposure bracketing works well in capturing HDR images on professional photography, it is not suitable for mobile photography, where computation resource is limited and the post processing pipeline is highly automated. To address the issue, Google AI presented a new pipeline, HDR+, for Google’s Pixel phone using constant exposure and new alignment and merging algorithms. In HDR+, the frames are captured with low enough exposure to avoid blowing out the highlights; the pipeline begins at Bayer raw frames rather than the demosaicked RGB frames[1].

The HDR+ pipeline can be broken down into the following steps:

1. Alignment: The alignment is done with a pairwise 4-level image pyramid. That is, each alternate frame in the burst is individually merged with the selected reference frame.

2. Merge: Merge is done with a temporal Weiner filter, then spatially de-noised with a Gaussian filter. The temporal filter is designed in the way that is robust against alignment failure.

3. White Balance, Demosaic, Chroma Denoise: The classic post processing pipeline on a bayer raw image. This include black level subtraction, lens shading correction. White balancing, demosaicking and chroma denoising

4. Local Tone Mapping (Exposure Fusion): This is the key step for creating a HDR image for display, without actually generating the full HDR image. From the demosaiced image, the pipeline generate two synthetic exposures, one long and one short, by applying gain and gamma correction to it. The two images is then merged with the exposure blending algorithm described in [2]. The idea of exposure fusion is to blend multi-exposure image sequences together by keeping the “best” parts in the images. The fusion is done per-pixel wise and the weight map is determined by the contrast, saturation and well-exposedness of the local area.

5. Finishing: The final step fine tone the image to look better. The steps include dehazing, global tone adjustment, chromatic aberration correction, sharpening. Hue-specific color adjustments and dithering for display.

Appendix

You can write math equations as follows: $y=x+5$

You can include images as follows (you will need to upload the image first using the toolbox on the left bar, using the "Upload file" link).