Google HDR pipeline Implementation in Matlab

Introduction

Smartphone cameras have evolved and improved a lot over the past decade. Nowadays, a lot of photography is performed using smartphone. However, small apertures and small sensor pixels of smartphone cameras normally lead to noisy images in low light and limited dynamic range. To fix these issues, Google proposes a complete computational photography pipeline which captures, aligns and merges a burst of frames to reduce noise in low-light setting and also increase dynamic range.

Background

In this section, we will introduce Google's HDR pipeline in the paper [1] in detail.

As shown in the figure above, the pipeline has several stages: capturing, aligning, merging, demosaicking, local tone mapping and finishing.

Capturing

A common approach to HDR imaging is to use bracketed exposure, which captures the same scene multiple times using different exposure length and merge them into a single image to make both bright region and dark region well-exposed. Although this method works well for stationary scenes, it does not work well for moving scenes. To address this issue, in google HDR pipeline a series of frames of constant exposure are captured, and the frames are intentionally made to be under-exposure so that fewer pixels saturate.

Aligning

Aligning refers to the process of estimating offset of a tile across different frames. The Google HDR pipeline does the aligning in a hierarchical manner, from coarse to fine. The alignment algorithm is applied to a grayscale image (obtained by averaging the color channels) instead of the Bayer raw frame. At coarse scale, the search radius is large, so computation is done in Fourier domain for higher speed.

Merging

After obtaining the offset for every tile in the image, a merging phase is needed to obtain one high-quality image that contains the information from all frames and has less artifact. The merging is done in Fourier domain. To merge two images, the Google pipeline controls the contribution of alternative frames according to the quality of alignment, so that merging is robust to alignment failures.

Demosaicking

After creating a single Bayer raw image with higher SNR using aliging and merging techniques, demosaicking step converts the image from a Bayer raw image to a full-resolution linear RGB image. The commonest method for demosaicking is binear interpolation. In Google's HDR pipeline, a more complex technique which includes edge directed interpolation with weighted averaging, constant-hue based interpolation is used.

Local Tone Mapping

For high dynamic range scenes, local tone mapping is needed to reduce the contrast between highlights and shadows while preserving local contrast. Google HDR Pipeline uses a variant of the exposure fusion[2], where one original image and one brightened image (that mimics a long-exposure shot) are merged to achieve a satisfied output.

Finishing

The finishing step aims at producing a high-quality aesthetically pleasing image. In the paper, several techiniques are introduced, including color correction, dehazing, global tone adjustment, chromatic aberration correction, sharpening and hue-specific color adjustments.

Appendix

You can write math equations as follows: $y=x+5$

You can include images as follows (you will need to upload the image first using the toolbox on the left bar, using the "Upload file" link).