Google HDR+ Image Processing Pipeline - Revision history

imported>Student2018: /* Methods and Implementation */

2018-12-17T04:24:25Z

Methods and Implementation

← Older revision		Revision as of 04:24, 17 December 2018
Line 22:		Line 22:

	== Methods and Implementation ==		== Methods and Implementation ==
	As exposure fusion is now a common technique to generate HDR images for low-resolution displays, we will focus on the robustness of the alignment algorithm proposed in the HDR+ paper. For our implementation, we have implemented the alignment algorithm and a simplified version of the merge algorithm as well as part of the post processing steps in order to view the generated images.		As exposure fusion is now a common technique to generate HDR images for low-resolution displays, we will focus on the robustness of the alignment algorithm proposed in the HDR+ paper. For our implementation, we have implemented the alignment algorithm and a simplified version of the merge algorithm as well as part of the post processing steps in order to view the generated images. We did this all in MATLAB, and our GitHub is referenced in 'Appendix I'.

	[[File:OurImplementation.PNG\|frame\|center\|'''Figure 2:''' Our Pipeline Implementation]]		[[File:OurImplementation.PNG\|frame\|center\|'''Figure 2:''' Our Pipeline Implementation]]

imported>Student2018: /* References */

2018-12-17T04:23:47Z

References

← Older revision		Revision as of 04:23, 17 December 2018
Line 168:		Line 168:
	3. HDR+ pipeline (http://timothybrooks.com/tech/hdr-plus/)		3. HDR+ pipeline (http://timothybrooks.com/tech/hdr-plus/)

	4. Our git implementation https://github.com/jenniferlin0902/HDRMerge
			== Appendix I ==
			Our git implementation https://github.com/jenniferlin0902/HDRMerge

			== Appendix II ==

			Jennifer Lin was in charge of the implementation of the code and analysis of different merge algorithms. Warren worked on the implementation and analysis of Fast L2. Linda Banh worked on the test images and analyzing the post-processing part of the pipeline. We worked together on the write-up.

imported>Student2018: /* Introduction */

2018-12-17T04:20:56Z

Introduction

← Older revision		Revision as of 04:20, 17 December 2018
Line 1:		Line 1:
	== Introduction ==		== Introduction and Background ==

	[[File:Googlepipeline.PNG \| 1150px \|thumb\|center\|'''Figure 1:''' Google's HDR+ Image Pipeline [1]]]		[[File:Googlepipeline.PNG \| 1150px \|thumb\|center\|'''Figure 1:''' Google's HDR+ Image Pipeline [1]]]

imported>Student2018: /* Shifts and Rotations */

2018-12-15T04:38:36Z

Shifts and Rotations

← Older revision		Revision as of 04:38, 15 December 2018
Line 132:		Line 132:
	Below is the original image with the '1' rotated by clockwise by 15 degrees and shifted down 300px and left 100 px.		Below is the original image with the '1' rotated by clockwise by 15 degrees and shifted down 300px and left 100 px.

	The images are displayed in the following order: original image ~~-->~~ after alignment ~~-->~~ after temporal average merge		The images are displayed in the following order: original image(left), after alignment(middle), after temporal average merge(right)

	[[File:shifted_and_rotated_original.png\|400px]] [[File:shifted_and_rotated_only_align.png \| 400px]] [[File:shifted_and_rotated_with_merge.png \| 400px]]		[[File:shifted_and_rotated_original.png\|400px]] [[File:shifted_and_rotated_only_align.png \| 400px]] [[File:shifted_and_rotated_with_merge.png \| 400px]]

imported>Student2018: /* Testing Limits of Translational Shifts */

2018-12-15T04:37:25Z

Testing Limits of Translational Shifts

← Older revision		Revision as of 04:37, 15 December 2018
Line 138:		Line 138:
	It can be seen that our algorithm seems to do well with translational shifts and rotations. In the final image (after temporal average merge), there is some noise at the bottom of the '1' but it is quite negligible. Therefore in the following sections, we will look at where the limitations of our algorithm lies for shifts and rotations.		It can be seen that our algorithm seems to do well with translational shifts and rotations. In the final image (after temporal average merge), there is some noise at the bottom of the '1' but it is quite negligible. Therefore in the following sections, we will look at where the limitations of our algorithm lies for shifts and rotations.

	===== Testing Limits of ~~Translational Shifts~~ =====		===== Testing Limits of Translations =====

imported>Student2018: /* Testing Limits of Translational Shifts */

2018-12-15T04:37:08Z

Testing Limits of Translational Shifts

← Older revision		Revision as of 04:37, 15 December 2018
Line 141:		Line 141:


	[[File:shift.PNG \|frame\|'''Figure 4''': Shifts\|center ]]		[[File:shift.PNG \|frame\|'''Figure 4''': Image after Different Type of Shifts\|center ]]

	===== Testing Limits of Rotations =====		===== Testing Limits of Rotations =====

imported>Student2018: /* Conclusions */

2018-12-15T04:24:30Z

Conclusions

← Older revision		Revision as of 04:24, 15 December 2018
Line 157:		Line 157:

	== Conclusions ==		== Conclusions ==
	The series of tests with alignment algorithm indicate that the proposed alignment algorithm is robust against most translational movement and a limited degree of rotational movement in the burst image. However, it does not handle random white noises well. Although the merge algorithm is robust against alignment failure, we are worried that this issue may manifest itself if the steps were to deployed in low-light condition where the SNR is significantly lower, reducing the noise reduction benefit from the burst photography technique.		The series of tests with alignment algorithm indicate that the proposed alignment algorithm is robust against most translational movement and a limited degree of rotational movement in the burst image. However, it does not handle random white noises well. Although the merge algorithm is robust against alignment failure, we are worried that this issue may manifest itself if the steps were to be deployed in low-light condition where the SNR is significantly lower, reducing the noise reduction benefit from the burst photography technique.

	In this project, we investigated the robustness of alignment with only part of the HDR+ pipeline. Most of the testing is done on a post processed jpg or png images. Although we can get a preliminary conclusion of the performance of the alignment algorithm, we were not able to investigate the alignment robustness with regard to the HDR+ pipeline. In particular, the goal of alignment is to reduce the noise in the merge and the HDR compression stage of the pipeline. Without the proper merge and exposure fusion step, it is hard to conclude the affect of alignment failure to the resulting HDR image. Moving forward, we would like to test the alignment with the complete pipeline. To achieve this, we would like to work on completing the post processing pipeline including the exposure fusion step as well as the merge algorithm.		In this project, we investigated the robustness of alignment with only part of the HDR+ pipeline. Most of the testing is done on a post processed jpg or png images. Although we can get a preliminary conclusion of the performance of the alignment algorithm, we were not able to investigate the alignment robustness with regard to the HDR+ pipeline. In particular, the goal of alignment is to reduce the noise in the merge and the HDR compression stage of the pipeline. Without the proper merge and exposure fusion step, it is hard to conclude the affect of alignment failure to the resulting HDR image. Moving forward, we would like to test the alignment with the complete pipeline. To achieve this, we would like to work on completing the post processing pipeline including the exposure fusion step as well as the merge algorithm.

imported>Student2018: /* Conclusions */

2018-12-15T04:23:38Z

Conclusions

← Older revision		Revision as of 04:23, 15 December 2018
Line 157:		Line 157:

	== Conclusions ==		== Conclusions ==
	The series of ~~test~~ with alignment algorithm ~~indicates~~ that the proposed alignment algorithm is robust against most translational movement and a limited degree of rotational movement in the burst image. However, it does not handle random white noises well. Although the merge algorithm is robust against alignment failure, we are worried that this issue may manifest itself if the steps were to deployed in low-light condition where the SNR is significantly lower, reducing the noise reduction benefit from the burst photography technique.		The series of tests with alignment algorithm indicate that the proposed alignment algorithm is robust against most translational movement and a limited degree of rotational movement in the burst image. However, it does not handle random white noises well. Although the merge algorithm is robust against alignment failure, we are worried that this issue may manifest itself if the steps were to deployed in low-light condition where the SNR is significantly lower, reducing the noise reduction benefit from the burst photography technique.

	In this project, we investigated the robustness of alignment with only part of the HDR+ pipeline. Most of the testing is done on a post processed jpg or png images. Although we can get a preliminary conclusion of the performance of the alignment algorithm, we were not able to investigate the alignment robustness with regard to the HDR+ pipeline. In particular, the goal of alignment is to reduce the noise in the merge and the HDR compression stage of the pipeline. Without the proper merge and exposure fusion step, it is hard to conclude the affect of alignment failure to the resulting HDR image. Moving forward, we would like to test the alignment with the complete pipeline. To achieve this, we would like to work on completing the post processing pipeline including the exposure fusion step as well as the merge algorithm.		In this project, we investigated the robustness of alignment with only part of the HDR+ pipeline. Most of the testing is done on a post processed jpg or png images. Although we can get a preliminary conclusion of the performance of the alignment algorithm, we were not able to investigate the alignment robustness with regard to the HDR+ pipeline. In particular, the goal of alignment is to reduce the noise in the merge and the HDR compression stage of the pipeline. Without the proper merge and exposure fusion step, it is hard to conclude the affect of alignment failure to the resulting HDR image. Moving forward, we would like to test the alignment with the complete pipeline. To achieve this, we would like to work on completing the post processing pipeline including the exposure fusion step as well as the merge algorithm.

imported>Student2018: /* Noise */

2018-12-15T04:23:15Z

Noise

← Older revision		Revision as of 04:23, 15 December 2018
Line 154:		Line 154:
	Looking at all the images, our algorithm does not seem that affected by translational shifts. However, it does seem to affected when there are rotations. Looking at the reconstructed image for 90 degree rotation, the '1' has a white streak in the middle and the reconstructed image for the 135 degree rotation has missing pixels at the bottom of '1'.		Looking at all the images, our algorithm does not seem that affected by translational shifts. However, it does seem to affected when there are rotations. Looking at the reconstructed image for 90 degree rotation, the '1' has a white streak in the middle and the reconstructed image for the 135 degree rotation has missing pixels at the bottom of '1'.

	In addition, the alignment algorithm cannot compensate pixel-wise noise well. When reconstructed, there appears to be even more noise in the image. This makes sense since our algorithm can only align and merge. When the algorithm is aligning, it cannot create a new black pixel where black pixels are missing in the alternate image. Therefore, whenever our algorithm hits a white pixel, it makes sense that our algorithm may get confused at where it is in the image and choose the wrong pixel during alignment. In ~~the~~ image processing pipeline random noise like this is typically removed by applying ~~denoising~~ filter, which is included in the merge step of the HDR+ pipeline.		In addition, the alignment algorithm cannot compensate pixel-wise noise well. When reconstructed, there appears to be even more noise in the image. This makes sense since our algorithm can only align and merge. When the algorithm is aligning, it cannot create a new black pixel where black pixels are missing in the alternate image. Therefore, whenever our algorithm hits a white pixel, it makes sense that our algorithm may get confused at where it is in the image and choose the wrong pixel during alignment. In an image processing pipeline, random noise like this is typically removed by applying a de-noising filter, which is included in the merge step of the HDR+ pipeline.

	== Conclusions ==		== Conclusions ==

imported>Student2018: /* Introduction */

2018-12-15T04:14:45Z

Introduction

← Older revision		Revision as of 04:14, 15 December 2018
Line 1:		Line 1:
	== Introduction ==		== Introduction ==

	[[File:Googlepipeline.PNG \| 1150px \|thumb\|center\|'''Figure 1:''' Google HDR+ Image Pipeline [1]]]		[[File:Googlepipeline.PNG \| 1150px \|thumb\|center\|'''Figure 1:''' Google's HDR+ Image Pipeline [1]]]

	'''HDR Overview'''		'''HDR Overview'''