PengStarobinets - Revision history

imported>Projects221: /* Background */

2013-03-20T18:30:49Z

Background

← Older revision		Revision as of 18:30, 20 March 2013
Line 11:		Line 11:
	The performance of long-distance imaging systems can often be strongly affected by atmospheric turbulence caused by variation of refractive index along the optical transmission path. Such turbulence can produce geometric distortion, space and time-variant defocus blur, and motion blur. An example is shown in the following video of the moon.		The performance of long-distance imaging systems can often be strongly affected by atmospheric turbulence caused by variation of refractive index along the optical transmission path. Such turbulence can produce geometric distortion, space and time-variant defocus blur, and motion blur. An example is shown in the following video of the moon.

	Below is the ~~exmaple~~ of atmospheric turbulence that was used for this project:		Below is the example of atmospheric turbulence that was used for this project:
	[[File:Moon_out_tj.gif\|center\|Moon with turbulence]]		[[File:Moon_out_tj.gif\|center\|Moon with turbulence]]
	<center> Recording of the moon with effects of turbulence<sup>1</sup></center>		<center> Recording of the moon with effects of turbulence<sup>1</sup></center>

imported>Projects221 at 06:45, 20 March 2013

2013-03-20T06:45:30Z

Show changes

imported>Projects221 at 06:10, 20 March 2013

2013-03-20T06:10:31Z

← Older revision		Revision as of 06:10, 20 March 2013
Line 287:		Line 287:

	==Code==		==Code==
	[[File:~~CodeFile~~.zip]]		[[File:Code_JPTS.zip]]

	==Data==		==Data==

imported>Projects221 at 06:07, 20 March 2013

2013-03-20T06:07:56Z

Show changes

imported>Projects221 at 05:09, 20 March 2013

2013-03-20T05:09:09Z

← Older revision		Revision as of 05:09, 20 March 2013
Line 113:		Line 113:

	= Results =		= Results =
			After a number of iterations to minimize the cost function, the new deformation matrix <math> \mathbf{W(x; } \overleftarrow{\mathbf{p}})</math> for a single subframe can look like this:

			[[File:deform_after_iter.png\|300px\|thumb\|center\|Deformation matrix after 10 iterations]]

	Using our process flow, we generated a set of sub-frames. As can be seen in the figure below, each individual sub-frame suffers from edge artifacts. Overlapping removes these artifacts and the final stitched image is also shown.		Using our process flow, we generated a set of sub-frames. As can be seen in the figure below, each individual sub-frame suffers from edge artifacts. Overlapping removes these artifacts and the final stitched image is also shown.
	[[File:before_after_stitching.PNG\|400px\|thumb\|center\|Frame before and after the stitching algorithm]]		[[File:before_after_stitching.PNG\|400px\|thumb\|center\|Frame before and after the stitching algorithm]]

imported>Projects221 at 22:22, 19 March 2013

2013-03-19T22:22:16Z

← Older revision		Revision as of 22:22, 19 March 2013
Line 13:		Line 13:
	Below is the exmaple of atmospheric turbulence that was used for this project:		Below is the exmaple of atmospheric turbulence that was used for this project:
	[[File:Moon_out_tj.gif\|center\|Moon with turbulence]]		[[File:Moon_out_tj.gif\|center\|Moon with turbulence]]
	<center> Recording of the moon with effects of turbulence </center>		<center> Recording of the moon with effects of turbulence<sup>1</sup></center>

	There have been many approaches to solving this problem that attempt to restore a single high-quality image from an observed frame sequence distorted by air turbulence. As in the video, these approaches, and the approach addressed in this paper, work under the assumption that the scene and the image sensor are both static and that observed motions are due to the air turbulence alone.		There have been many approaches to solving this problem that attempt to restore a single high-quality image from an observed frame sequence distorted by air turbulence. As in the video, these approaches, and the approach addressed in this paper, work under the assumption that the scene and the image sensor are both static and that observed motions are due to the air turbulence alone.

imported>Projects221 at 22:20, 19 March 2013

2013-03-19T22:20:12Z

← Older revision		Revision as of 22:20, 19 March 2013
Line 37:		Line 37:
	== Our Method ==		== Our Method ==

	To correct geometric distortion and reduce space and time-varying blur, we are proposing a new approach in this work capable of restoring a single high-quality image from a given image sequence distorted by atmospheric turbulence. This approach uses two major steps. For the first step, we use a B-spline based non-rigid image registration algorithm to register each observed frame with respect to a reference image ~~as described in [~~1~~](see References)~~. To improve registration accuracy, a symmetry constraint is introduced to the cost function to penalize inconsistency between the forward and backward deformation parameters during the estimation process. A fast Gauss-Newton method is used to reduce the computational cost of the registration algorithm.		To correct geometric distortion and reduce space and time-varying blur, we are proposing a new approach in this work capable of restoring a single high-quality image from a given image sequence distorted by atmospheric turbulence. This approach uses two major steps. For the first step, we use a B-spline based non-rigid image registration algorithm to register each observed frame with respect to a reference image<sup>1</sup>. To improve registration accuracy, a symmetry constraint is introduced to the cost function to penalize inconsistency between the forward and backward deformation parameters during the estimation process. A fast Gauss-Newton method is used to reduce the computational cost of the registration algorithm.
	Our new idea and an addition to this approach involves separating a single frame into a set of overlapping sub-frames and running the earlier-described algorithm on those sub-frames. In addition to reducing computational load and time, this improves on the original method by reducing the effect of systematic errors on the frame, since the errors are localized and isolated to the particular block rather than affecting the whole image. Another benefit of this approach is that scaling the number of the test points causes a much smaller processing time increase as opposed to the processing time that would be required for the same number of test points if applied to the whole image. The second step of our proposal is to stitch the undeformed back into a single image and then, using a high-pass sharpening filter produce a final, high-quality image.		Our new idea and an addition to this approach involves separating a single frame into a set of overlapping sub-frames and running the earlier-described algorithm on those sub-frames. In addition to reducing computational load and time, this improves on the original method by reducing the effect of systematic errors on the frame, since the errors are localized and isolated to the particular block rather than affecting the whole image. Another benefit of this approach is that scaling the number of the test points causes a much smaller processing time increase as opposed to the processing time that would be required for the same number of test points if applied to the whole image. The second step of our proposal is to stitch the undeformed back into a single image and then, using a high-pass sharpening filter produce a final, high-quality image.

imported>Projects221 at 22:19, 19 March 2013

2013-03-19T22:19:28Z

← Older revision		Revision as of 22:19, 19 March 2013
Line 37:		Line 37:
	== Our Method ==		== Our Method ==

	To correct geometric distortion and reduce space and time-varying blur, a new approach ~~is proposed~~ in this ~~paper~~ capable of restoring a single high-quality image from a given image sequence distorted by atmospheric turbulence. This approach uses two major steps. For the first step, we use a B-spline based non-rigid image registration algorithm to register each observed frame with respect to a reference image. To improve registration accuracy, a symmetry constraint is introduced to the cost function to penalize inconsistency between the forward and backward deformation parameters during the estimation process. A fast Gauss-Newton method is used to reduce the computational cost of the registration algorithm. ~~We improved the computational efficiency further by applying our method~~ to ~~multiple small blocks~~ and ~~stitching them together~~. In addition to reducing computational load and time, this improves on the original method by reducing the effect of systematic errors on the ~~system~~, ~~because an error will be~~ isolated to the particular block rather than affecting the whole image. In the second step~~, we restore~~ a ~~high quality~~ image ~~from the registered observed frames~~ using a high-pass sharpening filter.		To correct geometric distortion and reduce space and time-varying blur, we are proposing a new approach in this work capable of restoring a single high-quality image from a given image sequence distorted by atmospheric turbulence. This approach uses two major steps. For the first step, we use a B-spline based non-rigid image registration algorithm to register each observed frame with respect to a reference image as described in [1](see References). To improve registration accuracy, a symmetry constraint is introduced to the cost function to penalize inconsistency between the forward and backward deformation parameters during the estimation process. A fast Gauss-Newton method is used to reduce the computational cost of the registration algorithm.
			Our new idea and an addition to this approach involves separating a single frame into a set of overlapping sub-frames and running the earlier-described algorithm on those sub-frames. In addition to reducing computational load and time, this improves on the original method by reducing the effect of systematic errors on the frame, since the errors are localized and isolated to the particular block rather than affecting the whole image. Another benefit of this approach is that scaling the number of the test points causes a much smaller processing time increase as opposed to the processing time that would be required for the same number of test points if applied to the whole image. The second step of our proposal is to stitch the undeformed back into a single image and then, using a high-pass sharpening filter produce a final, high-quality image.


Line 44:		Line 45:

	==== Non-rigid Image Registration ====		==== Non-rigid Image Registration ====
	First, we obtain a reference image without image deformation, generally by averaging the frame sequence. The math is specified below in the equations section. However, in general, the geometric distortion between an observed distorted image and the reference image is obtained by the movement of control points placed in a grid on the images. B-spline interpolation is then used to map all the points from the reference image to the deformed image<sup>1</sup>. A cost function is then calculated to minimize both the error between the mapped points and maximize symmetry between forward and backward deformations. After this cost function is minimized, a motion vector from the deformed image to the reference image is generated, and ~~we can~~ then ~~register~~ the ~~deformed~~ image and ~~apply a sharpening filter to create~~ a ~~high-quality image~~.		First, we obtain a reference image without image deformation, generally by averaging the frame sequence. The math is specified below in the equations section. However, in general, the geometric distortion between an observed distorted image and the reference image is obtained by the movement of control points placed in a grid on the images. B-spline interpolation is then used to map all the points from the reference image to the deformed image<sup>1</sup>. A cost function is then calculated to minimize both the error between the mapped points and maximize symmetry between forward and backward deformations. After this cost function is minimized, a motion vector from the deformed image to the reference image is generated. This process is repeated for each sub-frame and then the sub-frames are recombined into a single image.

			The figure below shows an example of the sub-frame implementation of turbulence compensation.

			[[File:Turbulance_test_points.PNG\|400px\|thumb\|center\| (Left) Test points and their deformed positions for each sub-frame. (Right) Deformed location of each pixel for a single subframe.]]

	= Equations=		= Equations=
Line 109:		Line 114:
	= Results =		= Results =
	Using our process flow, we generated a set of sub-frames. As can be seen in the figure below, each individual sub-frame suffers from edge artifacts. Overlapping removes these artifacts and the final stitched image is also shown.		Using our process flow, we generated a set of sub-frames. As can be seen in the figure below, each individual sub-frame suffers from edge artifacts. Overlapping removes these artifacts and the final stitched image is also shown.
	[[File:before_after_stitching.PNG\|~~200px~~\|thumb\|center\|Frame before and after the stitching algorithm]]		[[File:before_after_stitching.PNG\|400px\|thumb\|center\|Frame before and after the stitching algorithm]]

	The images below show the frames as the progression from distorted image to the final improved one.		The images below show the frames as the progression from distorted image to the final improved one.
	[[File:Turb_cancelation_flow.PNG\|~~200px~~\|thumb\|center\|Process Flow]]		[[File:Turb_cancelation_flow.PNG\|400px\|thumb\|center\|Process Flow]]

	= Conclusions =		= Conclusions =

imported>Projects221 at 22:02, 19 March 2013

2013-03-19T22:02:19Z

← Older revision		Revision as of 22:02, 19 March 2013
Line 12:		Line 12:

	Below is the exmaple of atmospheric turbulence that was used for this project:		Below is the exmaple of atmospheric turbulence that was used for this project:
	[[File:Moon_out_tj.gif\|center\|~~alt=~~Moon with turbulence]]		[[File:Moon_out_tj.gif\|center\|Moon with turbulence]]
	<center> Recording of the moon with effects of turbulence </center>		<center> Recording of the moon with effects of turbulence </center>

Line 107:		Line 107:
	where <math> \frac{\part R(\mathbf{W(x; } \overrightarrow{\mathbf{p}}^l \mathbf{)})}{\part \mathbf{W}} = [r_x(\mathbf{W}),r_y(\mathbf{W})]</math> and is the gradient of the averaged image R at the deformed pixel location <math> \mathbf{W(x; } \overrightarrow{\mathbf{p}}^l \mathbf{)}) </math>.		where <math> \frac{\part R(\mathbf{W(x; } \overrightarrow{\mathbf{p}}^l \mathbf{)})}{\part \mathbf{W}} = [r_x(\mathbf{W}),r_y(\mathbf{W})]</math> and is the gradient of the averaged image R at the deformed pixel location <math> \mathbf{W(x; } \overrightarrow{\mathbf{p}}^l \mathbf{)}) </math>.

	= Results - ~~What you found =~~		= Results =
			Using our process flow, we generated a set of sub-frames. As can be seen in the figure below, each individual sub-frame suffers from edge artifacts. Overlapping removes these artifacts and the final stitched image is also shown.
			[[File:before_after_stitching.PNG\|200px\|thumb\|center\|Frame before and after the stitching algorithm]]

			The images below show the frames as the progression from distorted image to the final improved one.
			[[File:Turb_cancelation_flow.PNG\|200px\|thumb\|center\|Process Flow]]

	= Conclusions =		= Conclusions =

	In this project, we proposed a new framework for restoration of a single high-quality frame from an image sequence distorted by air turbulence. The proposed algorithm uses a multi-frame reconstruction approach with the addition of sub-frame evaluations with overlap and sub-image stitching to create a final image. The algorithm registers the sub-frames to suppress spatial and geometric deformation using a B-spline based non=rigid image registration method. Next the frames are stitched together using an edge-correlation algorithm. Finally, a sharpening high-pass filter is applied to the final image to generate a final high-quality output. We feel that our results show that our method is both an effective implementation of the multi-frame reconstruction framework and possibly improves on the process in some ways, by preventing localized errors from affecting the whole image.		In this project, we proposed a new framework for restoration of a single high-quality frame from an image sequence distorted by air turbulence. The proposed algorithm uses a multi-frame reconstruction approach with the addition of sub-frame evaluations with overlap and sub-image stitching to create a final image. The algorithm registers the sub-frames to suppress spatial and geometric deformation using a B-spline based non-rigid image registration method. Next the frames are stitched together using an edge-correlation algorithm. Finally, a sharpening high-pass filter is applied to the final image to generate a final high-quality output. We feel that our results show that our method is both an effective implementation of the multi-frame reconstruction framework and possibly improves on the process in some ways, by preventing localized errors from affecting the whole image.

	= References =		= References =
Line 138:		Line 143:
	[[File:MoonSurface.zip \| MoonSurface data]]		[[File:MoonSurface.zip \| MoonSurface data]]

	= Appendix II - Work partition ~~(if a group project)~~ =		= Appendix II - Work partition =
	~~Brian and Bob gave the lectures. Jon mucked around~~ on the wiki.		Both of us have been working together on the project/wiki as a team with even work distribution.

imported>Projects221: /* Conclusions */

2013-03-19T21:48:06Z

Conclusions

← Older revision		Revision as of 21:48, 19 March 2013
Line 111:		Line 111:
	= Conclusions =		= Conclusions =

	~~Here~~ is ~~where you say what your~~ results ~~mean~~.		In this project, we proposed a new framework for restoration of a single high-quality frame from an image sequence distorted by air turbulence. The proposed algorithm uses a multi-frame reconstruction approach with the addition of sub-frame evaluations with overlap and sub-image stitching to create a final image. The algorithm registers the sub-frames to suppress spatial and geometric deformation using a B-spline based non=rigid image registration method. Next the frames are stitched together using an edge-correlation algorithm. Finally, a sharpening high-pass filter is applied to the final image to generate a final high-quality output. We feel that our results show that our method is both an effective implementation of the multi-frame reconstruction framework and possibly improves on the process in some ways, by preventing localized errors from affecting the whole image.

	= References =		= References =