Benbrook Chen Leckie Lopez - Revision history

imported>Student221: /* Results */

2019-12-16T01:06:48Z

Results

← Older revision		Revision as of 01:06, 16 December 2019
Line 65:		Line 65:

	{\|style="margin: 0 auto;"		{\|style="margin: 0 auto;"
	\|[[File:alignedcars.png\|thumb\|center\|~~400px~~\|upright\|alt=A\|Fig.4 Aligned Test Images 1]]		\|[[File:alignedcars.png\|thumb\|center\|390px\|upright\|alt=A\|Fig.4 Aligned Test Images 1]]
	\|[[File:alignedchess.png\|thumb\|center\|~~400px~~\|upright\|alt=A\|Fig.5 Aligned Test Images 2]]		\|[[File:alignedchess.png\|thumb\|center\|390px\|upright\|alt=A\|Fig.5 Aligned Test Images 2]]
	\|}		\|}

	{\|style="margin: 0 auto;"		{\|style="margin: 0 auto;"
	\|[[File:alignedstreet.png\|thumb\|center\|~~400px~~\|upright\|alt=A\|Fig.6 Aligned Test Images 3]]		\|[[File:alignedstreet.png\|thumb\|center\|390px\|upright\|alt=A\|Fig.6 Aligned Test Images 3]]
	\|[[File:alignedmiguel.png\|thumb\|center\|~~400px~~\|upright\|alt=A\|Fig.7 Aligned Test Images 4]]		\|[[File:alignedmiguel.png\|thumb\|center\|390px\|upright\|alt=A\|Fig.7 Aligned Test Images 4]]
	\|}		\|}


	{\|style="margin: 0 auto;"		{\|style="margin: 0 auto;"
	\|[[File:alignedteapot.png\|thumb\|center\|~~400px~~\|upright\|alt=A\|Fig.8 Aligned Test Images 5]]		\|[[File:alignedteapot.png\|thumb\|center\|390px\|upright\|alt=A\|Fig.8 Aligned Test Images 5]]
	\|}		\|}

imported>Student221: /* Results */

2019-12-16T01:05:19Z

Results

← Older revision		Revision as of 01:05, 16 December 2019
Line 65:		Line 65:

	{\|style="margin: 0 auto;"		{\|style="margin: 0 auto;"
	\|[[File:alignedcars.png\|thumb\|center\|~~450px~~\|upright\|alt=A\|Fig.4 Aligned Test Images 1]]		\|[[File:alignedcars.png\|thumb\|center\|400px\|upright\|alt=A\|Fig.4 Aligned Test Images 1]]
	\|[[File:alignedchess.png\|thumb\|center\|~~450px~~\|upright\|alt=A\|Fig.5 Aligned Test Images 2]]		\|[[File:alignedchess.png\|thumb\|center\|400px\|upright\|alt=A\|Fig.5 Aligned Test Images 2]]
	~~\|[[File:alignedstreet.png\|thumb\|center\|450px\|upright\|alt=A\|Fig.6 Aligned Test Images 3~~]]
	\|}		\|}

	{\|style="margin: 0 auto;"		{\|style="margin: 0 auto;"
	\|[[File:alignedmiguel.png\|thumb\|center\|~~450px~~\|upright\|alt=A\|Fig.5 Aligned Test Images 4]]		\|[[File:alignedstreet.png\|thumb\|center\|400px\|upright\|alt=A\|Fig.6 Aligned Test Images 3]]
	\|[[File:alignedteapot.png\|thumb\|center\|~~450px~~\|upright\|alt=A\|Fig.5 Aligned Test Images 5]]		\|[[File:alignedmiguel.png\|thumb\|center\|400px\|upright\|alt=A\|Fig.7 Aligned Test Images 4]]
			\|}


			{\|style="margin: 0 auto;"
			\|[[File:alignedteapot.png\|thumb\|center\|400px\|upright\|alt=A\|Fig.8 Aligned Test Images 5]]
	\|}		\|}

imported>Student221: /* Methods */

2019-12-16T00:44:00Z

Methods

← Older revision		Revision as of 00:44, 16 December 2019
Line 21:		Line 21:

	Images in black and white are described by 2D array (x by y by 1) of uint8 values in Matlab, where uint8 is an 8 bit integer between 0 and 255. Colored images are described by a 3D array (x by y by z) in Matlab, where each pixel has 3 associated uint8 values representing R,G, and B.		Images in black and white are described by 2D array (x by y by 1) of uint8 values in Matlab, where uint8 is an 8 bit integer between 0 and 255. Colored images are described by a 3D array (x by y by z) in Matlab, where each pixel has 3 associated uint8 values representing R,G, and B.

			The scripts on this page request installation of the Computer Vision Toolbox from Mathworks, and the IAT toolbox which can be found here:
			https://sites.google.com/site/imagealignment/home

	== Algorithms ==		== Algorithms ==

imported>Student221: /* Methods */

2019-12-16T00:41:33Z

Methods

← Older revision		Revision as of 00:41, 16 December 2019
Line 20:		Line 20:
	The alignment algorithms we tested rely on identifying a delta_x and delta_y for each pixel, that describe the difference in "identical" pixels between the image we want to align to, called the target image, and the image we are trying to align, called the warped image. The scripts provided provide plots showing the target and warped images, and then the error of each algorithm and the alignment results. Once we align images, we use mean squared error (MSE) and structural similarity index (SSIM) as metrics to compare the results between algorithms. The optical flow map for each algorithm is found to help visualize the delta_x and delta_y values (e.g. the apparent direction of alignment).		The alignment algorithms we tested rely on identifying a delta_x and delta_y for each pixel, that describe the difference in "identical" pixels between the image we want to align to, called the target image, and the image we are trying to align, called the warped image. The scripts provided provide plots showing the target and warped images, and then the error of each algorithm and the alignment results. Once we align images, we use mean squared error (MSE) and structural similarity index (SSIM) as metrics to compare the results between algorithms. The optical flow map for each algorithm is found to help visualize the delta_x and delta_y values (e.g. the apparent direction of alignment).

	Images in black and white are described by x by y by 1 uint8 ~~vectors~~ in Matlab, where uint8 is an 8 bit integer between 0 and 255. Colored images are described by x by y by ~~3 vectors~~ in Matlab, where each pixel has 3 associated uint8 values representing R,G, and B.		Images in black and white are described by 2D array (x by y by 1) of uint8 values in Matlab, where uint8 is an 8 bit integer between 0 and 255. Colored images are described by a 3D array (x by y by z) in Matlab, where each pixel has 3 associated uint8 values representing R,G, and B.

	== Algorithms ==		== Algorithms ==

imported>Student221: /* Methods */

2019-12-16T00:38:53Z

Methods

← Older revision		Revision as of 00:38, 16 December 2019
Line 19:		Line 19:

	The alignment algorithms we tested rely on identifying a delta_x and delta_y for each pixel, that describe the difference in "identical" pixels between the image we want to align to, called the target image, and the image we are trying to align, called the warped image. The scripts provided provide plots showing the target and warped images, and then the error of each algorithm and the alignment results. Once we align images, we use mean squared error (MSE) and structural similarity index (SSIM) as metrics to compare the results between algorithms. The optical flow map for each algorithm is found to help visualize the delta_x and delta_y values (e.g. the apparent direction of alignment).		The alignment algorithms we tested rely on identifying a delta_x and delta_y for each pixel, that describe the difference in "identical" pixels between the image we want to align to, called the target image, and the image we are trying to align, called the warped image. The scripts provided provide plots showing the target and warped images, and then the error of each algorithm and the alignment results. Once we align images, we use mean squared error (MSE) and structural similarity index (SSIM) as metrics to compare the results between algorithms. The optical flow map for each algorithm is found to help visualize the delta_x and delta_y values (e.g. the apparent direction of alignment).

			Images in black and white are described by x by y by 1 uint8 vectors in Matlab, where uint8 is an 8 bit integer between 0 and 255. Colored images are described by x by y by 3 vectors in Matlab, where each pixel has 3 associated uint8 values representing R,G, and B.

	== Algorithms ==		== Algorithms ==

imported>Student221: /* Optical Flow */

2019-12-16T00:36:29Z

Optical Flow

← Older revision		Revision as of 00:36, 16 December 2019
Line 53:		Line 53:

	=== Optical Flow ===		=== Optical Flow ===
	Optical flow is the pattern of apparent motion of a visual scene. The optical flow can be calculated by finding the delta_x and delta_y of each pixel for two or more 2D image frames captured at different times, or from different angles. This method relies on differentiation since it uses the Taylor series approximations of the image signal. It can be useful to visualize the optical flow because in some cases of alignment, we only care about seeing which "direction" an image frame is moving.		Optical flow is the pattern of apparent motion of a visual scene. The optical flow can be calculated by finding the delta_x and delta_y of each pixel for two or more 2D image frames (voxels for 3D image frames) captured at different times, or from different angles. This method relies on differentiation since it uses the Taylor series approximations of the image signal. It can be useful to visualize the optical flow because in some cases of alignment, we only care about seeing which "direction" an image frame is moving.

	== Results ==		== Results ==

imported>Student221: /* Conclusions */

2019-12-16T00:26:43Z

Conclusions

← Older revision		Revision as of 00:26, 16 December 2019
Line 88:		Line 88:

	{\|style="margin: 0 auto;"		{\|style="margin: 0 auto;"
	\|[[File:sanmiguel_OpticalFlow.png\|thumb\|center\|450px\|upright\|alt=A\|Fig.11]]		\|[[File:sanmiguel_OpticalFlow.png\|thumb\|center\|450px\|upright\|alt=A\|Fig.11 - Optical Flow map for Architecture Scene]]
	\|}		\|}

imported>Student221: /* Conclusions */

2019-12-16T00:24:46Z

Conclusions

← Older revision		Revision as of 00:24, 16 December 2019
Line 85:		Line 85:
	{\|style="margin: 0 auto;"		{\|style="margin: 0 auto;"
	\|[[File:Results.png\|thumb\|center\|450px\|upright\|alt=A\|Fig.11]]		\|[[File:Results.png\|thumb\|center\|450px\|upright\|alt=A\|Fig.11]]
			\|}

			{\|style="margin: 0 auto;"
			\|[[File:sanmiguel_OpticalFlow.png\|thumb\|center\|450px\|upright\|alt=A\|Fig.11]]
	\|}		\|}

imported>Student221: /* Test Metrics */

2019-12-16T00:11:11Z

Test Metrics

← Older revision		Revision as of 00:11, 16 December 2019
Line 42:		Line 42:

	=== Mean Square Error (MSE) ===		=== Mean Square Error (MSE) ===
	MSE is a quantitative measure of signal quality. MSE is a simple to implement and is widely used to gauge ~~estimator~~ accuracy. However, it is not well matched to the human perception of images. Therefore, its effectiveness as a judge of image alignment is limited~~. MSE is always a positive value that measures the quality of an estimator and values closest to zero are better~~.		MSE is a quantitative measure of signal quality. MSE is always a positive value that measures the quality of an estimator and values closest to zero are better. MSE is a simple to implement and is widely used to gauge error signal accuracy. It describes a physical quantity, which sit eh energy of the error signal. The MSE is convention in the field of signal processing, optimization, and image alignment. However, MSE is not well matched to the human perception of images. Therefore, its effectiveness as a judge of image alignment is limited.

	MSE is computed by averaging the squared intensity differences of distorted and reference image pixels, t two distorted images with the same MSE may have very different types of errors, some of which are much more visible than others. Most perceptual image quality assessment approaches proposed in the literature attempt to weight different aspects of the error signal according to their visibility, as determined by psychophysical measurements in humans or physiological measurements in animals.		MSE is computed by averaging the squared intensity differences of distorted and reference image pixels, but two distorted images with the same MSE may have very different types of errors, some of which are much more visible than others. Most perceptual image quality assessment approaches proposed in the literature attempt to weight different aspects of the error signal according to their visibility, as determined by psychophysical measurements in humans or physiological measurements in animals. Therefore, MSE is probably not the best metric for perceived alignment quality.


	===Structural Similarity Index (SSIM) ===		=== Structural Similarity Index (SSIM) ===
	The SSIM quantifies the degradation of image quality after processing or losses during data transmission.The SSIM takes two images from the same image capture and measures the perceptual difference. SSIM was created as an improvement to other traditional methods like peak signal-to-noise ratio (PSNR) and mean squared error (MSE). SSIM is meant to predict the human perception of quality. Because of this, it only measures the perceptual difference, so the reference image must be of high quality. Therefore it is more suited to judge the quality of image alignment algorithms. Desired values are as close to 1 as possible.		The SSIM quantifies the degradation of image quality after processing or losses during data transmission.The SSIM takes two images from the same image capture and measures the perceptual difference. SSIM was created as an improvement to other traditional methods like peak signal-to-noise ratio (PSNR) and mean squared error (MSE). SSIM is meant to predict the human perception of quality. Because of this, it only measures the perceptual difference, so the reference image must be of high quality. Therefore it is more suited to judge the quality of image alignment algorithms. Desired values are as close to 1 as possible (meaning the structural similarity is 100%).

	While MSE and PSNR estimate absolute errors, SSIM calculates changes in structural information and takes into account image perception through luminescence masking and contrast masking components. Structural information is defined as the strong correlation between pixels that are spatially close. The correlations can elucidate structural information of the image scene, like objects. Luminance masking is the concept that distortions are more visible in darker regions. Contrast masking is the concept that distortions are less noticeable in areas of the image that are highly textured.		While MSE and PSNR estimate absolute errors, SSIM calculates changes in structural information and takes into account image perception through luminescence masking and contrast masking components. Structural information is defined as the strong correlation between pixels that are spatially close. The correlations can elucidate structural information of the image scene, like objects. Luminance masking is the concept that distortions are more visible in darker regions. Contrast masking is the concept that distortions are less noticeable in areas of the image that are highly textured.

			=== Optical Flow ===
			Optical flow is the pattern of apparent motion of a visual scene. The optical flow can be calculated by finding the delta_x and delta_y of each pixel for two or more 2D image frames captured at different times, or from different angles. This method relies on differentiation since it uses the Taylor series approximations of the image signal. It can be useful to visualize the optical flow because in some cases of alignment, we only care about seeing which "direction" an image frame is moving.

	== Results ==		== Results ==

imported>Student221: /* Methods */

2019-12-16T00:00:10Z

Methods

← Older revision		Revision as of 00:00, 16 December 2019
Line 16:		Line 16:
	== Methods ==		== Methods ==

	The main goal of this project is to experiment with existing image alignment algorithms and analyze their performance. We will focus on comparing algorithms that utilize feature-based, linear transformation models. We used data from the ISET3D software on simulated images taken by various cameras and sensors, and images where the object is moving as well as the global scene. Once we align images, we ~~can~~ use ~~root~~ mean squared error ~~or SSIM~~ (structural similarity) as metrics to compare the results between algorithms.		The main goal of this project is to experiment with existing image alignment algorithms and analyze their performance. We will focus on comparing algorithms that utilize feature-based, linear transformation models. We used data from the ISET3D software on simulated images taken by various cameras and sensors, and images where the object is moving as well as the global scene. The image alignment algorithms will work with other image formats and types as well, such as iPhone images.

			The alignment algorithms we tested rely on identifying a delta_x and delta_y for each pixel, that describe the difference in "identical" pixels between the image we want to align to, called the target image, and the image we are trying to align, called the warped image. The scripts provided provide plots showing the target and warped images, and then the error of each algorithm and the alignment results. Once we align images, we use mean squared error (MSE) and structural similarity index (SSIM) as metrics to compare the results between algorithms. The optical flow map for each algorithm is found to help visualize the delta_x and delta_y values (e.g. the apparent direction of alignment).

	== Algorithms ==		== Algorithms ==

← Older revision		Revision as of 01:06, 16 December 2019
Line 65:		Line 65:

	{\|style="margin: 0 auto;"		{\|style="margin: 0 auto;"
	\|[[File:alignedcars.png\|thumb\|center\|~~400px~~\|upright\|alt=A\|Fig.4 Aligned Test Images 1]]		\|[[File:alignedcars.png\|thumb\|center\|390px\|upright\|alt=A\|Fig.4 Aligned Test Images 1]]
	\|[[File:alignedchess.png\|thumb\|center\|~~400px~~\|upright\|alt=A\|Fig.5 Aligned Test Images 2]]		\|[[File:alignedchess.png\|thumb\|center\|390px\|upright\|alt=A\|Fig.5 Aligned Test Images 2]]
	\|}		\|}

	{\|style="margin: 0 auto;"		{\|style="margin: 0 auto;"
	\|[[File:alignedstreet.png\|thumb\|center\|~~400px~~\|upright\|alt=A\|Fig.6 Aligned Test Images 3]]		\|[[File:alignedstreet.png\|thumb\|center\|390px\|upright\|alt=A\|Fig.6 Aligned Test Images 3]]
	\|[[File:alignedmiguel.png\|thumb\|center\|~~400px~~\|upright\|alt=A\|Fig.7 Aligned Test Images 4]]		\|[[File:alignedmiguel.png\|thumb\|center\|390px\|upright\|alt=A\|Fig.7 Aligned Test Images 4]]
	\|}		\|}


	{\|style="margin: 0 auto;"		{\|style="margin: 0 auto;"
	\|[[File:alignedteapot.png\|thumb\|center\|~~400px~~\|upright\|alt=A\|Fig.8 Aligned Test Images 5]]		\|[[File:alignedteapot.png\|thumb\|center\|390px\|upright\|alt=A\|Fig.8 Aligned Test Images 5]]
	\|}		\|}

← Older revision		Revision as of 01:05, 16 December 2019
Line 65:		Line 65:

	{\|style="margin: 0 auto;"		{\|style="margin: 0 auto;"
	\|[[File:alignedcars.png\|thumb\|center\|~~450px~~\|upright\|alt=A\|Fig.4 Aligned Test Images 1]]		\|[[File:alignedcars.png\|thumb\|center\|400px\|upright\|alt=A\|Fig.4 Aligned Test Images 1]]
	\|[[File:alignedchess.png\|thumb\|center\|~~450px~~\|upright\|alt=A\|Fig.5 Aligned Test Images 2]]		\|[[File:alignedchess.png\|thumb\|center\|400px\|upright\|alt=A\|Fig.5 Aligned Test Images 2]]
	~~\|[[File:alignedstreet.png\|thumb\|center\|450px\|upright\|alt=A\|Fig.6 Aligned Test Images 3~~]]
	\|}		\|}

	{\|style="margin: 0 auto;"		{\|style="margin: 0 auto;"
	\|[[File:alignedmiguel.png\|thumb\|center\|~~450px~~\|upright\|alt=A\|Fig.5 Aligned Test Images 4]]		\|[[File:alignedstreet.png\|thumb\|center\|400px\|upright\|alt=A\|Fig.6 Aligned Test Images 3]]
	\|[[File:alignedteapot.png\|thumb\|center\|~~450px~~\|upright\|alt=A\|Fig.5 Aligned Test Images 5]]		\|[[File:alignedmiguel.png\|thumb\|center\|400px\|upright\|alt=A\|Fig.7 Aligned Test Images 4]]
			\|}


			{\|style="margin: 0 auto;"
			\|[[File:alignedteapot.png\|thumb\|center\|400px\|upright\|alt=A\|Fig.8 Aligned Test Images 5]]
	\|}		\|}