FeiYuePsych2012Project - Revision history

imported>Psych2012: /* Background */

2012-03-21T19:49:37Z

Background

← Older revision		Revision as of 19:49, 21 March 2012
Line 3:		Line 3:
	In today's computer graphics, a large amount of textures are mapped onto surfaces of objects to add visual details and provide realism. These textures can consume a large portion of system and video memory. An efficient compression algorithm can reduce the amount of memory required for storing the textures and allow better performance by reducing the memory bandwidth used to fetch textures. An efficient real-time compression algorithm is important for dynamic texture generation and for texture streaming where the source compression format is different than the GPU hardware compression format.		In today's computer graphics, a large amount of textures are mapped onto surfaces of objects to add visual details and provide realism. These textures can consume a large portion of system and video memory. An efficient compression algorithm can reduce the amount of memory required for storing the textures and allow better performance by reducing the memory bandwidth used to fetch textures. An efficient real-time compression algorithm is important for dynamic texture generation and for texture streaming where the source compression format is different than the GPU hardware compression format.

	The DXT compression format is common across most major GPU hardware. DXT1, stores 16 input pixels in 64 bits of output. The output consists of two 16-bit RGB 5:6:5 color values and a 4x4 two bit lookup table. [1] The compression ratio is 6:1 for 24 bit source images. DXT5, converts 16 input pixels into 128 bits of output. The output consists of 64 bits of alpha channel data, followed by 64 bits of color data [1]. The compression ratio is 4:1.		The DXT compression format is common across most major GPU hardware. DXT1, stores 16 input pixels in 64 bits of output. The output consists of two 16-bit RGB 5:6:5 color values and a 4x4 two bit lookup table [1]. The compression ratio is 6:1 for 24 bit source images. DXT5, converts 16 input pixels into 128 bits of output. The output consists of 64 bits of alpha channel data, followed by 64 bits of color data [1]. The compression ratio is 4:1.

	== Real-Time YCoCg-DXT Compression ==		== Real-Time YCoCg-DXT Compression ==

imported>Psych2012: /* References */

2012-03-21T19:49:12Z

References

← Older revision		Revision as of 19:49, 21 March 2012
Line 96:		Line 96:
	= References =		= References =

	~~List references~~. ~~Include links to papers that are online~~.		[1] http://en.wikipedia.org/wiki/S3_Texture_Compression

			[2] http://developer.download.nvidia.com/whitepapers/2007/Real-Time-YCoCg-DXT-Compression/Real-Time%20YCoCg-DXT%20Compression.pdf

			[3] http://wiki.multimedia.cx/index.php?title=YCoCg

	= Appendix I - Code and Data =		= Appendix I - Code and Data =

imported>Psych2012: /* Background */

2012-03-21T19:48:25Z

Background

← Older revision		Revision as of 19:48, 21 March 2012
Line 3:		Line 3:
	In today's computer graphics, a large amount of textures are mapped onto surfaces of objects to add visual details and provide realism. These textures can consume a large portion of system and video memory. An efficient compression algorithm can reduce the amount of memory required for storing the textures and allow better performance by reducing the memory bandwidth used to fetch textures. An efficient real-time compression algorithm is important for dynamic texture generation and for texture streaming where the source compression format is different than the GPU hardware compression format.		In today's computer graphics, a large amount of textures are mapped onto surfaces of objects to add visual details and provide realism. These textures can consume a large portion of system and video memory. An efficient compression algorithm can reduce the amount of memory required for storing the textures and allow better performance by reducing the memory bandwidth used to fetch textures. An efficient real-time compression algorithm is important for dynamic texture generation and for texture streaming where the source compression format is different than the GPU hardware compression format.

	The DXT compression format is common across most major GPU hardware. DXT1, stores 16 input pixels in 64 bits of output. The output consists of two 16-bit RGB 5:6:5 color values and a 4x4 two bit lookup table. [] The compression ratio is 6:1 for 24 bit source images. DXT5, converts 16 input pixels into 128 bits of output. The output consists of 64 bits of alpha channel data, followed by 64 bits of color data. [] The compression ratio is 4:1.		The DXT compression format is common across most major GPU hardware. DXT1, stores 16 input pixels in 64 bits of output. The output consists of two 16-bit RGB 5:6:5 color values and a 4x4 two bit lookup table. [1] The compression ratio is 6:1 for 24 bit source images. DXT5, converts 16 input pixels into 128 bits of output. The output consists of 64 bits of alpha channel data, followed by 64 bits of color data [1]. The compression ratio is 4:1.

	The DXT compression format loses some image quality and introduces blocking artifacts. Several attempts have been made to improve image quality for this format. The state-of-art in real-time DXT compression is "Real-Time YCoCg-DXT compression" []. This algorithm works in the YCoCg space, which represents the luma (Y), orange chroma (Co), and green chroma (Cg) components. The YCoCg space can be encoded similar to YCbCr, but it is invented for better compression of the RGB space. It is possible to losslessly transform from RGB to YCoCg when using 2 more bits for YCoCg representation than for RGB [].		== Real-Time YCoCg-DXT Compression ==
			The DXT compression format loses some image quality and introduces blocking artifacts. Several attempts have been made to improve image quality for this format. The state-of-art in real-time DXT compression is "Real-Time YCoCg-DXT compression" [2]. This algorithm works in the YCoCg space, which represents the luma (Y), orange chroma (Co), and green chroma (Cg) components. The YCoCg space can be encoded similar to YCbCr, but it is invented for better compression of the RGB space. It is possible to losslessly transform from RGB to YCoCg when using 2 more bits for YCoCg representation than for RGB [3].

	The YCoCg-DXT compression first converts from RGB to YCoCg. Then, it operates on every 4x4 pixel blocks. The algorithm finds the maximum and minimum values from each of the Y, Co, and Cg channels. These values are to be stored during the encoding process. The algorithm calculates six intermediate values for the Y channel, and two for the color channels. Each pixel of the 4x4 block is quantized to the calculated values. For the color channel, two combinations of max/min values exist: max Co to min Cg, or max Cg to min Co. The algorithm determines which of the two combination provides better results for quantization.		The YCoCg-DXT compression first converts from RGB to YCoCg. Then, it operates on every 4x4 pixel blocks. The algorithm finds the maximum and minimum values from each of the Y, Co, and Cg channels. These values are to be stored during the encoding process. The algorithm calculates six intermediate values for the Y channel, and two for the color channels. Each pixel of the 4x4 block is quantized to the calculated values. For the color channel, two combinations of max/min values exist: max Co to min Cg, or max Cg to min Co. The algorithm determines which of the two combination provides better results for quantization.

	The YCoCg-DXT compression algorithm can increase the RGB-PSNR by 6dB or more [], there are other steps one could take to further improve the image quality. This project focuses on three additional steps to reduce compression artifacts on top of the Real-Time YCoCg-DXT Compression algorithm. The artifacts we want to reduce in this effort are banding and blocking. As well, we want to address the loss of details in images after compression.		The YCoCg-DXT compression algorithm can increase the RGB-PSNR by 6dB or more [2], there are other steps one could take to further improve the image quality. This project focuses on three additional steps to reduce compression artifacts on top of the Real-Time YCoCg-DXT Compression algorithm. The artifacts we want to reduce in this effort are banding and blocking. As well, we want to address the loss of details in images after compression.

	= Methods =		= Methods =

imported>Psych2012: /* Methods */

2012-03-21T19:46:57Z

Methods

← Older revision		Revision as of 19:46, 21 March 2012
Line 42:		Line 42:


	After these steps, the rest of the steps to complete the compression are the same as the Real-time YCoCg-DXT5 algorithm []. For each 4x4 blocks, the maximum and minimum values for each of the Y, Co and Cg channels are determined. Lastly, quantization is applied, and the image is encoded and decoded to complete the compression pipeline.		After these steps, the rest of the steps to complete the compression are the same as the Real-time YCoCg-DXT5 algorithm. For each 4x4 blocks, the maximum and minimum values for each of the Y, Co and Cg channels are determined. Lastly, quantization is applied, and the image is encoded and decoded to complete the compression pipeline.

	= Results =		= Results =

imported>Psych2012: /* Results */

2012-03-21T19:46:09Z

Results

imported>Psych2012: /* Methods */

2012-03-21T19:44:12Z

Methods

← Older revision		Revision as of 19:44, 21 March 2012
Line 15:		Line 15:
	The base algorithm in this project is the YCoCg-DXT compression algorithm. The entire pipeline of this algorithm is implemented in code for comparison and evaluation purposes. On top of that, three new operations are added to the pipeline to make the improvements. These operations are described below.		The base algorithm in this project is the YCoCg-DXT compression algorithm. The entire pipeline of this algorithm is implemented in code for comparison and evaluation purposes. On top of that, three new operations are added to the pipeline to make the improvements. These operations are described below.

	The first step after transforming from RGB to YCoCg space is to do a non-linear remap of the chroma components after mapping Co and Cg to the range of -1 to 1. Using the function described in ~~Equation 1~~, we expand the near zero region of the chroma value to increase precision in less saturated colors, at the expense of a reduction of precision in the more saturated colors. This is to take advantage of the property of the human vision system in that the eye is more sensitive to chroma differences in less saturated colors compared to more saturated colors. The effects of ~~Equation 1~~ is illustrated by Figure 1.		The first step after transforming from RGB to YCoCg space is to do a non-linear remap of the chroma components after mapping Co and Cg to the range of -1 to 1. Using the function described in the equation below, we expand the near zero region of the chroma value to increase precision in less saturated colors, at the expense of a reduction of precision in the more saturated colors. This is to take advantage of the property of the human vision system in that the eye is more sensitive to chroma differences in less saturated colors compared to more saturated colors. The effects of the equation below is illustrated by Figure 1.


	Co_out = sqrt(abs(Co_in))*sign(Co_in) ~~------------ (Equation 1)~~		Co_out = sqrt(abs(Co_in))*sign(Co_in)


Line 24:		Line 24:


	The second step is to apply sharpening to the image. Sharpening is done by adding a scaled amount of the difference between the pixel and a lowpass filter applied to the pixel. The lowpass kernel is described in ~~Equation 2~~ below. The sharpening is only done to the luma (Y) channel, and is limited to one Y quantization level (-/+ 1.0/512.0). This effort is to shape some of the quantization artifacts on the image such that detail is not lost.		The second step is to apply sharpening to the image. Sharpening is done by adding a scaled amount of the difference between the pixel and a lowpass filter applied to the pixel. The lowpass kernel is described in the equation below. The sharpening is only done to the luma (Y) channel, and is limited to one Y quantization level (-/+ 1.0/512.0). This effort is to shape some of the quantization artifacts on the image such that detail is not lost.


	[1 2 1;		[1 2 1;
	2 4 2;		2 4 2;
	1 2 1] ~~------------ (Equation 2)~~		1 2 1]

imported>Psych2012: /* Methods */

2012-03-21T19:43:20Z

Methods

← Older revision		Revision as of 19:43, 21 March 2012
Line 18:		Line 18:


	Co_out = sqrt(abs(Co_in))*sign(Co_in) (Equation 1)		Co_out = sqrt(abs(Co_in))*sign(Co_in) ------------ (Equation 1)


Line 29:		Line 29:
	[1 2 1;		[1 2 1;
	2 4 2;		2 4 2;
	1 2 1] (Equation 2)		1 2 1] ------------ (Equation 2)

imported>Psych2012: /* Methods */

2012-03-21T19:42:55Z

Methods

← Older revision		Revision as of 19:42, 21 March 2012
Line 18:		Line 18:


	Co_out = sqrt(abs(Co_in))*sign(Co_in)		Co_out = sqrt(abs(Co_in))*sign(Co_in) (Equation 1)


Line 29:		Line 29:
	[1 2 1;		[1 2 1;
	2 4 2;		2 4 2;
	1 2 1]		1 2 1] (Equation 2)

imported>Psych2012: /* Methods */

2012-03-21T19:42:23Z

Methods

← Older revision		Revision as of 19:42, 21 March 2012
Line 15:		Line 15:
	The base algorithm in this project is the YCoCg-DXT compression algorithm. The entire pipeline of this algorithm is implemented in code for comparison and evaluation purposes. On top of that, three new operations are added to the pipeline to make the improvements. These operations are described below.		The base algorithm in this project is the YCoCg-DXT compression algorithm. The entire pipeline of this algorithm is implemented in code for comparison and evaluation purposes. On top of that, three new operations are added to the pipeline to make the improvements. These operations are described below.

	The first step after transforming from RGB to YCoCg space is to do a non-linear remap of the chroma components after mapping Co and Cg to the range of -1 to 1. Using the function described in Equation 1, we expand the near zero region of the chroma value to increase precision in less saturated colors, at the expense of a reduction of precision in the more saturated colors. This is to take advantage of the property of the human vision system in that the eye is more sensitive to chroma differences in less saturated colors compared to more saturated colors. The effects of Equation 1 is illustrated by Figure[].		The first step after transforming from RGB to YCoCg space is to do a non-linear remap of the chroma components after mapping Co and Cg to the range of -1 to 1. Using the function described in Equation 1, we expand the near zero region of the chroma value to increase precision in less saturated colors, at the expense of a reduction of precision in the more saturated colors. This is to take advantage of the property of the human vision system in that the eye is more sensitive to chroma differences in less saturated colors compared to more saturated colors. The effects of Equation 1 is illustrated by Figure 1.


Line 21:		Line 21:


	[[File:chromaRemap.png \|thumb\|center\|537px\| Figure 1: Non-Linear Remap of Chroma ~~Channels~~]]		[[File:chromaRemap.png \|thumb\|center\|537px\| Figure 1: Non-Linear Remap of Chroma]]

imported>Psych2012: /* Methods */

2012-03-21T19:41:55Z

Methods

← Older revision		Revision as of 19:41, 21 March 2012
Line 21:		Line 21:


	[[File:chromaRemap.png \|thumb\|center\|537px\| Figure x: ]]		[[File:chromaRemap.png \|thumb\|center\|537px\| Figure 1: Non-Linear Remap of Chroma Channels]]

← Older revision		Revision as of 19:46, 21 March 2012
Line 50:		Line 50:
	== Retention of Image Details ==		== Retention of Image Details ==

	The following set of example illustrate the effect of the micro-sharpening operation. Figure [] shows the original picture, and we transformed the picture such that we can see the details better, shown in Figure [].		The following set of example illustrate the effect of the micro-sharpening operation. Figure 2 shows the original picture, and we transformed the picture such that we can see the details better, shown in Figure 3.

	[[File:A-origin.png \|thumb\|center\|800px\| Figure x: Original Image ]]		[[File:A-origin.png \|thumb\|center\|800px\| Figure 2: Original Image ]]

	[[File:A-base.png \|thumb\|center\|800px\| Figure x: Original Image, transformed]]		[[File:A-base.png \|thumb\|center\|800px\| Figure 3: Original Image, transformed]]


	Figure [] shows a cropped version of the original picture, emphasizing on the wall details on the right hand side. In Figure [], we applied the DXT5 compression method, losing all the details of the brick pattern on the right hand side wall. In Figure [], our version of the algorithm retains the brick pattern. This is because of the micro-sharpening adjustments from our algorithm.		Figure 4 shows a cropped version of the original picture, emphasizing on the wall details on the right hand side. In Figure 5, we applied the DXT5 compression method, losing all the details of the brick pattern on the right hand side wall. In Figure 6, our version of the algorithm retains the brick pattern. This is because of the micro-sharpening adjustments from our algorithm.

	[[File:A-base-crop.png \|thumb\|center\|800px\| Figure x: Original Image, transformed, cropped ]]		[[File:A-base-crop.png \|thumb\|center\|800px\| Figure 4: Original Image, transformed, cropped ]]

	[[File:A-dxt5-crop.png \|thumb\|center\|800px\| Figure x: Real-Time YCoCg-DXT Compression (DXT5), loss of details of brick pattern on right hand side]]		[[File:A-dxt5-crop.png \|thumb\|center\|800px\| Figure 5: Real-Time YCoCg-DXT Compression (DXT5), loss of details of brick pattern on right hand side]]

	[[File:A-fei-crop.png \|thumb\|center\|800px\| Figure x: High Quality Real-Time DXT Compression (Our Method), details recovered on right hand side]]		[[File:A-fei-crop.png \|thumb\|center\|800px\| Figure 6: High Quality Real-Time DXT Compression (Our Method), details recovered on right hand side]]


	== Reduction of Blocking Artifacts ==		== Reduction of Blocking Artifacts ==

	The following example set illustrates blocking artifacts. For the ease of viewing, we applied the same high contrast to all images to amplify the artifacts. We also magnified all the images with a 4x factor. The original image, shown in Figure[], has no blocking artifacts nor significant noise. After the DXT5 compression (Figure[]), severe blocking can be seen. Our algorithm attempts to mitigate the problem through the noise dithering operation. In Figure[], the blocking artifacts are reduced, at the expense of adding noise.		The following example set illustrates blocking artifacts. For the ease of viewing, we applied the same high contrast to all images to amplify the artifacts. We also magnified all the images with a 4x factor. The original image, shown in Figure 7, has no blocking artifacts nor significant noise. After the DXT5 compression (Figure 8), severe blocking can be seen. Our algorithm attempts to mitigate the problem through the noise dithering operation. In Figure 9, the blocking artifacts are reduced, at the expense of adding noise.

	[[File:A-1.png \|thumb\|center\|800px\| Figure x: Original Image, no blocking artifacts, no noise]]		[[File:A-1.png \|thumb\|center\|800px\| Figure 7: Original Image, no blocking artifacts, no noise]]

	[[File:A-2.png \|thumb\|center\|800px\| Figure x: Real-Time YCoCg-DXT Compression (DXT5), heavy blocking artifacts]]		[[File:A-2.png \|thumb\|center\|800px\| Figure 8: Real-Time YCoCg-DXT Compression (DXT5), heavy blocking artifacts]]

	[[File:A-3.png \|thumb\|center\|800px\| Figure x: High Quality Real-Time DXT Compression (Our Method), blocking removed, but with noise]]		[[File:A-3.png \|thumb\|center\|800px\| Figure 9: High Quality Real-Time DXT Compression (Our Method), blocking removed, but with noise]]


	== Removal of Banding ==		== Removal of Banding ==

	The following example set illustrates how banding is eliminated for the compressed image. Figure [] is the original image. The sky consists of a smooth gradient with orange and yellow tints. After the DXT5 compression (Figure []), banding can be seen from the orange and yellow part, and the gradient is no longer smooth. Using our method (Figure []), the sky does not have any banding, and it looks identical to the original image. The removal of this artifact is contributed by the added precision in the chroma channels from the non-linear remap.		The following example set illustrates how banding is eliminated for the compressed image. Figure 10 is the original image. The sky consists of a smooth gradient with orange and yellow tints. After the DXT5 compression (Figure 11), banding can be seen from the orange and yellow part, and the gradient is no longer smooth. Using our method (Figure 12), the sky does not have any banding, and it looks identical to the original image. The removal of this artifact is contributed by the added precision in the chroma channels from the non-linear remap.

	[[File:Example1-base.png \|thumb\|center\|800px\| Figure x: Original Image, no banding]]		[[File:Example1-base.png \|thumb\|center\|800px\| Figure 10: Original Image, no banding]]

	[[File:Example1-dxt5.png \|thumb\|center\|800px\| Figure x: Real-Time YCoCg-DXT Compression (DXT5), with banding]]		[[File:Example1-dxt5.png \|thumb\|center\|800px\| Figure 11: Real-Time YCoCg-DXT Compression (DXT5), with banding]]

	[[File:Example1-fei.png \|thumb\|center\|800px\| Figure x: High Quality Real-Time DXT Compression (Our Method), banding removed]]		[[File:Example1-fei.png \|thumb\|center\|800px\| Figure 12: High Quality Real-Time DXT Compression (Our Method), banding removed]]

	= Conclusions =		= Conclusions =