Simulating Vision through Retinal Prothesis - Revision history

imported>Projects221: /* References - Resources and Related Work */

2014-03-20T18:47:02Z

References - Resources and Related Work

← Older revision		Revision as of 18:47, 20 March 2014
Line 118:		Line 118:
	[1] "Project Xense Retinal Implant Simulation" etc.cmu.edu. Carnegie Mellon University, 2012. Web. 14 Mar 2014. <http://www.etc.cmu.edu/projects/tatrc>.		[1] "Project Xense Retinal Implant Simulation" etc.cmu.edu. Carnegie Mellon University, 2012. Web. 14 Mar 2014. <http://www.etc.cmu.edu/projects/tatrc>.

	[2] "The Argus® II Retinal Prosthesis System" 2-sight.eu~~/en/product-en~~. The Argus II Retinal Prosthesis System, 2014. Web. 15 Mar 2014. <http://2-sight.eu/en/about-us-en>		[2] "The Argus® II Retinal Prosthesis System" 2-sight.eu. The Argus II Retinal Prosthesis System, 2014. Web. 15 Mar 2014. <http://2-sight.eu/en/about-us-en>

	[3] "Holographic display system for restoration of sight to the blind" G A Goetz et al 2013 J. Neural Eng. 10 056021		[3] "Holographic display system for restoration of sight to the blind" G A Goetz et al 2013 J. Neural Eng. 10 056021

imported>Projects221 at 18:45, 20 March 2014

2014-03-20T18:45:58Z

imported>Projects221: /* Computer Vision Assistance */

2014-03-19T00:25:08Z

Computer Vision Assistance

← Older revision		Revision as of 00:25, 19 March 2014
Line 102:		Line 102:
	Regarding edge detection, I found that only Canny edge detection seemed to be useful, at least in theory. My other two high pass filter implementations, Sobel and Laplacian, failed to sharply distinguish edges from noise, enough to be recognized in latter stage pixellation once the low frequency content is lost. With that said, Canny edge detection still outputs an image that is difficult to interpret with large enough pixel sizes, and fails to communicate edge information upon pixillation of the image. In practice, the implementation of edge detection was not successful. Sobel Edge Detection accentuates noise too much to really communicate image content, they may work in a pinch in events of very low dynamic range. The other two don't define edges strongly enough to persist through pixillation.		Regarding edge detection, I found that only Canny edge detection seemed to be useful, at least in theory. My other two high pass filter implementations, Sobel and Laplacian, failed to sharply distinguish edges from noise, enough to be recognized in latter stage pixellation once the low frequency content is lost. With that said, Canny edge detection still outputs an image that is difficult to interpret with large enough pixel sizes, and fails to communicate edge information upon pixillation of the image. In practice, the implementation of edge detection was not successful. Sobel Edge Detection accentuates noise too much to really communicate image content, they may work in a pinch in events of very low dynamic range. The other two don't define edges strongly enough to persist through pixillation.

	[[File:threshold.jpg\|~~800px~~\|center\| ~~Threshold comparisons~~]]		[[File:threshold.jpg\|frame\|center\| From left to Right: No threshold, Canny Edge Detection, Sobel Edge Detection, Laplacian Edge Detection. Pixellation renders only the first and third columns interpretable.]]
	Regarding thresholding, I found Otsu thresholding to be more effective at communicating information than the other two threshold implementations I use: Gaussian and mean adaptive thresholding. At very large pixel sizes, the image is still able to maintain the 2 contiguous shapes generated by the Otsu method. In contexts where key information, such as the presence of a face or large object, is held in the foreground, Otsu binarization does an adequate job of still relaying that information at very low resolutions. While a high dynamic range gray scale will generally communicate information better than a low dynamic range, the prothesis will have difficulty meeting the level of dynamic range displayed in the left image. In cases of very limited dynamic range, Otsu binarization is an adequate means of communicating information.		Regarding thresholding, I found Otsu thresholding to be more effective at communicating information than the other two threshold implementations I use: Gaussian and mean adaptive thresholding. At very large pixel sizes, the image is still able to maintain the 2 contiguous shapes generated by the Otsu method. In contexts where key information, such as the presence of a face or large object, is held in the foreground, Otsu binarization does an adequate job of still relaying that information at very low resolutions. While a high dynamic range gray scale will generally communicate information better than a low dynamic range, the prothesis will have difficulty meeting the level of dynamic range displayed in the left image. In cases of very limited dynamic range, Otsu binarization is an adequate means of communicating information.
	[[File:otsu1.png\|400px\|center\| Unfiltered image]]		[[File:otsu1.png\|400px\|center\| Unfiltered image]]

imported>Projects221: /* Computer Vision Assistance */

2014-03-19T00:21:52Z

Computer Vision Assistance

← Older revision		Revision as of 00:21, 19 March 2014
Line 102:		Line 102:
	Regarding edge detection, I found that only Canny edge detection seemed to be useful, at least in theory. My other two high pass filter implementations, Sobel and Laplacian, failed to sharply distinguish edges from noise, enough to be recognized in latter stage pixellation once the low frequency content is lost. With that said, Canny edge detection still outputs an image that is difficult to interpret with large enough pixel sizes, and fails to communicate edge information upon pixillation of the image. In practice, the implementation of edge detection was not successful. Sobel Edge Detection accentuates noise too much to really communicate image content, they may work in a pinch in events of very low dynamic range. The other two don't define edges strongly enough to persist through pixillation.		Regarding edge detection, I found that only Canny edge detection seemed to be useful, at least in theory. My other two high pass filter implementations, Sobel and Laplacian, failed to sharply distinguish edges from noise, enough to be recognized in latter stage pixellation once the low frequency content is lost. With that said, Canny edge detection still outputs an image that is difficult to interpret with large enough pixel sizes, and fails to communicate edge information upon pixillation of the image. In practice, the implementation of edge detection was not successful. Sobel Edge Detection accentuates noise too much to really communicate image content, they may work in a pinch in events of very low dynamic range. The other two don't define edges strongly enough to persist through pixillation.

	[[File:threshold.~~jpeg~~\|800px\|center\| Threshold comparisons]]		[[File:threshold.jpg\|800px\|center\| Threshold comparisons]]
	Regarding thresholding, I found Otsu thresholding to be more effective at communicating information than the other two threshold implementations I use: Gaussian and mean adaptive thresholding. At very large pixel sizes, the image is still able to maintain the 2 contiguous shapes generated by the Otsu method. In contexts where key information, such as the presence of a face or large object, is held in the foreground, Otsu binarization does an adequate job of still relaying that information at very low resolutions. While a high dynamic range gray scale will generally communicate information better than a low dynamic range, the prothesis will have difficulty meeting the level of dynamic range displayed in the left image. In cases of very limited dynamic range, Otsu binarization is an adequate means of communicating information.		Regarding thresholding, I found Otsu thresholding to be more effective at communicating information than the other two threshold implementations I use: Gaussian and mean adaptive thresholding. At very large pixel sizes, the image is still able to maintain the 2 contiguous shapes generated by the Otsu method. In contexts where key information, such as the presence of a face or large object, is held in the foreground, Otsu binarization does an adequate job of still relaying that information at very low resolutions. While a high dynamic range gray scale will generally communicate information better than a low dynamic range, the prothesis will have difficulty meeting the level of dynamic range displayed in the left image. In cases of very limited dynamic range, Otsu binarization is an adequate means of communicating information.
	[[File:otsu1.png\|400px\|center\| Unfiltered image]]		[[File:otsu1.png\|400px\|center\| Unfiltered image]]

imported>Projects221: /* Computer Vision Assistance */

2014-03-19T00:15:48Z

Computer Vision Assistance

← Older revision		Revision as of 00:15, 19 March 2014
Line 102:		Line 102:
	Regarding edge detection, I found that only Canny edge detection seemed to be useful, at least in theory. My other two high pass filter implementations, Sobel and Laplacian, failed to sharply distinguish edges from noise, enough to be recognized in latter stage pixellation once the low frequency content is lost. With that said, Canny edge detection still outputs an image that is difficult to interpret with large enough pixel sizes, and fails to communicate edge information upon pixillation of the image. In practice, the implementation of edge detection was not successful. Sobel Edge Detection accentuates noise too much to really communicate image content, they may work in a pinch in events of very low dynamic range. The other two don't define edges strongly enough to persist through pixillation.		Regarding edge detection, I found that only Canny edge detection seemed to be useful, at least in theory. My other two high pass filter implementations, Sobel and Laplacian, failed to sharply distinguish edges from noise, enough to be recognized in latter stage pixellation once the low frequency content is lost. With that said, Canny edge detection still outputs an image that is difficult to interpret with large enough pixel sizes, and fails to communicate edge information upon pixillation of the image. In practice, the implementation of edge detection was not successful. Sobel Edge Detection accentuates noise too much to really communicate image content, they may work in a pinch in events of very low dynamic range. The other two don't define edges strongly enough to persist through pixillation.

	[[File:threshold.~~jpg~~\|800px\|center\| Threshold comparisons]]		[[File:threshold.jpeg\|800px\|center\| Threshold comparisons]]
	Regarding thresholding, I found Otsu thresholding to be more effective at communicating information than the other two threshold implementations I use: Gaussian and mean adaptive thresholding. At very large pixel sizes, the image is still able to maintain the 2 contiguous shapes generated by the Otsu method. In contexts where key information, such as the presence of a face or large object, is held in the foreground, Otsu binarization does an adequate job of still relaying that information at very low resolutions. While a high dynamic range gray scale will generally communicate information better than a low dynamic range, the prothesis will have difficulty meeting the level of dynamic range displayed in the left image. In cases of very limited dynamic range, Otsu binarization is an adequate means of communicating information.		Regarding thresholding, I found Otsu thresholding to be more effective at communicating information than the other two threshold implementations I use: Gaussian and mean adaptive thresholding. At very large pixel sizes, the image is still able to maintain the 2 contiguous shapes generated by the Otsu method. In contexts where key information, such as the presence of a face or large object, is held in the foreground, Otsu binarization does an adequate job of still relaying that information at very low resolutions. While a high dynamic range gray scale will generally communicate information better than a low dynamic range, the prothesis will have difficulty meeting the level of dynamic range displayed in the left image. In cases of very limited dynamic range, Otsu binarization is an adequate means of communicating information.
	[[File:otsu1.png\|400px\|center\| Unfiltered image]]		[[File:otsu1.png\|400px\|center\| Unfiltered image]]

imported>Projects221: /* Computer Vision Assistance */

2014-03-19T00:14:57Z

Computer Vision Assistance

← Older revision		Revision as of 00:14, 19 March 2014
Line 100:		Line 100:
	[[File:mask.png\|400px\|center\| Image after pixillation demonstrates the difficulty of recognizing a face]]		[[File:mask.png\|400px\|center\| Image after pixillation demonstrates the difficulty of recognizing a face]]
	[[File:maskmeme.png\|400px\|center\| Superimposing the mask symbol helps distinguish the presence of a face]]		[[File:maskmeme.png\|400px\|center\| Superimposing the mask symbol helps distinguish the presence of a face]]
	Regarding edge detection, I found that only Canny edge detection seemed to be useful, at least in theory. My other two high pass filter implementations, Sobel and Laplacian, failed to sharply distinguish edges from noise, enough to be recognized in latter stage pixellation once the low frequency content is lost. With that said, Canny edge detection still outputs an image that is difficult to interpret with large enough pixel sizes, and fails to communicate edge information upon pixillation of the image. In practice, the implementation of edge detection was not successful		Regarding edge detection, I found that only Canny edge detection seemed to be useful, at least in theory. My other two high pass filter implementations, Sobel and Laplacian, failed to sharply distinguish edges from noise, enough to be recognized in latter stage pixellation once the low frequency content is lost. With that said, Canny edge detection still outputs an image that is difficult to interpret with large enough pixel sizes, and fails to communicate edge information upon pixillation of the image. In practice, the implementation of edge detection was not successful. Sobel Edge Detection accentuates noise too much to really communicate image content, they may work in a pinch in events of very low dynamic range. The other two don't define edges strongly enough to persist through pixillation.

			[[File:threshold.jpg\|800px\|center\| Threshold comparisons]]
	Regarding thresholding, I found Otsu thresholding to be more effective at communicating information than the other two threshold implementations I use: Gaussian and mean adaptive thresholding. At very large pixel sizes, the image is still able to maintain the 2 contiguous shapes generated by the Otsu method. In contexts where key information, such as the presence of a face or large object, is held in the foreground, Otsu binarization does an adequate job of still relaying that information at very low resolutions. While a high dynamic range gray scale will generally communicate information better than a low dynamic range, the prothesis will have difficulty meeting the level of dynamic range displayed in the left image. In cases of very limited dynamic range, Otsu binarization is an adequate means of communicating information.		Regarding thresholding, I found Otsu thresholding to be more effective at communicating information than the other two threshold implementations I use: Gaussian and mean adaptive thresholding. At very large pixel sizes, the image is still able to maintain the 2 contiguous shapes generated by the Otsu method. In contexts where key information, such as the presence of a face or large object, is held in the foreground, Otsu binarization does an adequate job of still relaying that information at very low resolutions. While a high dynamic range gray scale will generally communicate information better than a low dynamic range, the prothesis will have difficulty meeting the level of dynamic range displayed in the left image. In cases of very limited dynamic range, Otsu binarization is an adequate means of communicating information.
	[[File:otsu1.png\|400px\|center\| Unfiltered image]]		[[File:otsu1.png\|400px\|center\| Unfiltered image]]
	[[File:otsu2.png\|400px\|center\| ~~Unfiltered~~ image]]		[[File:otsu2.png\|400px\|center\| Filtered image]]

	= Conclusions and Future Work =		= Conclusions and Future Work =

imported>Projects221: /* Computer Vision Assistance */

2014-03-19T00:05:38Z

Computer Vision Assistance

← Older revision		Revision as of 00:05, 19 March 2014
Line 100:		Line 100:
	[[File:mask.png\|400px\|center\| Image after pixillation demonstrates the difficulty of recognizing a face]]		[[File:mask.png\|400px\|center\| Image after pixillation demonstrates the difficulty of recognizing a face]]
	[[File:maskmeme.png\|400px\|center\| Superimposing the mask symbol helps distinguish the presence of a face]]		[[File:maskmeme.png\|400px\|center\| Superimposing the mask symbol helps distinguish the presence of a face]]
	Regarding edge detection, I found that only Canny edge detection seemed to be useful, at least in theory. My other two high pass filter implementations, Sobel and Laplacian ~~derivatives~~, failed to sharply distinguish edges from noise, enough to be recognized in latter stage pixellation once the low frequency content is lost. With that said, Canny edge detection still outputs an image that is difficult to interpret with large enough pixel sizes, and fails to communicate edge information upon pixillation of the image. In practice, the implementation of edge detection was not successful		Regarding edge detection, I found that only Canny edge detection seemed to be useful, at least in theory. My other two high pass filter implementations, Sobel and Laplacian, failed to sharply distinguish edges from noise, enough to be recognized in latter stage pixellation once the low frequency content is lost. With that said, Canny edge detection still outputs an image that is difficult to interpret with large enough pixel sizes, and fails to communicate edge information upon pixillation of the image. In practice, the implementation of edge detection was not successful

	Regarding thresholding, I found Otsu thresholding to be more effective at communicating information than the other two threshold implementations I use: Gaussian and mean adaptive thresholding. At very large pixel sizes, the image is still able to maintain the 2 contiguous shapes generated by the Otsu method. In contexts where key information, such as the presence of a face or large object, is held in the foreground, Otsu binarization does an adequate job of still relaying that information at very low resolutions. While a high dynamic range gray scale will generally communicate information better than a low dynamic range, the prothesis will have difficulty meeting the level of dynamic range displayed in the left image. In cases of very limited dynamic range, Otsu binarization is an adequate means of communicating information.		Regarding thresholding, I found Otsu thresholding to be more effective at communicating information than the other two threshold implementations I use: Gaussian and mean adaptive thresholding. At very large pixel sizes, the image is still able to maintain the 2 contiguous shapes generated by the Otsu method. In contexts where key information, such as the presence of a face or large object, is held in the foreground, Otsu binarization does an adequate job of still relaying that information at very low resolutions. While a high dynamic range gray scale will generally communicate information better than a low dynamic range, the prothesis will have difficulty meeting the level of dynamic range displayed in the left image. In cases of very limited dynamic range, Otsu binarization is an adequate means of communicating information.

imported>Projects221: /* Results */

2014-03-19T00:04:03Z

Results

← Older revision		Revision as of 00:04, 19 March 2014
Line 81:		Line 81:
	[[File:Face.png\|400px\|center\| Unfiltered image]]		[[File:Face.png\|400px\|center\| Unfiltered image]]
	[[File:filteredface.png\|400px\|center\| Image after undergoing pixillation, color removal, and Otsu binarization]]		[[File:filteredface.png\|400px\|center\| Image after undergoing pixillation, color removal, and Otsu binarization]]
	These specifications were met. A user of the simulator has control of pixel density, color, as well as how dynamic range is expressed. To elaborate dynamic range can be expressed either by pixel color or pixel radius, and this dynamic range can span the entire spectrum of grays, to just two colors, as demonstrated by the Otsu Thresholding. If the user chooses to pixellate the image, there are three options from which to choose: square, dot, and radial		These specifications were met. A user of the simulator has control of pixel density, color, as well as how dynamic range is expressed. To elaborate dynamic range can be expressed either by pixel color or pixel radius, and this dynamic range can span the entire spectrum of grays, to just two colors, as demonstrated by the Otsu Thresholding. If the user chooses to pixellate the image, there are three options from which to choose: square, dot, and radial.

			Here is some information on specific parameters:
			Pixellation yields pixel blocks of 80X80, 40X40, 20X20, 10X10, 8X8, and 4X4 pixels.
			The neighborhood size used by the blurring method to determine central pixel intensity ranges from 1 to 30
			Frame rate ranges from the camera's advertised fps (~30fps in the case of my webcam) to 1/4 fps
			Regarding the track bar governing Facial Recognition Accuracy vs. Speed, that parameter n corresponds a resizing of the image examined for faces by 1/n. This resizing allows for the code to run faster, although it does compromise the accuracy of facial recognition. This bounds of this parameter are somewhat arbitrary, provided I don't shrink down the image so much that facial recognition if completely compromised. I set this maximum to n = 10.
	[[File:ThreeOptions.jpeg\|800px\|center\| Three types of pixel expressions]]		[[File:ThreeOptions.jpeg\|800px\|center\| Three types of pixel expressions]]

imported>Projects221: /* Introduction */

2014-03-18T23:46:05Z

Introduction

← Older revision		Revision as of 23:46, 18 March 2014
Line 5:		Line 5:
	= Introduction =		= Introduction =
	[[File:PalankerDevice.jpg\|400px\|right]]		[[File:PalankerDevice.jpg\|400px\|right]]
	Retinal degenerative diseases such as age-related macular degeneration or retinitis pigmentosa are among the leading causes of blindness in the developed world. These diseases lead to a loss of photoreceptors, while the inner retinal neurons survive to a large extent. Electrical stimulation of the surviving retinal neurons has been achieved either epiretinally, in which case the primary targets of stimulation are the retinal ganglion cells (RGCs), or subretinally to bypass the degenerated photoreceptors and use neurons in the inner nuclear layer (bipolar, amacrine and horizontal cells) as primary targets [1]. Other fully optical approaches to restoration of sight include optogenetics, in which retinal neurons are transfected to express light-sensitive Na and Cl channels, small- molecule photoswitches which bind to K channels and make them light sensitive or photovoltaic implants based on thin-film polymers.		Retinal degenerative diseases such as age-related macular degeneration or retinitis pigmentosa are among the leading causes of blindness in the developed world. These diseases lead to a loss of photoreceptors, while the inner retinal neurons survive to a large extent. Electrical stimulation of the surviving retinal neurons has been achieved either epiretinally, in which case the primary targets of stimulation are the retinal ganglion cells (RGCs), or subretinally to bypass the degenerated photoreceptors and use neurons in the inner nuclear layer (bipolar, amacrine and horizontal cells) as primary targets [3]. Other fully optical approaches to restoration of sight include optogenetics, in which retinal neurons are transfected to express light-sensitive Na and Cl channels, small- molecule photoswitches which bind to K channels and make them light sensitive or photovoltaic implants based on thin-film polymers.

	Recent clinical studies with epiretinal and subretinal prosthetic systems have demonstrated improvements of the visual function in certain tasks, with some patients being able to identify letters with equivalent visual acuity of up to 20/550. Despite progress in improving visual acuity, normal vision at this resolution lacks much functionality. Simulating vision through retinal prothesis, as well as processing the image through various means could determine better methods in transferring information through the retina at this limited bandwidth. In order to aid the development of future image processing software, this group will simulate vision through the retinal prothesis developed by the Palanker Lab.		Recent clinical studies with epiretinal and subretinal prosthetic systems have demonstrated improvements of the visual function in certain tasks, with some patients being able to identify letters with equivalent visual acuity of up to 20/550. Despite progress in improving visual acuity, normal vision at this resolution lacks much functionality. Simulating vision through retinal prothesis, as well as processing the image through various means could determine better methods in transferring information through the retina at this limited bandwidth. In order to aid the development of future image processing software, this group will simulate vision through the retinal prothesis developed by the Palanker Lab.

imported>Projects221: /* Introduction */

2014-03-18T23:45:25Z

Introduction

← Older revision		Revision as of 23:45, 18 March 2014
Line 5:		Line 5:
	= Introduction =		= Introduction =
	[[File:PalankerDevice.jpg\|400px\|right]]		[[File:PalankerDevice.jpg\|400px\|right]]
	Retinal degenerative diseases such as age-related macular degeneration or retinitis pigmentosa are among the leading causes of blindness in the developed world. These diseases lead to a loss of photoreceptors, while the inner retinal neurons survive to a large extent. Electrical stimulation of the surviving retinal neurons has been achieved either epiretinally, in which case the primary targets of stimulation are the retinal ganglion cells (RGCs), or subretinally to bypass the degenerated photoreceptors and use neurons in the inner nuclear layer (bipolar, amacrine and horizontal cells) as primary targets. Other fully optical approaches to restoration of sight include optogenetics, in which retinal neurons are transfected to express light-sensitive Na and Cl channels, small- molecule photoswitches which bind to K channels and make them light sensitive or photovoltaic implants based on thin-film polymers.		Retinal degenerative diseases such as age-related macular degeneration or retinitis pigmentosa are among the leading causes of blindness in the developed world. These diseases lead to a loss of photoreceptors, while the inner retinal neurons survive to a large extent. Electrical stimulation of the surviving retinal neurons has been achieved either epiretinally, in which case the primary targets of stimulation are the retinal ganglion cells (RGCs), or subretinally to bypass the degenerated photoreceptors and use neurons in the inner nuclear layer (bipolar, amacrine and horizontal cells) as primary targets [1]. Other fully optical approaches to restoration of sight include optogenetics, in which retinal neurons are transfected to express light-sensitive Na and Cl channels, small- molecule photoswitches which bind to K channels and make them light sensitive or photovoltaic implants based on thin-film polymers.

	Recent clinical studies with epiretinal and subretinal prosthetic systems have demonstrated improvements of the visual function in certain tasks, with some patients being able to identify letters with equivalent visual acuity of up to 20/550. Despite progress in improving visual acuity, normal vision at this resolution lacks much functionality. Simulating vision through retinal prothesis, as well as processing the image through various means could determine better methods in transferring information through the retina at this limited bandwidth. In order to aid the development of future image processing software, this group will simulate vision through the retinal prothesis developed by the Palanker Lab.		Recent clinical studies with epiretinal and subretinal prosthetic systems have demonstrated improvements of the visual function in certain tasks, with some patients being able to identify letters with equivalent visual acuity of up to 20/550. Despite progress in improving visual acuity, normal vision at this resolution lacks much functionality. Simulating vision through retinal prothesis, as well as processing the image through various means could determine better methods in transferring information through the retina at this limited bandwidth. In order to aid the development of future image processing software, this group will simulate vision through the retinal prothesis developed by the Palanker Lab.

← Older revision		Revision as of 18:45, 20 March 2014
Line 58:		Line 58:
	== Facial Recognition ==		== Facial Recognition ==
	For facial recognition, I used the OpenCV training set and cv2.cascadeClassifier method for determining whether or not there was a face on the screen. The algorithm is not robust enough to be able to distinguish a particular face; it can only detect the presence of a face. With face detection in place, we could superimpose over the face an object that would be much easier to detect. In this particular case, we used meme faces as the objects with which to superimpose over the face.		For facial recognition, I used the OpenCV training set and cv2.cascadeClassifier method for determining whether or not there was a face on the screen. The algorithm is not robust enough to be able to distinguish a particular face; it can only detect the presence of a face. With face detection in place, we could superimpose over the face an object that would be much easier to detect. In this particular case, we used meme faces as the objects with which to superimpose over the face.
	== ~~Pixellation~~ ==		== Pixelation ==
	I employed 4 methods of ~~pixellation~~:		I employed 4 methods of pixelation:

	In the first, I shrink down the source image by an adjustable constant, and then restore the original image to its original size. the result is a computationally quick method of ~~pixellation~~ that doesn't perfectly constrain image coloring to well defined pixel blocks. However, it is passable as a means of simulating visual acuity of restored sight, and can be used to test the effectiveness of visual acuity with and without the assistance computer vision algorithms.		In the first, I shrink down the source image by an adjustable constant, and then restore the original image to its original size. the result is a computationally quick method of pixelation that doesn't perfectly constrain image coloring to well defined pixel blocks. However, it is passable as a means of simulating visual acuity of restored sight, and can be used to test the effectiveness of visual acuity with and without the assistance computer vision algorithms.

	In the second, I iterate through an image via blocks of an adjustable size. Within each block, I take the mean intensity value of the block and assign that value to each pixel in the block. This method is computationally more expensive, although it does allow for sharply defined pixel blocks.		In the second, I iterate through an image via blocks of an adjustable size. Within each block, I take the mean intensity value of the block and assign that value to each pixel in the block. This method is computationally more expensive, although it does allow for sharply defined pixel blocks.
Line 80:		Line 80:
	The main features I found I needed to incorporate into my simulation were: encoding the dynamic range applied voltages of individual pixels, varying resolution in response to pixel density on the retinal prothesis, and removing color from images (as it's doubtful our prothesis will be able to transmit color information).		The main features I found I needed to incorporate into my simulation were: encoding the dynamic range applied voltages of individual pixels, varying resolution in response to pixel density on the retinal prothesis, and removing color from images (as it's doubtful our prothesis will be able to transmit color information).
	[[File:Face.png\|400px\|center\| Unfiltered image]]		[[File:Face.png\|400px\|center\| Unfiltered image]]
	[[File:filteredface.png\|400px\|center\| Image after undergoing ~~pixillation~~, color removal, and Otsu binarization]]		[[File:filteredface.png\|400px\|center\| Image after undergoing pixelation, color removal, and Otsu binarization]]
	These specifications were met. A user of the simulator has control of pixel density, color, as well as how dynamic range is expressed. To elaborate dynamic range can be expressed either by pixel color or pixel radius, and this dynamic range can span the entire spectrum of grays, to just two colors, as demonstrated by the Otsu Thresholding. If the user chooses to ~~pixellate~~ the image, there are three options from which to choose: square, dot, and radial.		These specifications were met. A user of the simulator has control of pixel density, color, as well as how dynamic range is expressed. To elaborate dynamic range can be expressed either by pixel color or pixel radius, and this dynamic range can span the entire spectrum of grays, to just two colors, as demonstrated by the Otsu Thresholding. If the user chooses to pixelate the image, there are three options from which to choose: square, dot, and radial.

	Here is some information on specific parameters:		Here is some information on specific parameters:
	~~Pixellation~~ yields pixel blocks of 80X80, 40X40, 20X20, 10X10, 8X8, and 4X4 pixels.		pixelation yields pixel blocks of 80X80, 40X40, 20X20, 10X10, 8X8, and 4X4 pixels.
	The neighborhood size used by the blurring method to determine central pixel intensity ranges from 1 to 30		The neighborhood size used by the blurring method to determine central pixel intensity ranges from 1 to 30
	Frame rate ranges from the camera's advertised fps (~30fps in the case of my webcam) to 1/4 fps		Frame rate ranges from the camera's advertised fps (~30fps in the case of my webcam) to 1/4 fps
Line 98:		Line 98:

	Utilizing facial recognition, I was able to substitute faces for symbols that are generally easier to recognize through low resolution vision as a face. There was the caveat that some threshold and edge detection filters compromised the effectiveness of this face substitution		Utilizing facial recognition, I was able to substitute faces for symbols that are generally easier to recognize through low resolution vision as a face. There was the caveat that some threshold and edge detection filters compromised the effectiveness of this face substitution
	[[File:mask.png\|400px\|center\| Image after ~~pixillation~~ demonstrates the difficulty of recognizing a face]]		[[File:mask.png\|400px\|center\| Image after pixelation demonstrates the difficulty of recognizing a face]]
	[[File:maskmeme.png\|400px\|center\| Superimposing the mask symbol helps distinguish the presence of a face]]		[[File:maskmeme.png\|400px\|center\| Superimposing the mask symbol helps distinguish the presence of a face]]
	Regarding edge detection, I found that only Canny edge detection seemed to be useful, at least in theory. My other two high pass filter implementations, Sobel and Laplacian, failed to sharply distinguish edges from noise, enough to be recognized in latter stage ~~pixellation~~ once the low frequency content is lost. With that said, Canny edge detection still outputs an image that is difficult to interpret with large enough pixel sizes, and fails to communicate edge information upon ~~pixillation~~ of the image. In practice, the implementation of edge detection was not successful. Sobel Edge Detection accentuates noise too much to really communicate image content, they may work in a pinch in events of very low dynamic range. The other two don't define edges strongly enough to persist through ~~pixillation~~.		Regarding edge detection, I found that only Canny edge detection seemed to be useful, at least in theory. My other two high pass filter implementations, Sobel and Laplacian, failed to sharply distinguish edges from noise, enough to be recognized in latter stage pixelation once the low frequency content is lost. With that said, Canny edge detection still outputs an image that is difficult to interpret with large enough pixel sizes, and fails to communicate edge information upon pixelation of the image. In practice, the implementation of edge detection was not successful. Sobel Edge Detection accentuates noise too much to really communicate image content, they may work in a pinch in events of very low dynamic range. The other two don't define edges strongly enough to persist through pixelation.

	[[File:threshold.jpg\|frame\|center\| From left to Right: No threshold, Canny Edge Detection, Sobel Edge Detection, Laplacian Edge Detection. ~~Pixellation~~ renders only the first and third columns interpretable.]]		[[File:threshold.jpg\|frame\|center\| From left to Right: No threshold, Canny Edge Detection, Sobel Edge Detection, Laplacian Edge Detection. Pixelation renders only the first and third columns interpretable.]]
	Regarding thresholding, I found Otsu thresholding to be more effective at communicating information than the other two threshold implementations I use: Gaussian and mean adaptive thresholding. At very large pixel sizes, the image is still able to maintain the 2 contiguous shapes generated by the Otsu method. In contexts where key information, such as the presence of a face or large object, is held in the foreground, Otsu binarization does an adequate job of still relaying that information at very low resolutions. While a high dynamic range gray scale will generally communicate information better than a low dynamic range, the prothesis will have difficulty meeting the level of dynamic range displayed in the left image. In cases of very limited dynamic range, Otsu binarization is an adequate means of communicating information.		Regarding thresholding, I found Otsu thresholding to be more effective at communicating information than the other two threshold implementations I use: Gaussian and mean adaptive thresholding. At very large pixel sizes, the image is still able to maintain the 2 contiguous shapes generated by the Otsu method. In contexts where key information, such as the presence of a face or large object, is held in the foreground, Otsu binarization does an adequate job of still relaying that information at very low resolutions. While a high dynamic range gray scale will generally communicate information better than a low dynamic range, the prothesis will have difficulty meeting the level of dynamic range displayed in the left image. In cases of very limited dynamic range, Otsu binarization is an adequate means of communicating information.
	[[File:otsu1.png\|400px\|center\| Unfiltered image]]		[[File:otsu1.png\|400px\|center\| Unfiltered image]]
Line 109:		Line 109:
	= Conclusions and Future Work =		= Conclusions and Future Work =

	This experiment of simulating yielded some insights into the difficulty of conveying important information to to the wearer, as well as the difficulty in developing an adequate simulator. First off, the limited information regarding what restored vision looks like from patient testimonials made confidence in any particular strategy of simulating restored vision difficult. The dot array and ~~pixillation~~ strategies I did employ however did convey the difficulty of perceiving objects at low resolutions. In light of this development, the edge detection, thresholding, and face detection strategies I employ, while arguably improving perception, left a lot of room for improvement, and justified the need for more advanced computer vision algorithms to improve how restored vision functionality.		This experiment of simulating yielded some insights into the difficulty of conveying important information to to the wearer, as well as the difficulty in developing an adequate simulator. First off, the limited information regarding what restored vision looks like from patient testimonials made confidence in any particular strategy of simulating restored vision difficult. The dot array and pixelation strategies I did employ however did convey the difficulty of perceiving objects at low resolutions. In light of this development, the edge detection, thresholding, and face detection strategies I employ, while arguably improving perception, left a lot of room for improvement, and justified the need for more advanced computer vision algorithms to improve how restored vision functionality.

	In the future, we could, accompanied with a better understanding of what corrected vision looks like, develop a more accurate model of what patients actually see. Once we have a better base of pixels from which to work, we could add more sophisticated computer vision algorithms to incorporate features such as object detection and edge enhancement. Additionally, having a smart camera determine from context what information to send to the patient, such as deciding to emphasize a bathroom sign if one is caught in the patients visual field, could do wonders in enhancing the functionality of restored vision. Hopefully, this simulation can be used to better illustrate the the quality of vision patients have to work with, and can be used as a tool in developing computer vision algorithms that can improve the functionality of this retinal prosthesis and future methods of restoring eyesight.		In the future, we could, accompanied with a better understanding of what corrected vision looks like, develop a more accurate model of what patients actually see. Once we have a better base of pixels from which to work, we could add more sophisticated computer vision algorithms to incorporate features such as object detection and edge enhancement. Additionally, having a smart camera determine from context what information to send to the patient, such as deciding to emphasize a bathroom sign if one is caught in the patients visual field, could do wonders in enhancing the functionality of restored vision. Hopefully, this simulation can be used to better illustrate the the quality of vision patients have to work with, and can be used as a tool in developing computer vision algorithms that can improve the functionality of this retinal prosthesis and future methods of restoring eyesight.