Mazzochette - Revision history

imported>Projects221: /* Conclusions: */

2013-03-20T23:59:32Z

Conclusions:

← Older revision		Revision as of 23:59, 20 March 2013
Line 167:		Line 167:
	The mean kernel and Gaussian kernel threshold methods both improved the performance of thresholding, but had significant drawbacks. The increased computation time is too high for the available window for the entire skeletonization of the worm. The success rate of finding the worm was also too low. The effective filtering of the image frequently blurred the noise eso much that the contours around it because longer than around the worm itself. The increased block size of the kernels improved preservation of the worm's body outline.		The mean kernel and Gaussian kernel threshold methods both improved the performance of thresholding, but had significant drawbacks. The increased computation time is too high for the available window for the entire skeletonization of the worm. The success rate of finding the worm was also too low. The effective filtering of the image frequently blurred the noise eso much that the contours around it because longer than around the worm itself. The increased block size of the kernels improved preservation of the worm's body outline.

	Future work in this project includes first constructing the next, more stable version of the optical system. Once constructed the improved Otsu thresholding method will be implemented, and further investigation into tracking the head and tail will be started. Other hardware including an automatic x,y microscope stage and cantilever control will also be integrated for a fully functional system.		Future work in this project includes first constructing the next, more stable version of the optical system. Once constructed the improved Otsu thresholding method will be implemented, and further investigation into improved methods for tracking the head and tail will be started. Other hardware including an automatic x,y microscope stage and cantilever control will also be integrated for a fully functional system.

	== References ==		== References ==

imported>Projects221: /* Appendix: Code */

2013-03-20T23:56:53Z

Appendix: Code

← Older revision		Revision as of 23:56, 20 March 2013
Line 183:		Line 183:
	== Appendix: Code ==		== Appendix: Code ==

	A note on the code: This code provided here is the implementation function for finding the worm. It is the highest level function in a complete module of functions. It includes all the threshold functions and contour finding functions provided by the OpenCV library. The other custom functions for finding the head and tail, segmenting and skeletonizing are outside the scope of this project so I did not include them in this appendix. There are also a series of structures used to store the images, data and other parameters that are defined outside of this function, but in my worm analysis module, but have not been included here either. The main function simply loads an image and calls the FindWorm function to find the worm in the image and then writes the data to a csv file and saves the processed image.		A note on the code: This code provided here is the implementation function for finding the worm written in C++. It is the highest level function in a complete module of functions. It includes all the threshold functions and contour finding functions provided by the OpenCV library. The other custom functions for finding the head and tail, segmenting and skeletonizing are outside the scope of this project so I did not include them in this appendix. There are also a series of structures used to store the images, data and other parameters that are defined outside of this function, but in my worm analysis module, but have not been included here either. The main function simply loads an image and calls the FindWorm function to find the worm in the image and then writes the data to a csv file and saves the processed image.

	<code>		<code>

imported>Projects221: /* Appendix: Code */

2013-03-20T23:56:34Z

Appendix: Code

← Older revision		Revision as of 23:56, 20 March 2013
Line 183:		Line 183:
	== Appendix: Code ==		== Appendix: Code ==

	A note on the code: This code provided here is the implementation function for finding the worm. It is the highest level function in a complete module of functions. It includes all the threshold functions and contour finding functions provided by the OpenCV library. The other custom functions for finding the head and tail, segmenting and skeletonizing are outside the scope of this project so I did not include them in this appendix. There are also a series of structures used to store the images, data and other parameters that are defined outside of this function, but in my worm analysis module, but have not been included here either.		A note on the code: This code provided here is the implementation function for finding the worm. It is the highest level function in a complete module of functions. It includes all the threshold functions and contour finding functions provided by the OpenCV library. The other custom functions for finding the head and tail, segmenting and skeletonizing are outside the scope of this project so I did not include them in this appendix. There are also a series of structures used to store the images, data and other parameters that are defined outside of this function, but in my worm analysis module, but have not been included here either. The main function simply loads an image and calls the FindWorm function to find the worm in the image and then writes the data to a csv file and saves the processed image.

	<code>		<code>

imported>Projects221: /* Appendix: Code */

2013-03-20T23:55:29Z

Appendix: Code

← Older revision		Revision as of 23:55, 20 March 2013
Line 182:		Line 182:

	== Appendix: Code ==		== Appendix: Code ==

			A note on the code: This code provided here is the implementation function for finding the worm. It is the highest level function in a complete module of functions. It includes all the threshold functions and contour finding functions provided by the OpenCV library. The other custom functions for finding the head and tail, segmenting and skeletonizing are outside the scope of this project so I did not include them in this appendix. There are also a series of structures used to store the images, data and other parameters that are defined outside of this function, but in my worm analysis module, but have not been included here either.

	<code>		<code>

imported>Projects221: /* Results: */

2013-03-20T23:51:30Z

Results:

← Older revision		Revision as of 23:51, 20 March 2013
Line 141:		Line 141:


	To demonstrate the effectiveness of the adaptive thresholding methods, we must take a look at the results of the contour finding ~~as well as the time needed for thresholding and~~ the percentage of frames that correctly identified the worm as the largest contour. The Gaussian kernel was much more effective at creating the worm as the largest contour. It makes sense that this method is more accurate because of the way it weights the pixels in the neighborhood. The pixels closest to the center are weighted higher than those farther from the center. It is also remarkable that the score decreases as the block size gets really big. I suspect this is because the filtering becomes too high, effectively smearing the background making other objects such as the cantilever, making that contour bigger that the one surrounding the worm. The smearing also brings out the entire worm body. Therefore, the contour around the worm is a nice outline of the body, as we desire.		To demonstrate the effectiveness of the adaptive thresholding methods, we must take a look at the results of the contour finding to see the percentage of frames that correctly identified the worm as the largest contour. The Gaussian kernel was much more effective at creating the worm as the largest contour. It makes sense that this method is more accurate because of the way it weights the pixels in the neighborhood. The pixels closest to the center are weighted higher than those farther from the center. It is also remarkable that the score decreases as the block size gets really big. I suspect this is because the filtering becomes too high, effectively smearing the background making other objects such as the cantilever, making that contour bigger that the one surrounding the worm. The smearing also brings out the entire worm body. Therefore, the contour around the worm is a nice outline of the body, as we desire.

	[[File:AdaptiveBlockSizeComparison.png]]		[[File:AdaptiveBlockSizeComparison.png]]
Line 151:		Line 151:


	To demonstrate the correlation between computation time and the two dimensional nature of the algorithm, I have plotted the block size versus the computation time for each method in the figure below. The Gaussian kernel method computation time increases linearly with block size, but, surprisingly, the mean kernel remains constant. I was surprised by this result, given that the amount of computation to find the threshold is effectively the same in both methods: weight each pixel by some value and then sum the weighted pixels. In the Gaussian kernel method though, we need to calculate what those weights are based on the size of the block, which is increases in complexity as the block size increases. This is the result of the linear increase in threshold finding time with block size that we see in these results.		To demonstrate the correlation between computation time and the two dimensional nature of the algorithm, I have plotted the block size versus the computation time for each method in the figure below. The Gaussian kernel method computation time increases linearly with block size, but surprisingly, the mean kernel remains constant. I was surprised by this result, given that the amount of computation to find the threshold is effectively the same in both methods: weight each pixel by some value and then sum the weighted pixels. In the Gaussian kernel method though, we need to calculate what those weights are based on the size of the block, which is increases in complexity as the block size increases. This is the result of the linear increase in threshold finding time with block size that we see in these results.

	[[File:BlockSizeVsTime.png\|400px]]		[[File:BlockSizeVsTime.png\|400px]]

imported>Projects221: /* Results: */

2013-03-20T23:49:29Z

Results:

← Older revision		Revision as of 23:49, 20 March 2013
Line 141:		Line 141:


	To demonstrate the effectiveness of the adaptive thresholding methods, the results of the contour finding as well as the ~~threshold~~ time and the percentage of frames that correctly identified the worm as the largest contour. The Gaussian kernel was much more effective at creating the worm as the largest contour. It makes sense that this method is more accurate because of the way it weights the pixels in the neighborhood. The pixels closest to the center are weighted higher than those farther from the center. It is also remarkable that the score decreases as the block size gets really big. I suspect this is because the filtering becomes too high, effectively smearing the background making other objects such as the cantilever, making that contour bigger that the one surrounding the worm. The smearing also brings out the entire worm. Therefore, the contour around the worm is a nice outline of the body, as we desire.		To demonstrate the effectiveness of the adaptive thresholding methods, we must take a look at the results of the contour finding as well as the time needed for thresholding and the percentage of frames that correctly identified the worm as the largest contour. The Gaussian kernel was much more effective at creating the worm as the largest contour. It makes sense that this method is more accurate because of the way it weights the pixels in the neighborhood. The pixels closest to the center are weighted higher than those farther from the center. It is also remarkable that the score decreases as the block size gets really big. I suspect this is because the filtering becomes too high, effectively smearing the background making other objects such as the cantilever, making that contour bigger that the one surrounding the worm. The smearing also brings out the entire worm body. Therefore, the contour around the worm is a nice outline of the body, as we desire.

	[[File:AdaptiveBlockSizeComparison.png]]		[[File:AdaptiveBlockSizeComparison.png]]

imported>Projects221: /* Results: */

2013-03-20T23:38:39Z

Results:

← Older revision		Revision as of 23:38, 20 March 2013
Line 127:		Line 127:
	[[File:OtsuComparisonResults.png]]		[[File:OtsuComparisonResults.png]]

	Methods 2 and 3, which use the Otsu threshold were 100% successful in identifying the largest contour as the worm, the contour is not the desired outline of the body of the worm. Instead the side walls of the worm are illuminated creating portions of the worm to be left out and the contour not smooth. In order to use this method, the worm must be more evenly illuminated. This is a key finding from this investigation as it affects the hardware construction of the system. Instead of a single light source from one direction, I will employ a ring light that surrounds the sample, applying light from all directions so that the worm is more evenly illuminated.		Methods 2 and 3, which use the Otsu threshold were 100% successful in identifying the largest contour as the worm. However, the contour is not the desired outline of the body of the worm. Instead the side walls of the worm are illuminated creating portions of the worm to be left out and the contour not smooth. In order to use this method, the worm must be more evenly illuminated. This is a key finding from this investigation as it affects the hardware construction of the system. Instead of a single light source from one direction, I will employ a ring light that surrounds the sample, applying light from all directions so that the worm is more evenly illuminated.

imported>Projects221: /* Part 2: Smart Thresholding */

2013-03-20T23:31:02Z

Part 2: Smart Thresholding

← Older revision		Revision as of 23:31, 20 March 2013
Line 48:		Line 48:
	The first step in processing the images is to threshold to find the outline of the worm. In thresholding, all the pixels in the image are classified as either foreground (maxValue = 255) or background (minValue = 0) based on specified threshold. All thresholding methods follow the same basic structure as shown in the following equation. All techniques are structured around smartly picking <math> T(x,y) </math>.		The first step in processing the images is to threshold to find the outline of the worm. In thresholding, all the pixels in the image are classified as either foreground (maxValue = 255) or background (minValue = 0) based on specified threshold. All thresholding methods follow the same basic structure as shown in the following equation. All techniques are structured around smartly picking <math> T(x,y) </math>.

	<math>dst(x,y) = \{ \begin{matrix} 1 & src(x,y) > T(x,y) \\ 0 & else \end{matrix} </math>		<math>dst(x,y) = \{ \begin{matrix} maxValue & src(x,y) > T(x,y) \\ minValue & else \end{matrix} </math>

	As shown in the images above, some of the images include faint outline of the cantilever, which must be classified as background. In my original pass at the code, I arbitrarily guessed at thresholds until the images correctly divided into classes such that when contours were found, the largest contour was the worm. In this particular video, I had to crop out the cantilever to be able to find and process the worm.		As shown in the images above, some of the images include faint outline of the cantilever, which must be classified as background. In my original pass at the code, I arbitrarily guessed at thresholds until the images correctly divided into classes such that when contours were found, the largest contour was the worm. In this particular video, I had to crop out the cantilever to be able to find and process the worm.

imported>Projects221: /* Part 1: Designing stable hardware */

2013-03-20T23:30:19Z

Part 1: Designing stable hardware

← Older revision		Revision as of 23:30, 20 March 2013
Line 34:		Line 34:
	===Part 1: Designing stable hardware ===		===Part 1: Designing stable hardware ===

	In this investigation, I hope to design a more robust hardware system so that images can be taken with repeatable quality and control. The first step in image processing, if controllable, is to take high quality images. I have found through taking several videos with my current system that stability in the construction and repeatable light placement is imperative for good analysis of images.		In the first part of this investigation, I hope to design a more robust hardware system so that images can be taken with repeatable quality and control. The first step in image processing, if controllable, is to take high quality images. I have found through taking several videos with my current system that stability in the construction and repeatable light placement is imperative for good analysis of images.

	My imaging system has two key innovations. The first is an inverted configuration, where the light is collected from below the sample. The second is an obliquely oriented lighting source. This configuration leverages the relative optical properties of the C. elegans and the silicon, the material from which the cantilevers are made. The nematode will scatted the light as it enters the body of the worm, whereas the cantilever reflects light. With an oblique source, the light incident on the cantilever will reflect at such an oblique angle that it will not be captured by the optics below. The light incident on the worm however will be scattered in many directions, including downwards where the camera can capture it. The figure below shows a sketch of the system, as well as a set of sample images taken with a prototype of the system with two orientations of the light, above the sample and to the side of the sample.		My imaging system has two key innovations. The first is an inverted configuration, where the light is collected from below the sample. The second is an obliquely oriented lighting source. This configuration leverages the relative optical properties of the C. elegans and the silicon, the material from which the cantilevers are made. The nematode will scatted the light as it enters the body of the worm, whereas the cantilever reflects light. With an oblique source, the light incident on the cantilever will reflect at such an oblique angle that it will not be captured by the optics below. The light incident on the worm however will be scattered in many directions, including downwards where the camera can capture it. The figure below shows a sketch of the system, as well as a set of sample images taken with a prototype of the system with two orientations of the light, above the sample and to the side of the sample.

	[[File:OpticsSet1.png\|500px]]		[[File:OpticsSet1.png\|500px]]

	A prototype of this system was previously built. Below is a sample video taken on the system, as well as a video of processed images from the first revision of the software. The faint outline of the cantilever can be seen in the upper left corner of the image. The noise generated by the cantilever is large enough to create an object in the foreground of the image with a larger contour than the worm. The processing mis-identifies the worm in almost every frame. An improved hardware system as well as better thresholding techniques will improve this processing.		A prototype of this system was previously built. Below is a sample video taken on the system, as well as a video of processed images from the first revision of the software. The faint outline of the cantilever can be seen in the upper left corner of the image. The noise generated by the cantilever is large enough to create an object in the foreground of the image with a larger contour than the worm. The processing mis-identifies the worm in almost every frame. An improved hardware system as well as better thresholding techniques will improve this processing. By looking at the strengths and weaknesses of different thresholding techniques, I intend to improve my design.

	[[File:Video1.gif]] [[File:Video1Precessed.gif]]		[[File:Video1.gif]] [[File:Video1Precessed.gif]]

imported>Projects221: /* Introduction: */

2013-03-20T23:26:46Z

Introduction:

← Older revision		Revision as of 23:26, 20 March 2013
Line 12:		Line 12:
	We know that the worm has exactly six neurons that respond to gentle touch sensation. The morphology of these neurons is consistent across the species, with three of the cell bodies located near the animal’s tail and the other three cell bodies located about 50% of the length of the body from the head. Each of the neurons has a long process that runs from the cell body towards the head (Goodman 2006). There exists strong evidence that points to specific ion channels in the processes of the neurons that open and close as strain is applied to the neuron. (O'Hagan 2004) As those channels open and close, ions flow in and out of the cell, creating a change in electric potential.		We know that the worm has exactly six neurons that respond to gentle touch sensation. The morphology of these neurons is consistent across the species, with three of the cell bodies located near the animal’s tail and the other three cell bodies located about 50% of the length of the body from the head. Each of the neurons has a long process that runs from the cell body towards the head (Goodman 2006). There exists strong evidence that points to specific ion channels in the processes of the neurons that open and close as strain is applied to the neuron. (O'Hagan 2004) As those channels open and close, ions flow in and out of the cell, creating a change in electric potential.

	One method we are using to characterize the sense of touch is to quantify the behavioral response of the C. elegans to gentle touch. To do this, we probe the body of the worm with nano- and micro-scale forces and score whether or not the worm responds. ~~We’ve~~ built micro-scale cantlievers with an embedded strain gauge to apply calibrated forces and integrated them into a clamping system (Park 2011). We apply forces to a freely moving worm and observe the response through a video taken through a stereoscope. A behavioral response is scored positively when the worm’s trajectory changes immediately after a force application (Petzold 2013).		One method we are using to characterize the sense of touch is to quantify the behavioral response of the C. elegans to gentle touch. To do this, we probe the body of the worm with nano- and micro-scale forces and score whether or not the worm responds. We have built micro-scale cantlievers with an embedded strain gauge to apply calibrated forces and integrated them into a clamping system (Park 2011). We apply forces to a freely moving worm and observe the response through a video taken through a stereoscope. A behavioral response is scored positively when the worm’s trajectory changes immediately after a force application (Petzold 2013).

	Currently, this system can apply forces with high precision, but the low spatial and temporal precision of targeting and analyzing behavioral response limits the data throughput and quality. Targeting of the worm is done semi-manually and the behavioral response is scored visually frame by frame. The figure below shows the manual intervention done on each frame so that the software can differentiate the worm from the cantilever. The user must actually drawn the white lines that are the boundary between the worm and the cantilever. The user must also specify the threshold used to find the skeleton indicated by the dashed line along the worm.		Currently, this system can apply forces with high precision, but the low spatial and temporal precision of targeting and analyzing behavioral response limits the data throughput and quality. Targeting of the worm is done semi-manually and the behavioral response is scored visually frame by frame. The figure below shows the manual intervention done on each frame so that the software can differentiate the worm from the cantilever. The user must actually drawn the white lines that are the boundary between the worm and the cantilever. The user must also specify the threshold used to find the skeleton indicated by the dashed line along the worm.
Line 18:		Line 18:
	[[File:ImageProcessing.png\|300px]]		[[File:ImageProcessing.png\|300px]]

	I am currently building a second generation of the system to enable automated tracking of the worm in real time. In this project I seek to design a more robust version of the hardware system as well as investigate methods for thresholding the images, the first step in the image processing. Acquiring images and thresholding are the first two steps in the process of tracking the worm. It is imperative that they ~~must~~ be robust in order to accurately find the worm skeleton and target later in the processing pipeline.		I am currently building a second generation of the system to enable automated tracking of the worm's skeleton in real time. With automated tracking we can precisely target a location along the body of the worm, such as 25% of the length of the skeleton from the head, as well as automate the behavioral response scoring. In this project I seek to design a more robust version of the hardware system as well as investigate methods for thresholding the images, the first step in the image processing. Acquiring images and thresholding are the first two steps in the process of tracking the worm. It is imperative that they be robust in order to accurately find the worm skeleton and target later in the processing pipeline.

	Prior to starting this project, I already had a working prototype for acquiring a data set of video and images, as well as a basis code to skeletonize the worm. The hardware prototype consists primarily of a chemistry stand with a ring attached to it to hold the sample. The camera is placed below the sample. A light is held to the chemistry stand using a clamp. After capturing the images of the worm, I have written code based on the algorithm developed in (Leifer 2011) to segment the body of the worm, use the segments to skeletonize the worm and find specified target along the skeleton. The basis for my code development is a library of C++ functions called OpenCV, an open source computer vision platform that employs many novel algorithms in computer vision as well as a set of functions, classes and structures for computation with images.		Prior to starting this project, I already had a working prototype for acquiring a data set of video and images, as well as a basis code to skeletonize the worm. The hardware prototype consists primarily of a chemistry stand with a ring attached to it to hold the sample. The camera is placed below the sample. A light is held to the chemistry stand using a clamp. After capturing the images of the worm, I have written code based on the algorithm developed in (Leifer 2011) to segment the body of the worm, use the segments to skeletonize the worm and find specified target along the skeleton. The basis for my code development is a library of C++ functions called OpenCV, an open source computer vision platform that employs many novel algorithms in computer vision as well as a set of functions, classes and structures for computation with images.