2009 Jim Cummings
Back to Psych 204 Projects 2009
A Comparison of Segmentation Performance: Amateur, Experienced, and Algorithm
Segmentation of the brain scan images produced by functional magnetic resonance imaging is a fundamental first step for the production of functional activity maps. The most basic of modern segmentation techniques typically entails employing software that allows a user to differentially depict white matter, grey matter, cerebrospinal fluid from both one another and other structures not of interest. In turn, the ability to accurately map functional data is dependent upon proper initial segmentation of images. Even the most minor of mistakes in voxel classification can results in a topologically incorrect model of the brain (Fischl, Liu, Dale, 2001). Therefore, the motivation for this project was two-fold: 1) to allow an amateur with no hands-on knowledge to gain experience with a fundamental process of functional imaging and 2) to compare how alternate segmentation performances may differentially affect both the resultant three-dimensional meshes and the ability to accurately map functional data. This project specifically compared the results of segmentations produced by the amateur, a more experienced individual (as segmentation skill is thought to increase with practice), and an auto-segmenting program.
Methods
Image Acquisition
All segmentations conducted were based on a single brain scan stored on file in the VISTA Lab. This file was provided by Jonathan Winawer
Segmentation
The amateur segmentation of the greyscale brain image was conducted using ITK-Snap and following directions supplied within VISTA Lab manual. The experienced segmentation of the same greyscale image had been previously completed by a member of the VISTA Lab and was provided by Jonathan Winawer. Finally, the algorithmic segmentation was conducted using the FSL automated preprocessing tool as implemented through VISTASOFT and following the directions supplied within the VISTA Lab manual.
Additionally, the segmentations were then transformed into three-dimensional meshes using ITK-Snap's built-in mesh constructor, allowing for initial comparison between performances.
Mapping of Functional Data
The three segmentations were also then transformed into three separate three-dimensional meshes using mrVista and mrmViewer.
Functional data maps were then projected onto these meshes in order to compare how structural differences may result in distinctive mappings. The functional data was of vision-related activity during a word rotation task. The data was from the same brain as the greyscale image employed for segmentation. The data file was provided by Jonathan Winawer.
Results
All results were limited to a basic visual comparison between alternate meshes and activity patterns. No statistical analysis of the significance of noted differences between meshes or activity patterns has been conducted.
Segmentation and ITK-Snap Meshes
Initial comparison of the amateur and experienced segmentation performances revealed a large difference in the number of handles produced during segmentation. Handles are essentially cavities within areas that have been segmented as white matter. Fixing them depends on their nature of origin: handles produced merely by failing to segment a patch within a region should be filled in; handles that result from falsely segmenting too far outwards (towards gray matter) should be resolved by removing the extraneous segmentation that produced the cavity. The large number of handles produced by the amateur segmentation would suggest a less cautious or diligent effort during the fine segmentation stage. This finding would indicate that a mere visual comparison of the the resulting ITK meshes for the amateur and experienced segmentations was deceiving; indeed, these meshes were largely similar, though the some handles (of the second type mentioned above) were apparent.
The algorithm segmentation, too, produced a sizable number of handles. However, of real comparative interest is the visual distinction between the meshes produced by algorithm and the experienced segmenter. Indeed, a three-dimensional mesh of solely the regions segmented by the experienced segmenter but not the algorithm (Algorithm "Misses" as seen in Figure 3 below) reveals some differences quite obvious to the naked eye. Of particular interest is the finding that the algorithm failed to segmented a sizable (again, statistical analysis absent from this report) amount of the ventral right hemisphere (indicated in blue). Similarly, a mesh of the material segmented by the algorithm but not the experienced segmenter (Algoritm "False Alarms" as seen in Figure 4 below) indicates some interesting differences. In addition to the algorithm segmenting the cerebellum (which is not of interest and thus not segmented by an experienced human segmenter), in some regions it would appear to also segment white matter much closer to the brain surface than does the human. This is the case for the anterior brain. Whereas a human segmenter has the contextual knowledge to not segment white matter to this proximity to the brain surface and CSF (as he would factor in a layer of gray matter), the algoritm would appear to fail in this regard.
mrVista Meshes
As mentioned above, new meshes were then created using mrVista (which would allow for the eventual projection of functional maps). In light of the functional data file available, only right hemisphere meshes were created for each of the three segmentations. Figure 5 below offers a comparison of the occipital areas of the right hemisphere meshes created from the experienced and amateur segmentations. Of note are distinctions similar to those observed when comparing the ITK meshes: the amateur mesh tended to have more visible handles, more jagged fold curvature, and differences in general fold shape and structure.
A visual comparison of mrVista meshes created from the experienced and algorithm segmentations yields similar though somewhat less pronounced results. The algorithm mesh holds some visible handles, jagged edges, and some differences in the general fold structure. However, the algorithm mesh's distinctions from the experienced mesh are perhaps less severe that those between the experienced and the amateur, in that 1) much of the jaggedness is found within the cerebellum (and is thus not of concern), and 2) by and large, the relative differences in general fold structure are less dramatic.
Mapping of Functional Data
Again, the true purpose of creating the mrVista meshes was to take advantage of their allowance of the mapping of functional data. The three respective meshes were all inflated (inflation of the meshes allows activity along buried sulci to be visible from the surface (Wandell, Chial, & Backus, 2000; mrVista Tutorial)). After the meshes were inflated, the functional data was applied to the newly rendered surface. ROI labels were also applied to the meshes so as to provide reference in a visual comparison.
As mentioned above, the functional data used here was an activation set pertaining to a word rotation task. During such a task, a word rotates on the subject's screen; different orientations of the word are thought to correlated with different regions of activation. The activation for each orientation is depicted within the functional map with a given color, as seen in the color scheme presented in Figure 7.
As the visual cortex is organized into multiple visual field maps (Wandell, Dumoulin, & Brewer, 2007), we would expect relatively clear demarcations of activity based on the color scheme above: regions for a given word orientation perception should be represented by relatively solid strands of a given color. In comparing the functional meshes created from the experienced and amateur segmentations (Figure 8.), we see that this is largely the case with the experienced mesh; however, when the same activity data gets mapped onto the amateur mesh, it becomes distorted with splotches of color pervading regions of alternate types of activity. Additionally, when the activity is mapped onto the experienced mesh, it would appear to most closely fit the regional demarcations depicted by the ROI labels.
When visually comparing the functional meshes produced by the experienced and algorithm segmentations (Figure 9.), we see that the functional data mapping for the algorithm is far more similar to the end result produced by the amateur segmentation than the end result produced by the experienced segmentation. Not only does the algorithm mesh contain specks of activity color scattered throughout alternate color strands, but - also like the function mapping for the amateur mesh - the strands themselves fit the ROI labels less accurately relative to the those projected onto the experienced mesh.
Conclusions
The reiterate, the initial goal of this project was to allow a novice with no previous experience to be exposed to some procedural elements and software commonly employed in functional data images. Considering the production of the segmentations and meshes here via ITK-Snap, mrVista, mrmViewer, and Matlab, the project was a success in light of this goal.
Regarding the secondary goal of conducting a meaningful comparison of the segmentations, meshes, and functional mappings, this project offers some initial insights but requires further work. The results described above would be greatly served by some manner of statistical analysis in order to investigate the relative significance of any differences between the alternate meshes and mappings produced by the three different segmentation performances. Additionally, this project employed ROI labels created for a specific mesh; as such, it would be useful to employ the proper ROIs for each functional mesh created (as noted during the discussion session for this project's in-class presentation).
Nonetheless, regardless of the above mentioned limitations, some general conclusions can be made regarding the different segmentation performances and their resultant effects on modeling brain structure and activity. First, it is evident that experience pays off for human segmenters - due diligence and practice in the exercise of ITK segmentation would appear to result in significantly fewer handles and less jagged segmentation of white matter. Second, the FSL auto-segmenter would appear to be a powerful and useful tool, but it nonetheless makes some mistakes that may need to be corrected for. Therefore, in closing, if nothing else, this project would point towards the use of a combination-approach to segmenting: in order to balance demands on time and accuracy, it may be best to employ an auto-segementing algorithm and then apply an experienced human touch in order to correct for specific deficiencies.
Citations
References
Fischl, B., Liu, A., & Dale, A.M. (2001). Autmated Manifold Surgery: Constructing Geometrically Accurate and Topologically Correct Models of the Human Cerebral Cortex. IEEE Transactions on Medical Imaging, 20(1), 70-80.
Wandell, B.A., Chial, S., & Backus, B.T. (2000). Visualization and Measurement of the Cortical Surface. Journal of Cognitive Neuroscience, 12(5), 739-752.
Wandell, B.A., Dumoulin, S.O., & Brewer, A.A. (2007). Visual Field Maps in Human Cortex. Neuron, 56, 366-383.
Software
ITK-Snap - http://www.itksnap.org/pmwiki/pmwiki.php
FSL Software Libraries - http://www.fmrib.ox.ac.uk/fsl/
VISTASOFT - http://white.stanford.edu/newlm/index.php/Software#VISTASOFT_.28Matlab.29
mrVista - http://white.stanford.edu/newlm/index.php/MrVista
mrmViewer - http://white.stanford.edu/newlm/index.php/MrmViewer
Appendix I
A very sincere thanks to Jon Winawer for extended consultation and extensive patience in completion of this project.