Mazzochette

Introduction:

In the Microsystems Lab, we are studying the sense of touch, specifically how our bodies undergo mechanotransduction, the process in which mechanical signals such as forces and displacements are converted into electrochemical signals that our nervous system can understand. The human body is incredibly complex, so we are first tackling the touch sensation problem in a popular neuroscience model organism, the Caenorhabditis elegans (C. elegans).

C. elegans is a small nematode, about 1 mm in length and about 50 um wide. It consists of about 1000 somatic cells and exactly 302 neurons. It was the first animal to have its genome fully mapped, as well as its complete neural network. Finally, it is a convenient sample in a biology lab because it grows to become a full adult animal in only four days. Because of all these traits, the animal lends itself easily to genetic mutation, allowing us to modulate different aspects of the worm’s composition for study.

We know that the worm has exactly six neurons that respond to gentle touch sensation. The morphology of these neurons is consistent across the species, with three of the cell bodies located near the animal’s tail and the other three cell bodies located about 50% of the length of the body from the head. Each of the neurons has a long process that runs from the cell body towards the head. <citation – wormbase> There exists strong evidence that points to specific ion channels in the processes of the neurons that open and close as strain is applied to the neuron. <citation – o hagan> As those channels open and close, ions flow in and out of the cell, creating a change in electric potential.

One method we are using to characterize the sense of touch is to quantify the behavioral response of the C. elegans to gentle touch. To do this, we probe the body of the worm with nano- and micro-scale forces and score whether or not the worm responds. We’ve built micro-scale cantlievers with an embedded strain gauge to apply calibrated forces and integrated them into a clamping system. <citation – sj park force clamp system paper> We apply forces to a freely moving worm and observe the response through a video taken through a stereoscope. A behavioral response is scored positively when the worm’s trajectory changes immediately after a force application.

Currently, this system can apply forces with high precision, but the low spatial and temporal precision of targeting and analyzing behavioral response limits the data throughput and quality. Targeting of the worm is done semi-manually and the behavioral response is scored visually frame by frame, as indicated in the Figure below.

I am currently building a second generation of the system to enable automated tracking of the worm in real time. In this project I seek to design a more robust version of the hardware system as well as investigate methods for thresholding the images, the first step in the image processing. Acquiring images and thresholding are the first two steps in the process of tracking the worm. It is imperative that they must be robust in order to accurately find the worm skeleton and target later in the processing pipeline.

Prior to starting this project, I already had a working prototype for acquiring a data set of video and images, as well as a basis code to skeletonize the worm. After capturing the images of the worm, I have written code based on the algorithm developed in <C. FY paper> to skeletonize the worm and find specified target. The basis for my code development is a library of C++ functions called OpenCV, an open source computer vision platform that employs many novel algorithms in computer vision as well as a set of functions, classes and structures for computation with images.

Methods:

The imaging system has two key innovations. The first is an inverted configuration, where the light is collected from below the sample. The second is an obliquely oriented lighting source. This configuration leverages the relative optical properties of the C. elegans and the silicon, the material from which the cantilevers are made. The nematode will scatted the light as it enters the body of the worm, whereas the cantilever reflects light. With an oblique source, the light incident on the cantilever will reflect at such an oblique angle that it will not be captured by the optics below. The light incident on the worm however will be scattered in many directions, including downwards where the camera can capture it. The figure below shows a sketch of the system, as well as a set of sample images taken with a prototype of the system with two orientations of the light, above the sample and to the side of the sample.

In this investigation, I hope to design a more robust hardware system so that images can be taken with repeatable quality and control. The first step in image processing, if controllable, is to take high quality images. I have found through taking several videos with my current system that stability in the construction and repeatable light placement is imperative for good analysis of images.

The first step in processing the images is to threshold to find the outline of the worm. In thresholding, all the pixels in the image are classified as either foreground (maxValue = 255) or background (minValue = 0) based on specified threshold.

$dst(x,y)=$

As shown in the images above, some of the images include faint outline of the cantilever, which must be classified as background. In my original pass at the code, I arbitrarily guessed at thresholds until the images correctly divided into classes such that when contours were found, the largest contour was the worm. In this particular video, I had to crop out the cantilever to be able to find and process the worm.

In order to build a more robust processing algorithm, active thresholding techniques must be employed. To achieve this, I investigated three advanced methods for choosing T(x,y) such that body of the worm is designated as the picture foreground, while everything else in the image is specified as background. The first method is a popular, stochastically based algorithm called Otsu’s Method. The second and third methods are adaptive threshold techniques where a mean kernel and Gaussian kernel are used to determine a local threshold. All three of these methods are built into the OpenCV image processing library, simplifying implementation.

Otsu’s Method

The goal of Otsu’s method is to find the threshold that minimizes the variances of the histogram in each class. This can be shown to be equivalent of maximizing the between class variance, after making some assumptions including the process is stationary. We choose to maximize the between class variance because it can be calculated recursively. Total variance is the sum of the within class variance and the between class variance. Since the image is stationary, the variance is constant and minimizing one term is equivalent to maximizing the opposing term.

${\sigma }^2={\sigma }_w^2(T)+{\sigma }_b^2(T)$

${\sigma }_w^2(T)=P_f(T){\sigma }_f^2(T)+P_b(T){\sigma }_b^2(T)$

${\sigma }_b^2(T)=P_f(T)[1-P_f(T)][{\mu }_f(T)-{\mu }_b(T){]}^2$

Need to find $T_{opt}$ that maximizes ${\sigma }_b^2(T)$

In class probabilities:

$P_f(T)=\sum _{g=0}^{T}p(g),P_b(T)=\sum _{g=T+1}^{G}p(g)$

In class means:

${\mu }_f(T)=\sum _{g=0}^T\frac{gp(g)}{P_f^2},{\mu }_b(T)=\sum _{g=T+1}^G\frac{gp(g)}{P_b^2}$

In class variances:

${\sigma }_f^2=\sum _{g=0}^T[g-{\mu }_f(T){]}^2\frac{p(g)}{P_f(T)},{\sigma }_b^2=\sum _{g=T+1}^G[g-{\mu }_b(T){]}^2\frac{p(g)}{P_b(T)}$

Probability mass function:

$p(g)=\frac{n_g}{N},g=0,\mathrm{...},G$

where, $G$ is the maximum luminance value in the image. In my case this value if 255. $N$ is number of pixels, and $n_g$ is the number of pixels at value $g$ Note: $n_1+n_2+\mathrm{...}+n_G=N$

I implemented this algorithm in two ways. In the first strategy, from a set of 22 images from a single video, I found the Otsu threshold in the first image and then used that threshold for the rest of the images in the set. In the second strategy, I calculated the Otsu threshold for each image and used that threshold.

Adaptive Threshold Methods

In adaptive threshold methods, a unique threshold is found for each pixel in the image using a kernel of a specified block size centered on the pixel of interest. This method effectively smooths the local area before thresholding. OpenCV has two adaptive methods that I ran on my image samples, the mean kernel and the Gaussian kernel.

Mean Kernel

This method computes the local mean in the specified block size. The value of the pixel is above the local mean, the pixel is classified as foreground. If not, it is classified as background. To analyze this method, I ran the algorithm for a variety of block sizes and parameter C.

$T(x,y)=\frac{1}{BlockSiz{e}^2}\sum _{i=-\lfloor BlockSize/2\rfloor }^{\lfloor BlockSize/2\rfloor }\sum _{j=-\lfloor BlockSize/2\rfloor }^{\lfloor BlockSize/2\rfloor }src(x-1,y-j)-C$

Gaussian Kernel

In the Gaussian kernel method, the threshold T(x,y) is found by pointwise multiplying a Gaussian window of the specified block size with the neighborhood around the current pixel. The product is them summed over all the points in the neighborhood. If the value of the pixel is above the calculated threshold, it is classified as foreground, otherwise it is classified as background. To analyze this method, I ran the algorithm for a variety of block sizes and parameter C.

$T(x,y)=\sum _{i=-\lfloor BlockSize/2\rfloor }^{\lfloor BlockSize/2\rfloor }\sum _{j=-\lfloor BlockSize/2\rfloor }^{\lfloor BlockSize/2\rfloor }src(x-1,y-j)*G(x-i,y-j)-C$

Where $G(x,y)$ is a Gaussian window specified by the block size.

$G_i=\alpha *e^{-\frac{(i-\frac{BlockSize-1}{2}{)}^2}{(2\sigma {)}^2}},\sigma =0.3(\frac{BlockSize}{2}-1)+0.8$

$\alpha$ chosen such that $\sum _i{G}_i=1$

Results: