CZCRetinalProsthesis
Introduction
The project aims at modeling the electric stimulation elicited by a subretinal-implanted photovoltaic retinal prosthesis, given a light field input from outside the eye. The model would help people better understand prosthetic vision, and would be informative for optimizing the image processing workflow.
Background
Subretinal implanted photodiode arrays with micrometer-level pixels are considered as a promising approach to restore vision. The system comprises an array implanted behind the inner nuclear layer (INL) and a pair of laser projection goggles that selectively elicit a set of pixels with near infrared (NIR) laser beams [1]. The microscopic image of Figure 1 shows the front surface (facing the inner nuclear layer and "outside") of a photovoltaic array implant. Figure 2 schematically illustrates the workflow of the whole system. However, the projected light field in the human optics has yet been carefully studied. Together with the unique point spread function (SPF) associated with a photovoltaic pixel [2], this results in the primeval understanding of the prosthetic vision.
A MATLAB toolbox is available for modeling the front end of the human visual system, including the scene construction, the human optics and the retinal images. However, a major modification necessary is the refractive index of the crystalline lens for 880nm wavelength, which, to the best of my knowledge, has not been reported [3][4][5][6][9]. The mosaic sampling by cones would be replaced by the electric field elicited by the hexagonal photovoltaic pixels [2]. The technical path of the project would be to some extent similar to the one described in [7] and [8].
Based on the model, researchers may further explore the “matching space” for prosthetic vision, to make intuitive sense of the restoration quality. With the matching space, one may also study the image processing workflow on the goggles, to optimize for the best visual quality with limited computational power.
Methods
The project is built in MATLAB with the ISETBIO toolbox ( https://github.com/isetbio/isetbio ). The toolbox provides a convenient way of modeling the human lens' geometry, as well as converting a scene to an optical image on the retina. A photocurrent response is later calculated from the optical image. The current response is then convolved with the electric potential elicited by a single pixel, to determine the overall potential distribution in front of the device.
Refractive Index
The first step was to estimate the refractive index of the human lens at 850nm. Though it is not directly reported in the literature, one can see the trend from the IR end of the visible range that the refractive index is asymptotically converging to a 1st-order polynomial of , as is shown in Figure 3. A primal explanation of such relationship is given in the appendix with a quantum mechanical model. These observations give reasonable confidence that the refractive index at 850nm is predictable from the visible range. The function @wvfLCAFromWavelengthDifference provides three models to extrapolate the refractive index, 'hoferCode', 'thibosPaper' and 'iset' respectively, and they gives highly consistent results in the diopter error (w.r.t. 580nm standard wavelength), 0.7351, 0.7349 and 0.7350 respectively. The number for the "water eye" assumption is calculated to be 0.7915 [11], which is reasonably close to the other three. Therefore, 0.7350 is chosen to be the diopter correction at 850nm.
Geometry
The geometry of the human optics is best described with the Zernick wavefront basis, which has been provided in the ISETBIO toolbox for both a virtual eye model and real measurements. The pipeline is free to switch between different Zernick wavefront models.
Stiles Crawford Effect
The SCE correction is not applicable here, since the model is not with biological photoreceptors. The relevant correction would be setting . Thus, no masking is applied to the illumination angle.
Electric Potential Distribution
A computational model was run in COMSOL to determined the 3D electric potential distribution, with the illumination of one pixel. The optical image was then sampled in a hexagonal manner, corresponding to the arrangement of the photovoltaic pixels. Specifically, the activation at the center of a pixel is set proportional to the integral of all photon absorption inside the hexagon. The activation profile was then convolved with the electric potential distribution, and the result gave the final electric field response of the prosthetic vision.
Results
Optical Images
With the proper diopter correction, we can calculate the point spread function of the 850nm light using the wavefront measurement from any specific subject. Figure 4 shows the point spread function of Subject 1 in the ISETBIO wavefront dataset.
Moving forward, one may compare the distortion under different field of view (FOV), for several classic scenes. The scenes' distances are all set to be 1.2 meter.
Electric Stimuli
With the COMSOL model, we may plot the potential distribution elicited by the current from a single pixel. The pixel size is not important at this step, since the electric field scales linearly with the geometric size. Note that such potential distribution is 3-dimensional. For plotting purpose, we take the sum of each z-direction column, and plot the potential as a 2-D function as in Figure 8. A more sophisticated scheme may be chosen to convert 3-D potential to 2-D neural response, but the scope is beyond this study.
Now we have the essential tool to study how the pixel size on the retinal prosthesis effect perception quality. Figure 9 shows the comparison between 100um, 55um and 10um pixels. The scene is 1.2m away, with FOV=8, which is the challenging level one would expect in daily life. The hexagonal sampling feature is very obvious, especially for 100um and 55um devices. The face is hardly recognizable with 100um, recognizable but with major defects with 55um, and almost faithful to the optical image with 10um pixels.
Conclusions
1. With ISETBIO, one can build a pipeline, mimicking the visual information propagation from the scene to the electric stimulus of a photovoltaic prosthetic.
2. It is demonstrated that at some specific condition (FOV=8, distance=1.2m), a face is not recognizable with 100um-pixel implants, and the visual quality keeps improves all the way down to 10um pixels.
References
[1] Boinagrov, David, et al. "Photovoltaic pixels for neural stimulation: circuit models and performance." IEEE transactions on biomedical circuits and systems 10.1 (2016): 85-97.
[2] Flores, Thomas, et al. "Optimization of return electrodes in neurostimulating arrays." Journal of neural engineering 13.3 (2016): 036010.
[3] Palmer, D. A., and J. Sivak. "Crystalline lens dispersion." JOSA 71.6 (1981): 780-782.
[4] Smith, George, Barbara K. Pierscionek, and David A. Atchison. "The optical modelling of the human lens." Ophthalmic and Physiological Optics 11.4 (1991): 359-369.
[5] Liang, Junzhong, and David R. Williams. "Aberrations and retinal image quality of the normal human eye." JOSA A 14.11 (1997): 2873-2883.
[6] Jones, Catherine E., et al. "Refractive index distribution and optical properties of the isolated human lens measured using magnetic resonance imaging (MRI)." Vision research 45.18 (2005): 2352-2366.
[7] Golden, James R., et al. "Simulation of visual perception and learning with a retinal prosthesis." bioRxiv (2018): 206409.
[8] Cottaris, Nicolas, et al. "A computational observer model of spatial contrast sensitivity: Effects of wavefront-based optics, cone mosaic structure, and inference engine." bioRxiv (2018): 378323.
[9] Thibos, Larry N., et al. "The chromatic eye: a new reduced-eye model of ocular chromatic aberration in humans." Applied optics 31.19 (1992): 3594-3600.
[10] Berendschot, Tos TJM, Jan van de Kraats, and Dirk van Norren. "Wavelength dependence of the Stiles–Crawford effect explained by perception of backscattered light from the choroid." JOSA A 18.7 (2001): 1445-1451.
[11] Fernández-Prini, R., and R. B. Dooley. "Release on the refractive index of ordinary water substance as a function of wavelength, temperature and pressure." International Association for the Properties of Water and Steam (1997): 1-7.
Appendix
A quantum mechanical explanation of the refractive index's dependency on wavelength
Source code
First, to account for the different mechanism of SCE, line 190 of sceCreate.m is manually changed to: sceP.rho = 0;
The source code and other files may be found in here. The zip file includes:
1. A_optImage.m - Generate an optical image on the retinal. By commenting and uncommenting lines in the "add a scene" section, one may select different scenes to test with.
2. B_diodeResponse.m - Accumulate photon absorption on the surface of hexagonal pixels, and calculate the current density at the active electrode fo each pixel.
3. C_potential_generate.m - Generate the potential distribution elicited by a single pixel, with the right pixel size and resolution.
4. D_stimuli.m - Convert the optical image to the electric field response, by convolving photon current with the potential distribution of a single pixel.
The four files above should run in alphabetic order.
5. diodeMask.mat - Specify the 2-D geometry of a hexagonal pixel.
6. humanOptics_wvf.m - Creat a model of the human optics at 850nm NIR light.
7. IAPWS_model.m - Compute the refractive index with the water eye model.
8. Lenna.bmp - Test image.
9. refractiveError.m - Compare the difference of the refractive index from the respective extrapolation of three models, as described in "Methods".
10. RP_PSF.mat - Potential distribution elicited by a single pixel.
11. Wandell.bmp - Test image.