KonradMolner - Revision history

imported>Student2016: /* Introduction */

2016-12-19T21:14:21Z

Introduction

← Older revision		Revision as of 21:14, 19 December 2016
Line 8:		Line 8:
	Accommodation is an absolute monocular depth cue that refers to the state of the ciliary muscles, which focus the crystalline lens in each eye. As with vergence, the state of these muscles varies as a function of the distance to the point at which the eyes are focused. Accommodation combined with the associated visual cue, retinal blur or perceived depth of field (DOF), make up the focus cues, which are particularly useful for depth perception of objects nearby.		Accommodation is an absolute monocular depth cue that refers to the state of the ciliary muscles, which focus the crystalline lens in each eye. As with vergence, the state of these muscles varies as a function of the distance to the point at which the eyes are focused. Accommodation combined with the associated visual cue, retinal blur or perceived depth of field (DOF), make up the focus cues, which are particularly useful for depth perception of objects nearby.

	The brain is “wired” to interpret all of these visual and oculomotor stimuli in a consistent manner, because that is what a person with normal vision experiences in the physical world. Unfortunately, all commercially-available near-eye displays produce conflicting depth cues referred to as the vergence-accommodation conflict or VAC. Displaying a stereoscopic image pair on a near-eye display creates binocular disparity, which drives the vergence of the eye to a simulated object distance. However, conventional near-eye displays use static magnifying optics to create a virtual image of a physical micro-display that is perceived at a fixed optical distance that cannot be changed in software. Hence, retinal blur drives the users’ accommodation to the virtual image (for reference, 1.3m for Oculus Rift). The discrepancy between these depth cues (vergence and accommodation), hinders visual performance [Hoffman et. al. 2008] and creates visual discomfort and fatigue, and ~~comprised~~ visual clarity [Shibata et al. 2011].		The brain is “wired” to interpret all of these visual and oculomotor stimuli in a consistent manner, because that is what a person with normal vision experiences in the physical world. Unfortunately, all commercially-available near-eye displays produce conflicting depth cues referred to as the vergence-accommodation conflict or VAC. Displaying a stereoscopic image pair on a near-eye display creates binocular disparity, which drives the vergence of the eye to a simulated object distance. However, conventional near-eye displays use static magnifying optics to create a virtual image of a physical micro-display that is perceived at a fixed optical distance that cannot be changed in software. Hence, retinal blur drives the users’ accommodation to the virtual image (for reference, 1.3m for Oculus Rift). The discrepancy between these depth cues (vergence and accommodation), hinders visual performance [Hoffman et. al. 2008] and creates visual discomfort and fatigue, and compromised visual clarity [Shibata et al. 2011].
	In this work, we explore a new potential way of tackling the VAC. By taking advantage of the parallax on the retina due to eye movements during fixation, we hypothesize the potential to drive our accommodation to arbitrary distances. Our accommodation is driven by the retinal blur cue, created by objects in the scene exhibiting parallax over the pupil, and therefore retina. An out of focus objects is one that exhibits a “large” amount of parallax on the retina, meaning that the images as perceived by pinhole cameras on opposite sides of the pupil, do not overlap perfectly on the retina, giving the impression of a blurred, or out of focus object. An in-focus object would be one that exhibits no parallax (theoretically), where the images on opposing sides of the pupil overlap perfectly on the retina. When one changes focus, the crystalline lens of the eye bends, changing the objects in the scene that overlap on the retina.		In this work, we explore a new potential way of tackling the VAC. By taking advantage of the parallax on the retina due to eye movements during fixation, we hypothesize the potential to drive our accommodation to arbitrary distances. Our accommodation is driven by the retinal blur cue, created by objects in the scene exhibiting parallax over the pupil, and therefore retina. An out of focus objects is one that exhibits a “large” amount of parallax on the retina, meaning that the images as perceived by pinhole cameras on opposite sides of the pupil, do not overlap perfectly on the retina, giving the impression of a blurred, or out of focus object. An in-focus object would be one that exhibits no parallax (theoretically), where the images on opposing sides of the pupil overlap perfectly on the retina. When one changes focus, the crystalline lens of the eye bends, changing the objects in the scene that overlap on the retina.

imported>Student2016: /* OpenGL Pipeline */

2016-12-19T08:30:21Z

OpenGL Pipeline

← Older revision		Revision as of 08:30, 19 December 2016
Line 34:		Line 34:
	OpenGL is a cross-platform API for rendering 2D and 3D vector graphics. Using OpenGL we are able to render 3D scenes with parameters similar to that of the eye. We generate a rotational model of the eye that accurately represents what an eye might “see” during eye drift. Using OpenGL we render the viewpoint from the center of the pupil, making the strong assumption that the pupil is infinitely small (i.e. the pinhole model of the eye). Using OpenGL, we render two line stimuli at distances of 25 cm and 4m from the camera. The stimuli are scaled to have the same perceived size. To evaluate different amounts of drift we render the scene with varying amounts of eye rotations. We use 0.2º horizontal increments with a maximum rotation of 1º. For each rotation angle, we re-render the scene. We project the 3D scene onto a plane 1m , as indicated by d, away from the retina, with a 2º field of view as indicated by θ. We then use ISETBIO, as described in the next section, to see the amount of parallax generated on the retina itself.		OpenGL is a cross-platform API for rendering 2D and 3D vector graphics. Using OpenGL we are able to render 3D scenes with parameters similar to that of the eye. We generate a rotational model of the eye that accurately represents what an eye might “see” during eye drift. Using OpenGL we render the viewpoint from the center of the pupil, making the strong assumption that the pupil is infinitely small (i.e. the pinhole model of the eye). Using OpenGL, we render two line stimuli at distances of 25 cm and 4m from the camera. The stimuli are scaled to have the same perceived size. To evaluate different amounts of drift we render the scene with varying amounts of eye rotations. We use 0.2º horizontal increments with a maximum rotation of 1º. For each rotation angle, we re-render the scene. We project the 3D scene onto a plane 1m , as indicated by d, away from the retina, with a 2º field of view as indicated by θ. We then use ISETBIO, as described in the next section, to see the amount of parallax generated on the retina itself.

	~~[[File:Eye~~ model.~~png\|500px \| center]]~~		To create an accurate representation of what the eye may see when it drifts by some angle Φ, we must consider that there is both a slight shift in the pupil location, indicated by Δx and Δz, and a change in the viewing direction. Assuming an eye radius R of 12mm, the change in pupil positions are indicated by equations 2 and 3. The perspective shift is accounted for in equations 4-7, and is in terms of the change in viewing frustum. All of these variations are accounted for in our rendering model.



	To create an accurate representation of what the eye may see when it drifts by some angle Φ, we must consider that there is both a slight shift in the pupil location, indicated by Δx and Δz, and a change in the viewing direction. Assuming an eye radius R of 12mm, the change in pupil positions are indicated by equations 2 and 3. The perspective shift is accounted for in equations 4-7, and is in terms of the change in viewing frustum. All of these variations are accounted for in our rendering model.

	[[File:Model equations.png\|300px \| center]]		[[File:Eye model.png\|500px \| left \| thumb \| caption \| Rotational model of the eye. A slight change of angle effects both the viewing direction as well as causing a slight shift in position of the pupil.]]
			[[File:Model equations.png\|300px \| center \| thumb \| Rendering parameters that change as a function of the rotation angle.]]

	=== ISETBIO Pipeline ===		=== ISETBIO Pipeline ===

imported>Student2016: /* Introduction */

2016-12-19T08:27:08Z

Introduction

← Older revision		Revision as of 08:27, 19 December 2016
Line 4:		Line 4:
	In the natural environment, the human visual system relies on a variety of cues to determine the 3D layout of the scenes. Vergence and accommodation are two different oculomotor cues that the human visual system can use for estimating absolute and relative distance of objects. Vergence refers to the relative rotation angle of the two eyeballs. When we fixate on a nearby object, our eyes rotate inwards, whereas the opposite happens when we fixate on far objects. The brain uses the relaxation or contraction of the extraocular muscles that physically rotate the eyeball as a cue for the absolute distance of the fixated object (see Fig. x). The associated visual cue is known as binocular disparity, the relative displacement between the images of a 3D scene point projected on the two retinal images. Together, vergence and disparity make up stereopsis, which is generally considered a strong depth cue, especially for objects at intermediate distances (i.e. 1-10m).		In the natural environment, the human visual system relies on a variety of cues to determine the 3D layout of the scenes. Vergence and accommodation are two different oculomotor cues that the human visual system can use for estimating absolute and relative distance of objects. Vergence refers to the relative rotation angle of the two eyeballs. When we fixate on a nearby object, our eyes rotate inwards, whereas the opposite happens when we fixate on far objects. The brain uses the relaxation or contraction of the extraocular muscles that physically rotate the eyeball as a cue for the absolute distance of the fixated object (see Fig. x). The associated visual cue is known as binocular disparity, the relative displacement between the images of a 3D scene point projected on the two retinal images. Together, vergence and disparity make up stereopsis, which is generally considered a strong depth cue, especially for objects at intermediate distances (i.e. 1-10m).

	[[File:Perception.png\|600px \| ~~center~~]]		[[File:Perception.png\|600px \| right \| thumb \| caption \| Overview of relevant depth cues. Vergence and accommodation are oculomotor cues whereas binocular disparity and retinal blur are visual cues. In normal viewing conditions, disparity drives vergence and blur drives accommodation.]]

	Accommodation is an absolute monocular depth cue that refers to the state of the ciliary muscles, which focus the crystalline lens in each eye. As with vergence, the state of these muscles varies as a function of the distance to the point at which the eyes are focused. Accommodation combined with the associated visual cue, retinal blur or perceived depth of field (DOF), make up the focus cues, which are particularly useful for depth perception of objects nearby.		Accommodation is an absolute monocular depth cue that refers to the state of the ciliary muscles, which focus the crystalline lens in each eye. As with vergence, the state of these muscles varies as a function of the distance to the point at which the eyes are focused. Accommodation combined with the associated visual cue, retinal blur or perceived depth of field (DOF), make up the focus cues, which are particularly useful for depth perception of objects nearby.

imported>Student2016: /* Appendix I */

2016-12-19T08:20:10Z

Appendix I

← Older revision		Revision as of 08:20, 19 December 2016
Line 161:		Line 161:

	== Appendix I ==		== Appendix I ==
	~~Upload source~~ code, ~~test~~ images, ~~etc, and give a description of each link. In some cases, your acquired data may~~ be ~~too large to store practically~~. ~~In this case, use your judgement (or consult one of us) and only link the most relevant data~~. ~~Be sure to describe the purpose of your code and to edit the code for clarity~~. ~~The purpose of placing the code online is to allow others to verify your methods and to learn from your ideas~~.		The code used to implement the OpenGL eye rotation model as well the ISETBIO code (in Matlab), as well as some images generated using these procedures, can be found [https://www.dropbox.com/s/y908gq1jvjaulij/Code_Data.zip?dl=0 here].

	== Appendix II ==		== Appendix II ==
	Robert tackled the OpenGL portions of the simulation, while Keenan created the ISETBIO model and image processing pipeline to measure cone disparity. We collaborated on the presentation, future work, and write up.		Robert tackled the OpenGL portions of the simulation, while Keenan created the ISETBIO model and image processing pipeline to measure cone disparity. We collaborated on the presentation, future work, and write up.

imported>Student2016: /* Citations */

2016-12-19T07:10:04Z

Citations

← Older revision		Revision as of 07:10, 19 December 2016
Line 121:		Line 121:

	== Citations ==		== Citations ==
			* Hoffman, D., Girshick, A., Akeley, K., AND Banks, M. 2008. Vergence-accommodation conflicts hinder visual perfor- mance and cause visual fatigue. Journal of Vision 8, 3.

			* SHIBATA, T., KIM, J., HOFFMAN, D., AND BANKS, M. 2011. The zone of comfort: Predicting visual discomfort with stereo displays. Journal of Vision 11, 8, 11.

			* SUGIHARA, T., AND MIYASATO, T. 1998. 32.4: A lightweight 3-d hmd with accommodative compensation. SID Digest 29, 1, 927–930.

			* SCHOWENGERDT, B., AND SEIBEL, E. 2006. True 3-d scanned voxel displays using single or multiple light sources. J. SID 14, 2, 135–143.

			* DOLGOFF, E. 1997. Real-depth imaging: a new 3d imaging tech- nology with inexpensive direct-view (no glasses) video and other applications. Proc. SPIE 3012, 282–288.

			* AKELEY, K., WATT, S., GIRSHICK, A., AND BANKS, M. 2004. 688 A stereo display prototype with multiple focal distances. ACM 689-690
			Trans. Graph. (SIGGRAPH) 23, 3, 804–813.

			*ROLLAND, J., KRUEGER, M., AND GOON, A. 2000. Mul- tifocal planes head-mounted displays. Applied Optics 39, 19, 32093215.

			*LLULL, P., BEDARD, N., WU, W., TOSIC, I., BERKNER, K., 763 AND BALRAM, N. 2015. Design and optimization of a near-eye 764
			ZHAI, Z., DING, S., LV, Q., WANG, X., AND ZHONG, Y. 2009. Extended depth of field through an axicon. Journal of Modern Optics 56, 11, 1304–1308.
			multifocal display system for augmented reality. In OSA Imaging and Applied Optics.

			* LANMAN, D., AND LUEBKE, D. 2013. Near-eye light field dis- plays. ACM Trans. Graph. (SIGGRAPH Asia) 32, 6, 220:1– 220:10.

			* HUA, H., AND JAVIDI, B. 2014. A 3d integral imaging optical see- through head-mounted display. Optics Express 22, 11, 13484– 13491.

			* HUANG, F., CHEN, K., AND WETZSTEIN, G. 2015. The light field stereoscope: Immersive computer graphics via factored near-eye light field display with focus cues. ACM Trans. Graph. (SIG- GRAPH) 34, 4.

	* Watson, Andrew B., and John I. Yellott. "A unified formula for light-adapted pupil size." Journal of vision 12.10 (2012): 12-12.		* Watson, Andrew B., and John I. Yellott. "A unified formula for light-adapted pupil size." Journal of vision 12.10 (2012): 12-12.

imported>Student2016: /* Introduction */

2016-12-19T07:06:49Z

Introduction

← Older revision		Revision as of 07:06, 19 December 2016
Line 4:		Line 4:
	In the natural environment, the human visual system relies on a variety of cues to determine the 3D layout of the scenes. Vergence and accommodation are two different oculomotor cues that the human visual system can use for estimating absolute and relative distance of objects. Vergence refers to the relative rotation angle of the two eyeballs. When we fixate on a nearby object, our eyes rotate inwards, whereas the opposite happens when we fixate on far objects. The brain uses the relaxation or contraction of the extraocular muscles that physically rotate the eyeball as a cue for the absolute distance of the fixated object (see Fig. x). The associated visual cue is known as binocular disparity, the relative displacement between the images of a 3D scene point projected on the two retinal images. Together, vergence and disparity make up stereopsis, which is generally considered a strong depth cue, especially for objects at intermediate distances (i.e. 1-10m).		In the natural environment, the human visual system relies on a variety of cues to determine the 3D layout of the scenes. Vergence and accommodation are two different oculomotor cues that the human visual system can use for estimating absolute and relative distance of objects. Vergence refers to the relative rotation angle of the two eyeballs. When we fixate on a nearby object, our eyes rotate inwards, whereas the opposite happens when we fixate on far objects. The brain uses the relaxation or contraction of the extraocular muscles that physically rotate the eyeball as a cue for the absolute distance of the fixated object (see Fig. x). The associated visual cue is known as binocular disparity, the relative displacement between the images of a 3D scene point projected on the two retinal images. Together, vergence and disparity make up stereopsis, which is generally considered a strong depth cue, especially for objects at intermediate distances (i.e. 1-10m).

	[[File:Perception.png]]		[[File:Perception.png\|600px \| center]]

	Accommodation is an absolute monocular depth cue that refers to the state of the ciliary muscles, which focus the crystalline lens in each eye. As with vergence, the state of these muscles varies as a function of the distance to the point at which the eyes are focused. Accommodation combined with the associated visual cue, retinal blur or perceived depth of field (DOF), make up the focus cues, which are particularly useful for depth perception of objects nearby.		Accommodation is an absolute monocular depth cue that refers to the state of the ciliary muscles, which focus the crystalline lens in each eye. As with vergence, the state of these muscles varies as a function of the distance to the point at which the eyes are focused. Accommodation combined with the associated visual cue, retinal blur or perceived depth of field (DOF), make up the focus cues, which are particularly useful for depth perception of objects nearby.

imported>Student2016: /* OpenGL Pipeline */

2016-12-19T07:06:30Z

OpenGL Pipeline

← Older revision		Revision as of 07:06, 19 December 2016
Line 34:		Line 34:
	OpenGL is a cross-platform API for rendering 2D and 3D vector graphics. Using OpenGL we are able to render 3D scenes with parameters similar to that of the eye. We generate a rotational model of the eye that accurately represents what an eye might “see” during eye drift. Using OpenGL we render the viewpoint from the center of the pupil, making the strong assumption that the pupil is infinitely small (i.e. the pinhole model of the eye). Using OpenGL, we render two line stimuli at distances of 25 cm and 4m from the camera. The stimuli are scaled to have the same perceived size. To evaluate different amounts of drift we render the scene with varying amounts of eye rotations. We use 0.2º horizontal increments with a maximum rotation of 1º. For each rotation angle, we re-render the scene. We project the 3D scene onto a plane 1m , as indicated by d, away from the retina, with a 2º field of view as indicated by θ. We then use ISETBIO, as described in the next section, to see the amount of parallax generated on the retina itself.		OpenGL is a cross-platform API for rendering 2D and 3D vector graphics. Using OpenGL we are able to render 3D scenes with parameters similar to that of the eye. We generate a rotational model of the eye that accurately represents what an eye might “see” during eye drift. Using OpenGL we render the viewpoint from the center of the pupil, making the strong assumption that the pupil is infinitely small (i.e. the pinhole model of the eye). Using OpenGL, we render two line stimuli at distances of 25 cm and 4m from the camera. The stimuli are scaled to have the same perceived size. To evaluate different amounts of drift we render the scene with varying amounts of eye rotations. We use 0.2º horizontal increments with a maximum rotation of 1º. For each rotation angle, we re-render the scene. We project the 3D scene onto a plane 1m , as indicated by d, away from the retina, with a 2º field of view as indicated by θ. We then use ISETBIO, as described in the next section, to see the amount of parallax generated on the retina itself.

	[[File:Eye model.png\|~~300px~~ \| center]]		[[File:Eye model.png\|500px \| center]]

imported>Student2016: /* Introduction */

2016-12-19T07:06:15Z

Introduction

@@ Line 1: / Line 1: @@
 == Introduction ==
-Motivate the problem. Describe what has been done in the past.
+Immersive and experiential computing systems are entering the consumer market and have the potential to profoundly impact our society. Applications of these systems range from entertainment, education, collaborative work, simulation and training to telesurgery, phobia treatment, and basic vision research. In every immersive experience, the primary interface between the user and the digital world is the near-eye display. Thus, developing near-eye display systems that provide a high-quality user experience is of the utmost importance. Many characteristics of near-eye displays that define the quality of an experience, such as resolution, refresh rate, contrast, and field of view, have been significantly improved over the last years. However, a potentially significant source of visual discomfort prevails: the vergence-accommodation conflict.
 == Background ==

imported>Student2016: /* Results */

2016-12-19T07:02:31Z

Results

← Older revision		Revision as of 07:02, 19 December 2016
Line 87:		Line 87:

	The displacement calculations are illustrated in the following graphics:		The displacement calculations are illustrated in the following graphics:
	[[File:EyeDistances.png\|200px~~\|thumb~~\|center]]		[[File:EyeDistances.png\|200px\|center]]

	== Conclusions ==		== Conclusions ==

imported>Student2016: /* OpenGL Pipeline */

2016-12-19T07:01:27Z

OpenGL Pipeline

← Older revision		Revision as of 07:01, 19 December 2016
Line 23:		Line 23:
	OpenGL is a cross-platform API for rendering 2D and 3D vector graphics. Using OpenGL we are able to render 3D scenes with parameters similar to that of the eye. We generate a rotational model of the eye that accurately represents what an eye might “see” during eye drift. Using OpenGL we render the viewpoint from the center of the pupil, making the strong assumption that the pupil is infinitely small (i.e. the pinhole model of the eye). Using OpenGL, we render two line stimuli at distances of 25 cm and 4m from the camera. The stimuli are scaled to have the same perceived size. To evaluate different amounts of drift we render the scene with varying amounts of eye rotations. We use 0.2º horizontal increments with a maximum rotation of 1º. For each rotation angle, we re-render the scene. We project the 3D scene onto a plane 1m , as indicated by d, away from the retina, with a 2º field of view as indicated by θ. We then use ISETBIO, as described in the next section, to see the amount of parallax generated on the retina itself.		OpenGL is a cross-platform API for rendering 2D and 3D vector graphics. Using OpenGL we are able to render 3D scenes with parameters similar to that of the eye. We generate a rotational model of the eye that accurately represents what an eye might “see” during eye drift. Using OpenGL we render the viewpoint from the center of the pupil, making the strong assumption that the pupil is infinitely small (i.e. the pinhole model of the eye). Using OpenGL, we render two line stimuli at distances of 25 cm and 4m from the camera. The stimuli are scaled to have the same perceived size. To evaluate different amounts of drift we render the scene with varying amounts of eye rotations. We use 0.2º horizontal increments with a maximum rotation of 1º. For each rotation angle, we re-render the scene. We project the 3D scene onto a plane 1m , as indicated by d, away from the retina, with a 2º field of view as indicated by θ. We then use ISETBIO, as described in the next section, to see the amount of parallax generated on the retina itself.

	[[File:Eye model.png\|300px\| ~~thumb \| right~~]]		[[File:Eye model.png\|300px \| center]]