ISETBIO Baseball Simulation Experiment

Introduction

Hitting a pitched baseball places extreme demands on the visual system. In the few hundred milliseconds between release and home plate, a batter must detect the ball, track its motion, and judge its future location well enough to decide whether and where to swing. Understanding what information is actually available at the level of the eye before any higher-level processing can provide useful insight into how difficult different pitches are to track and discriminate.

For this project, we explore how different baseball pitch trajectories drive cone responses over time in a simplified model of the batter’s eye. We start from rendered synthetic image sequences of pitched balls and use ISETCam to model the optical image at the retina under controlled field-of-view and display conditions. We then use ISETBio’s cone mosaic framework to compute noise-free cone absorptions over time as the ball travels from release to home plate, summarizing responses in foveal, mid-peripheral, and far-peripheral regions. Finally, we analyze these cone absorption sequences to see how well they do with pitch classification and understand their relationship to pitch characteristics.

Background

Prior work on baseball vision has shown that hitters operate under severe visual constraints. The ball is small relative to viewing distance, it travels quickly, and for most of its flight it remains in peripheral or near-peripheral vision. A key result from Shapiro et al. (2010, PLoS ONE) demonstrated that when an object moves from peripheral vision into the fovea, the visual system can misinterpret the motion and produce an illusory “break.” This finding is often used to explain why hitters sometimes report a sudden directional change in pitches like curveballs even though the actual trajectory is smooth.

In addition to these perceptual factors, the underlying pitch physics are more complex than the standard Magnus-force explanation. Recent aerodynamic work has shown that the raised seams create asymmetric airflow patterns—seam-shifted wakes (Barton & Nathan, 2021; Driveline Baseball, 2020). Importantly, simply changing seam orientation can alter the ball’s movement even when velocity, spin rate, and spin axis are held constant. Two pitches that would be identical in a simplified model can therefore diverge in flight purely due to seam geometry.

There is also evidence linking hitting performance to core visuomotor abilities such as tracking accuracy, predictive saccades, and eye–hand coordination (Kato & Fukuda, 2021). This means that what the batter “sees” on any given pitch reflects a combination of the retinal input, the physical flight of the ball, and the batter’s gaze strategy.

Taken together, these findings motivate the central question of this project: given the perceptual distortions that can arise from the visual system itself, the aerodynamic differences introduced by seams, and the reliance on active gaze behavior, what does the signal actually look like at the level of the cone mosaic during a pitch? That is the specific gap this analysis aims to address.