ISETBIO Baseball Simulation Experiment

Introduction

Hitting a pitched baseball places extreme demands on the visual system. In the few hundred milliseconds between release and home plate, a batter must detect the ball, track its motion, and judge its future location well enough to decide whether and where to swing. Understanding what information is actually available at the level of the eye before any higher-level processing can provide useful insight into how difficult different pitches are to track and discriminate.

For this project, we explore how different baseball pitch trajectories drive cone responses over time in a simplified model of the batter’s eye. We start from rendered synthetic image sequences of pitched balls and use ISETCam to model the optical image at the retina under controlled field-of-view and display conditions. We then use ISETBio’s cone mosaic framework to compute noise-free cone absorptions over time as the ball travels from release to home plate, summarizing responses in foveal, mid-peripheral, and far-peripheral regions. Finally, we analyze these cone absorption sequences to see how well they do with pitch classification and understand their relationship to pitch characteristics.

Background

Prior work on baseball vision has shown that hitters operate under severe visual constraints. The ball is small relative to viewing distance, it travels quickly, and for most of its flight it remains in peripheral or near-peripheral vision. A key result from Shapiro et al. (2010, PLoS ONE) demonstrated that when an object moves from peripheral vision into the fovea, the visual system can misinterpret the motion and produce an illusory “break.” This finding is often used to explain why hitters sometimes report a sudden directional change in pitches like curveballs even though the actual trajectory is smooth.

In addition to these perceptual factors, the underlying pitch physics are more complex than the standard Magnus-force explanation. Recent aerodynamic work has shown that the raised seams create asymmetric airflow patterns—seam-shifted wakes (Barton & Nathan, 2021; Driveline Baseball, 2020). Importantly, simply changing seam orientation can alter the ball’s movement even when velocity, spin rate, and spin axis are held constant. Two pitches that would be identical in a simplified model can therefore diverge in flight purely due to seam geometry.

There is also evidence linking hitting performance to core visuomotor abilities such as tracking accuracy, predictive saccades, and eye–hand coordination (Kato & Fukuda, 2021). This means that what the batter “sees” on any given pitch reflects a combination of the retinal input, the physical flight of the ball, and the batter’s gaze strategy.

Taken together, these findings motivate the central question of this project: given the perceptual distortions that can arise from the visual system itself, the aerodynamic differences introduced by seams, and the reliance on active gaze behavior, what does the signal actually look like at the level of the cone mosaic during a pitch? That is the specific gap this analysis aims to address.

Methods

Pitch Trajectory Simulation

To generate realistic ball-flight trajectories, I implemented a JavaScript port of the aerodynamic model developed by Alan Nathan (Nathan, University of Illinois), which predicts pitch trajectories given physical parameters including:

• backspin, sidespin, and gyro spin
• initial velocity vector
• pitcher release point
• drag coefficient and lift coefficient
• spin axis direction

Nathan’s model numerically integrates the equations of motion under Magnus force, drag force, gravity and outputs 3D coordinates from release point to home plate. By embedding this model directly into my simulation backend, I was able to generate new pitch trajectories on-demand for arbitrary combinations of speed, spin rate, spin axis, while also altering seam orientation (for visual purposes only).

(Reference: https://baseball.physics.illinois.edu/trajectory-calculator.html)

Seam-Orientation Impacts

A baseball’s seam orientation - which is the 3D rotation matrix that maps the ball’s spin axis to the stitched seam geometry — is a major contributor to pitch movement. For example, four-seam fastballs typically use a seam alignment where the ball’s spin axis is perpendicular to the horseshoe seams, maximizing Magnus lift. While seam orientation is known to have impacts on pitch movement, my focus on this project is not to estimate those effects, but rather attempt to measure the effects on retinal capabilities as it relates to a batter facing different types of pitches with different orientations, speeds, movement.

To study how seam orientation alters the retinal image, I represented each orientation as a 3×3 rotation matrix applied to the baseball texture. Examples include:

• canonical orientations for the following four pitch types:
  • 4-Seam Fastball
  • Curveball
  • Slider
  • Changeup
• Rotated seam orientations for each pitch type

During rendering, the baseball texture and normal map are rotated according to this matrix so that identical pitch trajectories with different seam matrices produce different appearance sequences, even when spin rate and velocity are held constant.

This setup allowed me to isolate how the orientation of the baseball’s seams while keeping all physical pitch attributes (velocity, spin rate, movement profile) constant — affects the cone-level retinal response. As an example, the figure below shows the three seam-orientation conditions used for the four-seam fastball: 0° (the standard 4-seam look), 45°, and 90°. These rotations don’t change the physics of the pitch in this simulation, but they do change what part of the seam pattern the batter sees.

Visual Rendering of the Spinning Ball Trajectory

I used Three.js to render synthetic image sequences of each pitch as it traveled along the simulated trajectory. The renderer produced a frame-by-frame animation of the ball’s position, orientation, and seam rotation over time. For each pitch type (4-seam fastball, slider, changeup, curveball), I ran 27 total simulations by perturbing seam orientation, spin rate, and velocity around their “normal” values.

For example, for the 4-seam fastball I used:

A. Three seam orientations: normal, rotated 45°, rotated 90°
B. Three spin rates: 3000 RPM (baseline), +500 RPM, –500 RPM
C. Three velocities: 96 mph (baseline), +3 mph, –3 mph

I repeated this structure for all four pitch types, using pitch-specific baseline values (e.g., a slider has a different default seam orientation and a baseline velocity closer to 87 mph rather than 96 mph).

Each simulation produced a little over 100 PNG frames showing the ball from roughly 55 feet out all the way to the strike zone and which was used in the retinal simulation stage.