ISETHDR CV Experiment

Introduction

Autonomous vehicles rely heavily on robust perception systems to interpret complex driving environments. Although many sensor modalities exist such as LiDAR, radar, and ultrasonic the automotive industry increasingly prioritizes camera based vision systems due to their lower cost and rich spatial detail. Typical AV systems include 6–10 cameras compared with 7 LiDAR units, resulting in substantial cost savings for large scale deployment.
In this project, we examine how HDR (High Dynamic Range) and exposure settings in the camera image signal processing (ISP) pipeline affect the performance of a modern computer vision model YOLO-E when identifying objects in low light or highly variable lighting conditions. Our central research question is: How do different HDR and exposure settings influence YOLO’s confidence and reliability in low-light driving scenes?

Background

Autonomous Vehicle Imaging Systems

• Automatic emergency braking
• Speed assist
• Autopark / Autosteer
• Traffic sign awareness
• Navigation under Autopilot

A typical AV uses 8–9 cameras to provide a complete 360° view, making reliable image processing essential.

YOLO-E - You Only Look Once computer vision model

• Single-shot detector: Divides the image into grid cells and predicts bounding boxes + class probabilities.
• Confidence estimation: Uses a sigmoid activation function, outputting scores from 0 → 1.
• Bounding boxes: Predicted relative to grid cell → refined using anchor boxes.
• HDR sensitivity: YOLO relies on local contrast; loss of detail from over or under exposure reduces object confidence.

HDR - High Dynamic Range

• Dynamic range is influenced by exposure time, ambient illumination, and sensor design.
• HDR scenes preserve detail in both dark and bright regions.
• SDR collapses detail near shadows and highlights → degrading YOLO detection.

Our challenge: Represent and manipulate HDR conditions using ISETCam tools.

Methods

Image generation

First, we need to assemble a dataset of driving images to run the YOLO algorithm on. We acquired four driving scenes from the ISR HDR Sensor Repository. Each scene includes .exr files that contain radiance data for the sky, street lights, headlights, and other lights.

ISET HDR Scenes 1112201236, 1112184733, 1113094429, and 1114031438

To set up the scenes, we consider four lighting scenarios that commonly occur during driving. The light scenarios are defined by a vector of weights for headlights, streetlights, other lights, sky map, in order. The daytime scenario has strong illumination from the sky only. The nighttime scenario is illuminated almost only by headlights and streetlights. The dusk scenario falls between day and night; it has half of the daytime sky illumination combined with headlights and streetlights. Finally, the blind scenario represents a nighttime scenario with stronger artificial lighting; the headlights and streetlights are 10x greater than in the nighttime scenario.

Illumination vector: [ headlights, streetlights, other lights, sky map ]

• Day - [ 0, 0, 0, 50 ]

• Night - [ 0.2, 0.001, 0, 0.0005 ]

• Dusk - [ 0.2, 0.001, 0, 20 ]

• Blind - [ 2, 0.1, 0, 0.0005 ]

Scene 1114031438 under day, dusk, night, and blind light scenarios with an exposure time of 8 ms

Next, we need to set up the camera. We selected four sensors. The ar0123at and mt9v024 are single pixel sensors for automotive applications while the ov2312 is a split pixel sensor also for automotive applications. In contrast, the imx363 is a single pixel sensor for smartphone applications.

Dynamic range, field of view, and resolution are some sensor parameters that are relevant in driving applications. In this project, we examine the influence of dynamic range on object identification. A large field of view is advantageous; since the car needs to be able to sense 360 degrees around its surrounding, larger field of view means that fewer sensors are required. A higher resolution, however, is not necessarily advantageous. The objective is to simply identify objects, rather than distinguish fine details, so a high resolution may require excessive data transmission and processing. We consider the field of view and resolution (table below) but do not vary them in the experiment.

Sensor	Pixel type	Resolution	Dynamic range	Application	FOV
AR0132AT	Single	1.2 MP	115 dB	Automotive	76°
MT9V024	Single	0.4 MP	100 dB	Automotive	69°
IMX363	Single	12 MP	n/a	Smartphone	21°
OV2312	Split	2 MP	68 dB	Automotive	81°

Field of view is calculated for a 4 mm focal length

In order to vary dynamic range, we sweep exposure time across 14 values from 0.1 ms to 1000 ms. In all, we sweep 4x scenes, 4x light scenarios, 4x sensors, and 14x exposure times, for a total of 896 images.

Results

YOLO Output Characteristics

• Good HDR performance ⇒ confidence remains stable across exposure times
• Overexposure ⇒ removes gradients and edges → YOLO loses confidence
• Underexposure ⇒ insufficient signal → YOLO fails to detect smaller objects

Automotive Sensors (AR0132AT & MT9V024)

• Much more stable confidence across exposures
• Optimized for HDR → recover spatial gradients even at low exposures
• AR0132AT performs exceptionally in low-light (0.1–2 ms)
• Both struggle in extreme overexposure (blind scenario)

Smartphone Sensor (IMX363)

• Poor HDR behavior → saturates easier
• Narrow field of view limits contextual detection
• Performs visibly worse than automotive sensors

OmniVision OV2312 Split Pixel

• LPD-HCG / LPD-LCG = strong low-light performance
• SPD = handles bright/high-exposure regions
• Combined output = the most consistent performance across all exposure levels

Appendix