ISETHDR CV Experiment

Methods

Image generation

First, we need to assemble a dataset of driving images to run the YOLO algorithm on. We acquired four driving scenes from the ISR HDR Sensor Repository. Each scene includes .exr files that contain radiance data for the sky, street lights, headlights, and other lights.

ISET HDR Scenes 1112201236, 1112184733, 1113094429, and 1114031438

To set up the scenes, we consider four lighting scenarios that commonly occur during driving. The light scenarios are defined by a vector of weights for headlights, streetlights, other lights, sky map, in order. The daytime scenario has strong illumination from the sky only. The nighttime scenario is illuminated almost only by headlights and streetlights. The dusk scenario falls between day and night; it has half of the daytime sky illumination combined with headlights and streetlights. Finally, the blind scenario represents a nighttime scenario with stronger artificial lighting; the headlights and streetlights are 10x greater than in the nighttime scenario.

Illumination vector: [ headlights, streetlights, other lights, sky map ]

• Day - [ 0, 0, 0, 50 ]

• Night - [ 0.2, 0.001, 0, 0.0005 ]

• Dusk - [ 0.2, 0.001, 0, 20 ]

• Blind - [ 2, 0.1, 0, 0.0005 ]

Next, we need to set up the camera. We selected four sensors. The ar0123at and mt9v024 are single pixel sensors for automotive applications while the ov2312 is a split pixel sensor also for automotive applications. In contrast, the imx363 is a single pixel sensor for smartphone applications.

Dynamic range, field of view, and resolution are some sensor parameters that are relevant in driving applications. In this project, we examine the influence of dynamic range on object identification. A large field of view is advantageous; since the car needs to be able to sense 360 degrees around its surrounding, larger field of view means that fewer sensors are required. A higher resolution, however, is not necessarily advantageous. The objective is to simply identify objects, rather than distinguish fine details, so a high resolution may require excessive data transmission and processing. We consider the field of view and resolution (table below) but do not vary them in the experiment.

Sensor	Pixel type	Resolution	Dynamic range	Application	FOV
AR0132AT	Single	1.2 MP	115 dB	Automotive	76°
MT9V024	Single	0.4 MP	100 dB	Automotive	69°
IMX363	Single	12 MP	n/a	Smartphone	21°
OV2312	Split	2 MP	68 dB	Automotive	81°

Field of view is calculated for a 4 mm focal length

YOLO

Appendix