ISETHDR CV Experiment: Difference between revisions

From Psych 221 Image Systems Engineering
Jump to navigation Jump to search
← Older editNewer edit →
Lschul (talk | contribs)
16 edits
Lschul (talk | contribs)
16 edits
Lschul (talk | contribs)
16 edits
Line 5: Line 5:
== Methods ==
== Methods ==


Image generation
=== Image generation ===


First, we need to assemble a dataset of driving images to run the YOLO algorithm on. We acquired four driving scenes from the ISR HDR Sensor Repository. Each scene includes .exr files that contain radiance data for the sky, street lights, headlights, and other lights.  
First, we need to assemble a dataset of driving images to run the YOLO algorithm on. We acquired four driving scenes from the ISR HDR Sensor Repository. Each scene includes .exr files that contain radiance data for the sky, street lights, headlights, and other lights.  


Scene images <insert images>
{| class="wikitable" style="border:none; text-align:center;"
1112184733
|-
1112201236
| [[File:id1112201236_ar0132at_day_05_4.0ms.png|200px]]
1114031438
| [[File:id1112184733_ar0132at_day_05_4.0ms.png|200px]]
1113094429
| [[File:id1113094429_ar0132at_day_05_4.0ms.png|200px]]
| [[File:id1114031438_ar0132at_day_05_4.0ms.png|200px]]
|-
| colspan="4" style="text-align:center" | ISET HDR Scenes 1112201236, 1112184733, 1113094429, and 1114031438
|}


When setting up the scenes, we consider four lighting scenarios that commonly occur while driving. The light scenarios are described by a weight for each .exr file.  
To set up the scenes, we consider four lighting scenarios that commonly occur during driving. The light scenarios are defined by a vector of weights for headlights, streetlights, other lights, sky map, in order. The daytime scenario has strong illumination from the sky only. The nighttime scenario is illuminated almost only by headlights and streetlights. The dusk scenario falls between day and night; it has half of the daytime sky illumination combined with headlights and streetlights. Finally, the blind scenario represents a nighttime scenario with stronger artificial lighting; the headlights and streetlights are 10x greater than in the nighttime scenario.


<insert table>
: ''Illumination vector: [ headlights, streetlights, other lights, sky map ]''
Day - strong sky illumination
* Day -   [ 0, 0, 0, 50 ]
[0, 0, 0, 50]


Dusk - weak sky illumination with headlights and streetlights
* Night - [ 0.2, 0.001, 0, 0.0005 ]
[0.2, 0.001, 0, 20]


Night - headlights and streetlights
* Dusk - [ 0.2, 0.001, 0, 20 ]
[0.2, 0.001, 0, 0.0005]


Blind - extreme headlights and streetlights
* Blind - [ 2, 0.1, 0, 0.0005 ]
[2, 0.1, 0, 0.0005]




Next, we set up the camera. We select four sensors:
Next, we need to set up the camera. We selected four sensors. The ar0123at and mt9v024 are single pixel sensors for automotive applications while the ov2312 is a split pixel sensor also for automotive applications. In contrast, the imx363 is a single pixel sensor for smartphone applications.


{|
! style="text-align:left;"| Sensor
! Resolution
! Dynamic range
|-
|ar0132at
|1.2 MP
|>115 dB
|-
|mt9v024
|0.4 MP
|>100 dB
|-
|imx363
|12 MP
|n/a
|-
|ov2312
|2 MP
|n/a
|}


That sets up the scenes. Now we set up the cameras.
We use 4 different sensor types: three automotive and one smartphone.


Automotive sensors
Automotive sensors
Line 42: Line 60:
ar0123at: 1.2 MP, dynamic range >115 dB
ar0123at: 1.2 MP, dynamic range >115 dB
mt9v024: 0.36 MP, dynamic range >100 dB
mt9v024: 0.36 MP, dynamic range >100 dB
ov2312 split pixel: 2 MP, dynamic range >68 dB
ov2312: 2 MP, dynamic range >68 dB
 
Smartphone sensor
imx363: 12 MP
imx363: 12 MP


Revision as of 17:31, 8 December 2025

Introduction

Background

Methods

Image generation

First, we need to assemble a dataset of driving images to run the YOLO algorithm on. We acquired four driving scenes from the ISR HDR Sensor Repository. Each scene includes .exr files that contain radiance data for the sky, street lights, headlights, and other lights.

ISET HDR Scenes 1112201236, 1112184733, 1113094429, and 1114031438

To set up the scenes, we consider four lighting scenarios that commonly occur during driving. The light scenarios are defined by a vector of weights for headlights, streetlights, other lights, sky map, in order. The daytime scenario has strong illumination from the sky only. The nighttime scenario is illuminated almost only by headlights and streetlights. The dusk scenario falls between day and night; it has half of the daytime sky illumination combined with headlights and streetlights. Finally, the blind scenario represents a nighttime scenario with stronger artificial lighting; the headlights and streetlights are 10x greater than in the nighttime scenario.

Illumination vector: [ headlights, streetlights, other lights, sky map ]
  • Day - [ 0, 0, 0, 50 ]
  • Night - [ 0.2, 0.001, 0, 0.0005 ]
  • Dusk - [ 0.2, 0.001, 0, 20 ]
  • Blind - [ 2, 0.1, 0, 0.0005 ]


Next, we need to set up the camera. We selected four sensors. The ar0123at and mt9v024 are single pixel sensors for automotive applications while the ov2312 is a split pixel sensor also for automotive applications. In contrast, the imx363 is a single pixel sensor for smartphone applications.

Sensor Resolution Dynamic range
ar0132at 1.2 MP >115 dB
mt9v024 0.4 MP >100 dB
imx363 12 MP n/a
ov2312 2 MP n/a


Automotive sensors

ar0123at: 1.2 MP, dynamic range >115 dB mt9v024: 0.36 MP, dynamic range >100 dB ov2312: 2 MP, dynamic range >68 dB imx363: 12 MP


Now build the data set

use exposure time to alter dynamic range

times = 0.1, 0.5, 1, 2, 4, 6, 8, 12, 16, 20, 50, 100, 500, 1000 ms

Loop over: 4x scenes 4x lighting scenarios 4x sensors 14x exposures

total images - 896



YOLO

Results

Conclusions

Appendix

Retrieved from "http://vista.su.domains/psych221wiki/index.php?title=ISETHDR_CV_Experiment&oldid=145566"