AlexOliviaAudrey

Introduction

In the world of autonomous driving, the simulation of car headlamps serves practical purposes. It involves the application of principles from optics to study how light interacts with the components of vehicle sensors and sight. This process is integral to ensuring that headlights meet safety regulations, provide optimal visibility, and contribute to an efficient driving experience. Typically, projects involved with autonomous driving or assisted driving often use simulation to perform testing before bringing it into the real-world. For example, lane keeping and pedestrian detection rely heavily on cameras and headlights which are first simulated to ensure feasibility and test capability before being implemented in real world situations. Therefore, ensuring that a simulation can accurately represent the real world and get accurate 3D spectral radiance scenes is important, and is also challenging.

One motivation for our project was that there was a recent tragedy where Google Maps led a father to drive off the end of a broken bridge. This tragedy, although not caused by autonomous driving, could possibly have been avoided if the father could see with brighter or more properly aligned headlights. Thus, we want to further improve headlights on cars through simulation. And the first step is calibrating the simulation to ensure that simulated headlights are more representative of real-world situations. And, maybe in the future, this step can help ensure that headlights will adhere to a regulatory compliance standard, are more energy efficient, and we could ensure that headlights are more standardized.

For our project, we wanted to calibrate the ISET simulation to have more accurate beam patterns for different headlight patterns. With our project, we used computer graphics (PBRT) and MATLAB to evaluate camera and headlight design options when used with industry standard benchmark scenarios.

Background

In this project, we utilize a couple radiometric and photometric equations we learned both in PSYCH 221 and from our mentors for this project to analyze the data we collected from the PR-670 spectroradiometer. These equations were to calculate irradiance from the radiance spectrum and luminance from the radiance spectrum.

The PR-670 spectroradiometer (shown below) was designed by JADAK to take spectral based photometric and colorimetric light measurements.

It was designed to capture precise light measurements from different possible light sources ranging from display monitors and projectors to reflective surfaces to industrial testing applications. It is configured with 256 detector elements and has a range of detection from 0.2 fL to 2,500,000 fL and a spectral resolution of 1.56 nm per pixel. We were able to use the PR-670 camera to take accurate measurements while using MATLAB to save the data and analyze it.

We also used a couple of the ISET repositories such as ISET3d and ISETauto to develop our calibration methodology. ISET3d interfaces with PBRTv4 to render realistic, ray-traced 3D scenes. In order to better understand night-time driving scenes, we must port our measurements from the real world into simulation. Our project focuses on this process for a specific case: the calibration of headlight brightness.

With the help of Dave Cardinal, we chose two headlamps to simulate: a halogen bulb headlight and a LED headlight. The LED headlight has a low and a high beam setting, however, we decided to calibrate the LED headlight to the low beam setting.

Methods

Data Collection

In order to fully characterize a light source for simulation, we would like to know the spectral radiance of the light, i.e. how much power per square meter per steradian is emitted, and which wavelengths compose the light.

To take such measurements, one uses a spectroradiometer, which records exactly the spectral radiance of a source over some small angular portion of its field of view. The PSYCH221 teaching team owns a PR-670 spectroradiometer and has a MATLAB script that runs the spectroradiometer, which we used to take our data.

The data from the PR-670 comes in the form of a .mat file for each measurement, containing a dictionary with two fields – “wavelength” and “data” – encoding the wavelengths at which the spectral radiance is measured, and the corresponding spectral radiance at each wavelength, respectively.

Spectral radiances of the halogen beam (left) and LED beam (right) on a white color card. Note these spectra make sense; A halogen bulb is a black body radiator, so we expect a broad black body curve across many wavelengths, peaking somewhere in the infrared, exactly as shown in the data. White LEDs are made from blue LEDs coated in phosphors that fluoresce yellow when radiated with blue light. Hence we expect a spectral peak in blue and yellow, as we see from the spectrum.

In deciding how to take measurements which would characterize the headlight, we quickly realized that it would be ineffective to point the spectroradiometer directly at a bare headlight, as the sensor would quickly be saturated. We considered using a neutral density filter in front of the lamp and a short integration time on the spectroradiometer. In the end, however, we realized that in ISET ray-tracing simulations, the most easily measurable metric of “brightness” was the luminance of different points in the scene.

With this in mind, we decided to take an indirect measurement of brightness by shining the headlamp at a surface of characterized reflectance and then measuring the reflected spectral radiance at various sample points in that scene, and then map those observed luminances to an ISET scene simulating the same scenario. This can be seen in the two figures below where the first figure contains the Halogen light’s beam pattern and the second figure contains the LED light’s beam pattern.

We collected 24 data points for each beam, assembled in a 3x8 matrix as shown above. We did this manually, labeling each file name by its corresponding matrix indices (as is shown, although hard to make out, in the photos). Each measurement was taken at the center of the sticky note, and we would manually adjust the spectroradiometer’s focus between measurements. Sticky notes were used only for alignment, not for measurements. Once the spectroradiometer was focused on the center of the sticky note, one of us would hold the grey or white card against the wall for the actual measurement as shown in the figure below:

With this data taken, we moved on to try different methods of synthesizing and analyzing this data. We first tried to calculate irradiance from the spectral radiance to see if our collected data matches our expectations using the equations in the below figure:

With M being the emittance, ρ being the reflectance, and R being the radiance, we can calculate E which is the irradiance. From this measurement, we were able to plot the irradiance maps for both the Halogen and the LED headlight beam patterns:

With these maps, we confirmed that the data we captured were representative of our sampled beam patterns, but we realized that this was still a radiometric quantity. So, we decided to calculate a photometric quantity instead: luminance.

As we know, spectral radiance is a radiometric quantity: one where spectra are measured as they physically exist in the real world. Luminance, on the other hand, is a photometric quantity: one that describes how humans perceive spectra. In converting from a radiometric quantity to a photometric one, we must always take into account the photopic luminosity function, which describes how bright the human eye perceives different wavelengths. This calculation for spectral radiance is summarized in the below figure:

Caption: When working with real data, it is crucial that the units match. Rendered ISET scenes yield luminance in candelas per square meter, whereas our data was spectral radiance in watts per square meter per steradian per nanometer. Thus to convert our data into a form comparable to ISET simulations, we performed the above transform from spectral radiance to luminance.

With this equation, we then transform every data point in the same way, yielding the following luminance maps for the halogen and LED beams (incident on grey cards and white cards)

Calibration

PBRT lights in ISET have several properties that we can modify to change the appearance of light

1) Type

2) Scale

3) FOV

4) Power

In our simulations, the light “type” was set to be a [projection light], and the “FOV” was set to 40 degrees. The attributes “scale” and “power” both change the intensity of the light, but “power” changes it nonlinearly. We thus chose to scale the intensity of our simulated headlamp using the “scale” parameter.

In addition to these basic parameters, we can also give the headlamp a property “obj.lightMask”, where the light mask is a 2D matrix that matches the size of the resolution of the headlamp, thus specifying a pixel-by-pixel attenuation. This is what allows us to port the specific beam profiles into the simulation.

With this in mind, we established a calibration method consisting of five steps, summarized in the image below:

1) We calculate a normalized version of our luminance profile (divide all values by the peak value). This gives us a dimensionless “attenuation map” that specifies the beam profile shape by specifying relative intensity without setting absolute intensity.

2) We then render the scene with this uncalibrated attenuation map and some default settings for the light, and measure the luminance from the scene.

3) We then calculate the ratio of luminance in real life to luminance in simulation, which tells us by how much we need to scale the power in the simulated light to match the absolute intensity we saw in real life.

4) We then multiply the “scale” parameter in ISET by this calculated ratio.

5) We then re-render the scene with this scaled intensity and ensure that it matches our real life data.

We experimented with two different ways of calculating the scale factor:

We found these two ways of calculating the scale factor to be quite similar, but we surmise that the “peak luminance” method is more accurate because the mean luminance from ISET includes points outside the main beam pattern.

Interpolation

We coarsely sampled the beam patterns using 3x8 sticky note grids with a total of 24 spectroradiometer samples. Our sampling does not fully capture the details of the beam patterns, particularly the halogen pattern with its distinct geometry. To upscale, we tried multiple interpolation functions in Matlab to test which interpolation method would accurately match the beam patterns we saw.

Nearest Neighbor

Halogen vs. LED Visualization

Halogen vs. LED Render

Bilinear

Halogen vs. LED Visualization

Halogen vs. LED Visualization

Cubic

Halogen vs. LED Visualization

Spline

Halogen vs. LED Visualization

Halogen vs. LED Render

Conclusions

Overall, we were able to calculate the brightness of two headlamps for ISET simulations, which we have added to ISETauto as headlamp presets. The code creating the headlamp presets as well as the scene we rendered to calibrate them can be found on our forked versions of the ISET3d and ISETauto repositories.

Although we have successfully calibrated the brightness of these headlamps, there is more that can be done to improve their realism in simulation:

1) Implementation of the lamp spectrum in ISET as a light preset

Our current brightness calibration is done with the projection light spectrum as D65, which is not accurate for halogen or LED beams. Creation of custom light profiles for the halogen beam and LED beam would be straightforward, given that we have spectral radiance data from both lights, and is a viable next step.

2) Higher resolution and more efficient mapping of the beam profile

If one desired to calibrate a variety of different headlamps, with wide angular spreads, then doing this manually as we did would be very time consuming. Taking even 24 data points for each headlamp took us over three hours due to the spectroradiometer’s data collection time and reset time as well as the need to recenter the focus on the center of a specific location with each measurement.

In reality, this method is probably quite redundant, given that the beam profile should be largely spectrally uniform. This means that we should be able to take the spectrum once, and then from thereafter just take in a measure of radiance or brightness. In the future, this should be straightforward with the Google Pixel 4a sensor that the Wandell Lab has calibrated and can use to take RAW photos. One could use exposure bracketing to get an accurate picture of the beam profile, and then scale intensity as we did with the luminance data. In this case, there would be no need to interpolate, as the beam profile would already be high resolution (resolution of the phone camera).

Indeed, if one hoped to do realistic night time driving simulations, one would want to characterize more headlights. These cheap headlights we characterized are much smaller and dimmer than those used on most modern cars today. The halogen headlight has no high beam / low beam switching, and although the LED light does, the low-beam setting light is not particularly well defined to a low area. That being said, the actual light source is not as important as the process by which one would calibrate it, which is what we have shown in our project.

References

• https://www.nytimes.com/2023/09/21/us/google-maps-lawsuit-collapsed-bridge.html#:~:text=The%20family%20of%20a%20North,led%20directly%20to%20his%20death.

• https://www.pbr-book.org/3ed-2018/Light_Sources/Point_Lights

• https://www.jadaktech.com/product/spectrascan-pr-670/