The Statistical Fingerprint of AI-Generated Images
Introduction
The Blur Between Real and Artificial
In 2023, the Sony World Photography Awards awarded a prize in the Creative category to an image titled "The Electrician". The image presented a haunting, black-and-white portrait of two women. However, the artist, Boris Eldagsen, refused the award, revealing that the work was not a photograph at all, but a synthetic creation generated by AI. This event marked a significant turning point: synthetic imagery had crossed a threshold of fidelity where even expert judges could no longer distinguish images formed by photons from artificially generated pixels.[1] This blurring of reality is compounded by the unprecedented scale of production. Recent reports indicate that in just 1.5 years, generative AI models have produced as many images as traditional photography produced in its first 150 years (approximately 15 billion images).[2] We are rapidly entering an era where a significant portion of digital visual data is artificially generated, rather than captured on a physical image sensing system.
Research Question and Project Goals
For the general public, the inability to distinguish real from synthetic raises concerns about misinformation, as well as a loss of intrinsic human creativity. However, for Image Systems Engineering, artificially generated images also provoke a serious question of engineering safety. Developers of autonomous systems, such as autonomous vehicles (AVs), are increasingly turning to generative AI to create training data for perception systems to make up for the scarcity of expensive real-world data, especially edge-case datasets, such as car accidents and severe weather. Although these digital twins may appear realistic to human observers, they often fail to behave like real sensors, lacking the correct noise, spectral, or optical properties. Thus, artificially generated data introduces a high-risk domain gap where autonomous systems are trained on hallucinations rather than physics. This poses a critical question: Can we discern the real from the artificial?
This project conducts a forensic analysis to determine whether AI-generated images can be distinguished from camera-simulated images based on radiometric statistics. In this investigation, we ignore the semantic content (whether the car looks like a car) and focus entirely on the physical statistics (how the image was formed from the radiance source). By comparing a physical radiance dataset (ISET3D physically based ray tracing[3]) and camera photograph dataset (ISETCam[4]) against an artificially generated image dataset (Stable Diffusion v1.5[5]), we aim to quantify the statistical fingerprints of the AI across four domains:
- Spatial Statistics: Texture and frequency distribution.
- Photometric Statistics: Signal-dependent noise response.
- Spectral Statistics: Inter-channel color correlation.
- Optical Statistics: Point spread function (PSF) and diffraction.
Background and Related Work
The Proliferation of Data and Machine Learning Models
To understand the challenge of distinguishing real from artificial imagery, one must first understand the mechanisms that enable modern AI generation and the forensic tools previously developed to detect them. The recent explosion in synthetic imagery is driven by two converging factors: the aggregation of massive image datasets and the development of latent diffusion models.
First, the capabilities of modern generative models are directly tied to the scale of their training data. In the last decade, the internet has become a vast repository of visual information, allowing researchers to scrape billions of image-text pairs. Foundational models like Stable Diffusion were trained on subsets of LAION-5B (Large-scale Artificial Intelligence Open Network), a dataset containing over 5.85 billion clip-filtered image-text pairs.[6] Similarly, proprietary models like Adobe Firefly rely on massive libraries of professional stock photography (estimated at 300+ million images), while OpenAI’s DALL-E 3 is estimated to use hundreds of millions of licensed and public images.[2][7] This scale allows models to learn a statistical approximation of reality by ingesting virtually every visual concept captured in the history of digital photography.
Second, the availability of this massive data was unlocked by a fundamental shift in model architecture. While early generative models struggled with high-resolution synthesis, Latent Diffusion Models (LDMs) represented a breakthrough in efficiency and fidelity. The specific model analyzed in this study, Stable Diffusion v1.5, is based on the LDM architecture proposed by Rombach et al. (2022).[5] Unlike pixel-based diffusion, which is computationally expensive, LDMs operate in a compressed latent space. The process involves two main components: first, an autoencoder compresses the input image into a lower-dimensional latent representation, reducing the computational complexity while preserving semantic details; second, in the latent space, the model is trained to reverse a gradual noising process. Gaussian noise is iteratively added to the latent vector (Forward Diffusion), and a neural network (typically a time-conditional U-Net) is trained to predict and remove this noise (Reverse Diffusion) to reconstruct the clean latent vector. The U-Net uses cross-attention layers to condition the generation on text prompts (for example, "a car on a dark road"). Once the denoising is complete, the decoder reconstructs the final pixel-space image. Because the model reconstructs the image based on learned statistical patterns rather than physical light transport, it risks hallucinating textures that look plausible to the human eye but lack physical integrity.
Past Work in Image Forensics
The field of image forensics has a long history of detecting manipulated imagery, traditionally focusing on manual edits (photoshop) and, more recently, deepfakes. While effective in their respective domains, these methods rely on assumptions that may not hold for modern generative models:
Geometric Forensics: Pioneering work by Hany Farid and colleagues focused on physical inconsistencies in manually manipulated images. Their methods analyzed geometric constraints, such as the consistency of cast shadows and lighting directions.[8] For example, if a person was inserted into a photo, the shadow they cast often failed to converge on the same light source as other objects in the scene. While effective for cheap fakes, these geometric checks are often semantic and difficult to automate at scale for subtle synthetic generations.
Frequency and Pixel Statistics: With the rise of Generative Adversarial Networks (GANs), researchers pivoted to detecting invisible artifacts. Frequency analysis revealed that GANs often leave unnatural high-frequency patterns caused by upsampling layers in the Fourier domain.[9] More recently, methods like Diffusion Reconstruction Error (DIRE) attempt to detect diffusion-generated images by checking if an image can be easily reconstructed by a pre-trained diffusion model, considering that real images tend to have higher reconstruction errors than synthetic ones.[10]
The Radiometric Gap: While existing methods detect semantic errors (shadows) or digital artifacts (frequency peaks), they rarely interrogate the radiometric formation of the image. Real photographs are created from specific physical processes: photons hitting a sensor, shot noise governed by Poisson statistics, and diffraction limited by apertures. Generative AI, however, is born from denoising a latent vector. This project fills the gap by proposing radiometric forensics: detecting AI not by what it looks like, but by how physically plausible its light statistics are compared to a camera simulation.
Image Generation Methodology
To isolate the differences between physical radiance, image sensing, and generative AI, a dataset of 243 unique night-driving scenes was constructed into three aligned sets: Physical Ground Truth (Set A), Camera Simulation (Set B), and Generative AI (Set C).
Set A: Physical Radiance (Ground Truth)
The foundation of the dataset is a collection of high-dynamic-range (HDR) spectral radiance maps derived from the ISETHDR driving scene database.[3] Unlike standard RGB images, these scenes contain the full spectral energy distribution of light at every pixel, modeled via physically based ray tracing (PBRT). Nighttime driving scenes were specifically selected because they present the most difficult challenge for imaging systems: a high dynamic range where bright light sources (headlights, streetlights) can be five orders of magnitude more intense than the surrounding dark regions. This extreme contrast creates optical flare that can obscure vulnerable road users, such as pedestrians or cyclists, making accurate simulation critical for safety validation.[11] For each of the 243 scenes, four independent lighting components were extracted: headlights, streetlights, other environmental lights, and skylight. To create a realistic night-driving environment, these components were combined using a specific weighting vector , prioritizing vehicle headlights while maintaining low-light ambient conditions. This resulting spectral radiance map serves as the physical ground truth, representing the raw photons arriving at the camera lens before any optical or sensor degradation.
Set B: Camera Simulation (The Baseline)
To generate photorealistic baselines, the spectral radiance from Set A was passed through a virtual camera model using ISETCam [4]. This stage introduces the physical imperfections inherent to optical imaging systems.[11] The specifications of the camera model are described below:
- Optics: A 4mm focal length lens with a wide aperture was simulated. Crucially, the aperture was modeled with a triangular shape (3 sides) rather than a perfect circle, following the flare simulation methods validated in previous work.[11] This introduces a distinct 6-point starburst diffraction pattern on bright sources (like headlights), serving as a known optical fingerprint. Additionally, lens scattering was modeled by introducing randomized dust and scratches into the aperture function, as previously demonstrated for accurate nighttime flare simulation.[11]
- Sensor: The optical image was captured by a simulated Bayer RGGB sensor with a pixel size of , matching the specifications of commercial sensors analyzed in previous studies.[11]
- Processing: The raw sensor data underwent a standard Image Signal Processor (ISP) pipeline, including demosaicing and global tone mapping to mimic the non-linear response of consumer photography.
Set C: Generative AI (The Digital Twin)
The final dataset consists of synthetic reconstructions of Set B generated by Stable Diffusion v1.5.[5] An Image-to-Image (Img2Img) pipeline was used to ensure the AI respected the broad geometry of the original scene, filling in fine-level texture and lighting details without hallucinating completely new objects or structural deviations. The model was conditioned with the text prompt: "photorealistic night driving scene, city street, car headlights, streetlights, highly detailed, 8k, cinematic lighting" and a negative prompt to suppress artifacts: "blur, noise, grain, cartoon". A denoising strength of 0.5 was used. This critical parameter allows the model to modify approximately 50% of the pixel statistics while retaining the structural layout of the cars and roads. The generation was performed at resolution with a guidance scale of 7.5, creating an image that appears similar to the camera photo but is radiometrically synthetic.
The resulting dataset provides 243 aligned triplets. By subtracting the AI generation (Set C) from the camera simulation (Set B), some initial visual differences can be observed. As shown in Figure 7, differences are most pronounced around high-intensity light sources, where the AI fails to replicate the complex diffraction spikes created by the triangular aperture, defaulting instead to generic Gaussian blooms. Subtle statistical differences will be revealed in the following section.