Image Optimization for Optical Projector Arrays

Introduction

This project addresses the core challenges of geometric distortion and luminance inconsistency in projection systems, particularly at ultra-short throw ratios. Traditional projection lens arrays must be designed for a limited range of throw distances due to their static lens arrays, leading to issues in maintaining spatial fidelity and consistent brightness across the projection surface as the throw distance changes. To overcome these challenges, we consider a computational framework that improves projection quality without relying on expensive corrective optics. This approach demonstrates significant improvements in image clarity and uniformity. By dynamically adjusting geometric distortions and blending outputs from multiple projectors, we achieve projections with improved spatial fidelity at throw ratios as low as 0.10.

Background

Projection systems face challenges such as glare, geometric distortions, and uneven luminance, especially at ultra-short throw ratios. These issues limit projector placement to specific configurations—such as ceiling-mounted, floor-mounted, or ultra-short throw setups—while requiring precise alignment and costly optical lens arrays to correct distortions. Misalignment can lead to geometric warping, reduced coverage, diminished clarity, glare, and inconsistent luminance, making deployment in dynamic or unconventional scenarios particularly difficult.

Common projector orientations. Glare lines illustrate undesirable projection angles where reflected light reaches the audience's eyes.

To address these challenges, we can utilize a computational framework comprising two key methods:

• Perspective Alignment: Dynamically adjusts projector input to ensure consistent projection in suboptimal configurations.
• Multi-Projector Blending: Utilizes overlapping projections from multiple projectors to eliminate blind spots and improve luminance uniformity.

Methods

Projection quality is heavily influenced by the throw ratio, defined as the distance between the projector and the screen divided by the height of the projection screen. For this project, we analyzed throw ratios on a logarithmic scale ranging from 0.1 to 100. Testing was conducted using a base test image with a height of 180 pixels and a width of 270 pixels, upscaled by a factor of 2. The upscaling allows us to simulate the effects of decreasing throw distances while maintaining visible distortion patterns. Although higher upscaling ratios and resolutions could yield more precise results, they significantly increase computational costs. The chosen parameters strike a balance between computational efficiency and the accuracy required for this project. All image processing was conducted using a custom Python library built specifically for this work, providing lossless geometric transformations and total control over the image alignment process. The code is available in the appendix.

Projector spacing visualized at different throw ratios.

The test image used in experiments to evaluate projection quality.

Perspective Alignment

Perspective Alignment dynamically adjusts the input image to correct geometric distortions caused by suboptimal projector placement or low throw ratios. By using back-projection, this method computes the geometric distortions introduced by the projector’s position and orientation and applies transformations to the input image to counteract them. The result is a projection where the proportions and scaling of objects are preserved, even at extremely low throw ratios such as τ = 0.10, where spatial distortion is most severe.

While Perspective Alignment ensures that the relative size and shape of objects on the projection surface are maintained, it introduces some trade-offs. Portions of the original image that fall outside the accessible projection area at very short throw distances must be cropped. Additionally, luminance tends to decrease in areas of the display farther from the projector, and the projected pixel size increases in these regions, leading to noticeable pixelation. These limitations will be addressed with the introduction of additional projectors.

Overview of Perspective Alignment and its role in correcting geometric distortions.

Examples of projection quality with and without Perspective Alignment at various throw ratios.

Multi-Projector Blending

Multi-Projector Blending addresses luminance inconsistencies and blind spots inherent to single-projector setups by combining overlapping projection areas from multiple projectors. This technique leverages the aligned perspectives of projectors placed at different positions around the display, ensuring that regions inaccessible to some projectors are covered by others. The result is a more uniform and evenly lit projection, even at extreme throw ratios.

By blending the projections, the output image more closely matches the target image, with enhanced brightness and reduced gaps in coverage. However, some artifacts can still be observed, particularly at the edges of individual projections, and some vignetting occurs in regions covered by fewer projectors. These issues can be mitigated by increasing the simulation resolution to reduce aliasing and by adding additional projectors to improve luminance consistency. Despite these minor limitations, Multi-Projector Blending significantly enhances projection quality, especially in large-scale or ultra-short throw setups where single-projector solutions are insufficient without corrective lenses. Future refinements to the computational blending process, including improved algorithms, could further reduce these artifacts and optimize performance.

Overview of Multi-Projector Blending for enhanced coverage.

Examples of Multi-Projector Blending in action.

Results

The combination of Perspective Alignment and Multi-Projector Blending resulted in:

• Reduced spatial warping at ultra-short throw ratios.
• Improved luminance uniformity across the projection surface.
• Enhanced image quality even without advanced corrective optics.

To evaluate the effectiveness of these methods, ten different test images were tested at each of the ten selected throw ratios, ranging from 0.1 to 100. The average Delta E and SSIM were calculated for each image, and these values were then averaged across all ten images to produce the data for the plots below.

File:Perspective Alignment Delta E.png

Delta E analysis for Perspective Alignment.

Delta E analysis for Multi-Projector Blending.

Performance statistics for Normal Input, Perspective Alignment, and Multi-Projector Blending.

From the Delta E analysis, we observe that Perspective Alignment significantly reduced the average Delta E across the projection surface at low throw ratios. However, at very short throw distances, large portions of the display are excluded and regions far from the projector still exhibited notable Delta E values due to reduced luminance and geometric distortions.

By incorporating Multi-Projector Blending, some of this Delta E was recovered, leading to further improvements in luminance uniformity. Despite those improvements, overlapping projections introduced slight blurring, which reduced sharpness and contributed to increased Delta E in certain regions. Overall, Perspective Alignment and Multi-Projector Blending were most effective at reducing Delta E for throw ratios less than approximately 4, where spatial distortions and luminance inconsistencies are most severe.

Additionally, by examining the graphs of SSIM over throw ratios, we see that spatial error introduced by varying throw ratios loosely correlates with chromatic error. This relationship arises because as the throw ratio decreases, the projected pixel size and shape deviate from their ideal configuration. Spatial and chromatic information are condensed, as the average color value of the pixels covered by each projected pixel is applied to the entire covered area.

Conclusions

This project demonstrates a improvement in the spatial fidelity and luminance uniformity of projection systems in ultra-short throw environments through the combination of two computational methods: Perspective Alignment and Multi-Projector Blending.

• Perspective Alignment effectively mitigates geometric distortions by dynamically adjusting the input image to correct for suboptimal projector placement and orientation.
• Multi-Projector Blending enhances luminance consistency by overlapping projection areas from multiple projectors. This method eliminates blind spots and improves overall brightness, although overlapping regions introduce slight blurring that can increase Delta E in certain areas.

Together, these techniques are most effective at throw ratios less than approximately 4, significantly reducing spatial and chromatic distortions while enabling high-quality image projection without expensive corrective optics. Performance metrics such as Delta E and SSIM demonstrate that this computational approach significantly reduces spatial and chromatic distortions, particularly at extreme throw ratios. However, further refinement of the simulation is necessary to explore the full extent to which these methods can be optimized for distortion reduction.

Limitations

While the computational methods developed here show great promise, certain limitations remain:

• Minor aliasing artifacts are introduced due to the limited resolution of the simulation.
• Blurring in overlapping regions during Multi-Projector Blending contributes to slight reductions in sharpness.
• Computational cost increases with higher resolutions and the addition of more projectors, limiting scalability in real-time applications.

Future Work

Real-world testing and hardware development are ongoing to further refine these methods and validate their effectiveness. A 3D-printed projector array has been designed as part of this effort, and Perspective Alignment has been successfully tested in a real-world setting, but those results are beyond the scope of this project. Future work will focus on:

• Verifying results with real-world testing:

 - Building a multi-projector testbed to validate trends observed in simulations.
 - Designing custom projection hardware to accelerate image computation and improve real-time applicability.

• Optimizing individual projector inputs with machine learning:

 - Implementing convolutional neural networks (CNNs) to refine the accuracy of individual input adjustments.
 - Developing GAN or diffusion models to compute individual node inputs more efficiently.
 - Incorporating SSIM and S-CIELAB into the loss function for more perceptually accurate corrections.

3D-printed mini projector used for testing and hardware development.

Real-world test setup demonstrating Perspective Alignment in action.

This work provides a foundation for improving projection quality in constrained or unconventional setups. Further developments in real-world testing and algorithm refinement will determine the practical applicability of these techniques to scenarios such as large-scale displays and portable projection systems.

Appendix

• GitHub Repository