Can Lensless Cameras Redefine Depth of Field in Photography?

Contributed by Dr. Jose Reinaldo Cunha Santos A V Silva Neto, based on the IEEEXplore® article, “Extended Depth-of-Field Lensless Imaging Using an Optimized Radial Mask”, published in the IEEE Transactions on Computational Imaging in September 2023, and the SPS Webinar of the same name, available on the SPS Resource Center.

Introduction

Lensless cameras have garnered growing interest in recent literature due to their compact design, which replaces bulky lenses with thin coded masks like diffusers. They are cost-effective, support compressive sensing techniques, and offer the flexibility to modify or optimize coded masks with fewer constraints than traditional lenses. Unlike traditional cameras, lensless imaging requires post-processing to produce visually informative captures. Reconstruction algorithms rely on knowledge of the light pattern formed by the coded mask, known as the point spread function (PSF), to generate high-quality images.

A defining feature of lensless cameras is their fundamentally different depth of field compared to traditional lens-based cameras. In these systems, the point spread function and its depth dependence constrain the reconstruction process, thereby limiting the depth range at which objects can be effectively observed. However, this unique characteristic also enables the potential to extend the depth of field by designing a coded mask with a depth-independent pattern. We show that the radial code, with its scaling-invariance property, is an ideal candidate for this purpose.

Our goal is to leverage the design flexibility of lensless cameras and their unique depth of field characteristics to create cameras with inherently extended depth of field.

Proposed Method

Any radial pattern can theoretically extend the depth of field (DOF) of a lensless camera. However, poorly designed radial masks may result in inefficient optical transfer functions, reducing measurement quality and degrading reconstructed images. One solution is to perform the optimization of the coded pattern. Previous works modeled coded masks as 2D matrices with each element representing the light transmittance at a specific area of the mask plane, and assumed independence between mask elements during optimization which generated random-like patterns [2].

To address this, we propose a radial-shape-constrained optimization procedure for the coded mask. Our method divides the mask area into N predefined radial structures, constraining all elements within each structure to share the same light transmittance value. This reduces the 2D mask representation to a 1D vector, allowing optimization that not only improves the transfer function but also enforces a radial pattern. This design ensures the mask retains its extended-DOF properties while enhancing reconstruction quality.

Prototype Camera Experiment

To validate our claims, we demonstrate the optimized radial mask’s superior depth of field (DOF) and imaging quality through simulations and through experiments using a prototype lensless camera, providing both quantitative and qualitative evidence of its advantages. An extensive explanation of all the experiments can be found in our published manuscript [1].

Our prototype camera consists of an axial stack of a CMOS image sensor and a transmissive spatial light modulator. The modulator allows us to easily switch between coded mask patterns, enabling direct comparisons of their DOF performance. We evaluate our optimized radial mask against multiple baselines, including hand-crafted radial patterns [3], randomly generated radial patterns, non-radial Fresnel zone aperture (FZA) masks, and non-radial random patterns.

The results demonstrate that our proposed mask outperforms the baselines, achieving both extended DOF and superior image quality.

Conclusion

Our experiments demonstrate that radial patterns can effectively extend the depth of field (DOF) of lensless cameras. Specifically, we show through simulations and prototype captures that our optimized radial mask outperforms hand-crafted radial masks in reconstruction quality and achieves a larger DOF compared to non-radial masks.

For a detailed explanation of the optimization procedure and radial mask parameterization, as well as extensive experimental results with continuous scenes, discrete depth scenes, and transparent objects, please refer to our original manuscript [1].

References:

[1] J. R. C. S. A. V. Silva Neto, T. Nakamura, Y. Makihara and Y. Yagi, "Extended Depth-of-Field Lensless Imaging Using an Optimized Radial Mask," in IEEE Transactions on Computational Imaging, vol. 9, pp. 857-868, 2023, doi: https://dx.doi.org/10.1109/TCI.2023.3318992.

[2] R. Horisaki, Y. Okamoto, and J. Tanida, “Deeply coded aperture for lensless imaging,” Opt. Lett., vol. 45, no. 11, pp. 3131--3134, 2020, doi: https://doi.org/10.1364/OL.390810.

[3] T. Nakamura, S. Igarashi, S. Torashima, and M. Yamaguchi, "Extended depth-of-field lensless camera using a radial amplitude mask." in Imaging and Applied Optics Congress, OSA Technical Digest, Optica Publishing Group, 2020, doi: https://doi.org/10.1364/COSI.2020.CW3B.2.

[4] K. Monakhova, V. Tran, G. Kuo, and L. Waller, "Untrained networks for compressive lensless photography," Opt. Express 29, pp. 20913-20929, 2021, doi: https://doi.org/10.1364/OE.424075.