Motion Hologram: Jointly optimized hologram generation and motion planning for photorealistic 3D displays via reinforcement learning

Abstract

Holography is capable of rendering three-dimensional scenes with full-depth control, and delivering transformative experiences across numerous domains, including virtual and augmented reality, education, and communication. However, traditional holography presents 3D scenes with unnatural defocus and severe speckles due to the limited space bandwidth product of the spatial light modulator (SLM). Here, we introduce Motion Hologram, a novel holographic technique to accurately portray photorealistic and speckle-free 3D scenes, by leveraging a single hologram and learnable motion trajectory, which are jointly optimized within the deep reinforcement learning framework. Specifically, we experimentally demonstrated the proposed technique could achieve a 4~5 dB PSNR improvement of focal stacks in comparison with traditional holography and could successfully depict speckle-free, high-fidelity, and full-color 3D displays using only a commercial SLM for the first time. We believe the proposed method promises a new form of holographic displays that will offer immersive viewing experiences for audiences.

Framework

Here, we proposed a novel CGH technique called Motion Hologram to disrupt the spatial coherence of light source using reinforcement learning, which can thus render photorealistic and speckle-free 3D scenes by globally designing the holographic system. We reappraised the role of system motion, which was commonly believed detrimental in imaging and display, and conceptualized that the inherent speckles in holography could be neutralized and photorealistic 3D display could be achieved by spatially multiplexing the holograms via proper system motions. Specifically, a novel system design paradigm that leverages reinforcement learning to jointly generate the phase-only hologram and system’s motion trajectory were introduced, which promises an unparalleled viewing experience.

Experimental Case1: Continuous Camera Zooming

Experimental Case2: High-resolution Dynamic Contents

BibTeX

@misc{dong2024motionhologramjointlyoptimized,
      title={Motion Hologram: Jointly optimized hologram generation and motion planning for photorealistic and speckle-free 3D displays via reinforcement learning}, 
      author={Zhenxing Dong and Yuye Ling and Yan Li and Yikai Su},
      year={2024},
      eprint={2401.12537},
      archivePrefix={arXiv},
      primaryClass={physics.optics},
      url={https://arxiv.org/abs/2401.12537}, 
}

Acknowledgement

This website is adapted from Nerfies, licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.