Divide-Conquer-and-Merge: Memory- and Time-Efficient Holographic Displays

Abstract

Recently, deep learning-based computer-generated holography (CGH) has demonstrated tremendous potential in three-dimensional (3D) displays and yielded impressive display quality. However, most existing deep learning-based CGH techniques can only generate holograms of 1080p resolution, which is far from the ultra-high resolution (16K+) required for practical virtual reality (VR) and augmented reality (AR) applications to support a wide field of view and large eye box. One of the major obstacles in current CGH frameworks lies in the limited memory available on consumer-grade GPUs which could not facilitate the generation of higher-definition holograms. To overcome the aforementioned challenge, we proposed a divide-conquer-and-merge strategy to address the memory and computational capacity scarcity in ultra-high-definition CGH generation. This algorithm empowers existing CGH frameworks to synthesize higher-definition holograms at a faster speed while maintaining high-fidelity image display quality. Both simulations and experiments were conducted to demonstrate the capabilities of the proposed framework. By integrating our strategy into HoloNet and CCNNs, we achieved significant reductions in GPU memory usage during the training period by 64.3% and 12.9%, respectively. Furthermore, we observed substantial speed improvements in hologram generation, with an acceleration of up to 3× and 2 ×, respectively. Particularly, we successfully trained and inferred 8K definition holograms on an NVIDIA GeForce RTX 3090 GPU for the first time in simulations. Furthermore, we conducted full-color optical experiments to verify the effectiveness of our method. We believe our strategy can provide a novel approach for memory- and time-efficient holographic displays.

Framework

Here, we proposed a novel CGH generation framework to synthesize highquality and ultra-high-resolution holograms. The framework consists of two main components, the phase generator and the phase encoder. For the phase generator, we employ a dividing strategy where the image is initially divided into r² sub-images with reduced resolution. These lower-definition sub-images are then fed into the phase generator network to predict the corresponding phases at the target plane. The phase and amplitude are upsampled and merged to synthesize a complex-valued wave field, which is subsequently propagated to the SLM plane using the angular spectrum method (ASM). For the phase encoder, similar to the operations of the phase generator, we continue to apply the divide-and-conquer strategy to generate a phase-only hologram. Finally, the phase-only hologram is propagated back to the target plane, and the parameters of the networks are updated by calculating the loss between the ground truth (GT) and the reconstructed image.

Live Demo

BibTeX

@INPROCEEDINGS{10494141,
  author={Dong, Zhenxing and Jia, Jidong and Li, Yan and Ling, Yuye},
  booktitle={2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR)}, 
  title={Divide-Conquer-and-Merge: Memory- and Time-Efficient Holographic Displays}, 
  year={2024},
  volume={},
  number={},
  pages={493-501},
  keywords={Training;Image quality;Solid modeling;Three-dimensional displays;Computational modeling;Neural networks;Graphics processing units;Holography;User interfaces;Optical imaging;VR/AR;Holographic Displays;Computer-generated Hologram},
  doi={10.1109/VR58804.2024.00070}}

Acknowledgement

This website is adapted from Nerfies, licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.