The Future VR/AR Network - Towards Virtual Human/Object Teleportation

Virtual and augmented reality (VR/AR) hold tremendous potential to advance our society and are commonly seen as the 4^th major disruptive technology wave after PC, the Internet/Web, and mobile. Together with another pair of emerging technologies, 360° video and holographic video, they can suspend our disbelief of being at a remote location or having remote objects/people present in our immediate surrounding, akin to virtual human/object teleportation. Presently limited to offline operation and synthetic content and targeting gaming and entertainment, VR/AR are expected to reach their potential when deployed online and with real remote scene content, enabling novel applications in disaster relief and public safety, the environmental sciences, transportation, and quality of life (see Figure 1). Networked VR/AR applications will play a major role in the envisioned/emerging global Internet of Things framework and are expected to represent the foundation of the anticipated 5G tactile Internet ecosystem. However, there are considerable challenges ahead, in the form of technology limitations and infrastructure costs.
 

Figure 1. Remote 360° VR/AR immersion for diverse societal applications.

The raw data rates (5-60 Gbps) required to enable an online immersion experience indistinguishable from real life dramatically exceed the FCC requirements for future broadband networks. Thus, simply introducing more bandwidth (business as usual) will not bridge this gap as the scales demand vs. supply are very different. This necessitates exploring holistic solutions that go beyond the traditional networking domain and integrate the capture, coding, networking, and user navigation of VR/AR data. Moreover, emerging services, e.g., YouTube/Facebook 360 are extremely inefficient in bandwidth utilization and data management, thereby considerably degrading the user experience, due to their heuristic design choices. Finally, further critical aspects such as wireless operation, ultra-low latency, system scalability, edge computing, device and application security, and end-to-end reliability are yet to be considered.

The National Science Foundation (NSF) recently sponsored The Future VR/AR Network -- Towards Virtual Human/Object Teleportation, the first Visioning Workshop on Networked Virtual and Augmented Reality Communications, held in Washington, DC on April 23-24, 2018. The workshop was organized by Jacob Chakareski from the University of Alabama. The objectives of the workshop were to gather a forum of experts and interdisciplinary practitioners to help identify the most promising horizons to explore to address the above challenges and contemplate a community agenda that will integrate emerging threads of related work into collaboration.

By setting a rigorous long-term vision platform of science and technology, the workshop has helped chart the field technically. In turn, this will help accelerate the introduction and broad adoption of related societal applications. The workshop advances will also facilitate fundamental research in the general application area of high-volume high-speed/low-latency data transfer in emerging settings, where for the first time the temporal and spatial dimensions of the data capture need to be closely explored and tightly integrated with the user navigation actions, to maintain the desired quality of experience for the end user, given the limited available system resources.

The workshop comprised a broad set of participants, as several disciplines and closer collaboration of researchers of diverse backgrounds needed to be involved to realize its objectives. The workshop program comprised a mix of breakout sessions, panel discussions, and short talks by leading early investigators, scheduled over two days. The program included on the first day a panel discussion featuring industry leaders in the field and interdisciplinary practitioners, and an expert panel on the second day to identify the grand challenges in networked VR/AR over the next five years. The workshop participants were invited or selected from five different categories: Networks, Theory, Systems, Industry, and Applications, where the first three correspond to the respective NSF programs that supported the event: NeTS, CIF, and CSR.

There was strong agreement among the workshop participants that overcoming the present challenges and technology limitations is essential for enabling the next generation societal VR/AR applications. Similarly, there was uniform agreement that this will necessitate departing from traditional networking approaches that solely aim to increase data rates or lower transmission delays, as the performance gap between networked VR/AR requirements and present/upcoming conventional networking technologies is only expected to increase. Instead, holistic solutions should be investigated that go beyond the traditional networking domain to closely integrate capture, coding, networking, and user navigation of VR/AR data, while prospectively leveraging emerging/non-legacy networking technologies such as millimeter wave, free-space optics, and edge computing.

Finally, there was strong consensus at the workshop that it is essential to invest now in research that aims to meet this objective. Moreover, steady investment should be made in developing publicly available benchmark datasets, source code, evaluation settings, and testbeds, to help accelerate such research and drive reproducibility and standardization, at the same time.

More information about the workshop can be found on its website. A detailed report outlining the identified challenges of present networked VR/AR technologies and research horizons to explore towards enabling the envisioned next generation societal VR/AR applications has been compiled and included on the website.