AR with a Purpose
As part of DIDYMOS-XR, the team at the Vision and Robotics Lab at AUB developed what we call Human-in-the-loop mapping. This is a technology which enables a user wearing an AR headset to visualize a 3D map of an environment superimposed over reality in real time.
The benefit of a system like that is to improve the generation, maintenance, and update of digital twins. It allows the user to not only visualize the 3D map but also edit and annotate it. The user can add and delete map sections, as well as adding points of interest to the map.
Traditionally, this process is done offline and sometimes offsite, which complicates the corrections and makes the process time-consuming. Our research has shown that real-time edits using AR on-site significantly contribute to reducing time and effort.
This technology was recently tested at the idealworks warehouse in Munich. The plan is to integrate this system into the idealworks onboarding process to allow them faster and more convenient mapping of factories where their robots are deployed.
This functionality mitigates the limitations existing in mapping sensors and reduces the cost and time needed to generate accurate digital twins. This work has been recently submitted for review at one of the top robotics conferences (IEEE International Conference on Robotics and Automation 2024).
Currently, we working on advancing the technology by supporting maps’ mergers from numerous sources (robots, handheld, etc.…). In addition, we are working on optimizing the system performance by compressing the data transferred in order to reduce delays and bandwidth requirements.
Such a technology has broad applications to different domains. For example, the construction field can rely on technologies like this for mapping a construction site, identifying and localizing assets, and making corrections to as-built maps. Another potential use case from the DIDYMOS-XR project is applying the technology in the city on a large scale. Imagine a municipality worker who is able to scan and map the town as they go about their daily work while being able to locate assets, and points of interest, and identify tasks to be carried out at specific locations. All of which can now be done on-site and in real-time.
Clearly, every technology brings with it challenges and risks. As part of DIDYMOS-XR project, we are looking at ethical considerations of such technologies and possible negative impacts. Technically, there is the concern of privacy of data and the privacy of individuals in a particular scene being scanned. In addition, there is the issue of user fatigue from prolonged use of a head mount display device. Part of the innovation in DIDYMOS-XR is to establish a balance between the benefits of digital twins and the risks associated with them. Stay tuned for more on this in future blog posts.
By Imad H. Elhajj
