The first year of the Didymos-XR project has yielded some exciting results followed by new ideas and work for 2024!

The Spanish partners in DIDYMOS-XR, I2CAT and Neàpolis met at FICOSA’s facilities to review the first year’s results and future work. The visit included a tour of FICOSA’s installations, labs and the testing site (Fig 1).

Figure 1: Meeting in FICOSA’s demo lab. FICOSA’s research team welcomed the teams of Neàpolis and I2CAT in January 2024. 

FICOSA is an industrial group dedicated to the research, development, production and commercialization of systems and components for the automotive sector. Located in Barcelona, Spain, the group has a global team of more than 9,000 professionals and is represented in 19 countries around the world. Neàpolis is the Public Innovation Agency of the city of Vilanova i la Geltrú, near Barcelona. Their project lead, Mar Sibila, coordinates the implementation of disruptive technologies such as Digital Twins to improve the city’s management. I2CAT’s researchers, Dr Gianluca Cernigliaro and Dr Ivan Huerta, are responsible for the tasks of point-cloud compression and volumetric video processing applied to the creation of digital twins.

The DIDYMOS-XR project has allowed FICOSA to enter the new technology domain of Digital Twins. The company’s research team is responsible for vehicle sensor fusion and data capture and FICOSA’s vehicle onboard sensors record images of the vehicle’s surroundings, which are used to train and validate computer vision algorithms developed by other partners in the consortium. In the second year of the project, FICOSA will lead the City Inventory use case, which will be tested in Vilanova i la Geltrú.

For the generation of a cityscape digital twin, FICOSA’s team has equipped the company’s demo vehicle with a state-of-the-art camera-based SVS, a LiDAR and a dGPS. Additionally, significant effort from the FICOSA data engineering and tooling team has been done to develop a data infrastructure necessary for the implementation of multiple data pipelines, which process the data generated by the vehicle’s sensors. The integration of these sensors is combined with the state-of-the-art calibration and data synchronization techniques to generate 3D information of several city’s urban areas.

Data capturing was done over different seasons, weather and lighting conditions. The generated datasets are used for training and validating the algorithms developed by the project partners to obtain 3D reconstruction of the urban areas and update the city’s digital twin. The initial results of the 3D reconstruction are shown in Figure 2.

 

Figure 3: First reconstruction images from Vilanova i la Geltrú commuting area. Early fusion results showing an RGB pointcloud created using Ficosa’s camera-based Surround View System (SVS) and LiDAR sensor.

From the ADAS (Advanced Driver-Assistance System) point of view, the city’s digital twin will be a sophisticated tool that provides the vehicle with complementary information about the city’s traffic, or its surroundings that are not visible to its onboard sensors. This will include events such as the state of the road, traffic, parking locations and so on.

DIDYMOS-XR is an ambitious multidisciplinary project that connects teams from Europe and the Middle East, and which allows FICOSA and the rest of the partners to learn about each other’s technologies. Special attention is given to data privacy and fulfilling GDPR requirements. For this reason, FICOSA’s vehicle was upgraded according to the Ethical Board recommendations, as shown in Figure 3.

Figure 3: FICOSA’s demo car. Car setup with the onboard sensor and Ethical Board recommendations.

Following the first results, the plan for 2024 is to deploy a static camera in the city and synchronise it with FICOSA’s onboard vehicle sensors and improve data pipelines for faster and more efficient updates of the digital twin, which is one of the main objectives of DIDYMOS-XR .

As artificial Intelligence is revolutionizing the world of image processing and visual content, both in 2D and 3D, the DIDYMOS-XR project will build on these advancements and use them as an enabling technology for digitalization of the real world.

The next DIDYMOS-XR consortium meeting will be hosted by Neàpolis in Vilanova i la Geltrú in April 2024 and will include a visit to FICOSA’s facilities where we will be able to present new developments.

So, stay tuned!

By Isaac Agustí, Sara Chamadoira and Aleksandar Jevtic

 For further information and news from FICOSA follow them on LinkedIn, YouTube, X, and Facebook!