HARDWARE-IN-THE-LOOP CO-SIMULATION FOR SEAMLESS INTEGRATION OF PHYSICAL AND VIRTUAL ROBOTS IN 3D SCENES

Abstract

Three-dimensional (3D) robot simulators are widely used to prototype and test perception and control algorithms before deploying them on real platforms, but a persistent reality gap remains due to unmodelled dynamics, sensor noise and communication delays. Hardware-in-the-Loop (HIL) simulation helps to close this gap by inserting real hardware into the virtual loop. This paper presents a modular HIL co-simulation framework for integrating physical and virtual robots in 3D scenes. Real robot components are coupled to a high-fidelity simulator via a bidirectional interface that supports both sensor and actuator exchange and is independent of specific robot platforms or simulators. We outline the architecture, synchronisation mechanisms and implementation, and evaluate three execution modes: pure simulation, pure physical execution and HIL co-simulation. Results indicate that HIL co-simulation improves trajectory tracking and robustness to disturbances over pure simulation while maintaining safety and flexibility, offering a practical basis for robotic digital twins and mixed-reality applications.