
Bridging the Sim-to-Real Gap in Collective Robotic Construction - A Mixed Reality Reinforcement Learning Approach
The transition of collective robotic construction from controlled laboratory settings to real-world applications faces a significant barrier: limited autonomy. Advances in learning-based approaches offer promising potential to allow robots to autonomously discover optimal construction strategies through cost-effective, scalable, and safe simulation training. Nonetheless, simulation environments cannot entirely capture the complexities of real-world conditions, leading to the phenomenon known as the "reality gap." To address these limitations, this research proposes a mixed-reality reinforcement learning workflow for augmenting simulation-based training with real-world observations, action executions, and environment state changes. To this end, a sim-real-sim feedback loop is established between the Unity training environment (ML-Agents Toolkit), the Robot Operating System, and the OptiTrack motion capture system. The methodology is demonstrated through a case study on the assembly of a bending-active structure by a team of Roaming Autonomous Distributed robots. The agents are trained to execute the isolated task of manipulating a rod into a target position, first in a simulation-only environment for 25,000 episodes, followed by 5,000 episodes in a mixed-reality training setup. In this setup, robots operate in the physical world, with certain environmental aspects, such as spatial constraints and objectives, remaining virtual. Real-time tracked and monitored data provide observations that better reflect the real-world states of the robots and the rod. The effectiveness of the mixed-reality training is assessed by comparing its deployment performance to that of parallel simulation-only training with the same total number of episodes. The results obtained from 200 runs indicate that the mixed-reality-trained policy significantly outperforms its counterpart across several key metrics, including task completion time, success rate, and system stability. Notably, there is a substantial reduction in the number of immobilization and rod drop incidents. The findings confirm that the agents exhibit the emergence of error-recovery behaviors when confronted with previously unseen environmental states. Ultimately, the proposed workflow leverages the advantages of simulation training while incorporating essential real-world variability to diminish the reality gap and increase the feasibility of deploying CRC systems in on-site construction tasks.
ITECH M.Sc. Thesis Project 2024: Bridging the Sim-to-Real Gap in Collective Robotic Construction - A Mixed Reality Reinforcement Learning Approach
Niki Kentroti, Rabih Koussa, Che Chen Hu
Thesis Advisers: Lasath Siriwardena, Shermin Sherkat
Thesis Supervisor: Prof. Achim Menges
Second Supervisor: Prof. Thomas Wortmann