使用深度強化學習進行人形機器人手臂的球體平衡控制

Abstract

noneThis study investigates the application of deep reinforcement learning (DRL) for ball balancing on a tray using our adult-size humanoid robot(modified THORMANG3’s upper body). This task demands high-precision control, real-time adaptability, and robustness, which can provide advanced control capabilities for humanoid robots. To solve this problem, first, we develop a Dual-Actor Proximal Policy Optimization (DA-PPO) algorithm within the NVIDIA Isaac Gym simulation framework and demonstrate its superior performance in simulation through improved learning stability against standard algorithms like Proximal Policy Optimization (PPO). Afterward, we address the challenge of the simulation-to-reality (sim-to-real) task of balancing a ball on a tray. In this regard, we apply the PPO policy in NVIDIA Isaac Gym and integrate it with a vision-based observation pipeline and low-pass filtering to ensure stable control. To achieve successful results, we employ domain randomization and actuator parameter identification to bridge the sim-to-real gap. Experimental results show that the proposed framework has outstanding performance in simulation. On the other hand, we achieve robust ball balancing in the physical world, with success rates of 76% in 50 experiments with long-duration stability sustained up to 96.9 seconds under platform noise and actuation uncertainties. These findings highlight the potential of DRL in enabling humanoid robots to perform complex manipulation tasks reliably.