基於深度強化式學習之多目標人群導航機器人系統

Abstract

自主移動機器人（AMR）由於其多功能性，已成功引起了人們的關注，目前已廣泛應用於自動化工廠和人與機器人之共存環境，如機場和購物中心等。為了使機器人能夠在人群環境中進行導航，機器人必須具有社交意識並能夠預測行人的移動。然而，以往的方法，機器人都需要先預測行人未來軌跡，再規劃安全路徑，常會受到行人移動之高度隨機性的影響，導致計算成本增加和機器人凍結的問題。隨著深度學習的發展，許多與導航有關的研究都基於深度強化式學習，使機器人可以通過與環境的互動找到最佳策略。社交關注強化式學習（SARL）是一最先進的(state-of-the-art)方法，能夠提升機器人在人群環境中的導航能力。儘管SARL成功改善了人群環境下的導航效能，但它仍然存在幾項缺點。因此，本研究提出了一種基於深度強化學習的多目標人群導航機器人系統，藉由所提出之獎勵函數以實現多個導航目標，包括安全性、時間效率、避免碰撞和路徑平滑度等。為了解決人群環境中的導航延遲，我們也開發了一多目標雙重選擇注意力模組(MODSRL)，使機器人能夠做出更有效的決策，同時減少在導航初始階段的徘迴問題。實驗結果表示，所提出的MODSRL方法在五個不同的指標上優於現有的研究，展現了在複雜人群環境中導航的強健性。Autonomous mobile robots (AMRs) are gaining attention due to their versatile capabilities, making them suitable for various applications in automated factories and human-robot coexistence environments such as airports and shopping malls. To enable robots to navigate in crowded environments, robots must be socially aware and able to predict the movements of pedestrians. However, previous methods, such as predicting future trajectories of pedestrians and then planning a safe path, encountered challenges due to the high randomness of pedestrian movements, resulting in increased computational costs and the problem of robot freezing. With the development of deep learning, many navigation-related studies have been based on deep reinforcement learning, allowing machines to find the optimal strategy through interaction with the environment during navigation. Socially Attentive Reinforcement Learning (SARL) is a promising method for enhancing navigation in crowded environments. While SARL has been successful in improving navigation performance in crowded environments, it still has several shortcomings. This study proposes a deep reinforcement learning-based multi-objective crowd-aware robot navigation system. The proposed method uses a set of reward functions to reach multiple objectives, including safety, time efficiency, collision avoidance, and path smoothness during navigation. To address the challenge of hesitation in crowd environments, we develop a Multi-Objective Dual-Selection Attention Module called MODSRL, which enables the robot to make efficient decisions while reducing hesitation. Experimental results demonstrate that the proposed MODSRL method outperforms existing research studies in five different metrics, showcasing its robustness in navigating complex crowd environments.