基於深度強化學習之自動充電系統電腦視覺與機械手臂控制系統發展

Abstract

隨著科技與電動車技術的蓬勃發展，近年來電動車獲得更多關注，建設更為便利的充電設施已然是電動車主們一大需求。因此本論文提出一主僕式架構之自動充電系統，提供自動充電服務，整體架構由自主移動式充電機器人與電源拖車組成，並搭配本研究設計之充電行為流程、充電插座姿態辨識與使用深度強化學習(Deep Reinforcement Learning, RL)於機械手臂運動控制，最終實現可提供充電服務之自動充電系統。本論文於充電插座自動辨識上使用Yolo、PnP等電腦視覺技術，並搭配類神經網路進行座標補償。在機械手臂控制策略上使用深度強化學習之深度確定策略梯度（Deep Deterministic Policy Gradient, DDPG）與近端策略最佳化（Proximal Policy Optimization, PPO）進行模擬實驗，並最終使用PPO搭配PID補償器進行實作，此設計架構可有效補償PPO輸出之穩態誤差，在機械手臂運動控制方面，滿足系統執行自動充電服務的需求。With the rapid advancement of technology and electric vehicle (EV) technology, EVs have gained significant attention in recent years. The demand for more convenient charging facilities has become a major requirement for EV owners. Therefore, this thesis proposes an automated charging system based on a master-slave architecture, providing automatic charging services. The overall structure consists of a mobile charging robot and a power trailer, combined with a charging behavior process, charging socket posture recognition, and deep reinforcement learning (RL) for motion control of the robotic arm, ultimately achieving an automated charging system that offers charging services.In this thesis, automatic recognition of the charging socket is performed using computer vision techniques such as Yolo and PnP, along with coordinate compensation through neural networks. For the robotic arm control strategy, Deep Reinforcement Learning algorithms like Deep Deterministic Policy Gradient (DDPG) and Proximal Policy Optimization (PPO) were used for simulation experiments. PPO, combined with a PID compensator, was finally implemented. This design framework effectively compensates for steady-state errors in the PPO output, fulfilling the system’s requirements for executing automated charging services in robotic arm motion control.