許陳鑑Hsu, Chen-Chien李作庭Li, Tso-Ting2023-12-082025-09-012023-12-082022https://etds.lib.ntnu.edu.tw/thesis/detail/a65d5b0430ce408257c353a32796bb4a/http://rportal.lib.ntnu.edu.tw/handle/20.500.12235/120320本論文提出一任務監控系統，以確保人員操作程序與標準作業程序一致，避免意外或操作不當的情況發生，包含:影像目標偵測模組、手部動作辨識模組、用於穩定辨識結果之濾波器、以及程序比對演算法。作法係使用 SlowFast 動作辨識演算法，根據影像的取樣頻率，將輸入拆分為slow pathway 以及fast pathway，前者用於取得空間特徵，後者則增強對於時間特徵上之提取，使得模型可以取得更多時空間之資訊，進而實現精細動作的辨識，解決傳統動作辨識演算法只專注在單一取樣頻率進行空間特徵提取，不利於應用在連續動作辨識的限制。為了將該系統有效地實踐在實際應用場景，本論文亦使用YOLOv4偵測目標影像，濾除非目標事件之場景，當目標影像收集足夠的影像數量時，啟用SlowFast進行人員操作目標物之動作辨識，再使用一改良的濾波器用以降低辨識結果不穩定之情形，建立手部與施作工件等目標物件之相依動作行為之動作庫(action base)。隨後，利用一預先建立之標準作業程序動作庫，包含了操作物件以及相對應的動作，利用一比對演算法進行任務行為之檢測，判別人員操作程序流程是否符合規範。為驗證系統之性能，本論文將所提出之任務監控系統以一小型工作坊人機協作進行即時判斷，監督操作員的操作程序是否符合正確規範。In this paper, we propose a task monitoring system to ensure the operation procedure conducted by a human operator complies with a Standard Operating Procedure (SOP) to avoid accidental or improper operation. The system includes a target image detection module, hand action recognition module, filter module for stable recognition results, and procedure comparison algorithm. As far as action detection is concerned, the proposed system uses the SlowFast algorithm to split the input into a slow pathway and a fast pathway based on the sampling frequency of the image. The former is used to obtain spatial features while the latter enhances the extraction of temporal features. As a result, the model can obtain more spatial and temporal information to achieve fine-grained action recognition, solving the problems of continuous action recognition by traditional action recognition algorithms which generally focus on a single sampling frequency for spatial feature extraction. To solve practical real-world applications, this paper first uses YOLOv4 to detect the targets and filter out non-targeted scenes. When enough frames are collected from the target image, SlowFast is enabled to recognize the action of the human operator, an improved filter is used to reduce the instability of the recognition results, and an action base is created, which describes the behavior of the operator and the target object being interacted within the workspace. Then, Standard Operating Procedures with pre-established operation rules, including operative objects and their corresponding actions, are used to compare against the action base established by the proposed system to determine whether the procedure flow of the human operator complies with the rules. In order to verify the performance of the system, a task monitoring system exemplifying man-machine collaboration in a small workshop is demonstrated in this paper to validate whether the operator's operating procedures meet the correct SOP in real-time.深度學習動作辨識智慧型監控標準作業程序deep learningaction recognitionsmart surveillancestandard operation proceduresetd