以毫米波雷達為基礎的手勢辨識之研究

Abstract

隨著人機互動技術的快速發展，毫米波雷達因具備隱私性高、不受光線干擾、可穿透遮蔽物等優勢，已逐漸成為手勢辨識應用之新興感測技術。本文採用開酷科技所開發之 60GHz 毫米波雷達，並搭配其專用的視覺化平台 Ksoc Tool，此工具為專門配合該雷達設計，具備資料收集與標註功能。透過 Ksoc Tool 完成原始資料擷取與資料標註後，進一步進行資料前處理、模型訓練與即時辨識顯示，建構出一套具備即時性與彈性的手勢辨識系統流程。在影像資料方面，本文深入說明兩種常見雷達影像格式：RDI（Range-Doppler Image）與 PHD（Phase Difference Map），並透過圖像與實例詳細解析其物理意義與應用情境。為有效處理動態手勢資料，系統採用滑動視窗機制切割連續序列，並透過高斯函數生成 soft label，提升標註於手勢邊界區域的過渡敏感性。模型部分則採用三維卷積神經網路（3D CNN）以同時擷取空間與時間特徵，並搭配均方誤差（MSE）作為損失函數進行監督式訓練。為強化手勢段落之區分能力，本文提出雙門檻後處理機制，透過進入與離開閥值協助界定動作啟始與終止點，並架設圖形介面，實現雷達資料的即時推論與手勢顯示。實驗結果顯示，本系統可正確辨識包含背景、PatPat、Come 與 Wave 四類別手勢，整體準確率達 95.8%，展現本研究於準確性、即時性與可擴展性三方面之應用潛力。

With the rapid advancement of human-computer interaction technologies, millimeter-wave radar has emerged as a promising sensing modality for gesture recognition due to its high privacy, resistance to lighting conditions, and ability to penetrate occlusions. This study utilizes a 60 GHz mmWave radar developed by Kaiku Technology, along with its dedicated visualization platform, the Ksoc Tool—designed specifically for the radar—to perform data collection and annotation. Based on the collected data, we implement a full pipeline encompassing preprocessing, model training, and real-time recognition display.The proposed system incorporates two types of radar images—Range-Doppler Images (RDI) and Phase Difference Maps (PHD)—with detailed analysis of their physical meaning and usage scenarios. A sliding window mechanism is employed to segment continuous gesture data, and soft labels are generated using Gaussian functions to enhance sensitivity at gesture boundaries. A 3D Convolutional Neural Network (3D CNN) is adopted to capture spatiotemporal features, trained with Mean Squared Error (MSE) loss. To further improve gesture segmentation, a dual-threshold post-processing mechanism is introduced to detect gesture start and end points. A graphical user interface is implemented to support real-time inference and visualization.Experimental results demonstrate that the system accurately recognizes four gesture types—Background, PatPat, Come, and Wave—with an overall accuracy of 95.8%, highlighting its potential in real-time performance, accuracy, and scalability.