基於 Kinect 之台灣手語單字辨識

Abstract

手勢辨識一直被應用於人機互動介面相關的研究，而手語更是其中當熱門的研究之一。 以影像方法為基礎的單字手語辨識發展已有一段時間，並都獲得不錯的辨識結果，但仍有其限制及問題，故近期開始研究加入深度資訊來協助辨識。 然而，市面上的深度攝影機通常價格不斐或配戴不易，而微軟公司於2010年推出的 Kinect控制器則提供了新的選擇。因此本論文提出了一個以Kinect為基礎，可即時辨識台灣手語單字的方法。 手語中的手勢由位置、方向及形狀三個主要部分組成，我們利用Kinect本身的骨架追蹤及提供的深度資訊，對應不同部分的特性分別擷取如軌跡特徵及手型特徵，且利用不同方法來辨識。最後利用上述三個主要部分的辨識結果，並透過本論文所設計的單字決策方法，來達到最後辨識手語單字的目的。Gesture recognition systems are wildly used in human-computer interaction research problems, and sign language recognition is a popular research among one of the studies. Vision-based sign language recognition approaches have been developing for long time and achieved good results, but it’s still has limitations and problems, so recent research has started adding depth information to solve problems. However, the depth cameras usually endure with high cost and hard to fetch problems, while recently the Microsoft Kinect has offered an affordable depth camera which has made depth a viable option for more researchers. Therefore, we propose a Kinect-based Taiwan sign language recognition methods. Sign consists of three main parts: hand position, direction, and shape of the composition. We use Kinect skeleton tracking and depth information, extracting features for recognition, respectively. Finally, we use our proposed method to help us decides possible sign language vocabulary.