以視覺為基礎之即時指揮手勢追蹤系統
No Thumbnail Available
Date
2008
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
隨著網路視訊的普及，網路攝影機品質日趨優良、價格也相對低廉，本研究旨在提出一個「指揮者手勢追蹤系統」，代替鍵盤與滑鼠作為輸入單元，讓使用者能透過視訊攝影機(Webcam)及個人電腦、運用基本的指揮動作，能夠即時追蹤使用者手勢的軌跡與方向變化、偵測音樂節拍所在的時間點。
本研究可分為兩個主要階段：第一階段為目標物追蹤，採用CAMSHIFT演算法來實現物件追蹤。CAMSHIFT演算法為平均位移演算法的改良，此演算法利用使用者所感興趣的顏色機率分佈特性，經由平均位移迭代的方式，找出其機率分佈圖的峰值，此峰值即為可能性最高之影像區塊並得到物體移動路徑。第二階段則利用兩種方法計算：K-曲率法則以及垂直分量低點偵測。K-曲率法則利用物體移動路徑各點之區率並計算找出其方向轉變；而垂直分量低點偵測則是找出物體移動的垂直低點，將此低點定義為音樂的節拍點。
本研究所開發之系統可以讓使用者自行選定偵測目標（如指揮棒）並準確偵測移動的軌跡，將使用者的指揮動作上方向的改變，轉變成音樂檔的節拍事件，其準確率平均可達86.46%以上。
In recent years, interaction between humans and computers is becoming more important. “Virtual Orchestra” is an Human Computer Interface (HCI) software which attempts to authentically reproduce a live orchestra using synthesized and sampled instruments sounds. Compared with the traditional HCIs, using vision-based gesture can provide a touch-free interface which is less bounding than mechanical instruments. In this research, we design a vision-based system that can track the hand motions of a conductor from webcam and extract musical beats from motions. The algorithm used is based on a robust nonparametric technique for climbing density gradients to find the mode of probability distributions. For each frame, the mean shift algorithm converges to the mode of the distribution. Then, the CAMSHIFT algorithm is used to track the moving objects in a video scene. After acquiring the target center point continuously, we can form the trajectory of moving target (such as baton, conductor’s hand…etc). By computing an approximation of k-curvature for the trajectory, and the angle between these two motion vectors, we can compute the point of the change of direction. In this thesis, a system was developed for interpreting a conductor’s gestures and translating theses gestures into musical beats that can be explained as the major part of the music. This system does not require the use of active sensing, special baton, or other constraints on the physical motion of the conductor.
In recent years, interaction between humans and computers is becoming more important. “Virtual Orchestra” is an Human Computer Interface (HCI) software which attempts to authentically reproduce a live orchestra using synthesized and sampled instruments sounds. Compared with the traditional HCIs, using vision-based gesture can provide a touch-free interface which is less bounding than mechanical instruments. In this research, we design a vision-based system that can track the hand motions of a conductor from webcam and extract musical beats from motions. The algorithm used is based on a robust nonparametric technique for climbing density gradients to find the mode of probability distributions. For each frame, the mean shift algorithm converges to the mode of the distribution. Then, the CAMSHIFT algorithm is used to track the moving objects in a video scene. After acquiring the target center point continuously, we can form the trajectory of moving target (such as baton, conductor’s hand…etc). By computing an approximation of k-curvature for the trajectory, and the angle between these two motion vectors, we can compute the point of the change of direction. In this thesis, a system was developed for interpreting a conductor’s gestures and translating theses gestures into musical beats that can be explained as the major part of the music. This system does not require the use of active sensing, special baton, or other constraints on the physical motion of the conductor.
Description
Keywords
手勢追蹤, Gesture Tracking, CAMSHIFT