學習導向黑板教學影片結構化之研究

Abstract

現今的教室仍然經常使用黑板，而以黑板授課的教學影片亦相當普遍，但黑板教學影片的內容分析在多媒體語意分析的領域深具挑戰性但極少被討論。本論文提出一個基於注意力模型的教學影片結構化方法，用以提醒學習者要投注多少注意力在不同時段的教學影片上。本研究分別設計視覺和聽覺注意力模型以分析影片內容，並利用兩種模型的混合結果產生一個以學習為中心的模型。在視覺分析方面，則以講者寫在黑板上的字跡和講者的講述姿態為分析的內容；而在聽覺分析方面，則以將分析講者的講述語音分析為主。 藉由混合多重注意力模型的結果，產生一個以學習為中心的注意力曲線。以學習為中心的注意力值反映出此段黑板教學影片中講者期望學生投注注意力的程度，也反映語意的強度。因此，學習者可以有彈性且結構化地讀取黑板教學影片，去找到他們應該理解的片段。實驗結果呈現提出的方法可以有效地結構化黑板教學影片，也擷取出與以學習為中心注意力值關聯的教學內容。

Since blackboards are the standard in the classrooms and are still used today, blackboard lecture videos are common in the lecture video recordings. However, it has been known that content-based blackboard lecture video analysis is challenging and thereby rarely touched upon in the field of multimedia semantics analysis. In this thesis, we proposed a new structuring method for blackboard lecture videos by estimating the learning focus that learners should pay more attention to. Both visual and aural analysis for blackboard lecture videos are utilized and integrated to develop a learning-focused attention model. As for the visual analysis, the fluctuation of written lecture content on the blackboard and the posture of lecturers are analyzed. On the other hand, the speech of lecturers is used for aural analysis. Finally, a learning-focused attention curve can be generated by fusing multiple attention models. In a sense, the values of the learning-focused attention reflect the strength of attention or semantics that the learners should pay to the blackboard lecture videos and can be used for indicating the importance of the extracted lecture content at the corresponding time. Therefore, learners can easily access the blackboard lecture video with good flexibility to find what the lecture content they should understand and video frames to watch from the well-structured video. The experimental results show that the proposed method can effectively structure blackboard lecture videos and extract the lecture content with associated learning-focused attention values.