探討提升自動英語口語評估準確性之方法- 以會話測試為例

Abstract

由於全球化與網路的普及，人們需要學習第二語言的需求急劇增加，尤其是英文作為最主要的知識傳遞語言。雖然現今有許多免費或付費的英文教學影片、補習班等資源可供選擇，然而語言教師的增加速度卻跟不上學習者的需求。因此，為了解決此問題，我們需要有效率的方式處理學習者在語言學習過程中獲得的資訊，協助非母語者在沒有足夠語言教師的情況下，仍能順利地學習第二語言。在各種補足人力的方法中，電腦作為人力輔助的角色最為適合，尤其是語音辨識技術已經成熟，並出現許多商業應用案例，如電腦輔助語言學習 (Computer Assisted Language Learning, CALL) 的錯誤發音偵測與診斷 (Mispronunciation Detection and Diagnosis, MDD)、可讀性評量，以及我們本研究的主題：自動口說評量。自動口說評量是英文評量中的一個方面，透過受訪者的口說聲音和內容來進行能力評估，但需要英文專家花費時間進行評分。如果可以藉由電腦完成相同任務，將節省大量的人力、時間和金錢。然而，目前在此領域的研究遇到幾個問題，例如不同等級的語者數量不平衡，尤其是在最高和最低等級的語者數量和其他等級之間呈倍數差距，以及自由口說容需要考慮更細緻的子句關係代名詞關係和面試官的資訊。我們嘗試從資料、訓練技巧和模型架構等方面入手，提升整體效能，同時兼顧可解釋性，使本研究能夠真正在實際應用中被接受。模型的程式碼在 \url{https://github.com/a2d8a4v/HierarchicalContextASA/}、資料前處理的程式碼在 \url{https://github.com/a2d8a4v/local_for_nict_jle}。

Due to globalization and the prevalence of the Internet, there has been a sharp increase in the demand for second language learning, especially in English, which is the primary language of knowledge transfer. While there are many free or paid resources such as English tutorial videos and cram schools available today, the rate of increase in language teachers cannot keep up with the demand of learners. Therefore, to address this problem, we need an efficient way to process the information acquired by learners in the language learning process, to assist non-native speakers in successfully learning a second language without sufficient language teachers. Among various methods to supplement manpower, the computer plays the most suitable role as a human assistant, especially since speech recognition technology has matured and many commercial applications have emerged, such as Mispronunciation Detection and Diagnosis (MDD) in Computer Assisted Language Learning (CALL), readability assessment, and the topic of our research: automatic speaking assessment. Automatic speaking assessment is an aspect of English assessment that evaluates the ability of respondents through their oral speech and content, but requires English experts to spend time grading. If the same task can be completed by a computer, it will save a lot of manpower, time, and money. However, current research in this field has encountered several problems, such as the imbalance of the number of speakers in different levels, especially the multiple differences in the number of speakers between the highest and lowest levels and other levels, and the need to consider more detailed clause relationships, pronoun relationships, and interviewer information in free speaking content. We attempted to improve the overall performance of our research from the aspects of data, training techniques, and model architecture, while also considering interpretability so that our research can be truly accepted in practical applications. The model's implementation code is available at \url{https://github.com/a2d8a4v/HierarchicalContextASA/}, and the code for the data preprocessing stage is at \url{https://github.com/a2d8a4v/local_for_nict_jle}.