英語聊天機器人對台灣高中生英語學習之效益探討

No Thumbnail Available

Date

2013

Journal Title

Journal ISSN

Volume Title

Publisher

Abstract

隨著電腦科技的演進及網路的普及,近年有越來越多的線上學習資源能夠擔任輔助台灣學生英語學習的工具。其中,英語聊天機器人(chatbots)能夠處理鍵盤輸入的語言並以螢幕呈現文字的方式回應,此種科技已經在部分英語系國家的線上訂票系統及商店中擔任諮詢者的角色,顧客能於網路上提出問題,而線上的英語聊天機器人能夠運用關鍵字、資料庫查詢、句型轉換等技術來及時提供適切的回答,文獻探討指出目前主要有五種建構聊天機器人的程式語言,然而僅有其中三種提供完整的對談記錄;而先前的研究主要是針對使用者的觀感做分析,迄今卻鮮有針對英語聊天機器人對英語學習成效上的實證性研究。本研究之主要目的即是探討先後與三種英語聊天機器人(Cleverbot, Skynet-AI, 及Meg)進行開放性互動是否能提升學習者英文語言使用上的句型複雜度、文法準確度、訊息適切度及流暢度。本研究也比較是否學習者的語言使用以及學習者對於「不同聊天機器人對於英語學習」的看法是否會因為聊天機器人的選擇而有不同。 本研究的受試者由南投一社區高中的42位高一學生所組成,在實驗之初,所有受試者都在電腦教室中與三個機器人各進行十五分鐘的互動,研究者回收聊天記錄檔案並分析受試者在句型複雜度、文法準確度、訊息適切度及流暢度上的起始能力。接著在六個星期的過程中,受試者分別自行與所有三個機器人進行兩次三十分鐘的開放式聊天並針對每次選用的機器人填寫一份問卷。至實驗結束時,所有的受試者再次在電腦教室中與三個相同的機器人各進行十五分鐘的互動,研究者同樣回收聊天記錄並以統計比較實驗後受試者的句型複雜度、文法準確度、訊息適切度及流暢度是否有變化。在六個星期的過程中,受試者必須在與機器人互動後將聊天記錄及問卷繳回給實驗者。 然而,在回收資料的過程中,研究者發現僅有20位受試者與每個機器人進行兩次三十分鐘的開放式互動,因此,在資料分析時,研究者將受試者分為在六個星期的實驗中完整與機器人互動共三個小時的「實驗處理組」(20人)及平均跟機器人互動不到一小時的「部份參與組」(12人)兩組,剩下的10人並未完成前測或後測而被排除。 為探索學習成效,採用組內比較同樣的受試者在前、後測中的表現是否有顯著差異。在確認「實驗處理組」及「部份參與組」兩組之可比性後,研究者也進行跨組的比較;此外,本研究也使用組內重複量數來比較相同學生在與三種機器人互動時是否有不一樣的表現,藉此觀察機器人的選用是否會影響受試者的英文使用。在受試者對於不同機器人的看法方面,本研究以描述性統計以及單因子變異數分析來呈現並比較學習者對於不同機器人的看法是否有差異。 本研究的結果摘要如下: 1. 與英語聊天機器人定期互動一段時間後,學生的句型複雜度並無顯著變化。 2. 定期與英語聊天機器人聊天能提升英語學習者的文法精確度,但對於訊息適切度上卻沒有顯著影響。學生與聊天機器人的互動中可能零星地從機器人的語句中學到一些新的文法概念或是藉由模仿機器人的用語而提升文法水平。 3. 與英語聊天機器人互動達三小時對學生英文流暢度有顯著助益。學生也在問卷中指出使用聊天機器人能提供他們更多使用英文的機會,因此,英文流暢度的提升可能是機器人提供了更多練習機會的結果。 4. 不同程式語言建構出的聊天機器人僅對學生少數的語言複雜度指標產生影響,但對流暢度沒有影響。 5. 學生對「不同聊天機器人對於英語學習」的看法指出Skynet-AI的反應速度快於Cleverbot,因此,程式語言的選用可能影響機器人的回應速度;學習者也指出Meg在訊息適切度上明顯低於Skynet-AI以及Cleverbot。 關鍵字:聊天機器人、自然語言處理、語料庫分析
With the advancement of computer technology and the popularity of the Internet, there are gradually more online learning resources that can serve as English-learning tools for Taiwanese students. Among those resources, chatbots can process typed messages and respond by showing texts on the screen. Chatbot technology has been incorporated by some English-speaking countries, whose online ticket reservation system and stores adopt conversational agents to serve as information staff. Customers can pose questions online and chatbots can immediately offer appropriate responses using keyword matching, database searching, or pattern transformation. Literature showed that there are now five major types of chatbot construction mechanisms, though only three of which offer complete interaction transcripts. Previous studies on chatbots mainly focused on user perception. Few studies were conducted on chatbots’ effects on language learning. The main purpose of this study is to explore the effects of open-ended chats with three chatbots on English learners’ syntactic complexity, grammar accuracy, message appropriateness, and fluency. This study also examines learners’ interactions with different chatbots and investigates if learners’ perception of chatbots for English learning changes after chatting with different chatbots. The participants were a cohort of 42 10th graders from a community-based senior high school in Nantou. This study lasted for eight weeks. In Week 1, all students spent forty-five minutes chatting with three chatbots. The transcripts were collected and analyzed for learners’ syntactic complexity, grammar accuracy, message appropriateness, and fluency as initial proficiency. Between Week 2 to Week 7, participants chatted in an open-ended manner with all three chatbots for two thirty-minute sessions on their own and filled out a perception questionnaire for each bot they used. In Week 8, all participants chatted again with all three chatbots for fifteen minutes each. The transcripts were once again collected and analyzed for syntactic complexity, grammar accuracy, message appropriateness, and fluency to compare if pre- and post-tests were significantly different. During the six weeks from Week 2 to Week 7, participants were required to turn in their transcripts and questionnaires to the researcher to prove their participantion. However, during data collection, the researcher found that only 20 students finished the two thirty-minute sessions with each chatbot; hence, participants were put into two groups based on their degree of participation. Those who spent three hours chatting with bots were labeled as complete experiment group (CEG), while those who finished less than one hour were categorized as partial participation group (PPG). The rest of 10 students failed to finish either pre- or post-tests and were excluded. To examine the effects of bots, paired sample t-tests were first used to explore if within group differences in the pre- and post-tests achieved significance. After ensuring comparability between CEG and PPG, cross group analyses were computed to investigate the effects of interacting with chatbots on English learning. Moreover, repeated measure ANOVAs were adopted to compare if the same students performed differently when interacting with different chatbots. As for perception, descriptive statistics and one-way ANOVAs were computed to check if students had different perception of different chatbots. The five major findings are summarized below. 1. After interacting with bots for three hours, students’ syntactic complexity did not change. 2. Regularly chatting with bots raised English learners’ grammar accuracy, but did not significantly influence message appropriateness. In the interactions with bots, students might sporadically pick up some new grammar concepts or simply mimic chatbots’ language and developed their grammar level as a result. 3. Interacting with bots for three hours significantly increased students’ fluency. Students indicated in the questionnaire that using chatbots offered them more time to use English for communication; thus, fluency might have improved due to increased time to practice English. 4. Chatbots made using different construction mechanisms had some influence on syntactic complexity, but they did not affect fluency. 5. Students’ perception indicated that Skynet-AI responded at a faster rate than Cleverbot. Learners also indicated that Meg’s message appropriateness was much weaker than Skynet-AI and Cleverbot, suggesting that different mechanisms might also influence bots’ processing speed and chatbots’ message appropriateness. Keywords: chatbots, natural language processing (NLP), corpus analysis

Description

Keywords

聊天機器人, 自然語言處理, 語料庫分析, chatbots, natural language processing, corpus analysis

Citation

Collections