呈現不同視覺媒材對英語學習者多模態聽力理解與認知負荷之影響

Abstract

視覺媒材與第二語言聽力表現的關係，長期以來備受關注。在眾多視覺媒材之中，以影片做為聽力測驗媒材最為普遍。然而，過往的研究結果呈現分歧。為深入瞭解分歧的起因，探究視覺媒材與第二語言學習者語言程度之間的複雜關係尤為重要。本研究旨在探討不同第二語言程度(高或低)的學習者，在接觸不同視覺媒材(動態影片或連續靜態影像)後，其聽力表現是否呈現差異。為豐富研究成果，本研究同時測量學習者的主觀認知負荷(內在、外在與增生)以及對視覺媒材的整體感受。研究結果顯示，在68名學習者中，學習者的第二語言程度與所接觸的視覺媒材皆對聽力表現產生直接影響。其中第二語言程度的影響尤為顯著。高成就學習者在不同視覺媒材條件下的聽力表現皆顯著高於低成就學習者。進一步分析，縱使低成就學習者的聽力表現顯著低於高成就學習者，低成就學習者仍對於視覺媒材展現正向態度且仔細留意不同非語言訊息之意涵。就視覺媒材而言，動態影片相較於連續靜態影像更能顯著提升聽力表現。此差異可能源於動態影片提供較細膩的非語言訊息且引發較低的外在認知負荷與較高的增生認知負荷所致。此外，結果亦可推論動態影片為較合適的多模聽力視覺素材。本研究結果再次呼應第二語言學習者之語言程度與所接觸之視覺媒材對於聽力表現的影響。同時，本研究為聽力教學與測驗中，多模態資源的運用提出建議，希冀透過教學與測驗方法，培養學習者面對真實世界的聽力技巧與策略。The interaction between visual components and L2 listening performance has garnered considerable attention, with video as a prevalent assessment tool, albeit with discrepant results. The inconsistency necessitates a nuanced examination of the intricacies between diverse visual formats and learners' L2 proficiency. In light of the existing findings and assumptions, the present study investigated the interplay of L2 proficiency levels (i.e., high vs. low) and contrasting visual formats (i.e., dynamic video vs. sequential static imagery). The study also gauged learners' perceived cognitive load at different proficient levels while encountering various visual formats to further enrich the analysis. The manipulation of L2 proficiency and visual formats resulted in four listening conditions (i.e., high-proficient + dynamic video, low-proficient + dynamic video, high-proficient + sequential static imagery, and low-proficient + sequential static imagery). A cohort of 68 participants, stratified by proficiency and visual format, were assigned to one of the four listening conditions. They then completed the listening comprehension test and questionnaires measuring their perceived cognitive load (i.e., intrinsic, extraneous, and germane) and overall perception.Results elucidated that learners' L2 proficiency exerted a more profound impact on listening performance than visual formats. Specifically, high-proficient learners consistently outperformed their low-proficiency counterparts, irrespective of visual formats. While low-proficient learners' listening performance was inferior to that of high-proficient learners, they generally reported positive attitudes toward visual inclusion, and paid close attention to various nonverbal information. Visual formats, on the other hand, demonstrated a comparatively limited impact on learners' listening performance; they nonetheless exhibited statistically significant effects, with dynamic video yielding superior outcomes compared to sequential static imagery. The enhanced performance was derived from the detailed nonverbal information in dynamic video, the low extraneous cognitive load, and the high germane cognitive load. Additionally, dynamic video might be an optimal visual material for eliciting learners' multimodal listening ability. The present study underscores the role of learners' L2 proficiency and visual formats in shaping listening performance within a multimodal listening environment. The study's implications are far-reaching for L2 listening instruction and assessment. By judiciously considering the role of visual components and learners' proficiency levels, instructors can apply effective strategies to cultivate learners' listening skills in real-world, multimodal contexts.