BRR-CAT對能力估計與測驗焦慮之影響

Abstract

傳統電腦化適性測驗(computerized adaptive test; CAT)中，提供分區式的回溯 (Block-Review；BR)機制，可給予受試者修正誤解題意、輸入錯誤及計算錯誤的機會，有助受試者發揮其部分知識，展現最大能力表現，亦滿足受試者對回溯及改答機會的需求，舒緩其因無法回溯及改答而引發的挫折與焦慮。然而，受試者改答後，隨之改變的作答反應(response pattern)可能會形成不合理的情況，進而影響能力重新估計的結果。受試者改答後，於變動的作答反應中加入重排程序(rearrangement procedure)，調整為合理的作答反應是否可行？重排後的作答反應是否增加受試者能力重估計的精確度？皆為BR-CAT中待解決的問題。

本研究旨在提出於BR-CAT中加入作答反應重排程序(rearrangement procedure)之可行性，及探究分區重排式CAT(BRR-CAT)對受試者能力估計之精確度與效率，以及測驗焦慮之影響。第一階段模擬實驗，比較BRR-CAT、BR-CAT與傳統CAT於能力估計值、測驗標準誤及施測長度的差異情形。研究結果顯示，與傳統CAT相比，當受試者為中高能力考生，BRR-CAT和BR-CAT演算法測得的能力估計值更為接近受試者的真實能力值，而測量標準誤（standard error, SE）則無顯著差異。此外，當測驗結束條件較寬鬆時(例SE .35), BRR-CAT和BR-CAT之施測長度與CAT無顯著差異。第二階段為實徵實驗，即依模擬研究結果，建置BRR-CAT、BR-CAT與CAT之線上測驗系統，蒐集真實受試者之作答反應及測驗焦慮資料，比較BRR-CAT、BR-CAT與傳統CAT對受試能力估計及測驗時間的差異情形，以及受試者對測驗焦慮的反應狀況。資料分析結果顯示，與CAT和BR-CAT相較，BRR-CAT演算法測得的能力估計值與SE皆無顯著差異，而接受BRR-CAT和BR-CAT時，學生的擔心和緊張程度較低。

The block-review computerized adaptive test (BR-CAT) offers the review and change mechanism, provides examinees with the opportunities to check answers and revise key-in errors or mistakes after completing a block items. It also helps examinees to mitigate their test anxiety. However, changing answers would make an unsuitable estimator in ability re-estimation if the new response pattern was unreasonable. Is it practicable to incorporate the rearrangement procedure into BR-CAT so as to arrange a reasonable order of new response patterns and improve the precision of ability re-estimation? We would like to find a solution to this problem.

In this study, the BRR-CAT algorithm incorporating the rearrangement procedure into BR-CAT was designed. Two phase experiments were carried out to investigate the precision and the efficiency of BRR-CAT on ability estimation and the effect of BRR-CAT on examinees’ test anxiety. In Phase 1 (the simulated experiment), compared with CAT, the estimations of BRR-CAT were closer to examinees’ true ability and kept the equal SE when examinees’ ability was middle and high. Moreover, the precision of BRR-CAT and BR-CAT was equal. The efficiency of BRR-CAT would not be decreased when a moderate SE (SE .35) as a stopping criterion was set. In Phase 2 (an empirical experiment), three versions of testing system were implemented and the response patterns and test anxiety records from 112 participants were collected. The analytical results showed that the estimators and SE of BRR-CAT on ability estimation was equal to those of BR-CAT and CAT. Additionally, compared with CAT, the participants have lower worry and tenseness in a reviewable CAT environment such as BRR-CAT and BR-CAT.