跨向度轉換程序對多向度多階段適性測驗測量精準度的影響

Abstract

多階段適性測驗（computerized multistage adaptive testing, MSAT）為電腦化適性測驗（computerized adaptive testing, CAT）的一種特例，它擁有CAT的優點，相較於線性測驗，可以使用較少的題數達到與CAT相近的測量精準度。本研究探討題間二向度MSAT與題內四向度MSAT，在不同跨向度轉換程序對於測量精準度的影響，分為三個子研究。研究一為提出題間二向度MSAT設計，在已知受試者樣本分佈狀態且向度能力間相關正確，分別使用單向度二參數對數模式（unidimensional two-parameter logistic model, 2PL）與多向度二參數IRT 模式（multidimensional item response model, M2PL）估計能力，研究發現M2PL模式測量精準較高。其次，在跨向度測驗的轉換程序，利用答對題數或迴歸模型進行適性，受試者的能力估計均方根差（root mean square error, RMSE）較小。研究二為提出題內四向度MSAT設計，在已知受試者樣本分佈狀態且向度能力間的相關正確，設計不同跨向度的轉換程序，研究發現利用迴歸模型進行跨向度的轉換程序時，當向度間能力相關程度越高，模板1–3–3–3的設計最佳，模板1–3–2–3的設計為其次，均優於無跨向度轉換程序的設計，受試者的能力估計RMSE較小。研究三探討當受試者樣本分佈狀態未知且不一定正確，不同能力組合之受試者進行題間二向度MSAT與題內四向度MSAT，對於測量經準度的影響。研究發現，在極端能力受試者的能力估計RMSE較大，而中等能力受試者的能力估計RMSE較小。當受試者能力越不符合向度間能力相關程度時，其能力估計RMSE越大。在進行多向度MSAT時，利用向度間能力的相關進行適性，可以有效的降低受試者的能力估計RMSE，受試者僅需作答部分的試題，就能達到良好的測量精準度，節省測驗時間。Computerized multistage adaptive testing (MSAT) is a particular case of computerized adaptive testing (CAT). MSAT has the merits of CAT, and the measurement accuracy is similar to CAT with fewer items than linear tests. This study explored the effect of two-dimensional MSAT in between-item structure and four-dimensional MSAT in within-item structure with different routing modules between dimensions on the measurement accuracy. The study is divided into three sub-studies.In Study 1, a two-dimensional MSAT in a between-item structure was proposed. The unidimensional two-parameter logistic model (2PL) and multidimensional item response model (M2PL) were employed to estimate the ability of the subjects with known sample distribution status and the correct correlation between the dimensional abilities. It was found that the M2PL model was more accurate. Secondly, in the routing modules between dimensions, the root mean square error (RMSE) in the estimation of the ability is fewer when the number of correct answers or the regression model was used for adaptation.In Study 2, a four-dimensional MSAT in a within-item structure was proposed. With the correct correlation between the dimensional abilities and the known sample distribution status of the subjects, it was found that using the regression model for routing modules between dimensions, the panel 1–3–3–3 design was superior to the panel 1–3–2–3 design, and all better than the design without routing module between dimensions, and the ability estimation error was minimal.In study 3, the effect of the two-dimensional MSAT in between-item structure and four-dimensional MSAT in within-item structure on the measurement accuracy for the subjects with different ability combinations when their sample distribution status was unknown. It was found that the RMSE was the highest for the extreme ability examinees and the lowest for the moderate ability examinees. When the examinees' ability did not match the cross-directional ability correlation, their RMSE of ability estimation was larger. In sum, the two MSAT designs can utilize between-dimension correlation to improve testing efficiency. The examinees only need to answer some of the items and save testing time.