英語能力測驗難度的控管－以臺師大英語會考為例
No Thumbnail Available
Date
2015-03-??
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
國立臺灣師範大學師資培育與就業輔導處
Office of Teacher Education and Career Service
Office of Teacher Education and Career Service
Abstract
本研究分為二大目的，目的一:呈現本人近年來對難度變數的研究發現；目的二：與有志於撰寫能力測驗者分享本人多年實務經驗的心得，希望藉此協助他們控管試題的品質。其實要控制試題難度並不是一件容易的事，因為難度並不是一個不變的定數，它會隨著考生的程度、考試的情境以及試題本身的種種因素而改變，命題者需要累積許多經驗才能正確地判斷變數運作的方向和力道。本研究的資料取自臺師大三年的英語會考試題，考生為非英語系的大一新生，程度從高級到初級皆有，共超過2700 人，試題共285 題，經統計考驗，試題信度都超過0.82 。研究步驟首先透過文獻探討，列出常見會影響閱讀測驗和聽力測驗通過率的變數。接著把所有試題按照通過率高低排列，逐一列舉並審視每一題牽涉到的變數的種類和數目，結果發現了以下的趨勢一一試題包含的難度變數愈多，難度就愈高。但其中有一些例外，本研究也進一步檢視並分析其原因。最後，根據分析的結果，提出一些命題原則和步驟，同時還設計出一份可以粗略計算難度的表格，供命題者參考。
For many test writers, it is common sense that the level of difficulty must be appropriate in order to discriminate test takers of different English levels. However, to know is one thing, and to do is another. According to my past experience as a test reviewer, quite a few test writers failed to put their testing knowledge into real practice, so the items they wrote were not good enough in terms of either facility or discriminability. In response to this gap, this study is intended to explore how to well control the facility value of a test. In my opinion, the failure of the test writers to control the level of difficulty can be largely attributed to a misconception, i.e. they regard it as something static rather than dynamic. In fact, the reverse is true. The facility value always varies with a lot of factors, such as the test taker's English competence, the testing environment and condition, and a variety of variables that are involved with the test itself. Therefore test writers need much experience so that they can correctly judge which variables will be active enough to affect the test result, while others will not. The data for this study comprise 3 English proficiency tests administered to all the non-English-major freshmen of NTNU from 2003 to 2005. The 3 tests are all statistically proved to be reliable. There are altogether 285 items and over 2700 subjects with different levels of English abilities. To begin with, this study lists all the relevant variables of facility/difficulty through literature survey. Next, all the items are arranged in order of their facility values, and each of them is scrutinized to see what variables of facility or difficulty are involved, how these variables interact with each other, and if there is any hierarchical relationship among the variables. A general tendency for predicting the facility value is found. Finally, some specific guidelines are proposed based on the result of the analysis. Besides, a table listing all the rel
For many test writers, it is common sense that the level of difficulty must be appropriate in order to discriminate test takers of different English levels. However, to know is one thing, and to do is another. According to my past experience as a test reviewer, quite a few test writers failed to put their testing knowledge into real practice, so the items they wrote were not good enough in terms of either facility or discriminability. In response to this gap, this study is intended to explore how to well control the facility value of a test. In my opinion, the failure of the test writers to control the level of difficulty can be largely attributed to a misconception, i.e. they regard it as something static rather than dynamic. In fact, the reverse is true. The facility value always varies with a lot of factors, such as the test taker's English competence, the testing environment and condition, and a variety of variables that are involved with the test itself. Therefore test writers need much experience so that they can correctly judge which variables will be active enough to affect the test result, while others will not. The data for this study comprise 3 English proficiency tests administered to all the non-English-major freshmen of NTNU from 2003 to 2005. The 3 tests are all statistically proved to be reliable. There are altogether 285 items and over 2700 subjects with different levels of English abilities. To begin with, this study lists all the relevant variables of facility/difficulty through literature survey. Next, all the items are arranged in order of their facility values, and each of them is scrutinized to see what variables of facility or difficulty are involved, how these variables interact with each other, and if there is any hierarchical relationship among the variables. A general tendency for predicting the facility value is found. Finally, some specific guidelines are proposed based on the result of the analysis. Besides, a table listing all the rel