An Empirical Study Of The Content Validity Of Cloze Test In NMET (Hunan)

The purpose of this thesis is to use corpora to study the content validity of Cloze One in National Matriculation English Test (NMET) (Hunan 2010-2014). Usually content validation employs expert judgment. However, as it is a logical or cognitive process it has often been doubted for its subjectivity. Corpus, as a database storing abundant and authentic linguistic texts, is deemed as a good supplement to traditional methods in the way that it makes a data-based approach possible to language analysis which was previously simply not practical.Based on the theory of Progressive Validity Argument and from the basic viewpoint of Progressive Validity, this thesis intends to apply corpora in studying the following three research questions concerning the content validity of Cloze One in NMET (Hunan 2010-2014):(1) How is the correctness of the test content in Cloze One? (2) How is the representativeness of the test content in Cloze One? (3) How are the test points distributed in Cloze One?Correctness refers to the correctness of language form and meaning. This study only focuses on language form, or rather, on collocations of the options and corresponding contexts, to check if they are acceptable. Thus context-based searches in BNC and COCA were conducted for the first question. The results show that some options may not be collocable with corresponding contexts as their collocations could not be retrieved in the two corpora:"0" result was found. To inspect the representativeness of the test content, a Word List for all the options was firstly generated to check if they were relevant to the test domain. Then word frequencies of the 240 options were retrieved in BNC. Results show that some words’ frequencies are obviously much lower than the majority’s, which indicates that there may be some problems about their representativeness. Conversely, some words’frequencies are comparatively too high and they are proved to be overused by learners. It is noteworthy because in one item if common words are put together with uncommon words as options the guessability of that item is due to increase. For the third question, a Domain was set up to investigate each year’s distribution of test points. Generally in the period of 2010-2014 proportions were relatively stable and reasonable except that in some year the amount of nouns exceeded the domain which led to the insufficiency of adjective test points.Based on a corpus-based approach this study mainly uses word frequencies to study content validity. In the future study, logical judgments such as expert judgments or questionnaires and so on are still recommended so data could be collected from different aspects which could make the conclusions of this study more solid.