Fairness In Language Testing:Conceptualization And Application

Testing is a practical activity which plays a significant role in personal development as well as social progress(Liao,2003:1).It has been acknowledged that the fairness of testing is one of the most important aims of tests thus it should be maintained throughout the entire process of testing.In recent years,the topic has aroused more concerns and has become a hot issue.As part of educational testing,language tests are used to evaluate on testees' language proficiency as well as learning achievement.What makes language tests different from other tests is that language is the content of tests and at the same time the media.This trait makes fairness of language tests more complicated.According to Angoff,scholars began to notice the problem with fairness in the field of educational assessment in the 1960s(Angoff,1993).However,the discussions on language testing fairness only started from 1990s(Kunnan,2000).It is acknowledges that the annual meeting of LTRC in 1997 is the millennium of study on language testing fairness.Up till now,the science of language testing has evolved into a mature discipline and has gained wide concerns in the world of America and Europe as it can be proved that most influencing major works on the subject have been produced by experts and scholars in the West.In China,it still remains to be a field that is seldom occupied.As it can be traced back,the study on testing in China begun from the 1980s,and later there appeared research on fairness of tests including language testing fairness(Dai,2010:27).Among all the research,discussions have been highly focused on fairness in educational testing from the perspective of educational system and policy;there has been very limited exploration of fairness in language testing(Xie&Chen,2016:143).Up till now,the research on the subject has never been sufficient and efficient.At present,it has been accepted that fairness is important and indispensable for a valid and reliable test,but it seems difficult to reach agreement on certain principles,for example:Is fairness of test really measurable?How can fairness of a test be measured?...etc.This research will make an attempt to find solutions based on the discussions of these topics and would try to work out a framework on language testing fairness which is practical for application.Among language testing standards which have been developed and enforced in different testing contexts,there has been no similar works from Chinese testing contexts.To fulfill the purpose of developing a framework that can be applied to the testing contexts in China,this study is intended to address the research questions which are listed in the following:1)What are the fairness review standards which have already been developed and enforced in language testing standards,and what are the implications of these items if we are to construct similar standards to the present testing situation?This research will review and evaluate the major present conceptualization on language testing fairness in hope of providing an exploratory mode that can be used in the Chinese context.2)What is stake holders' perception on language testing fairness,and what are the implications if we are to specify fairness checklist?This research will investigate the most important stake holders-testees' perception and evaluation on fairness to explore on basic elements of fairness.3)What are possible guidelines of the fairness review standards,and how can we apply these standards to the testing operational works?This research will explore on a framework on fairness which can guide the real practice in language testing process.To clarify these three research questions,this study will proceed on four steps.First,we will collect major works on language testing standards and review those specifically on fairness.The review on the standards or items will help establish a panoramic view on the fairness issue,thus it can offer ideas on constructing a preliminary framework which demonstrate the fundamental considerations in fairness review process.Second,an empirical investigation was conducted with a view to displaying a representative picture of the current perception of students on fairness issues in language tests.According to the survey,students on the whole had positive perceptions of fairness issue and heightened the necessity and significance of developing standards on fairness of testing development and enforcement.Then at step three,a review checklist was drafted on the basis of the studies at the previous two steps.Since the rights and responsibilities of different stakeholders can not be specifically defined and balanced,it is impossible to reach an agreement on fairness theoretically and operationally.Therefore,it is practical to make the mechanisms inspirational rather than mandatory.In other words,it aims at raising the awareness of the professional as well as the public,and communicating to all the feasible good practices.In Chinese context,the evaluation framework on language testing fairness should clarify responsibilities and rights of all stake-holders such as designers,users and testees.To improve the quality of testing,the fairness evaluation should encompass check both on technical fairness and social justice.For the sake of operational application,it is essential to undertake validation,bias review and sensitivity review.It is also suggested that the result of fairness evaluation should be applied to improve fairness under public supervision to ensure it's ongoing test quality and social justice.At the fourth step,we will make an attempt to check the fairness of a paper using the constructed model.Based on both qualitative and quantitative means,we aim to test that fairness review is possible.During the research,the following methods have been adopted:1)Major works on language testing were collected,reviewed and evaluated.These works encompassed research from scholars as well as major codes drafted and developed by testing organizations from all over the world.Careful survey was made on those standards which were recognized as having important impact on the international language testing community.The review of the standards helped to demonstrate fundamental considerations in the hope of exploring the multifaceted factors which may shed influence on fairness in language tests.2)A self-developed questionnaire survey was conducted in the quantitative research design to look into the perception of students with regard to fairness in language testing.The questionnaire was designed according to Likert's five-point scale.More than 200 college students took part in the study.The survey identified the areas and issues which were recognized as in need of improvement in testing practices which might pose threat to test fairness.3)Case study was adopted in the qualitative research plan.Analysis on various testing items were conducted in a view to portraying a representative picture of the current fairness issue in language testing in China.4)Participants of in-depth interview were three teachers.Based on the checklist of fairness review,three participants conducted careful survey of the sample test paper.They were required to evaluate items on their personal judgment and explain further if they found anything unfair.Through literary analysis,questionnaire survey,case study and in-depth interview as the research methods to investigate the fairness issue in language testing,the researcher reckons that fairness in testing refers to the construct-relevant factors which ensure the same chance for different groups as well as individuals.It should encompass the measurement of fairness and the concept of social fairness.The fairness issue in essence relies on value judgment,while it is an essential factor that must be taken into consideration on testing quality.Therefore,the research on fairness should display its objectivity in science as well as it feasibility in operation:In theory,standards on fairness should specify responsibilities of different stake-holders while in practice more consideration should be given on its feasibility.The major contribution of this research lies in that will a framework on fairness has been constructed which can guide the real practice in language testing process based on empirical evidence.What's more,a checklist on fairness review makes abstract guidelines operable in real practice(Zieky,2006).