A Study Of Genre-based Discursive Competence Of Business English: Construct Definition And Test Development

More than600universities offer undergraduate business English programs orundergraduate English programs with a focus on business English. And62universities have been approved by the Department of Higher Education, MOE, toofficially offer undergraduate business English programs. There exists an urgent needfor an achievement test to inform and improve the teaching and learning ofundergraduate business English majors. Business English test design and validationare particularly important as these may offer an insight into language learning andteaching, thereby facilitating business English teaching programs. However, testing ofEnglish for business purposes is still in its rudimentary stage and lacks awell-established system to assess business English competence.The fundamental consideration in test development is to clearly define theconstructs being measured, which constitutes the theoretical foundation in testdevelopment. Furthermore, test designers should provide evidence of howrequirements of practicality, reliability, validity and washback are met by their test. Itis out of these considerations that the current research seeks to explore the constructof language tests for business English majors, with attentions being drawn to testdesign and validation.The research embodies three main aims. One is define the construct of“Genre-based Discursive Competence of Business English” based on an extensivereview of relevant theories and practices in applied linguistics. The second aim is todevelop the Test for Business English Majors–Band8(TBEM-8) which isspecifically designed to meet the needs of assessing business English teaching andlearning. The third aim is to establish the validity, namely construct validity, contentvalidity, criterion-related validity, scoring validity and washback validity, of thenewly-developed test.To achieve the above aims, the research focuses on the following aspects withrespect to test development and validation.To start with, a framework was proposed to describe genre-based discursive competence of business English. It is based on a comprehensive review of theoriesand principles of the socio-cognitive perspective of literacy, genre studies, languagetesting, English for Specific Purposes, text linguistics, corpus linguistics anddiscourse analysis. The framework comprises five components: textual knowledge,pragmatic knowledge, generic knowledge, topical knowledge and the organizationalcompetence for business situations.Genre-based discursive competence of business English refers to the graduatesshould be able to carry out the negotiation of meaning by constructing, interpretingand exploiting academic and professional genres, in compliance with the rules, normsand conventions of the discourse community of applied linguistic studies and thecommunity of international business practitioners. In particular, they should be able toaccomplish tasks of oral and written intercultural communication by incorporating thecontextual factors of specific communicative tasks and activating the textual,pragmatic generic and topical knowledge with the organizational competence forbusiness situations.Based on the construct definition, an achievement test, namely the Test forBusiness English Majors-Band8, was designed. The guidelines that govern goodtesting practices were formulated based on an extensive review of research findings ofthe Educational Testing Service in the US and the Cambridge ESOL in the UK.The TBEM-8is a criterion-referenced achievement test of English for businessEnglish majors in Heilongjiang University. It aims to measure genre-based discursivecompetence of business English majors, and to examine whether they have met therequirements specified in the curriculum of the undergraduate business Englishprogram. The guiding principles of test design call for clear specifications of testdomains, adequate coverage of the content in the curriculum, authenticity andrepresentativeness of test tasks, high reliability and validity, as well as positivewashback on learning and teaching.The later parts of the thesis were devoted to the validation of the TBEM-8. Inlight of the influence that the test might exert on language learners and teachers inbusiness English program, validation should be a continuous process through which a variety of types of evidence about test score interpretation and test use is produced.Based on the framework of a unitary conceptualization of validity in language testing(Messick1989), validity of the TBEM-8was evaluated in terms of test construct,content, external criterion, rating of constructed-response items, and washback effects.Evidence for construct validity was gathered by employing a combination ofquantitative and qualitative approaches.341senior students majoring in businessEnglish from five universities took the TBEM-8prototype test and answered aquestionnaire after the test. Quantitative analyses investigated the internal structure ofthe test by assessing the extent to which the internal components of the test match thedefined construct. Correlational studies, exploratory factor analysis and analysis oftextual features of candidates’ essays were conducted to support the validityarguments in this regard. Qualitative analysis attempted to supply evidence for thesubstantive aspect of construct validity by revealing the extent to which expectedknowledge was utilized and hypothesized processes were involved in candidates’performances on the test.Results of correlation analysis suggested that the test had satisfactory internalstructure and items were of a homogeneous nature. Exploratory factor analysisrevealed that11tasks loaded heavily on three factors, namely listening, reading andgeneric competence. The results lent support to the argument that the structure of thetest was a good representation of the proposed construct framework. Analysis ofverbal reports provided additional support regarding test construct by examining theextent to which writing performances fit the defined construct. In sum, the studyprovided ample evidence of the validity of the TBEM-8in measuring the proposedconstruct.Content-related validity was investigated on the basis of experts’ evaluations ofthe extent to which the test’s content represents the specified content domain. Sixexperts were invited to fill in a questionnaire designed to gather their views ofrelevance, importance, clarity, and lacks of bias in test items or tasks, as well asprocesses involved in completing the tasks. Results indicated that test contentrepresented the knowledge and competences stipulated in the test’s specifications. Criterion-related validity evidence was achieved from correlational studies of therelationships between the TBEM-8and the external criteria, including candidates’achievements, ranking order provided by language teachers, and BEC (higher) scores.Data were obtained from163senior students majoring in business English. TheTBEM-8was proved to be a valid measure in light of its relationships to othermeasures that the test should, theoretically, have high correlations with.Rating-related validity attempted to investigate the ways in which raters used therating scales as intended. The marking schemes of the TBEM-8translating andwriting consist of both analytic and holistic rating scales. Two teams of eight ratersparticipated in the scoring session and the inter-rater correlations were over0.80.T-test results demonstrated that no significant difference existed among ratersapplying the same rating scale. These findings provided evidence in regard to thereliability of translating and writing scores as well as the validity of the rating scalesused in the test.Questionnaires and interviews were conducted to explore washback effects of theTBEM-8on teaching and learning.341students took the test and answered thequestionnaire, and15language teachers took part in the interview sessions. Resultssuggested that the TBEM-8could exert positive effects on language teaching andlearning, and feedback provided by the test scores would improve the quality ofteaching and learning.The thesis consists of eight chapters.Chapter One is an overview of the study. The rationale is that the developmentand validation of an English test for Business purposes are imperative. An in-depthanalysis of business English testing is crucial for the design of the test and the insightsgained in this study could promote language teaching and learning.Chapter Two reviews literature on theoretical models of language competence,and theories and studies on test validity, construct, and validation.Chapter Three presents the overall design of the current study, including researchpurposes, questions, and procedures of test development and validation.Chapter Four and Five lay a solid theoretical foundation for the study. A thorough study of business English discourse, business English genres andgenre-based discourse practice of the Major is described in Chapter four. Based on anextensive literature review, the framework of “Genre-based Discursive Competence ofBusiness English” is provided in Chapter Five.Chapter Six elaborates on the development of the TBEM-8, an achievement testfor business English majors.Chapter seven reports the validation of TBEM-8based on the evidence-basedvalidation framework. Empirical evidence used in support of the validity argumentswas collected from different perspectives, including test content, internal structure,response process, external criterion, and washback effects.Conclusions are drawn in Chapter Eight. Limitations of the present study andsuggestions for further research in this field are also provided.