Study Of Assessment Tool For Evaluating The Child Observation And Assessment Skill Of Kindergarten Teachers

As an indispensable skill of kindergarten teacher professional development and a critical part to enhance ECEC process quality,the importance of the observation and assessment skill has been gradually recognized.However,the current level of this skill among kindergarten teachers is far below satisfaction.Specific on-demand training is required to offer support.Thus,it is urgent to develop an effective evaluation tool to assess this skill and stratify kindergarten teachers for training.However,the implicit feature of the skill posed a challenge on designing an objective assessment and the urgent need for developmental teacher evaluation,personalized professional development and quality monitoring system also lead to the insufficiency of related tools.Therefore,this study analyzed the structure of the skill and builds up a new test paradigm based on ideas of Situational Judgement Tests(SJTs),designed 80 items and finished the quality analysis.Four applicable paper-and-pencil tests were formed.Besides,considering of the limitations of the traditional test paradigm,the study proposed it was necessary to develop computerized adaptive test to meet the need for more personalized and intelligent assessment and makes a preliminary attempt at designing such a test.The whole study consists of three interwoven sub-studies,with five-phased tests involving 8,300 subjects from 18 cities,specifically including:Research 1: This sub-study constructs and operationally verifies the test paradigm based on the SJTs and Zone of Proximal Development(ZPD)theory,which includes:(1)Clarifying the test content and demand based on literature review and field investigation;(2)After analyzing the suitability,the study builds up a test paradigm which combines SJTs and Zone of Proximal Development(ZPD)theory;(3)Improving the details of test items based on the operational verification to the pretest results.Research 2: This sub-study develops an paper-and-pencil test based on classical testing theory and accomplishes quality analysis.It practices and verifies the hypothesis raised by Research 1 and illustrates the necessity of Research 3.The sub-study includes:(1)Developing and revising test items through the cases collected from the continuous observation made by teachers in an organized workshop.The formation of 30 available items follows this logic " referring to the theories + case observation + group discussion";(2)Formalizing the first set of the evaluation tool.The study chooses to organize the items to form different versions of tests and uses two pretests to examine the quality of each version and improve the suitability.Besides,the study uses 2464 data from 9 cities to further verify the quality of the tool(with the reliability of 0.867,and the analysis of Classical Testing Theory and Item Response theory meets the standard criterion).Through this process,one version of the tool has settled;(3)Analyzing the distinguishing efficiency of the test.The study combines the test scores and teachers’ oral reports(including group interviews and 533 individual interview data)to test the distinguishing efficiency of the tool from different perspectives;(4)Increasing items and making three counterparts of the tests.The study respectively collects 429,458,and 402 valid data from 4-5 cities to verify the quality;(5)Based on the reflection of the trend to conduct developmental teacher evaluation and the limitation of paper-andpencil test,the study illustrated that using computerized adaptive assessment is the future direction.Research 3: On the basis of modern test theory,this sub-study finished the preliminary research on computerized adaptive assessment,including:(1)Designs three “anchor item" plans and forms three tests,and collects 1377,1264 and 1156 valid data respectively;(2)Used equivalent analysis to examine the four anchor tests(including the first test)and achieves the quality equivalence of 6443 teachers.It shows that besides a few items,the overall informative level of the test and most items are acceptable.The score of teachers’ ability range from-2.14 to 2.02,and the test is more reliable when test subjects whose ability score falls between-1.5 1.5(account for89.63%),which covers the majority of teachers.(4)According to the result,the study proposed the concept of constructing a test item bank and setting rules for adaptive tests.Besides,the study proposed a continuous development cycle of future assessment which includes several steps: “applying the test and items in training,collecting feedback during the training to improve the items and new observation cases,forming new items and enriching the item bank and applying the new test in teacher training”.To sum up,this study has designed an objective test to stratify kindergarten teachers’ child observation and assessment skill.The large-scale test illustrates the reliability and availability of the test and provides a basis for the establishment of the complete computerized adaptive test system.Meanwhile,this study analyzes the current situation and characteristics of teachers’ child observation and assessment ability and gives suggestions on professional development and training strategies.