Research On Collaborative Problem Solving Assessment Based On Conversation Agent—Approach Design

In the 21 st century,the ever-changing of the global economy and the social conditions,as well as the rapid development of technological transformation,are reshaping work and life,and redefining the skills necessary for contemporary success.With the interconnection of knowledge and culture,the fine differentiation of the social division of labor,situations where non-routine problems are asking for communication and collaboration has appeared in more and more places.The reports from the UNESCO,the OECD,and many other organizations and countries have pointed out the importance of problem solving skill and interpersonal skill.Collaborative problem solving is a combined competency of these two core skills,how to evaluate and develop it has received wide attention.Due to the complexity of the problem solving and the situatedness of collaboration,the implicitness of thinking,find an approach to obtain data onto evaluating the collaborative problem solving is the key issue to be solved.Collaboration problem solving activity is not a specific flow path,but rather an abstract concept of a process.Both communication and problem solving,the core content of collaborative problem solving assessment,are nonlinear sequence activities.Traditional methods of gifted evaluation and data collection methods are difficult to capturing the contour of the activity comprehensively.No matter which fields,research is based on the premise that the object in the study can be described.For instance,the research on the charged particles was promoted after the cloud chamber had invented,which make the particles movement track visualized.Therefore,seeking and exploring new approaches to describe and trace the process of collaborative problem solving can provide strong evidence and support for establishing an effective evaluation system.The assessment based on conversational agent,which can standardize and regulate the active environment,and guide the individual to make fuller express through dialogues becoming a new way for collaborative problem solving assessment to obtain supplementary evidence.Based on a large quantity of related literature,such as collaborative learning,conversation analysis and so on,this research attempts to use dialogue as an entry point to design and develop a tool which uses the Dialogue Management module to construct conversational agents to obtain the process data of collaborative problem solving activity.This article teases out the components of collaborative problem solving activity and their relationships from a cognitive perspective,pre-defines the code of the individual's performance in the collaborative problem solving activities from “output” and “feedback” respectively,designs the core logic of dialogue management according to the “context” and constraints agent's performance through “limited cognitive space”.These form the tool capable of supporting the normatively externalize the process of collaborative problem solving.To verify whether the design of the tool can simulate a real-world collaborative problem situation,and whether the process data collected through the tool can present individual collaboration problem solving ability,a prototype based on this design was developed.After encoding the process data collected through prototype,collaborative problem solving activity process is visualized from the perspectives of team and individual.The prototype proved that the dialogue agent could simulate the real collaboration to some extent,and the team gain different performances shows relatively obvious difference in the process data.This research provides an exploration to develop conforming to the actual situations,normative,and adaptable assessments,as well as an attempt to involve task design,human-computer interaction,and interactive assessment.