A 3D Mandible Surgery Simulation System Based On Pen Interaction

The traditional mandible surgery simulation systems use the WIMP interaction which is based on desktop environment. It requires surgeons to use the traditional input devices for surgical simulation operation. However, there are two problems in using it. Firstly, the WIMP paradigm, which is based on the desktop metaphor, is imperative and coarse-grained. But doctors want the system to satisfy their normal working ways doing while thinking when they do the operation. Therefore it requires the system to support the fine-grained interaction. Secondly.surgical simulation is a conceptual creative activity and not the programmed activities supported by the WIMP paradigm. Doctors may need to capture the vague, uncertain and transient changed thinking activities and record the diagnostic results during the surgical operation. Input devices under the WIMP paradigm are lack of nature and doctors have to frequently switch the input devices in order to achieve their goals, which will decentralize users'attention and reduces their productivity.In view of the problems existing in mandible surgery simulation system, mandible surgery simulation which is based on the pen-based interaction was proposed in this thesis. At the same time, we designed and implemented a 3D virtual mandible surgery simulation system with pen-based interaction, which offered a possible solution for improving effectiveness and efficiency of the interaction in the mandible surgery simulation system. The main work includes:(1) We analyzed the common interaction techniques and characteristics of Post-WIMP. According to the features of interaction tasks under jaw surgery and using User-Centered Design, we applied pen interaction into the jaw surgery simulation system. As a complement to the WIMP, pen-based interaction is consistent with doctors'work habits as it is based on pen-paper metaphor. Users can imitate surgery operation such as surface trimming by using virtual scalpel which is generated by pen digital information.(2) We did research on the characteristics and tasks of 3D interaction in the virtual jaw surgery. In order to control object precisely in 3D scene,we used 3D mouse as an auxiliary input device which is fit for human perception and control principle.(3) We designed and implemented a 3D mandible surgery simulation prototype system and an editor based on text and gesture recognition techniques of pen interaction. Thus users can input text by the editor and make appropriate changes by using gestures since thinking activities and diagnostic results don't need too many words. Thereby it reduces frequency of switching the input devices and improves nature of interaction. Eventually, this thesis gave the system assessment to verify validity and rationality of the prototype system.