Interactive Pruning Operation On Virtual Fruit Tree Based On Kinect

Pruning is the key technology in the fruit trees management, which can promote the growth of fruit trees effectively and keep the structure of trees to improve fruit yield and quality. At present, the new generation of growers and students in pomology major study the pruning technique by reading books, watching videos, or consulting with an expert. While fruit tree pruning depends on high technology and has flexible pruning methods. In particular,once the branches have been cut off from trees, it cannot be reversed. Thus, beginners seldom have the chance to practice in the groves. It is difficult for the farmers and fruit tree professional students to use the correct method for pruning. Therefore, the study of a convenient, simple and intuitive fruit trees pruning practice platform has important application value.Based on the analysis of traditional teaching methods of fruit tree pruning techniques, the kinect sensor was introduced in this thesis. A rapid virtual fruit tree pruning method and the implementation of natural human-computer interaction technology was studied, and a virtual fruit tree pruning system with gesture interaction was developed. The major work and conclusions of this thesis are as follows:(1) Study on rapid pruning method of virtual tree branches.Several commonly used 3D picking methods were analyzed. According to the characteristics of the existing 3D fruit tree model and the demand of picking precision of virtual fruit tree pruning, a method of ray pick-up combined with frame buffer object was proposed. First, the selected stem index was got by creating an invisible virtual screen using frame buffer object, and then the intersection of the selected branch and the pick-up ray was calculated to get the coordinate information of picking point. The collection of pruning branches was got according to the picking point coordinates and topology of fruit tree model.Finally, the branches of virtual fruit tree were pruned by modifying data of the fruit tree model.This method had a high picking accuracy, and it avoided a large number of intersection tests in 3D objects in the traditional ray pick-up method, which can improve the speed of operation effectively.(2) Design of gesture interaction of virtual fruit tree pruning.Nine kinds of interaction gesture were designed according to the virtual fruit tree trim applications and human-computer interaction design principles. The state machine matching was used to recognize types of hand gestures based on skeleton information got by the kinect sensor. The hand joint was transformed from the skeleton coordinate system to the mouse absolute coordinate system. Finally, a mapping relation among hand gestures, mouse events and keyboard events were found to replace the mouse and keyboard interaction with hand gesture interaction. The skeleton smoothing and object snap were used to solve the problem of mouse cursor jitter.(3) Design and implementation of virtual fruit tree system.In this thesis, a virtual fruit tree pruning system with gesture interaction was developed.The virtual fruit tree pruning module and human-computer interaction module were realized by using C++ and C# language in Visual Studio 2010 development platform. The system interface design was designed based on WPF framework. The system included virtual fruit tree pruning practice module, fruit tree pruning knowledge module and system introduction module. The gesture recognition rate and interactive real-time performance of the system were tested. The gesture recognition rate was 97%. The response time of gesture control commands was between 32.5ms to 191.9ms. Experimental results showed that the virtual fruit tree pruning system had the merits on high recognition rate, and it can satisfy the requirement of real-time. The proposed method in this thesis was suitable to the virtual fruit tree pruning.This study provided a new approach for the study of fruit tree pruning technology.