Hand Interaction Based On Optimal Query

The virtual-reality-based flight simulator,using some softwares,builds the virtual environment of the cockpit for various aircrafts to construct a training platform,which can accomplish many flight training tasks with low using and maintence cost.The virtual helmet used in cockpit for visual scene display,hinders the hand interaction due to the visual obstruction from itself,which has become the bottleneck for the development of this VR simulator.In the field of human-computer interaction,the technology of vision-based hand interaction has been widely researched and applied due to its comfortability and naturalness.Meanwhile,the vision-based method is more easily affected by the external environment and the object itself,which influences the performance of interaction.This paper researched mainly on the hand tracking and gesture recognition included in the hand interaction and proposed a technology of hand interaction based on optimal query for increasing their robustness,by the analysis of the properties and problems among the interaction.Hand tracking belongs to the domain of object tracking,the complexity of environment and the variousness of object have brought many challenges for the robust and real-time tracking.This paper proposed an algorithm based on the optimal query for multiple features in the framework of correlation filter.It aims to improve the adaptability for the change of environment and object itself,with utilizing the multiple features for filters training.And this paper utilized the Pareto-Optimality method to fuse the multiple filter responses for the optimum.Moreover,against the possible failure during the tracking,this paper proposed a mechanism based on the keypoints match for object redetection.When judging a tracking failure,the global object redetection is implemented,using the updated keypoint database,to correct the tracking result,for preventing the error accumulation.Compared with multiple trackers on the various scenes,the experiment shows the robustness of proposed tracking algorithm.Because of the high freedom,there are the self-occlusion problem and the issue of the huge difference between the varieties of the same gesture during the gesture recognition.Thus,this paper proposed a recognition framework under orthographic double views,based on deep learning network.The gesture features are extracted through the deep network and the shallow and deep convolution features are concatenated by one dimension-reduction and fusing method,which aims to get more robust features.The gesture images from two views are inputted into the two single recognition networks for the class-probability vectors,and the final result is obtained with the Pareto-Optimality method to optimize the two vectors.It is seen that the proposed method can improve the recognition accuracy effectively from the experiment result.