The Research Of Projection-floor Interactive System Based On Binocular Vision

With the development of Human Computer Interaction Techniques and Virtual Reality Technology,the demand for patterns of human-computer interaction has become more diverse,and the research and application of an interactive system based on camera-projection in human-computer interaction has become more and more extensive.The implementation of current camera-projection interaction system is using infrared camera and infrared laser to assist location.Although the principle of this plan is simple,it is susceptible to the interference of environmental lights.To solve this problem,this paper designs and implements an interactive floor projection system based on binocular vision,using binocular camera to achieve object detection instead of locating by infrared camera and infrared laser.The main research work of this paper includes three parts:(1)The research and implementation of real-time foot detection algorithm.The biggest difficulty of the camera-projection interactive system is the positioning and recognition of interactive objects in the projection scene.And the deep convolutional neural network applied to the field of object recognition has obvious advantages such as high real-time performance and high accuracy.This paper focuses on YOLO target detection algorithm based on the convolutional neural network,and analyzes its shortcomings,such as the training speed is slow and the ability to distinguish pictures details is poor,and corresponding improvements are proposed for these deficiencies,including batch normalizing data,adding Inception model to the network,and improving gradient functions,etc.Finally,using the improved YOLO algorithm to realize the identification and positioning of feet.(2)The estimation of toe coordinates and toe coordinates mapping from image coordinates to PC mouse coordinates.On the basis of the implementation of foot detection,a method of estimating toe coordinates is designed by combining the orientation information of the feet with respect to the camera,and based on the principle of linear interpolation,a coordinate mapping method is designed to achieve the coordinates of the toe in the image coordinates mapped to the coordinates of the PC mouse.In this way,the mouse API is invoked to trigger a mouse event to implement user interaction with the system.(3)The implementation of interactive floor projection system.Under the Linux and QT development environment,this paper uses C++ and OpenCV to program and implements the synchronous acquisition of binocular cameras through multi-thread technology.And combining the position and depth information of toes to realize the recognition of the foot actions based on the principle of binocular vision.With the actions are as a correspondence of mouse events,a fully functional set of projection-floor interactive system is implemented.Experimental results show that the interactive floor projection system designed by this paper can detect the movement of user's feet in real time,regonize the defined foot actions,and reduce the interference of environmental lights.And it demonstrates that the system has good real-time performance and high robustness.