Research On Any-wall Touch Technology Based On 3D Sensor

At present,human-computer interaction technology based on the big screen has drawn more and more people's attention.Large-screen human-computer interaction technologies nowadays are mainly divided into two types: a voice-based human-computer interaction technology and a machine vision-based human-computer interaction technology.Voice-based human-computer interaction technology is often limited by complex and noisy environments,so the applications are relatively small and mostly applied in the field of small-scale human-computer interaction.The human-computer interaction system based on machine vision is mainly based on the sensor and other equipment to capture images or signals,and then the interactive software is employed to capture the image or signal processing,and then the target from the background is segmented.Then a series of operations as target tracking,gesture recognition and so on are applied to complete the corresponding human-computer interaction.Therefore,the cost of human-computer interaction based on machine vision is relatively low,and the interaction effect is well obtained.Therefore,human-computer interaction technology based on machine vision is widely used in large-screen human-computer interface.There are three major issues about the large screen human-computer interaction system currently.The first is how to get the exact mapping relationship between the projector and the camera.The second is how to accurately segment the dynamic gesture area when there exists complex areas,and locate and track dynamic gestures accurately.The third is how to effectively filter the interference in the depth image collected by the interactive sensor.Based on the above issues,a human-computer interaction method based on Kinect sensor is presented in this paper,which can achieve remarkable human-computer interaction effect on any wall.The method utilizes a sensor to capture gestures or infrared rays emitted by an infrared pen.After transmitting the infrared signal or gesture signal information to a computer for filtering,such human-computer interaction can be achieved.The method is easy and simple to operate,whose interaction effect can be greatly guaranteed.The main innovations in this paper are as follows:(1)The current major gesture tracking algorithms can not overcome the problem of skin-color interference tracking in complex environments well.Therefore a gesture tracking method based on mean shift algorithm combined with particle filtering is proposed in this paper,which takes into account both the real-time and accuracy of tracking and tracking.Canny operator is employed to segment the area of the gesture,and then the ellipse window skin color model is created for hand gesture tracking,and finally a federated filter which combines particle filter and mean shift algorithm is used to track the dynamic gesture model.This algorithm can be a good way to overcome the tracking dynamic gesture which contains color interference problem.(2)In the depth image acquired by the sensor,there are not only the spots of the infrared pen required by the system,but also the spots formed by sunlight.Therefore it is necessary to filter the depth image.For the interference noise in the depth image collected by the sensor,a joint filtering algorithm based on dynamic adaptive filtering and morphological filtering is designed in this paper.The original collected data is processed firstly and processing is divided into two parts.The first part is dynamic adaptive filtering that facilitates the elimination of external noise.The second part is the image erosion operation of the preprocessed data,which makes the image smooth and removes the rough edges of the image.