| In view of the unbalanced development of educational resources in different regions and the shortage of high-quality teachers in poverty-stricken areas,an intelligent pan-tilt human-computer interaction system based on head-eye posture,which can be applied into distance education is designed in this paper.Based on the common camera on the teacher end,the real-time head-eye posture information of the remote teacher is obtained.The data is sent to the pan-tilt on the student end to drive the pan-tilt to track and rotate to the corresponding field of view in real time,and the real-time status value is returned to the upper computer on the teacher end to realize the human-computer interaction between the remote teacher and the pan-tilt.The system designed in this paper is composed of the data collection terminal and the pan-tilt interaction terminal,which simulates the teacher end and student end in the remote education.The main research work of this paper is as follows:Firstly,the facial feature points are located at the data acquisition terminal.In this paper,the face alignment method of ERT(Ensemble of Regression Trees)is used to extract the feature points of real-time frames,and the coordinate information of feature points is stored in the point set.Secondly,by traversing and filtering the face contour area,the target object is selected,and the head pose of the target object is estimated.Based on the 3D-2D pose estimation method,six representative feature points are selected from the feature points set,and the 2D face image is matched with the standard3 D face model.According to the conversion relationship between 3D feature points and corresponding 2D points,the real-time head pose data is obtained.Then,the head-eye offset relationship is obtained.In this paper,according to the coordinate information of the feature points of the eye,the eye area of interest is cropped to obtain the smallest rectangular image of the eye.Histogram equalization,morphological processing,contour extraction and other methods are combined to obtain the largest connected area of the iris.Based on the principle of image moments,the center coordinates of the smallest rectangle of the eye and the center of gravity coordinates of the largest connected area of the iris are calculated,the head-eye offset vector is obtained,and the combined head-eye posture information is output.Finally,the head-eye posture output by the data collection terminal is combined with the data for processing.The first step,the data is converted into decimal and the format of the output data is defined.The second step,the pan-tilt control interface function is called to output control commands to the interactive terminal to drive the pan-tilt rotate biaxially to the specified viewing angle.The last step,the pan-tilt interactive terminal returns the real-time value to the upper computer of the data acquisition terminal,and adjusts the step variable according to the deviation between the returned value and the sent value to obtain the real-time video of the corresponding viewing angle.In order to verify the effectiveness of the algorithm proposed in this paper,multiple experimenters are selected to conduct interactive experiments and evaluate the accuracy.The experimental data shows that the average recognition rate of the head-eye pose combination algorithm can reach 98.33%,and the average accuracy of the PTZ interactive experiment can reach 94.35%.The experimental results show that the intelligent pan-tilt human-computer interaction method based on the combination of head-eye posture proposed in this paper can achieve better interaction results under different posture conditions.The method designed in this paper breaks the limitation of the field of view in a remote space,allowing the experiencer to freely obtain a multi-dimensional full-view video according to their own posture transformation,and can be combined with more application scenarios such as education,medicine,and transportation in the future. |
PDF Full Text Request