Research And Application On Head Pose Estimation In Natural Environment

Head pose estimation (HPE) is important in artificial intelligence and computer vision. Recently, more and more researchers have begun to pay close attention to head pose estimation. Head pose is the key to the study of human behavior and attention. In class, head pose is also the key to understanding students’learning performance, and is the important content of class information extraction in the modern intelligent classroom teaching management system. In the era of big data for education, a modern classroom teaching management system not only has the functions of a traditional teaching management system, but also provides new function modules such as classroom behavior extraction, intelligent monitoring, appraisal decision etc. Head pose estimation is an important part and technology to realize information extraction and intelligent monitoring, and to achieve the basic objective for decision appraisal.In computer vision, head pose estimation is the process of inferring the orientation of a human head from digital imagery. The objective of head pose estimation is to obtain the parameters of head pose using the methods of image and pattern recognition in a global coordinate system, which includes head pose orientation and position. Due to its practical signification and challenges, there is a fair amount of work developed fast and reliable algorithms for head pose estimation. However, most of the work has reported good results in constrained environment, the performance could be decreased due to the high variations in unconstrained natural environment, especially in a wide classroom scene, such as, facial appearance, poses, illumination, occlusion, expression, motion interaction and make-up.In a practical application of teaching management, head pose estimation for intelligent system needs efficient and robust performance under the unconstrained natural environment. And multi-person head pose estimation and analysis in a natural environment is difficult and important. In this paper, in order to develop a modern intelligent teaching management system based on head pose estimation and analysis, concretely, the research objective is to estimate head pose in practice under the natural environment. Our research content includes four aspects, (1) Discrete head pose estimation using a hierarchical learning algorithm in unconstrained environment; (2) Robust head pose estimation based on hybrid feature and weighted vothing decision in a wide scene; (3) Continuous head pose estimation using a iterative regression approach in natural environment; (4) Class teaching management system development based on head pose estimation and analysis in a classroom.In order to extract learning behavior in a classroom, this paper firstly researches on head pose estimation under the natural environment, especially the wide classroom, and presents a modern intelligent teaching management system based on head pose estimation and analysis. Our contributions are given in the following,(1) A Dirichlet-tree distribution enhanced Random Forest approach (D-RF) is proposed in this paper to estimate head pose efficiently and robustly under various conditions. And facial positive/negative patch classification is proposed to eliminate the influence of noise in natural environment. The proposed method has been evaluated with different data sets spanning from-90° to 90° in vertical and horizontal directions under various conditions. The experimental results demonstrate the approach’s robustness and efficiency.(2) In order to eliminate the influence of occlusion and noise in a wide scene, more powerful combined texture and geometric features (i.e., Gabor feature-based PCA, Sobel, LBPH and two geometric features) are extracted to estimate head pose. The performance shows that it has improved the accuracy and efficiency of the D-RF approach.(3) We proposed an iterative regression D-RF approach for continuous head pose estimation in natural environment. The approach includes two steps. Firstly, cascaded D-RF approach is used to detect facial feature in natural environment under various head poses; Secondly, the continuous head pose is updated based on the estimated facial features, iteratively, the facial feature localization is refined based on the updated head pose by the proposed iterative regression D-RF. The experimental results demonstrate that the proposed approach is robust and efficient for head pose and facial feature detection.(4) We present a class teaching management system based on head pose estimation and analysis in a natural classroom, which is based on three modules, i.e., intelligent attendance management, teacher-student (T&S) communication, head pose estimation and analysis. And a novel VFOA model is proposed to recognize multi-person VFOA simultaneously based on a monocular camera from estimated head pose. In order to evaluate our approach and system, we collect a new available database on the natural classroom. Experimental results show that our proposed system and method can provide an objective and concise information needed for multi-student VFOA monitoring and learning interaction analysis.