Study On Action Evaluation Based On Adaptive Joint Weighting And Interpolation Wavelet

Embodied action similarity evaluation has gained an increasing attention in the virtual teaching of agriculture, rural security guard, natural human-computer interaction and medical rehabilitation projects. The embodied action similarity evaluation system usually consists of three flows:motion capture, action pre-processing and action similarity calculation. The motion capture refers to the records of human actions with hardware facilities and software, and the action pre-processing refers to the format conversion and noise reduction of the action, and the action similarity calculation focus on obtaining the similarity of the reference action and comparison action by calculating the distance between two actions. Therefore, this paper explores the methods of similarity calculation of the embodied action based on human skeleton structure extracted from the depth camera and the reference action in the template library is explored, so as to lay a foundation for its application scenarios.In this paper, a set of embodied action similarity evaluation method based on human skeleton structure is proposed. It covers two parts, including the pre-processing of action data and similarity evaluation. At last, a prototype system was implemented based on these methods. More details as following:(1) Pre-processing of action. This paper proposed the noise reduction method based on the combination of Faber-Schauder interpolation wavelet and mean filter. The noise reduction of actions was implemented by constructing the adaptive interpolation basic function, and its convergence has been proved. The interpolation wavelet defined in the Banach space has the features of rapid operational speed, small boundary effect and integrity keeping of action, but its interpolation property keeps both the action details and some noises. This method made use of the interpolation property of the interpolation wavelet, which could keep the details of actions especially for drastic actions, and lower down some noises kept by the interpolation property through mean filter, so improve the noise reduction effect of the action. In addition, Faber-Schauder basis function is compactly supported, which could improve the calculation efficiency effectively. The experiments choose different types of noises added into different types of actions. The results showed that this method has stable noise reduction effect for most actions, and it has better noise reduction effect for irregular and drastic actions.(2) Adaptive joint weight action similarity evaluation based on DTW. This method firstly segmented the comparison action and reference action, assigns higher weight for joints with drastic action, and the rest joints are assigned with equal weight. Later, with the joint cascade quaternion and joint weight as the feature vector, the distance between two actions were calculated based on DTW. Finally, the k-nearest neighbor classifier was used to estimate the accuracy of the method for action evaluation. Classification accuracy was employed as the action evaluation accuracy. The results showed that this method has high classification accuracy and could distinguish most actions, includes some similar actions.(3) Action similarity evaluation method based on the key frame extraction by interpolation wavelet. In this paper, key frames of each quaternion component sequence of the drastic joints in reference action were extracted based on multi-scale Faber-Schauder interpolation wavelet and difference interval interpolation wavelet. Later, each quaternion component sequence of key frames was combined, and the threshold was chosen for eliminating the key frames with high similarity. Finally, the comparison action and reference action were matched through DTW method, and the key frames of comparison action was obtained, and the action similarity was obtained after the normalization of average distance of key frames. The multi-scale feature of interpolation wavelet could extract key frames from different scales according to the drastic degree of actions, calculate the drastic part in the action much more precisely, so as to obtain more accurate action similarity evaluation. The results showed that this method has high classification accuracy and more precise action similarity evaluation.(4) The action similarity evaluation method based on the interpolation wavelet key frame extraction and adaptive joint weight. Adaptive joint weight action similarity evaluation mainly improved the action feature of the human skeleton structure, and the action similarity evaluation method based on the interpolation wavelet key frame extraction mainly improves the temporal feature of action. These two methods improved the action features in different dimensions, so this paper proposed a method to combine both two methods together. The key frames of the action are extracted based on interpolation wavelet, then the reference action and comparison action were matched through DTW, in which the frame distance of two actions is calculated with the adaptive joint weight method. Finally, the action similarity was obtained by the normalization of the average distance between matched key frames of the two actions. The results showed that the combination of the adaptive joint weight method and the interpolation wavelet key frame extraction method could achieve higher classification accuracy than both two methods.(5) Prototype system based on the action similarity evaluation method was proposed. A motion sensing interactive application system oriented for rural public cultural activities based on the adaptive joint weight and interpolation wavelet key frame extraction was established as the prototype system. The system chose the popular square dance as the theme under the rural public culture background, and captured the real-time human action with ASUS Xtion. Users could dance by following the standard dance movement of virtual roles, meanwhile the application evaluated the performance of action of users in real-time. Compared with the traditional square dance, the application allows users to know the performance of their dance, with strong interactions and high illusion of immersion. In addition, this system is a digital product, so the contents can be shared rapidly and utilized repeatedly, and can be updated according to user requirements.