Research On Applicability Retrieval Technology For Motion Capture Data

Three-dimensional(3D)human animations created using motion capture(MoCap)technology,with their realistic,delicate,and rich expressiveness,have been widely applied in fields such as film,gaming,sports simulation,virtual and augmented reality.The proliferation of MoCap technology has led to an accumulation of a vast amount of motion data,presenting challenges in managing this data.Research into content-based MoCap data retrieval techniques has become particularly vital to efficiently display and reuse the existing data.Similar to many retrieval tasks,the solution approach to MoCap data retrieval involves designing and extracting features from motion data,and obtaining query results based on the similarity ranking of these features.Currently,most MoCap methods simplify or limit the problems they address during this process,such as requiring manual intervention or extensive data annotations,using only homogeneous data with the same skeleton and data structure,or retrieving only segmented complete motion sequences.Although these methods have achieved impressive results,these constraints mean they don’t fully address the challenges encountered in real-world applications.Additionally,due to the barriers posed by MoCap systems,obtaining the desired query samples isn’t easy for most ordinary users.The aforementioned issues lead to the following requirements for enhancing the applicability:(1)fully unsupervised techniques that lower user barriers;(2)sub-sequence retrieval that supports partial similarity matching;(3)retrieval of heterogeneous data with different skeleton structures,data structures,and motion subjects;and(4)cross-modal retrieval that adaptive to different data modalities.In response to the varied requirements for applicabilities mentioned above,three applicable methods for MoCap data retrieval is proposed in this thesis.Specifically,the main work and contributions of the thesis are as follows:1.Addressing the issue of sub-sequence retrieval,an applicable retrieval method via unsupervised posture encoding and strided temporal alignment(PESTA)is proposed.It effectively retrieves unsegmented and unlabeled data at the sub-sequence level,achieves robustness against singular frames and enables control of tradeoff between precision and efficiency.It firstly learns a dictionary of encoded postures utilizing unsupervised adversarial autoencoder techniques and,based on which,compactly symbolizes any MoCap sequence.Secondly,it conducts strided temporal alignment to align a query sequence to repository sequences to retrieve the best-matching sub-sequences from the repository.Further,it extends to find matches for multiple sub-queries in a long query at sharply promoted efficiency and minutely sacrificed precision.Outstanding performance of the proposed method is well demonstrated by experiments on two public MoCap datasets and one MoCap dataset captured by ourselves.Furthermore,the framework of the method can be easily extended to time series retrieval tasks on other datatypes.2.Addressing the issue of heterogeneous MoCap data retrieval,a motion signature based applicable retrieval method of MoCap data is proposed.It works on MoCap data of arbitrary subject types and arbitrary marker attachment and labelling conventions.Specifically,a novel motion signature is proposed to statistically describe both the high-level and the low-level morphological and kinematic characteristics of a MoCap sequence,and the content-based retrieval is conducted by measuring and ordering the motion signature distance between the query and every item in the database.The distance between two motion signatures is computed by a weighted sum of differences in separate features contained in them.For maximum retrieval performance,a scheme to pre-learn an optimal set of weights for each type of motion in the database through biased discriminant analysis is proposed,and a good set of weights for any given query is adaptively chosen at the run time.Excellence of the proposed method is experimentally demonstrated on three specialized datasets.Moreover,if the step of feature weighting is omitted,the method is able to operate in a completely unsupervised mode.3.Addressing the issue of cross-modal retrieval,an algorithmic framework is proposed for retrieval of MoCap data given a video query.The skeleton animations is reconstructed from video clips by 3D human pose estimation to narrow the representation gap between videos and MoCap data.A statistical motion signature is computed to extract features from the skeleton animations and the MoCap sequences uniformly.This as well ensures that the proposed scheme works on MoCap data with arbitrary skeleton structures.The retrieval is achieved by computing and sorting the distances between the motion signature of the query and those of the MoCap sequences which are pre-computed and stored in the MoCap database.In the experiments,a dual-modality dataset,concurrently capturing videos and MoCap data,is utilized to validate the effectiveness of the algorithm and to assess the performance degradation due to cross-modality.Furthermore,this framework offers flexibility;by swapping out 3D human pose estimation algorithm,improvements in performance can be garnered,or support can be extended to heterogeneous data of varying motion subjects.