Multimodal Unsupervised Domain Adaptation Model For Time Series Classification

Time series classification is a very important task in machine learning and data mining.With the popularity of deep learning,deep neural networks have been widely used in temporal classification tasks.But the deep learning framework requires a large amount of labeled data to train the model.However,it is difficult to collect all the labels of the data in some real data scenarios.In addition,the training data may be different from the actual data distribution,making it difficult for the models trained on a specific data set to better migrate to the new scenarios with different data distribution.Unsupervised domain adaptation is an ideal transfer learning method that can utilize labeled source data to improve the classification performance of unlabeled target data.However,the current time series classification method based on unsupervised domain adaptation uses only the time domain or the frequency domain data as input,neglecting to fuse them,leading to the problems of insufficient information extraction and inaccurate alignment of the target domain distribution in the source domain.This paper proposes a multi-modal unsupervised time series classification method,which mainly includes two parts: Multimodal Domain Adversarial Network(MDAN)and Time-Frequency Joint Maximum Mean Discrepancy(TF-JMMD),to solve the problem of multi-modal feature extraction and insufficient application from two aspects.On the one hand,the MDAN model aligned the distribution of source and target domains from two directions: temporal and frequency modes.It can learn the time-frequency joint distribution of data,fully extract and utilize the multimodal features of temporal data,align the source domain target domain more effectively in the process of adversarial training,and improve the accuracy of unsupervised classification of the target domain.On the other hand,the combined maximum mean difference(Joint Maximum Mean Discrepancy,JMMD)is improved to the TF-JMMD.The role of TF-JMMD is to calculate the difference between the source domain and the target domain,and it can mine the deep association of the time series in different feature spaces.It makes full use of the time-frequency domain characteristics of time series data.By reducing the distance measure,it can further shorten the distance between the two domains from the three directions of time domain,frequency domain and time and frequency domain combined,so that the target domain can share a large amount of effective information of the source domain.Finally,the experiment is compared on UCR and other real datasets.The experimental results show that the proposed multimodal domain adaptation method outperforms the traditional unimodal domain adaptation method in the time series classification task,which proves that this method can effectively improve the accuracy of the time series classification task.