Telecom Complaint Text Classification Based On Adversarial Training And Contrastive Learning

With the intensification of competition in the telecommunications market,telecom operators need to classify customer complaint texts to improve customer experience and avoid customer churn.Due to the complexity of telecom services,the classification of complaint texts requires significant investment of human and time resources.Thus,utilizing text classification models to automatically categorize complaint texts holds significant practical significance.According to different application scenarios,the demands of telecom operators for complaint text classification also vary.On one hand,in production environments where labeled data is relatively sufficient and real-time requirements are not high,telecom operators require higher accuracy of text classification models.However,currently mainstream text classification models have insufficient semantic feature representation and inadequate extraction of text feature information when performing telecom complaint text classification.In addition,these models are easily interfered by noisy data,resulting in weak robustness and generalization ability,thus affecting their classification performance.On the other hand,when labeled data is scarce and real-time requirements are high,telecom operators pay more attention to the model’s inference speed and performance under the condition of less labeled data.However,when labeled data is insufficient,traditional models may not be able to capture enough features and rules,thereby affecting the model’s generalization ability and accuracy.Meanwhile,currently well-performing semi-supervised models generally rely on deep pre-trained models,resulting in slow model inference speed.This thesis analyzes the challenges encountered in two distinct usage scenarios for the telecom complaint text classification task,and proposes corresponding solutions.The specific research topics are as follows:To meet the demands in production environments where labeled data is relatively sufficient and real-time requirements are not high,a supervised BRAC text classification model has been proposed.Firstly,the dynamic word vector representation of text is learned using the BERT model to enhance the model’s semantic feature representation ability.Secondly,the RCNN model is used for feature extraction,obtaining contextual information through the bidirectional recurrent structure,and the important features are extracted through the maximum pooling layer to improve the model’s feature extraction ability.Then,a contrastive learning method is proposed.This method uses adversarial training to perturb the model’s word embedding matrix,generating more challenging and difficult-to-learn adversarial samples,and treats the original samples and adversarial samples as positive sample pairs for contrastive learning.This way,the model can learn to normalize noise-invariant representations,enabling it to distinguish noisy samples,thereby significantly improving the model’s generalization ability and robustness.Finally,experiments on three different datasets demonstrate that the BRAC model has excellent classification performance.To meet the needs of production environments with limited labeled data and high real-time requirements,a semi-supervised SSAC-KD text classification model has been proposed.Firstly,a pre-trained language model BERT is employed as a teacher model.To deal with unlabeled samples,two sets of positive samples are constructed by using dropout and adversarial training techniques,and two rounds of contrastive learning are performed separately.This method enables the model to undergo consistent training on a large amount of unlabeled data,thus constraining its predictions on noisy samples.At the same time,a small amount of labeled data is used for supervised training of the teacher model,allowing the model to learn classification knowledge.Then,in order to further improve the model’s performance on the target task,a large amount of unlabeled data is directly used for knowledge distillation tasks,and labels are added by the teacher model to guide the training of the student model.In this way,the performance of the student model surpasses that of the teacher model.Finally,experimental results show that the model not only demonstrates excellent classification performance with limited labeled data,but also has fast inference speed.