| Text generation models like Chat GPT have shown significant improvement in performance,making text generation tasks highly desirable in the field of natural language processing.Controlled text generation involves extracting feature information from input data and generating corresponding textual representations for downstream tasks.Obtaining excellent latent space representations from input data is crucial for text generation tasks,and the model’s representation learning ability determines the satisfaction and fluency of the generated text to a certain extent.While text generation tasks have achieved excellent results with sufficient training data,obtaining training data corpus in many scenarios can be challenging,resulting in poor latent space representation of the model when acquiring input text under low-resource conditions.Although migration learning of downstream tasks using large-scale pre-trained models makes the models work well under low resources,it neglects the impact of differences in representation structure in latent space on downstream tasks during migration.Therefore,current research focuses on how to obtain more robust and interpretable latent variable information from text with low resources.This paper investigates the latent space text feature representation to address this difficulty.The main work includes:First,the paper focuses on improving the structural representation of text latent variables under low-resource conditions.Specifically,we propose a method that aligns the structural representation of text latent variables to enhance the model’s ability to obtain latent space text features.In the text summarization task,the model may acquire redundant information in the latent representation due to differences between the input text and downstream task.To address this issue,the paper introduces a low-resource text summarization model based on latent space structure alignment.This model converts the input text and target summary in latent space structure representation into a graph matching problem,and aligns the latent space representation distribution of both using the optimal transmission idea.The alignment not only associates the word information related to the summary content in the input text but also considers the contextual relationship segments embedded in the text.The experiments conducted on a low-resource summary task combined with transfer learning demonstrate the effectiveness of the proposed method in improving the model’s ability to obtain key information from the input text and in generating accurate summaries.Second,the paper focuses on the interpretability of the latent space representation.Sampling the common representation between the input text and the target text in the latent representation enhances the model to focus on the common information when extracting features from the input data and improves the interpretability of the latent representation.In machine translation tasks,where the source language and the target language have the same semantic meaning,it is convenient to investigate the interpretability of the text,and currently it is common to use variational encoders to obtain semantic information from the latent representation for translation generation.However,this approach ignores the impact of the difference in semantic structure between the two languages on the translation,and this difference has a more significant impact on the translation under low resources.To solve this problem,this paper proposes a latent space semantic enhancement method.The method improves the model translation capability by reducing the structural difference between the latent space semantics of two languages.The method uses the idea of dual learning to model the translation models of two languages simultaneously,sampling the respective semantic representations from the hidden-space text representations of the two languages.The semantic representations of the two languages are made to learn from each other through constraints to reduce the impact of translation due to structural differences in semantics between the languages and to improve the translation capability of the model.The effectiveness of the proposed approach is demonstrated on two low-resource machine translation datasets,and the results show that the approach in this paper not only outperforms the baseline,but also has significant improvement under the same language model migration.In summary,in this paper,we investigate the information of text representation in latent space,and improve the structure of latent variable representation as well as the interpretability of latent variable representation,respectively.The results show that the quality of text generated by downstream tasks is effectively enhanced by improving the representation of text in latent space. |
