Research On Text Understanding And Generation Techniques Based On Ethical Lexicon

With the rapid development of artificial intelligence(AI)technology,more and more companies and organizations are applying AI to various domains,ranging from finance,education,healthcare,agriculture to manufacturing,among others.The rapid advancement and widespread application of AI technology have accelerated the transition of the economy and society towards intelligence,bringing numerous conveniences to human life and production.However,as the scope of AI application continues to expand,the emerging ethical issues have drawn extensive attention from various sectors of society.Natural language processing,as an integral part of AI,has become one of the research hotspots in the field of AI ethics,focusing on the identification and handling of ethical issues present in textual data.The utilization of pre-trained models trained on large-scale corpora and fine-tuning them in the domain of ethics can effectively facilitate the learning of ethical concepts and connotations embedded in text.Despite the substantial advantages demonstrated by pre-trained models in natural language processing tasks,their performance falls slightly short when faced with complex ethical issues.To address this concern,this paper proposes a solution based on enhancing pre-trained models with ethical knowledge,aiming to reinforce their understanding of ethical concepts through external ethical knowledge.The main research objectives of this paper are outlined as follows:(1)A Chinese ethics lexicon containing ethical polarity and ethical strength is constructed.In view of the lack of publicly available Chinese ethics knowledge bases,an ethics lexicon containing 33 k words is constructed.On the basis of labeling ethical polarity,ethical strengths of all words is calculated using similarity measurement techniques,achieving fine-grained segmentation.(2)An ethics lexicon adapter(ELA)for Transformer-based pre-trained language models is proposed.As a underlying encoder,ELA can flexibly and efficiently integrate the ethics lexicon into the bottom layer of pre-trained language models,thus enhancing the sensitivity and accuracy of models in recognizing ethical content without retraining the entire model.(3)Machine learning models for various ethics-related tasks such as ethical behavior classification,ethical story understanding,and ethical generation are proposed.Starting from the practical needs of improving the performance of NLP models,ELA-based enhancement schemes for representative pre-training models such as BERT,T5,and GPT-2 are proposed.(4)Comprehensive experimental comparisons between the newly proposed models and SOTA models are conducted on the ETH-News and STORAL-ZH datasets.Experimental results demonstrate the rationality and effectiveness of the newly proposed solutions in enhancing the ethical understanding and application capabilities of KEPTMs.