Research And Application Of Sensitive Information Detection Based On Multimodal Feature Fusion

With the continuous development of social networks,the content that users share on social networks is becoming more and more diverse,not only limited to text,but also involves multi-modal data,including some inappropriate,sensitive or illegal content,including but not limited to pornography,violence,hatred,terrorism,harassment,etc.It has a serious impact on the security and healthy development of the Internet.Therefore,detecting and processing multimodal sensitive information in the Internet is of great significance for ensuring Internet security,maintaining public order and protecting user interests.The biggest problem of traditional detection methods is the lack of semantic understanding of multimodal information fusion,which gives criminals some opportunities to bypass.The current deep learning algorithm has a good feature extraction and feature fusion mechanism,and has achieved good results in natural language processing,computer vision and other fields.In this thesis,we mainly use deep learning to extract and fuse multi-scale features of text and image to detect multimodal sensitive information in social networks.The main research contents are as follows.(1)Research the text information detection method that integrates the context semantic tendency feature.In the feature extraction stage,on the basis of the traditional Word2 Vec semantic feature method,improve the understanding of semantics,propose to extract the context semantic tendency feature,and combine the long-term and short-term memory network and the self-attention mechanism to obtain the semantic tendency classification model.This model classifies and predicts the text semantic tendency,and then obtains a two-classification result.On this basis,a sensitive information detection model integrating semantic orientation features is proposed.The text detection results and the classification results of semantic orientation features are fused to obtain the final detection results.Experiments show that this method is superior to the traditional sensitive information detection method.(2)The sensitive information detection method integrating multimodal features is studied.On the basis of(1)research,the understanding of image semantics is added,the label word vector of the image is extracted using VGG-19 network model,the BoVW feature of the image is extracted using SIFT algorithm,and the above features and text features are fused and classified to realize the detection of multimodal information.According to the actual needs of the project and the actual application scenario,the sensitive information detection subsystem is designed and implemented,which mainly includes user management module,data processing and analysis module and humancomputer interaction module.The user management module has three main functions:user login,user information management and user authority management;The data processing and analysis module is mainly the application of the sensitive information detection model that integrates multimodal features to realize the training and call of the model,and realize the sensitive information detection;The human-computer interaction module mainly displays the detection results and historical record query information to users.