Research On Food Safety Emergency Cross-media Information Semantic Analysis And Classification

At present, China’s food safety emergencies happen frequently, which results in a rapid growth of the relevant information on the Internet. The form of these data has the cross-media characteristics. Images, as the main part of such data, become an important source of the relevant information for the public due to their richness of semantics. Although the traditional semantic analysis and classification techniques have achieved certain progress on semantic understanding of images, there are still many problems to be solved in the aspects of the quantitative representation and the feature selection of images. The cross-media information with the semantic background of food safety emergencies has heterogeneous feature spaces and closely related semantic spaces. How to use these cross-media information with a multi-modal form to mine the latent semantic relationships, as well as to transfer knowledge between different modal data, proposes new challenges to the semantic analysis and classification tasks. Thereby, it could help the understanding of the semantics contained in the images. Consider the problems of quantitative representation and feature selection of images, mining the latent semantic relationships and transferring knowledge between heterogeneous data sources, this dissertation studies on the semantic analysis and classification techniques for the cross-media information of the food safety emergencies from three aspects such as image quantitative representation, semantic annotation and semantic classification. The main contributions and innovations of this dissertation are as follows:(1)This dissertation proposes a Distance Optimization based Locally Linear Embedding (DO-LLE) feature dimension reduction algorithm, which could solve the difficulty in determining the neighbor point parameters and instability of the dimension reduction results. Regarding the difficulty in determining the size of the visual vocabulary, it proposes a feature Adaptive Clustering algorithm (AC). The AC algorithm could adaptively generate the optimized cluster centers through the iterative computation on the features set according to the max-min distance rule and the Davies-Bouldin Index (DBI). The DO-LLE algorithm and the AC algorithm are verified via the image classification experiment on the Lazebnik Schmid and Ponce (LPS) image classification dataset. The DO-LLE algorithm gains an average classification accuracy result of81.9%which is3.3%more than the traditional LLE algorithm. The AC algorithm could adaptively determine the number of feature clusters (the number is298), and its average classification accuracy is82%, which gains an average classification accuracy promotion of10.7%,5.4%,0.3%and1.1%compared to the man-made cluster numbers of100,200,300and400, respectively. The experimental results show that the DO-LLE algorithm and AC algorithm could effectively enhance the image semantic classification performance.(2) In order to solve the effective quantitative representation problem of images, it proposes a Bag-Of-Visual-Words model based Robust Image Representation approach (BOVW-RIR). In order to enhance the characterization of the low-level image features, it fuses the Speed Up Robust Feature (SURF) and the Multiresolution Histogram Moment feature (MRHM) to generate a low-level compound feature, which contains the local information and the structural information of each image. It successively does feature dimension reduction and feature clustering. It uses the feature clustering results to construct the visual vocabulary and represent each image using the visual vocabulary. The BOVW-RIR approach achieves the average classification accuracy results of89.1%,83.9%and82%on the Oliva and Torralba (OT), the Fei-fei and Perona (FP), and the LPS image classification datasets, respectively. It also achieves an average classification accuracy result of70%on the food safety image dataset. The BOVW-RIR approach is experimentally compared with three famous image representation approaches (Fei-fei, Bosch, Lazebnik) on the OT, FP and LPS datasets. The BOVW-RIR approach gains a classification accuracy promotion of4.4%,6.6%and7.3%on the OT, FP and LPS datasets compared to the best of the three comparison approaches respectively, which shows that the BOVW-RIR approach could more effectively make quantitative representation of each image.(3) This dissertation proposes a Latent Semantic Topic Weighted Fusion based Image Semantic Annotation model (LSTWF-ISA). This method models the latent semantic topics for the semantic keywords and the image visual words of the training data, in order to get the latent topic distributions of the textual modal data and visual modal data. It utilizes the entropy of the visual words distribution to calculate the weighting parameter, and fuses each latent topic distribution of the textual modal data and visual modal data through this weighting parameter. It generates the fusion latent semantic topic distribution, and establishes the LSTWF-ISA model based on this distribution. The LSTWF-ISA model achieves the average F-measure results of0.71and0.22on the49best words subset and the most commonly used263words subset of the Core15K dataset, respectively. It also gets the average F-measure result of0.36on the image annotation dataset oriented for the20categories of the food safety emergencies. The LSTWF-ISA model is experimentally compared with four well-known image semantic annotation models (TM, CMRM, CRM, and PLSA-WORDS). The signed-rank test is used to check the comparison results. The LSTWF-ISA model gains an average F-measure promotion of11%and29%on the49best words subset and the most common used263words subset of the Core15K dataset compared to the best of the three comparison approaches respectively, which shows that the LSTWF-ISA model could enhance the annotation performance by using the latent semantic relationship between the textual modality and visual modality.(4) This dissertation gives out the formal definition of the text-image co-occurrence data, which describes the characteristics of the documents containing both texts and images within the event constraint. A Text-Image Feature Mapping (TIFM) algorithm is proposed, which conducts the feature mapping from the text feature space to the image feature space based on the text-image co-occurrence data. The proposed TIFM algorithm gains an average Euclidean distance of0.024, an average cosine similarity of0.84and an average K-L divergence value of0.075between the approximated image feature distribution and the ground truth image feature distribution on a text and image dataset oriented for20categories of food safety emergencies, which shows that the TIFM algorithm could effectively map the text feature distribution into the image feature distribution.(5) In order to solve the problem of using text data to aid the image semantic classification, this dissertation proposes a Feature Transferring based Image Semantic Classification (FT-ISC) approach. Regarding the calculating difficulty of the text features caused by their massiveness and sparseness, it proposes an Information Gain based Text Semantic Feature Selection (IGTSFS) approach, which figures out the effective text topic features by computing the information gain of each latent semantic topic extracted from the text data. The proposed FT-ISC model achieves an average classification accuracy result of76%over the whole food safety dataset and a classification accuracy result of86%on the "Crayfish" food safety semantic category. The FT-ISC model is compared to the Bayesian classification model and the Label Query classification model on the food safety dataset. The FT-ISC model gains a classification accuracy promotion of8%and5%compared to the Bayesian classification model and the Label Query classification model, respectively.(6) This dissertation proposes a Feature Weighting based Image Semantic Classification (FW-ISC) approach. This approach combines the Filter-based Feature Weighting mechanism (FFW) with the support vector machine to achieve image semantic classification. To solve the problem of the uneven distribution of features and the difficulty in measuring the closeness of the features with semantic categories in the feature weighting procedure, it proposes a Conditional Mutual Information based Feature Diversity Measuring approach (CMIFDM). It calculates the weight values of the features of each semantic category through iterative computation. It designs the feature weighting kernel function and combines the FFW feature weighting mechanism with SVM. The feature weighted results are used in the training procedure of this classifier. The FW-ISC approach achieves an average classification accuracy of87%and75%on the LPS image dataset and the food safety image dataset, respectively. The FW-ISC approach is compared with the traditional support vector machine classifier. The signed-rank test is used to check the experimental results. The experimental results show that the FW-ISC method gains a classification accuracy promotion of5%and8%compared to the traditional support vector machine classifier on the LPS dataset and food safety dataset respectively, which indicates that the FW-ISC method could effectively enhance the image semantic classification performance.