Image Material Recognition Via Hierarchical Multi-feature Fusion And Ensemble Learning

Material information often exists on the surface of objects or scenes.It contains high-value semantics.Hence,image material recognition is a basic problem in the field of computer vision,and it has high theoretical and practical values.This paper focuses on the research of image material recognition and uses three methods to complete the image material recognition:correlation analysis models,ensemble learning strategies,and hierarchical multi feature fusion strategy.The main work is shown as follows:(1)Image material recognition based on correlation analysis models:Aiming at the problems of large apparent variations,high similarity between classes and large differences within classes in image material recognition,this paper adopts the early feature fusion method called correlation analysis models including CCA,grad KCCA and DCA to mine the relevant information among image features.Cross modal semantics(CMS)is generated in turn.Finally,nine general classification models are employed to complete the material recognition.Experimental results show that:among these three correlation analysis models,the DCA model performs best and is more suitable for image material recognition,and the real-time efficiency of DCA?CMS is the best,but compared with a single feature,the corresponding recognition accuracy needs to be improved.(2)Image material recognition based on discriminant correlation analysis model and ensemble learning strategies:Based on the best DCA?CMS and the above classification models,ensemble learning methods such as hard voting,soft voting,and stacking are introduced.The implicit correlations among different models are fully considered,the predictions of each model are ensembled together to complete the final image material recognition.Experimental results show that the performance of these ensemble learning strategies can be improved in terms of average accuracy;the performance of the two voting strategies has their own advantages:on the coarse-grained dataset,they can all improve recognition accuracy,which means that the predictions of each classification model can be balanced when the voting strategy is utilized.However,on the fine-grained dataset,the peak performance of these ensemble learning strategies decreases.This is because the DCA model only focuses on mining the correlations between two types of heterogeneous features whereas ignores the potential complementary information among multiple image features.(3)Image material recognition based on hierarchical multi feature fusion and ensemble learning strategy:By improving the traditional MDCA model and adopting a hierarchical multi-feature fusion strategy with an ensemble learning strategy,we propose the HMF~2 model for image material recognition.Experimental resuls show that the HMF~2 model is a comprehensive model with high effectiveness,efficiency,robustness,and simplicity.Compared with the MDCA model,the HMF~2 model can make full use of the correlations among heterogeneous features and increase the key discriminant information in the cross modal semantics,which helps to improve the recognition performance.Our recognition results are better than those mainstream baselines,even simple linear classification models can achieve high recognition accuracy:excellent results were obtained under both types of cross-validation methods,On the coarse-grained dataset,the HM8-top5-soft combination performed best,and on the fine-grained dataset,the HM7-top7-hard combination performed best.In addition,the proposed HMF~2 model contains a small amount of parameters without large-scale parameter modulation.