| Material recognition has very broad application prospects,such as clothing recognition,automatic picking by a robot,industrial inspection,etc.,so it has high theoretical and application value.However,the material image may be prone to visual variations and resulting in different visual characteristics affected by different conditions,such as the light intensity,shooting angle of view,and shooting distance.Different visual characteristics usually point to the same semantics for the images with the same material category.Hence,mining robust and efficient image features is one of the most important factors to handle these visual variations.However,the implicit material knowledge obtained using the simple feature fusion method is insufficient to fully represent the material images,and the existing material image datasets has some problems,such as sample scarcity and image distortion.Therefore,this paper focuses on resolving the above problems of the material recognition research.It adopts the advanced deep learning heterogeneous-layer features,improved cluster canonical correlation analysis(Cluster-CCA)algorithm,ensemble learning strategy,and improved efficient range gene selection(ERGS)algorithm to carry out non-end-to-end material recognition in the field of traditional machine learning(DML).Then it extends the research to deep mutual learning,and multi-level feature fusion to complete end-to-end material recognition in the field of deep learning.The main works are shown as follows:(1)Material recognition combining heterogeneous-layer feature fusion of SENet and ensemble learning: There are still some key problems in material recognition.For example,the existing mainstream models usually use a large number of heterogeneous features to complete material recognition,which reduces the practicality of these models.Moreover,material images are prone to change,with large intra-class differences and small inter-class differences.The effective complementary information among different features has not been fully mined.To address these problems,this paper only uses the squeeze-andexcitation network(SENet)to extract image features,and employs the early feature fusion method to modify the Cluster-CCA model.Then it mines the implicit cluster canonical correlations between the heterogeneous image features to generate deep-level visual semantic(DVS).Finally,the ensemble learning-based voting strategy is utilized to train five general machine learning classification models to complete the material recognition.Experiments results demonstrate that the proposed material recognition model based on the heterogeneouslayer SENet feature fusion and ensemble learning strategy obtains the best performance in terms of recognition accuracy and real-time efficiency.Notably,it is more suitable for finegrained material recognition,which means that further mining the effective information hidden between heterogeneous features is valuable.(2)Material recognition based on progressive feature fusion: Based on the work in(1),to make fully use of the complementary information hidden among different features,the well-known middle feature fusion method,namely the improved ERGS model,are introduced to further mine the feature-shared knowledge between different DVSs.Feature-shared knowledge refers to the discriminative information between different features,which are complementary and often point to the same material semantics.Experimental results show that the combination of early feature fusion and middle feature fusion is more effective than the combination of early feature fusion and late feature fusion,which can further mine the implicitly effective information between diverse features,significantly improving the final recognition accuracy in both fine-grained and coarse-grained material recognition tasks.Moreover,the corresponding real-time recognition efficiency of the proposed model also outperforms those mainstream material recognition models.Therefore,the feature-shared knowledge among different DVSs can be used to complete the middle feature fusion,which helps further mine the implicit material knowledge and finally improve the recognition performance.However,compared with the fine-grained material images,the corresponding material information contained in the coarse-grained material images is distorted and insufficient.So,it is necessary to enhance the quality and diversity of the image samples from the perspective of the material samples themselves.In addition,only mining deep learningbased features to complete feature fusion and using machine learning-based methods are not enough to fully mine material knowledge.Hence,end-to-end deep learning network should be adopted to continuously optimize the ability of the recognition model to better represent material images,so as to improve the effectiveness and efficiency of material recognition.(3)Superimposed sample augment and deep-level visual semantic fusion for material recognition: Based on the research in(1)and(2),in this section,this paper firstly proposes a new superimposed sample augment algorithm independently to generate highquality but diverse material image samples.Secondly,it introduces the well-known DML framework to complete mutual learning between differnt neural networks.Then it proposes the the residual learning idea for multi-level feature fusion on the basis of heterogeneous SENet features.Experimental results demonstrate that the superimposed sample enhancement algorithm is effective and robust.It can complete high-quality sample augmentation on both fine-grained and coarse-grained material datasets.And the DML framework makes full use of the complementarity between heterogeneous SENet features,further improving the recognition performance of each SENet network.Notably,the multi-level feature fusion strategy can adaptively mine the multi-level deep-level visual semantic(MDVS)with more powerful representational ability.Finally,through the end-to-end deep learning,the recognition accuracies on the coarse-grained and fine-grained material image datasets are significantly improved,and the model complexity is reduced to a certain degree,speeding up the deployment of the proposed model. |
