Research On Tongue Image Segmentation And Classification Method Based On Deep Learning

Tongue examination is an extremely important part of the diagnosis of Chinese medicine.The health of the human body can be preliminarily judged through tongue examination.Traditional tongue examination mainly uses the doctor’s naked eye to observe the tongue.There is a disadvantage of strong subjective dependence on the doctor,so the tongue Modern research on diagnosis is an inevitable development trend of tongue diagnosis.This paper combines tongue segmentation with deep learning technology to realize automatic tongue segmentation,build a multi-task learning network based on feature intersection,and apply it in the tongue image classification process.The main research contents are as follows:(1)Regarding the current lack of open source data sets with labeled tongue image labels,a batch of high-quality tongue image data sets totaling 1257 were constructed,and the tongue image data sets were manually segmented and classified data set construction.The image data has both tongue color and moss color labels.(2)For the problem of poor segmentation effect and slow segmentation speed of traditional tongue segmentation methods,the Mask-RCNN algorithm is used to achieve accurate tongue segmentation,and compared with traditional methods.The results show that the segmentation effect of this method is better than the traditional tongue segmentation method.(3)Tongue image classification currently uses single-task classification or multi-label classification,ignoring the relationship between different tasks.In this paper,multi-task learning is introduced into tongue image classification,which can perform tongue color and fur color classification tasks at the same time.L2 regularization is used to constrain model parameters between different classification tasks,and a multi-task learning network model based on soft sharing mechanism is constructed.(4)Currently commonly used multi-task methods usually only consider the relationship between features or the relationship between model parameters.For this problem,this paper proposes a multi-task tongue based on feature fusion and feature intersection based on soft shared multi-task learning.Like a classification network,it improves the feature extraction ability of the model,and refers to the idea of cross-stitch network to construct a feature cross unit to select the best shared features between different tasks.The results show that the multi-task network can make use of the connections between various tasks,perform more balanced on the classification task,and greatly improve the classification accuracy of the moss color classification task.The accuracy rate of tongue color classification was 92.22%,and the accuracy rate of fur color classification was 88.89%.(5)Intelligent tongue diagnosis system design,The tongue diagnosis system is constructed based on the development interface of Qt Designer with python as the basic language.It can gradually display image selection,tongue body segmentation,region cropping,moss zone,tongue classification,and result display.The tongue diagnosis processing method proposed in this article can realize tongue body segmentation,tongue image classification and systematic diagnosis,and has obtained good experimental results,which has certain reference value for the modern research of tongue diagnosis.