Research On Artificial Intelligence Methodology Of Tongue Recognition In Traditional Chinese Medicine Based On Convolutional Neural Network

Objective:This study was based on comprehensive,accurate and objective tongue image collection,with two tongue characteristics——tooth-marked tongue and greasy tongue fur as a entry point,TCM experts common understanding were the diagnostic gold standard for tongue characteristics,and then built a tongue classification dataset;we constructed an artificial intelligence detection model based on the deep learning convolutional neural network(CNN)algorithm for tooth-marked tongue and greasy tongue fur.An artificial intelligence technology suitable for the treatment of tongue diagnosis of Traditional Chinese Medicine was proposed,and a new method of objectively assisted diagnosis of traditional Chinese medicine was initially explored.Methods:In this study,1760 tongue images(1680 with standard equipment and 80 with smart phones)were collected by our research group in the early stage and handed them to TCM experts to judge the characteristics in these tongue images,and classified tongue images according to their characteristics.We used a software named Colabeler(Ver.2.0.2)to label the outline of tongue to form a tongue image dataset containing different tongue image features and tongue outlines,and built a standard tongue image dataset for traditional chinese medicine.Using the tooth-marked tongue as a entry point,an artificial intelligence network framework based on deep learning convolutional neural networks(CNN)——ResNet34,was applied to this dataset to automatically extract image features and realized the classification of the tooth-marked tongue;at the same time,we used transfer learning and data amplification techniques to improve sample training efficiency,optimize data samples,and prevented model overfitting.Three methods were used to verify the effectiveness of the tooth-mark tongue recognition model:applied the popular VGG16 network framework in the classification task to the dataset of this research to compare the classification effect with ResNet34 network framework;compared our research model with other scholars's;used the tongue image collected by the mobile phone as the test dataset to verify the generalization ability of our model.Finally,we applied the same method to the greasy fur dataset,trained and verified the three-class task for non-greasy fur,greasy fur and thick greasy fur,and established an artificial intelligence recognition model for greasy fur.In this study,the parameters of accuracy(accuracy,Acc),sensitivity(Sensivity,Sens)and specificity(specificity,Spec)were used to evaluate the effectiveness of the model.Results:(1)Through data sorting and image labeling of the collected tongue images,4 datasets of tooth-marked tongue were finally formed,included the raw tongue-image dataset of tooth-marked tongues which collected with standard equipment(672 cases of tooth-marked tongues and 876 cases of non-tooth-marked tongues)and its corresponding tongue region dataset,the raw tongue image dataset of tooth-marked tongue(27 cases of tooth-marked tongue,23 cases of non-dented tongue)which collected with smartphone and its corresponding tongue region dataset;4 greasy tongues datasets were finally formed,included the raw tongue image dataset of standard which collected with standard equipment(642 cases of thick greasy fur,759 cases of greasy fur,85 cases of non-greasy fur)and its corresponding tongue region dataset,and raw tongue image dataset of greasy fur which collected with smartphone(19 cases of thick greasy fur,25 cases of greasy fur,and 6 cases of non-greasy fur)and its corresponding tongue region dataset.These datasets had laid the foundation for our research on the artificial intelligence recognition model of tooth-mark tongue and greasy fur;(2)The recognition result of tooth-marked tongueThe overall recognition accuracy of the tooth-marked tongue model on the raw tongue image was 90.50%,the sensitivity was 87.25%,and the specificity was 93.00%.These showed that our model had relatively good performance and strong robustness.Indicating that the model proposed by us had higher sensitivity and specificity and can better identify the tooth-mark tongue of different instruments and different shooting environments.The accuracy of the network framework was 91.47%,which was 0.97%higher than the average accuracy of using tongue image directly for feature recognition,it was suggested that other facial areas except the tongue would indeed affected the recognition accuracy of the artificial intelligence model of tooth-mark tongue;The average accuracy of the model on the test dataset which was taken by the mobile phone camera were 83.20%and 88.80%,and the overall accuracy of the model was 85.00%,which proveed that the model had strong generalization ability and can be extended to different devices in the future.The average accuracy of the ResNet34 tooth-mark tongue recognition model on the raw tongue image dataset and tongue region data set were 89.41%and 90.96%.The accuracy were 1.10%and 0.52%higher than that of VGG16.It can be seen that the ResNet34 algorithm architecture had excellent performance on two datasets,indicate that the ResNet34 architecture was better able to perform the task of identifying tooth-marks and tongue features.Compared with the similar learning tasks in existing research,the accuracy of the tooth-marked tongue recognition model proposed in this study was more than 10%higher.These results suggested that the CNN algorithm proposed in this research can distinguish the tooth-marked tongue more accurately and effectively;(3)The recognition result of greasy furThe recognition accuracy of the greasy fur recognition was 88.36%,and the accuracy of the tongue region is 87.08%.This result reminds us that in the recognition of greasy fur,other facial areas outside the tongue may had little effect on the recognition accuracy of the artificial intelligence model;the average accuracy of the model on the test dataset were 62.80%and 76.80%,indicating that the model in this scene was greatly affected by objective conditions such as other areas of the face and the shooting background;the average accuracy rates of the greasy fur recognition model built with the VGG16 algorithm architecture were 79.48%and 80.89%.The ResNet34 algorithm architecture' accuracy rate were 8.80%and 6.19%higher,this shows that the ResNet34 algorithm architecture can better meet the task of greasy moss feature recognition.Conclusion:This study uses a convolutional neural network model based on deep learning,which can automatically extract tongue features while reducing the steps of manual preprocessing of data,and can extract tongue features more quickly and conveniently.This is the key to transforming such tongue image recognition system into clinical practice.At the same time,our proposed model architecture has excellent performance and strong generalization ability,which can track disease progress from the perspective of TCM informatics and observe the evaluation of the efficacy of Traditional Chinese Medicines for changes in tongue condition provides a more objective and convenient new computer-assisted method.