Intelligent Tele-Consultation And Tele-Guidance Of Skin Tumor

Accurate diagnosis and treatment of skin tumor is challenging since it heavily relies on the expertise and experience of dermatologists.However,uneven socioeconomic development of regions leads to a severe shortage of experienced dermatologists in the areas with scarce medical resources.In recent years,researchers have tried to alleviate this problem by developing artificial intelligence automatic diagnosis technology and remote diagnosis and treatment technology.Although these efforts have achieved a lot of promising results,there are still several shortcomings.First,the scale of the publicly accessible medical datasets is small,and the deep learning model is prone to overfitting,resulting in poor generalization.Second,the synchronous tele-consultation system relies heavily on the network and consumes a lot of time for remote experts.Third,most of the tele-guidance systems use monitors to display expert annotations,therefore trainees are distracted by repetitive shift of attention between the operating area and the monitor.To address the above research shortcomings,we focus our research and innovation efforts on the following aspects:(1)To address the bottleneck induced by the small medical datasets and to improve the diagnostic accuracy of artificial intelligence model,this thesis innovatively proposes an automatic data augmentation method.This method increases the number and diversity of samples in the dataset by transforming the existing samples,and the generated data only exists during training process and does not occupy additional storage space.It can be used to alleviate the overfitting problem and improve the diagnostic accuracy of the model.The experimental results on four different medical datasets,including ISIC 2018 skin tumor dataset,show that this data augmentation method can universally improve the diagnostic performance of convolutional neural network models on different types of medical datasets.Advanced model structures can achieve better performance,and models with moderate capacity rather than maximum capacity in the same structure series can achieve the best performance.(2)To improve the efficiency and applicability of synchronous tele-consultation system,this thesis designs and builds an intelligent tele-consultation system for skin tumor,which has both the offline capability for automatic disease screening and the online capability for remote consultation.Control experimental results show that the diagnostic performance of the deep learning algorithm deployed in the system is comparable to that of dermatologists and can help them make more efficient and accurate decisions.Clinical experimental results show that the system has clinical applicability.(3)To improve the efficiency and accuracy of monitor-based tele-guidance system,this thesis innovatively proposes an augmented reality technology called co-axial projective imaging,and builds a tele-guidance system for skin tumor surgery that can project the virtual annotations drawn by a remote expert with great accuracy to the surgical field.The ex vivo study that compares our system and a monitor-based telementoring system shows that our system can reduce the focus shift and avoid subjective mapping of the instructions from a monitor to the real-world scene,thereby saving operation time and achieving precise teleguidance.The clinical feasibility of our system is validated in tele-guided skin cancer surgery.The research results of this thesis can help improve the level of diagnostic and treatment equipment in primary hospitals,and help patients in resource-limited areas conduct early screening and treatment for skin tumor and other diseases.