Segmentation And Fusion Of Multispectral Dental Images Based On Deep Neural Networ

Accurate and fast measurement of tooth color is an important part of the restorative dentistry process.Traditional tooth color measurement methods can only measure a single point on the tooth surface,which can lead to unstable results and low measurement efficiency.By taking multispectral images of teeth and fusing them,we can quickly and accurately obtain the color information of each point on the tooth surface.However,because of the inconsistent shooting time and conditions,there are significant differences in the images of teeth under different spectral illumination conditions,and it is necessary to segment the images of teeth before fusing the images of single teeth.In this paper,we propose a segmentation and fusion algorithm for dental multispectral images,which firstly performs instance segmentation of dental multispectral images,and then fuses the multispectral images of the segmented single teeth to obtain multi-point chromaticity coordinate values of teeth.This paper includes the following main contents:(1)A dental multispectral image acquisition platform is designed and built,and a dental multispectral image dataset containing 477 photos is created and labeled using this platform.(2)The segmentation of dental multispectral images was realized using an existing general image segmentation model,and a data augmentation method based on spectral reconstruction was proposed to enhance the self-created dental multispectral image dataset,which effectively improved the segmentation accuracy and robustness of the dental segmentation model under different spectral illumination conditions.The alignment of tooth images in different channels after segmentation and the removal of highlighted areas on the tooth surface are also achieved.(3)A color information fusion model is proposed to fuse and convert the RGB values of selected locations on the tooth surface in a set of aligned tooth multispectral images to the corresponding CIELAB chromaticity coordinate values.The ΔE and ΔE variations of the chromaticity coordinate values output by the color information fusion model are smaller than the chromaticity coordinate values obtained using spectrophotometer.In this study,accurate,reproducible,efficient,and low-cost dental color measurements using multispectral images of teeth were achieved.The method not only allows visible photographs of teeth to be taken,but also allows color measurements to be made in all visible positions of the teeth in a single measurement operation.This improvement will help improve the accuracy and efficiency of clinical diagnosis and provide more accurate data to support dental research.