Research On Surgical Instrument Segmentation And End Of Forceps Pose Estimation In Endoscope Images

The interactive safety early warning system for minimally invasive digestive endoscopic surgery aims to monitor the safety of operating force of forceps in minimally invasive digestive endoscopic surgery.The system is mainly composed of tumor segmentation,surgical instrument segmentation,pose estimation of the end of the operating forceps,and the interactive model design of the operating forceps and soft tissues,among which,surgical instrument segmentation and pose estimation of the end of the operating forceps are the foundation and key of the early warning system.However,There are few studies on endoscopic surgical instrument segmentation,and most of them put too one-sided emphasis on the prediction accuracy or prediction speed,and lack of effective balance on the overall network segmentation performance.In addition,most of the mainstream objective estimation algorithms are designed on the basis of CNN networks,which can only predict the ordinary rectangular estimation box representing the overall position of objectives,and lack the estimation of the objective direction information and the end position information.Furthermore,most of current pose estimation algorithms are designed based on key point detection algorithms,which demand the time-consuming and energy-consuming dataset notation.First,we designed a real-time high-performance surgical instruments segment network.We firstly set up a multi-instrument endoscopic minimally invasive surgery simulation platform to simulate instrument interaction in the operation.Then,we made a simulation dataset by sampling,notating and augmenting data.Lastly,based on some high-performance modules,such as light-weight feature extraction network,attention mechanism,feature pyramid network and so on,we designed a novel highprecision and real-time surgical instrument segmentation network with performance of more than 85% m Io U and 40 FPS or more.Second,we verified the network's segmentation performance based on the public MICCAI competition dataset.Further,based on the dataset,we employed some original designs,such as boundary loss function,lateral connection attention mechanism,gate mechanism of feature selection to further improve the network's performance.The network achived outstanding performance with 62.49% m Io U and 53.90 FPS on the three-class segmentation task of this dataset,superior to most mainstream segmentation networks.Last,we analyzed the shortcomings of the objective pose estimation algorithms based on key point detection,and designed an efficient end of forceps pose estimation algorithm based on high-precision segmentation.We firstly designed a multi-task learning network which can simultaneously segment instruments and detect forceps pose based on a key point detection algorithm and a segmentation network,and analyzed its merits and demerits.Secondly,in order to remove the burden of dataset notation,we designed a novel end of forceps pose estimation algorithm based on high-precision segmentation by fully utilizing the rich semantic information in instrument segmentation results.The algorithm can infer end of forceps position and oritation by the detection of the minimum circumscribed rectangle surrounding forceps and the design of pose-inference algorithm.It has many merits,such as learning unnecessary,real-time estimation based on segmentation maps,etc.Lastly,we implemented the algorithm and verified its validity.To sum up,this dissertation implemented a novel real-time high-precision surgical instrument segmentation algorithm and a real-time pose estimation algorithm of the end of the surgical forceps without learning,which has a strong practical value for the research of early warning system design.