Posture Detection Method Of A Magnetically Controlled Dual Hemisphere Capsule Robot Using Images

In the field of gastrointestinal tract detection,for deep areas of the gastrointestinal tract with a complex physiological environment,traditional endoscopes cannot effectively complete the detection.The detection process will cause harm and pain to patients.The appearance of capsule endoscopy solved the problem.In order to solve the motion uncontrollability of the passive capsule endoscopy,especially to solve the diagnosis and treatment problems in the ample environment such as the stomach and colon,our research group proposed and developed a dual hemisphere capsule robot(DHCR)which is controlled by magnetic drive and realized the separation of the two modes of robot walking and posture adjustment.The posture information of the capsule robot is very important when it is in the detection process.If we can obtain the actual posture information,it can lay the foundation for the feedback control of the DHCR posture.We can also improve the control accuracy of the axis orientation.It is important to assist the doctor in determining the lesion area.At present,the attitude information of capsule robots is mainly recognized by sensors and other hardware devices.This method is complicated to operate and occupies a large space,which limits the expansion of the function of the capsule.It is hard to obtain the posture information of capsule by images in abdomen environment.To this end,this paper proposes a posture detection method for DHCR robot based on magnetic vector and images.First,the structural characteristics and working principles of the capsule are introduced.Then,the attitude information of the DHCR is analyzed.In this method,the minimum error orientation,which is determined by analyzing the spatial rotating magnetic vector error model,is regarded as the initial image calibration position of the DHCR initial position.Then,a coordinate system is established for the DHCR and its embedded camera.It is determined that the attitude of the camera is equivalent to that of the DHCR.It is established the relationship between the image acquired by the camera and the attitude estimation of the DHCR.Finally,by recognizing and matching some feature points between the initial calibration image and the image after the posture adjustment transmitted and by the embedded wireless camera with corrected rotation angle information relative to the initial position,combined with the multi-view geometric related constraints,complete the detection of the axis orientation of the DHCR in a favorable environment such as the gastrointestinal tract and the colon.The experiments inside pig large intestine have verified the effectiveness of the method.The theory and experimental results show that the attitude of the DHCR in passive mode can be detected in term of the rotation,yaw and pitch angle using the image information obtained by the camera on the robot.Although the estimated value of the pitch angle or the yaw angle is not completely equal to the actual value,the rolling motion of the DHCR has a certain fault tolerance in the intestine.Then the estimated values with the accuracy still can be used to control the robot for rolling locomotion.Without any interference from magnetic field and any extra hardware resources,the visual pose detection method proposed in this paper can realize both posture detection and diagnosis and treatment by the embedded vision system.Therefore,the capsule can meet the real-time accuracy requirements for pose estimation without any structural changes.It laid the foundation for accurate diagnosis and treatment.