Research On Foreign Substances Vision-based Inspection Robot For Bottled Infusion Solution

Medical infusion is one of the five important preparations of the pharmaceuticalindustry in China. They are frequently-used drugs in medical institutions and play avery important role in modern clinic. However, some visible foreign substances mayappear in the bottled infusion during the processes of producing, filling and packaging.These foreign substances will turn out to be a serious threat to patients. At present,due to the lack of pivotal technologies and for cost reduction, most of the Chinesepharmaceutical manufacturers adopt the traditional manual inspection, instead ofbringing in the automatic medical inspection equipments in most of the bottled liquidpharmaceutical production lines. In the manual inspection, trained workers check thevisible foreign substances in bottled liquid in front of the light boxes in darkrooms.Obviously, the manual inspection is slow, complex and unreliable, and usually bringsin extra pollution to the drugs, thus is unsafe. Fortunately, vision based medicineinspection machine can achieve online, high speed and high accuracy, and automaticdetection. The research of vision based medicine inspection equipment for bottledinfusion in China is far behind the developed countries, which is vastlydisproportionate with the top-ranking Chinese infusion usage.Hence, to solve the existing problems in manual inspection and meet therequirements of pharmaceutical companies, this dissertation carried out a deepresearch on the intelligent vision based infusion inspection technologies. Theconcepts and requirements of vision based inspection of bottled infusion areintroduced firstly. Then the detection principle, mechanical structure, electricalcontrol system, optical illumination, image capturing system and detection algorithmsfor visible foreign substances are studied in detail. Accordingly, a sample visualinspection machine is developed, on which a series of indicators are tested andvalidated.Major achievements and innovations of this dissertation are listed below:1. Research background and significance of the visual infusion inspectionmachine are introduced. Status of Chinese pharmaceutical market and equipment arealso described. Then the producting process of the bottled infusion is introduced andthe problems in the manual inspection are pointed out. Meanwhile, some typicalmachine vision technologies of inspection machine are overviewed, such as light sources and lighting technology, visual imaging, as well as their applications in thearea of pharmaceutical, beverage, electronics and production line inspection. Finally,foreign similar development results of detection technologies and equipments areanalyzed and summarized.2. According to the requirements of pharmaceutical manufacturers’ onlineinspection, intelligent inspection machine’s technical feasibilities are analyzed withthe practical producting process. To simulate the manual inspection, a strategy--―high-speed rotation of drugs, emergency stop, tracking and image capturing‖isproposed. Meanwhile, a series of essential mechanical structures are designed, such asbottle grasper (bottle holding mechanism), rotating and twisting mechanism, defectiverejection mechanism,etc. The overall system consists of the control subsystem and theimage processing subsystem. In the control subsystem, a distributed control structurebased on parallel processing of multi-IPC is developed. The timing of rotation,braking, and tracking and snapshotting, the time control strategy of sub-regionalbottle rubbing in the multi-stage is proposed. In addition, the control method of thetracking arm motion is designed. In the image acquisition and processing subsystem,legible images of bottled infusion are acquired after plenty of repeated experimentsfrom the selection of the camera lens to the optical system design. In addition, theworking process and the software architecture of the visual inspection robot systemfor pharmaceutical infusion are analyzed in detail.3. An FFT frequency domain transform based image calibration and registrationmethod is proposed, testing a series of sequence images to complete the roughalignment of the sequence image parameters. On this basis, through Powelloptimization searching algorithms, the best matching parameters are w orked out.Considering the average cross-relation value of image subset as the cost function ofmatching, the precise image registration parameters are obtained, which achieves thealignment accuracy of the sub-pixel level. In order to reduce the computationalcomplexity of further processing, a method based on probability and statistics for theinterest area extraction of infusion image is proposed. Firstly, the histogramequalization operation is applied to the images to enhance the overall image contrast.Then, the2D maximum entropy threshold segmentation is implemented. Finally,through the method of probability and statistics, the position of the image containingthe liquid region is worked out. In order to suppress the interference of backgroundnoise, an adaptive mean filtering algorithm based on maximum value is proposed.Compared with the traditional filtering algorithms, our algorithm can remove noise successfully while keeping the edge and other details in the images, and relaxing thefuzziness, which can effectively filter noise within the detecting area of the infusionimage.4. Combined with the characteristics of the inspection for visible foreignsubstances of infusions, an infusion image segmentation algorithm based on CellularNeural Networks (CNN) is proposed. An appropriate CNN template is designed tochange the CNN weighted linear links. Meanwhile, the min/max linking weights ofnonlinear fuzzy operation is introduced, and the structure of Fuzzy Cellular NeuralNetworks (FCNN) is designed. Experiments showed that FCNN was more effectivethan traditional CNN, but FCNN’s effect on edge detection was not so perfect. Tosolve this problem, an Improved Fuzzy Cellular Neural Network (IFCNN) is proposedand its convergence and stability are studied. Experimental results showed thatIFCNN can effectively solve the problem appeared in the edge detection that may notbe solved by existing methods, which achieved better approximation to the originalimages.5. According to the features of complexity and multi-character of the foreignsubstances, a series of morphological, statistical and motion features are selectedfrom the continuous medical image sequences. The calculation method of every usedfeature is introduced. The feature parameters of every target in the experimentalimages are extracted and analyzed. Accordingly, an improved ReliefF algorithm withk nearest neighborhoods is proposed to extract the features. By filtering out irrelevantfeatures with the feature selection algorithm, the algorithm can replace the traditionalnearest neighbor value with the average value of distance to k nearest neighborhoods,to reduce effects of false features such as noises on weight values vastly, and to makefeature extraction more precisely. After analyzing SVM and Boosting algorithms,experiments are carried out to compare the advantages and disadvantages of the twomethods in the classification applications for a variety of foreign substances. Then theAdaBoosting multi-class classification algorithm based on SVM is proposed, andtested by related experiments. In addition, the algorithm is integrated in the softwareof the pharmaceutical defedct inspection line that achieves great outcomes.6. A sample bottled infusion inspection machine and the related softwareplatform are developed. From the point of engineering application, the softwaresystem of bottled infusion inspection machine is described in detail, ranging fromentrance structure, gripper, rubbing device to electrical control modules, etc. Thedetection and analysis software modules are developed. In the process of the system performance testing, the effectiveness of the bottled infusion inspection machine andrelated algorithms are validated with international verification method (Knapp-Kushner Test). On the other hand, the repeatability, detection accuracy and theperformance on detecting different types of foreign substances of the inspectionmethod are also tested, whose results show that sample machine can meet therequirements of the bottled infusion online inspection effectively.In this dissertation, through theoretical analysis and experiments, the validityand feasibility of the optical, mechanical, electrical structures and inspectionalgorithms in the proposed bottled infusion visual inspection machine are proved. Asone of the important outcomes of the National High Technology Research andDevelopment Program of China "Intelligent Inspecting and Packaging Robot inLarge-Scale High-Speed Pharmaceutical Automatic Production Line", the develo pedinspection machine is accepted by the experts of the Ministry of Science andTechnology on December9th,2011. The field applications of the inspection machineshow that the developed system can solve most problems in the field applications, andwill play an important role in the automatic detection of pharmaceutical industry, andhave a promising future in a wide range of applications.