Research On Imbalanced Data Problem Of Defect Inspection For IC Solder Joints

With the development of technology and mobile internet,electronic devices have been inseparable necessities in human daily life.The qualities and lifespans of electronic devices depend on the qualities of printed circuit boards(PCBs),which are the carriages of electronic components and integrated circuit(IC).The qualities of IC solder joints are especially significant as IC solder joints are the connecting bridge between PCBs and ICs.As the IC chip emerges towards miniaturization and PCB towards high density,the traditional manual inspection hardly meets the demand of accuracy and efficiency in the production line.In recent years,automatic optical inspection(AOI)system based on computer vision has been widely used to deal with practical problem faced by traditional manual inspection.IC solder joint inspection method is the core of this type of the AOI system.The inspection methods can be divided into three categories: traditional image-process-based,classifier-based and statistical-modeling-based.The former two often do not inspect well due to the problems of hardly extraction of significant features,imbalance data and many empirical thresholds.Comparably,the latter,similar to anomaly detection,can somewhat deal with the imbalanced data problem and achieve fair good inspection.But some problems still existed for it: 1)The ignorance of unqualified IC samples will possibly result in the mis-inspection of some anomaly qualified IC samples;2)Simple evaluation scheme based on weighted sum of pixels will lead to poor robustness for IC solder joint inspection since it is easily affected by the shift and noise existed in IC samples.To deal with the above problems of statistical-modeling-based inspection method,unqualified IC samples should be introduced into inspection while dealing the imbalance data problem.Thus,two different solutions are put forward in this thesis,as described below.The first utilizes a sampling solution to solve the data imbalance problem.That is,an inspection method is proposed based on local-to-global ensemble learning mechanism,which is divided into locally statistical modeling and global ensemble learning evaluation stages.At the former stage,a locally statistical model is established based on improve visual background extraction by a series of qualified IC samples.A corresponding defect image is obtained when comparing the IC sample to the model.At the latter stage,a balanced sub-training set is obtained by sampling,which can improve the performance of base classifiers.An adaptive weighted strategy is proposed to reasonably ensemble multiple classifiers,which can improve the robustness and accuracy of the inspection method.The evaluation model based on the ensemble learning mechanism is able to comprehensively evaluate the qualities of IC solder joints,which reduces the influence of the shift and noise existed in IC samples on inspection.The proposed inspection method is able to grasp the tiny difference of IC solder joints and comprehensively evaluate qualities of IC solder joints from a global perspective.The second applies a data augment solution to solve the problem of data imbalance.That is,an inspection method is proposed based on the optimized statistical modeling.First,based on the improved visual background extraction algorithm,a statistical model contained several templates is established by a series of qualified samples.The qualified training samples and the inspected samples are inspected by the similar template set obtained by a clustering method to obtain the corresponding defect images.For unqualified training samples,several corresponding defect images are obtained by one unqualified sample based on the multiple different random templates.The random template strategy can augment the number of unqualified samples without destroying the internal structure of the sample,which obtains a balanced data set.The evaluation model is based on the weighted sum of pixels to evaluate the qualities of IC solder joints.But the pixel weights are adaptively achieved by an optimal problem that is solved based on the least squares model regularized by elastic net.Based on the regulation of L1 and L2,the pixel weights can reasonably reveal the contribution of pixels to the evaluationFinally,the experimental results indicate that the two proposed inspection methods for IC solder joints can solve the problem of data imbalance to a certain extent and also outperform some state-of-the-art IC solder joints defect inspection methods in accuracy and robustness.