Research On TFT-LCD Mura Defect Recognition Based On Deep Learning

In recent years,applications of the industrial vision technology in the new display device field such as light-emitting diode(LED)and liquid crystal display(LCD)become more and more widely.Traditional ways to complete the quality inspection,verification and other works with humans are gradually being replaced by the automatic optical inspection(AOI)technology.Meanwhile,this technology has also brought new challenges to the traditional manufacturing industry.In order to better adapt to the needs of modern manufacturing and improve the competitiveness of products,it is particularly important to detect and analyze the quality problems in the production automatically.Mura,a kind of defect with low contrast and irregular shape,has long been considered as the core problem in the LCD defect inspection industry.Identification and analysis of Mura defects have important significance for the state monitoring of the production line.Aiming at the Mura defect recognition problem in process of TFT-LCD Cell stage,we utilized a mechanism of defect recognition from coarse-grained to fine-grained in this dissertation,and studied the problems of feature representation,classification in coarse-grained and fine-grained image recognition thoroughly.In this dissertation,three novel algorithms are proposed respectively.They include an algorithm for multi-modal intact feature representation in coarse-grained recognition,a multi-modal multi-task algorithm for discriminative feature classification in coarse-grained recognition,and a mixed Gaussian density transformation algorithm based on visual attention in fine-grained recognition.These three algorithms have some versatility in both the traditional image recognition datasets and the Mura defect dataset.They exhibit good recognition performance and are capable of solving the accurate recognition issue in TFT-LCD automatic optical inspection equipments.The main research effort and contributions of this dissertation are introduced as follows.Firstly,there are too many kinds of Mura defects,and their samples are usually very rare.Manually marking the types of Mura defects is generally difficult.Traditional methods which only use handcrafted features to describe defects in the production are not sufficient.Considering these issues,a multi-modal intact feature representation algorithm named JFR-DRF is proposed.This algorithm combines features extracted by the unsupervised learning based methods and traditional handcrafted descriptors.And it is capable of comprehensively describing Mura defect characteristics from different aspects,which solve the problem that traditionally used handcrafted features are insufficient to extract informations of the defective samples.For Mura defect recognition,the use of unsupervised learning strategies makes up the problem for handcrafted approaches to describe defects with high-level abstraction and difficult semantic characteristics.The combination of handcrafted features also makes up the problem for learning based methods to describe samples with a small number.The successful application of this algorithm further confirms the effectiveness of using multi-modal feature representation methods for image analysis.Secondly,considering the issue that Mura defects are easily confused and their representation features are not robust,a multi-modal multi-task model named M2 DNN for discriminative feature classification in coarse-grained recognition is proposed.This model utilizes the Siamese network architecture,and it makes use of feature selective properties of activation function to filter information from different modalities.This model also transforms features using high nonlinearity of the MLP neural network structure.In addition,the model introduces a constraint term which is similar to the Fisher discriminant criterion.By jointly optimizing the feature selection,transformation and classification tasks with end-to-end training,this model improves robustness of the fused multi-modal feature and discrimination of the classifier.The successful application of this method in TFT-LCD Mura defect recognition further improves the classification accuracy of defects.Thirdly,considering the issue that in the fine-grained classification problem of region Mura defects,the local discriminant regions need manual labeling and the traditional visual attention mechanism identification methods have too many parameters and large computational complexity,a mixed Gaussian density transformation algorithm named DA-MGDT based on visual attention is proposed.This algorithm can be used to synthesize global image context information and multiple local discriminant partial information for more accurate category prediction.Effectiveness of this proposed algorithm DA-MGDT is verified on a number of typically fine-grained image recognition datasets and the Mura defect dataset.Fourthly,a Mura defect dataset in TFT-LCD Cell process is produced.This dataset is the largest Mura defect dataset in the public literature as we know.In addition,we integrated and tested the proposed three algorithms in Mura defect automatic optical inspection equipment according to Mura defect inspection and recognition procedures.Experimental results show that the proposed coarse-grained to fine-grained recognition mechanism is able to satisfy the needs of Mura defect identification.The recognition performance is also significantly improved compared with traditional recognition methods.