| The external quality of apples is the most important and intuitional quality,which affects their point-of-sale value and consumer?s preference.In this paper,we aimed to develop algorithms to inspect the outer appearance quality and common defects of apples by using computer vision combining with hyperspectral imaging,multispectral imaging,image processing,pattern recognition,spectral analysis,and chemometrics methods.The proposed systems and methods could provide a potential support for developing fast in-line inspecting and grading equipment for apples according to their external quality.The main contents and conclusions were listed as follows:(1)Shape feature extracting and protruding stem removing algorithm were conducted by using image processing techniques.Edge detection algorithm based on Sobel Operator,the extracting methods of frequently-used shape features,protruding stem detecting method and removing algorithm were mainly studied.A hand-coded software for apple shape inspection was implemented in Visual C++ and Open Source Computer Vision(OpenCV).The results show that the protruding stem had effects on apple shape inspection,and protruding stem removing algorithm could remove the protruding stem from images and improve the inspection accuracy of apple shape.The proposed method and algorithm could be used in other fruits and vegetables? shape inspection.(2)Weight and volume of apples were non-contact measured by using computer vision and near-infrared linear-array structured light.3D height map of the upper half apple surface was obtained by using our proposed system and method.Projection area,and two different type height features would be extracted in our research.The first type height features were extracted from the 50 concentric annulus equally distributing in the height maps with an adaptive distance to the size of the inspected apples by averaging the height of all pixels in concentric annulus.The second type height features were extracted from the 50 vertical lines equally distributing in the height maps with an adaptive distance to the size of the inspected apples by averaging the height of all pixels in the vertical line.And the weight and volume evaluation models(including univariate models and composite variable models)were established by using PLS and LS-SVM methods.The results show that LS-SVM regression model based on projection area and first type height features could achieve a good prediction precision and stability.The correlation coefficient of prediction(Rp)and root mean square error of prediction(RMSEP)for the best prediction of apple weight by LS-SVM were 0.8234 and 11.4991;The Rp and RMSEP for the best prediction of apple volume by LS-SVM were 0.9032 and 10.1115.(3)Computer vision recognition of stem and calyx in apples was studied by using near-infrared linear-array structured light and 3D reconstruction in this paper.Automatic detection of common defects on apples by computer vision is still a challenge due to the similarity in appearance between true defects and stems/calyxes.Because the stem and calyx present a concave feature in apples,this paper proposes a novel stem and calyx recognition method using a computer vision system combined with near-infrared linear-array structured lighting and 3D reconstruction techniques to reveal this concavity.The 3D surface of the upper half of the inspected apples could be reconstructed by using a single multi-spectral camera and near-infrared linear-array structured light line by line on an adjustable speed conveyor belt.The height information for each pixel could be calculated by triangulation.Stems and calyxes would present a lower height than that of their neighbouring regions due to the local concave surface.In order to recognize the stems and calyxes efficiently,a standard spherical model(without stems and calyxes)is also constructed automatically,adapted to the size and boundary shape of the inspected apple.The difference between the 3D surface reconstruction and standard spherical model provides great potential for the recognition of stems and calyxes in apples.The final stem and calyx recognition algorithm was developed on the ratio images between 3D surface reconstruction images and standard spherical model construction images in gray level.The result had 97.5% overall recognition accuracy for the 100 samples(200 images),indicating that the proposed system and methods could be used for stem and calyx recognition.(4)Computer vision detection of defective apples by using automatic lightness correction and weighted RVM classifier was studied in this paper.Automatic detection of defective apples by computer vision system is still not available due to the uneven lightness distribution on the surface of apples and the similarity between the true defects,stems and calyxes.This paper presents a novel automatic defective apple detection method by using computer vision system combining with automatic lightness correction,number of the defect candidate(including true defect,stem and calyx)region counting,and weighted RVM classifier.Automatic lightness correction was used to solve the problem of the uneven lightness distribution,especially in the edge area of the apples.According to the fact that the calyx and stem cannot appear at the same view of image,some apples could be classified as sound(N=0)or defective(N?2)apples based on the number of defect candidate region(Labeled as N)in the preliminary step without any other complex processing.For the rest uncertain apples(N=1),further discrimination was conducted.Average color,statistical,and average textural features were extracted from each candidate region,the relevant features and their weights were also analyzed by using I-RELIEF algorithm.Finally,the defect candidate regions are classified as true defect or stem/calyx by the weighted RVM classifier,and the apples would be finally classified as sound or defective class according to the category of the candidate regions.The result with 95.63% overall detection accuracy for the 160 samples indicated that the proposed algorithm was effective and suitable for the defective apples detection.(5)Recognition of unobvious defects was studied by using hyperspectral imaging techniques in this paper.Hyperspectral imaging and successive projection algorithm(SPA)were applied in the recognition of the early decay in apples.80 average spectra of the rectangle region of interests(ROIs)were extracted from sound tissues and early decay tissues.The 563,611,816,and 966 nm were selected as the potential efficient wavelengths.The classification performance of the four wavelengths was evaluated by using PLS-DA.Finally,the recognition algorithm of the early decay in apples was developed by using MNF conducted on the images at the four selected wavelengths.The detection performance of our proposed methods was evaluated by using 20 sound apples and 80 decay apples.The overall detection accuracy was 98%.Similar method was conducted to detect the early bruise in apples by using MNF and I-RELIEF methods.The 590,660,720,and 960 nm were selected by I-RELIEF as efficient for early bruise detection.80 apples were used for the detection performance evaluation,and the overall accuracy was 98.75%.Results show that hyperspectral imaging and images at the efficient wavelengths could be used for early decay and bruises detection,this could provide foundation for developing multispectral computer vision system for unobvious defect detection.(6)Detection of common defects in apples was studied by using hyperspectral imaging techniques and band math methods in this paper.In order to reduce the effects caused by uneven lightness distribution and variable color on apples surface,and improve the common defect detection accuracy,we extracted spectra from sound and defective tissues,730 and 925 nm were selected as efficient wavelengths for common defect detection by analyzing the spectral trends,band math equation was developed and applied in image processing for common defect detection.Band math methods could reduce the noise and non-uniform distribution of lightness on the fruits to some extent due to the calculation between intensity of the corresponding pixels in the same position and the influence caused by shape and color are similar in the same position.The band math method could enhance the difference between the sound and defective tissues as well,this make it easier to detect the defects.109 hyperspectral images were used to extract images at 730 and 925 nm for the common defect detection performance evaluation,and the overall accuracy was 93.6%.Results show that band math method is efficient,could reduce the uneven lightness caused by color,shape,and stem concave,and increase the detection accuracy.(7)Detection of common defects in apples was studied by using multispectral imaging techniques and band math methods in this paper.Hyperspectral imaging was verified as a useful tool for common defect detection,however,hyperspectral image acquisition and processing is time consuming,it is not suitable for inline inspection.In order to promote the development of spectral imaging in fast or inline detection of fruits,we designed a filter wheel and a box,the selected filter were installed in hole of the filter wheel,then the filter wheel and camera were enclosed in the black box to develop a multispectral imaging system.The inspection software for common defect detection in apples was implemented in Visual C++ and OpenCV.A total of 68 apple samples with various types of defects were used in our research,1 to 3 multispectral images were captured,a total of 116 multispectral images were got and used for evaluated the detection performance of our system and algorithm,the overall detection accuracy is 91.38%.Results show that the selected wavelengths and algorithms developed from hyperspectral imaging system could be implanted into multispectral imaging system for common defects inspection.And this could provide foundation for developing multispectral computer vision system and grading equipment for common defect detection or grading. |
PDF Full Text Request