Knowledge-based machine vision system for outdoor plant identification

Identification of individual crop plants in the field and locating their exact position is one of the last untouched areas of automated farming. Aimed at solving this "untouchable" problem, this dissertation explored the theory, implementation, and real-time application of machine vision for automatic outdoor detection of individual field plants.; This research describes the need for a reliable outdoor field plant detection and location system in horticultural operations. After reviewing the current state of development in agricultural robotics, machine vision systems, and related areas, a color machine vision sensing system for an outdoor field robot was developed and evaluated.; All research findings were conducted on juvenile processing tomato plants (Lycopersicon esculentum) using video images collected in vivo under normal California commercial farming conditions.; The scope of this research included: (1) the study of commercial field operations and properties of outdoor field plants (crop plants and weeds) to select a suitable view for object sensing, and to find the best natural plant growth stage (or timing) for identification, (2) the development of outdoor real-time video image acquisition and image preprocessing methods, (3) the development of a robust color segmentation algorithm for outdoor image processing, and (4) the formulation of a special pattern recognition algorithm for plant identification, plant location determination and/or operation decision making. Three major topics were studied attentively: (1) Outdoor imaging conditions. Sunlight, uniform illumination, shadow problems, optional lighting sources, diffusers, and filtering were studied in the real-time field environment. The effects of travel speed, equipment vibration, and the position of the camera relative to farm machinery components were investigated. (2) Image segmentation. A robust, environmentally adaptive segmentation algorithm has been developed. Major outdoor imaging problems such as different light source temperatures, changes in light source position, and shadowing of the field of view were studied. A partially supervised learning procedure was applied in the construction of an adaptive classifier. (3) Plant identification. Semantic leaf-shape features, position, and orientation data were used to determine the position of the whole plant and the location where the stem enters the soil.; Based on the processing results of more than 270 frames of real-time images from six different outdoor fields, the algorithms developed could find and identify between 61 to 82 percent of all the individual crop plants studied. The results of the prototype system showed that a machine vision system could be developed for outdoor, real-time field operation using current machine vision technology. The number of computationally intensive procedures could be reduced by careful design of the machine vision system to provide high quality raw outdoor images.; This dissertation is composed of five chapters and more than 60 figures which show the image processing and pattern recognition procedures evaluated.