Image Understanding And Analysis In Automated Growth Status Observation Of Maize Tassels

Leaves,tassels and fruits are typical crop-related traits.These traits reflect the growth status of crops during different stages.Monitoring their growth status is of great importance for many farming operations,such as seed breeding,effective cultivation,growth stage prediction,and yield estimation.Current automated observation solutions for crop traits are only specific to plants under constrained environment,like potting plants.Regarding in-field crops,it still mainly depends on manual observation,however.Considering that ultimately crop cultivars bred in laboratories must be transferred to real-world greenhouses or fields,in-field observation matters in practice.How to bring an automated solution to these tasks becomes an urgent need in modern intelligent agriculture.Since most collected data are in the form of images and videos,Computer Vision thus plays a vital role in helping machines to understand in-field objects or scenes.To better address these practical problems,scientists are required returning back to Computer Vision per se to understand what types of visual challenges are brought by in-filed environment and in-field objects,and to ponder over how to cope with these field-specific challenges.Motivated by these,this thesis explores several typical visual problems during the growth process of in-field maize tassels and conducts fundamental studies from the perspective of Computer Vision.First,to address automated segmentation problems of objects with different color attributes under complex scenarios,this thesis proposed a region-based color modeling approach for segmentation.By fusing superpixels with different grains and learning ensemble neural network models,our approach could be applied to semantic segmentation tasks in which colors are presented as the main visual cue.Evaluations on the joint crop and maize tassel segmentation dataset validate the effectiveness,the generality and the expandability of the proposed method.Second,to tackle the in-field texture recognition problem of different flowering status,this thesis first introduced the concept of partially-flowering.We found that,when the visual differences between different flowering status reflect in the feature space,the differences are generally exhibited as large interclass and small intra-class variations.Inspired by the large-margin framework,this thesis proposed a metric learning approach to directly optimize the within-and between-class Euclidean distance between features when projected into low-dimensional feature space.Experimental results on the maize tassel flowering status recognition dataset justify that our proposed method can effectively improve the accuracy of flowering status recognition.Third,to solve the fine-grained visual categorization problem of different crop cultivars,this thesis first demonstrated that one can recognize crop cultivars based on the visual characteristics of tassels(compared to conventional recognition that based on the visual characteristics of seeds).In this task,different cultivars only present subtle visual differences.To highlight local variations of objects,this thesis proposed a filter-specific feature encoding and selection mechanism for convolutional activations,which can extract discriminative binary features.Results on the maize cultivar identification dataset show that our proposed method can recognize different maize cultivars effectively and efficiently.In addition,this thesis first considered a plant-related object counting problem under unconstrained field-based environment.To cope with variations exhibited by in-field objects,such as different appearance,varying poses,unpredictable scales,and changing physical sizes,this thesis built on the idea of local regression,proposed to map local image patches to local counts directly,and further constructed a local counts regression model via deep learning,which can learn the mapping in a data-driven manner.Experiments on the maize tassel counting dataset show that our method significantly outperforms other state-of-the-art approaches.Furthermore,this thesis realized that there exists an awkward situation in computer vision applications in agriculture.Limited by the natural growth rule of crops,the cost of collecting agricultural image data is very expensive and time-consuming.Scientists have to use historical data collected in previous years to build a model but need to apply the model to an as-yet non-existent scenario in the field.Intrinsic and extrinsic variations caused by years,geographical locations and cultivars lead to the mismatch of data distributions easily,and thus giving rise to significant performance degradation of the model.This thesis tried to correct these distribution shifts from the perspective of domain adaptation.According to an observation of a pattern rule that CNN features present in experiments,this thesis proposed an embarrassingly simple approach to visual domain adaptation.The approach can adapt domains without explicit adaptation and can be applied to any classification-orientated visual adaptation problems.Extensive experimental results on several standard domain adaptation benchmarks and the cross-domain flowering status recognition dataset confirm that our approach achieves high-quality adaptation with light computational cost.Finally,this thesis integrated several proposed techniques into an automated crop growth stage observation system,and showed an application of the system on automated detection of the tasseling stage and the flowering stage of maize.After compared to ground manual observation results recorded from2010 to 2015 across several meteorological observation station in China,we found that our system detected the growth stages accurately,with comparable performance to manual observation.This validates the studies covered in this thesis are of great practical application value.As a summary,we hope that,by instantiating above various visual problems on maize tassels,this thesis could highlight different visual challenges in the field to readers,because these challenges are representative and may appear in most in-field visual problems.Hence,Computer Vision is one of key technologies in agricultural automation,the work we have done in this thesis and acquired research achievements are of important theoretical significance and application prospect for automated,intelligent agriculture.