Research And Construction Of Harvester Data Preprocessing Platform

In the process of logging of Cut-to-length harvester with computer control system,the harvester control system and data acquisition equipment recorded a large amount of positioning,logging and material data in real time.These logging data are important for precise forestry and for forest management and researches based on big data.This research aiming to the actual needs of using harvester data in forest modeling studies,analyzes structure of harvester data sets and data noise source.According to the formation reasons of noise data,the research summarizes the data cleaning rules that meet the requirements of consistency,uniqueness and rationality,combined with the data cleaning rules carding processing harvester data preprocess.And then,a harvester data preprocessing platform is designed and implemented.The platform has functions of automatically data filtering with screening rules,suspective data identifying,missing data import,automatically calculating stem diameter at breast height and total tree height,cut simulating,generating tapper data,data visualization,metadata management,data query,data import and export.The platform provides a clean set of available data for subsequent research.According to system analysis and user requirements,harvester data preprocessing platform adopts B/S system architecture,selects Django as the server framework,Layui as the front-end page,MYSQL and Redis as the database,and adopts pandas to process data.In order to solve the problem of time consuming in the process of data cleaning,multithreading and matrix operation have been adopted to improve the efficiency of system operation.Task management with Celery to handle asynchronous tasks in task queue management way,to avoid congestion.After test,the system supports online data preprocessing,simulated cutting,tapper data generation and other functions,and has a good user experience.In this study,the design objectives of harvester data preprocessing and data set processing sharing were realized,and the working efficiency of researchers was improved.