Research On Autonomous Decision-Making Method Of Intelligent Loader Operation Based On Data-Driven

As the most prominent and widely used construction machinery,the loader plays an important role in energy mining,infrastructure construction,disaster relief and other tasks.Because of the growth of the demand in construction accuracy,efficiency and safety,traditional loaders have difficult in meeting the social development in the future.The intelligent loader with autonomous operation capability can improve operation efficiency and energy efficiency,which help achieve carbon peak in 2030 and carbon neutrality in 2060 and liberate drivers from the harsh driving environment.Thus,it is very necessary to carry out the research on the intelligent operation of the loader.The research background and significance of the intelligent operation technology of loader is discussed systematically.The foreign and domestic research status of the intelligent operation technology of construction vehicle based on physical model and data-driven is elaborated in detail.The related technical difficulties existing in the current intelligent operation of the loader are analyzed and discussed,and new explorations based on data-driven methods are carried out.The major work can be summarized as follows:（1）Aiming at the fact that existing physical model-based methods cannot accurately reflect the driver’s driving habits and the interaction between loaders and the environment,this paper proposed a prediction model based on deep learning,which predicts the state and throttle value of the loader by learning from the driving data.First,a test data acquisition platform was built based on the loader test prototype.The operating materials and test site were determined,and real driving data was obtained.Secondly,multiple long short term memory（LSTM）networks are used to extract the temporal features of different stages in the loader operation process.Then,two backpropagation neural networks（BPNNs）were designed,which take the temporal features extracted by LSTM as input and output the final prediction results of the throttle value and state respectively.Finally,the proposed prediction model was trained with real driving data under two different working conditions.The prediction results showed that the prediction model had good prediction accuracy and adaptability compared with the baseline model.In addition,the relationship between prediction performance and signal sampling frequency is investigated.（2）Deep learning-based methods need to label the training data and the performance is difficult to outperform humans.To address this problem,this paper introduces a reinforcement learning-based approach.On the basis of analyzing the working characteristics of the bucket-filling stage of the loader,the automatic buckerfilling problem of the wheel loader is transformed into a finite Markov decision process.A nonlinear nonparametric Markov prediction model is constructed using real data to realize indirect reinforcement learning and provide a simulation environment for the training of the reinforcement learning algorithm.（3）An automatic bucker-filling algorithm based on Q-learning was proposed and the action,state and reward are appropriately set.Then,the constructed Markov prediction model was used as a virtual simulation environment to train the proposed algorithm.Finally,part of the parameters of Q-learning in the source task are transferred to the Q-learning of the target task,and the effect of data variance on the results was explored.The simulation training results show that the algorithm can achieve policy self-learning and convergence without a physical model and outperform human drivers in terms of fuel consumption.In addition,the algorithm has good adaptability and transfer learning ability.