| With the continuous development of Self-Driving technology,more and more cars have realized the capability of Self-Driving.Pedestrians are one of the traffic participants that the Self-Driving system focuses on.How to detect them accurately and quickly is one of the criteria for testing whether a Self-Driving system is qualified.Although the pedestrian detection algorithm based on Deep Learning has achieved high accuracy,only detecting pedestrians cannot infer what he is going to do or where he is going,so simple pedestrian detection cannot provide more advanced information for the Self-Driving system,there is still some distance from the practical application.If the behavior of pedestrians can be further analyzed,then pedestrian detection will be more practical.However,pedestrian behavior is affected by many factors and is very complex,such as pedestrian orientation,pedestrian trajectory and pedestrian intention,which can reflect pedestrian behavior.Therefore,this dissertation will study the behavior of pedestrians from the aspects of pedestrian orientation recognition,pedestrian detection,pedestrian trajectory prediction and pedestrian intention estimation.Based on the above research content,the innovations and contributions of this dissertation are as follows:(1)For the discrete pedestrian orientation recognition problem,this dissertation regards it as an image classification problem to solve.To this end,a Graph Recurrent Attention Network(GRAN)is proposed.The whole network consists of a general convolutional backbone network,a graph recurrent encoder and a graph recurrent attention decoder.First,the efficient VGG11 is used in the backbone network to extract pedestrian visual features.Then build a body part graph on the feature map.Each node on the graph represents a body part of a pedestrian.Through this graph,the internal correlation among body parts can be established,and the feature expression ability among nodes is enhanced.Learning on the graph is achieved via a graph recurrent encoder-graph recurrent attention decoder,where an adjacency matrix with attentional edge weights is used in the decoding stage to represent graph node relationships,so that features can be automatically selected when aggregating neighbor node information.Attention edge weights are learnable parameters that are automatically updated while the network is training.There is no manual intervention from input to output,and the entire network can achieve end-to-end learning.Through experiments with other methods on three pedestrian orientation datasets,the results show the effectiveness and advancement of GRAN in this dissertation.(2)For the continuous pedestrian orientation recognition problem,this dissertation regards it as a regression problem to solve.Due to the complexity of continuous orientation,it is proposed to use discrete orientation classification to assist continuous orientation regression.To this end,a Pedestrian Orientation Regression Network based on Multi-Task Learning(PORN-MTL)with joint discrete orientations is proposed.In addition,global features are added to the network to enhance the feature expression ability.At the end of the network,the global features and local features are fused together,which enriches the expression of the final regression feature map.The ablation experiment and comparison with other methods were carried out on the continuous pedestrian orientation regression dataset(COR-ECP),and the results proved the effectiveness and advancement of the proposed network.(3)For the problem of pedestrian detection and orientation recognition,this dissertation defines it as pedestrian orientation detection,which means that it can also recognize his orientation while detecting the location of pedestrians.Therefore,a Multiwindow Transformer Parallel Fusion Feature Pyramid Network(MTPF-FPN)is proposed to realize the orientation detection of pedestrians.Since pedestrian orientation recognition pays more attention to detailed features,the detected pedestrians are often small,which leads to limited features that can be provided for orientation recognition.Therefore,the network enhances the expressive ability of features by improving the fusion method of feature pyramids.A sliding window is placed on each prediction map,the corresponding window area is spliced into a window patch,and Transformer is executed inside each window patch to realize feature extraction.Due to the existence of self-attention,it can be automatically selected during feature fusion,which enhances the learnability of the network.Finally,through the combination with GRAN,pedestrian detection and orientation recognition can be realized in one network at the same time.Experiments on three datasets show that the proposed MTPF-FPN can achieve high detection accuracy and orientation recognition accuracy,which proves the effectiveness and rationality of the scheme.(4)In order to further analyze pedestrian behavior,this dissertation will conduct research on pedestrian trajectories and intentions,and propose a Two-Stream Long Short-Term Memory(TS-LSTM)model to realize the prediction of pedestrian trajectories and intentions estimate.The model consists of a pedestrian trajectory prediction stream and an intent estimation stream,employing an LSTM-based encoderdecoder architecture in both streams.First,in the trajectory prediction stream,the encoder receives pedestrian orientations and historical trajectories as input,and the decoder outputs predicted trajectories at each step.The trajectory is expressed in the form of foot points,which can more truly reflect the location of pedestrians.Then,considering that the pedestrian’s intention will be affected by the vehicle’s driving,this dissertation proposes the concept of relative displacement for intention estimation,which can dynamically represent the distance change between the pedestrian and the vehicle.In the intent estimation flow,the encoder receives the historical relative displacement and the encoding of the orientation and historical trajectory in the predicted stream as input,and each step in the decoder receives the future relative displacement to adjust the output of the decoder,and the final output is the result of the intent estimation.In addition,the combination of TS-LSTM and MTPF-FPN can achieve end-to-end inference from input images to output trajectories and intent results.Finally,experiments on three datasets,and the results show that the proposed model is effective and advanced. |
