Research On Mobile Phone Sensor Behavior Recognition Based On Image Analysis

With the rapid development of mobile Internet technology in recent years and the continuous growth of the number of smart phone user groups,a variety of applications based on smart phone sensors have emerged.These applications have been integrated into all areas of people's daily life and work,bringing great convenience to people's lives.The realization of these applications is inseparable from the research of behavior recognition technology.At present,the research on user behavior recognition based on mobile phone sensors mainly has the following two problems:1.Most research work is based on a single sensor,namely an acceleration sensor,for behavior recognition,and does not make full use of other sensors built into the smartphone for fusion recognition;2.Some research work uses deep learning methods for behavior recognition.For time series data,it is relatively difficult to construct predictive models.To solve these problems,this thesis adopts the design idea of transforming the classification problem based on sensor data into the classification problem based on image data,and proposes a research scheme of behavior recognition based on multi-sensor image feature fusion.Firstly,the time series data of the two sensors in the smart phone are converted into four kinds of images,which are Recurrence Plot,Markov Transition Field,Grammy Angle Sum Field and Grammy Angle Difference Field;Then a fusion depth neural network based on multi-sensor image is designed.The network structure mainly includes three parts.The branch network mainly extracts the features of the transformed image,and then sends them to the fusion layer.In the fusion layer,the image features extracted by the two sensors in the branch network layer are fused by concatenation fusion method,Then the fusion features are sent into the behavior classification network to complete the classification and recognition;Finally,the feasibility of the proposed scheme based on multi-sensor image feature fusion is verified by comparative experiments.The experimental results show that the performance of the proposed method is improved by more than10%compared with the traditional machine learning experiment,and the performance is improved by more than 6%compared with the deep learning,and the accuracy and(6cro₁ can reach up to 93.9%.In addition,the feasibility of the four image coding methods based on time series data is verified by ablation experiments.The experimental results show that the recognition effect of Grammy Angle Sum Field is the best.The ablation experiments verify the rationality of the structure design of the fusion depth neural network based on multi-sensor image and the applicability of the fusion strategy.This thesis makes full use of the advantages of deep learning technology in image classification,and transforms the classification problem based on sensor data into the classification problem based on image data.At the same time,a variety of sensor data features are fused,and the recognition effect is significantly improved.It provides a new idea and method for the research of user behavior recognition based on mobile phone sensor in the future.