Research And Implementation Of Video-based Mmwave Sensing Data Generating Method

In recent years,there has been widespread adoption of human sensing technology in smart homes,gyms,and security systems.This has led to the development of various related sensors and algorithms.Additionally,as public awareness of privacy has increased,millimeter-wave(mmWave)radar has become a topic of extensive research due to its high-precision and high-privacy capabilities.However,the existing research on human sensing with mm Wave radar has been limited by the lack of available data.Therefore,a solution to the lack of mm Wave data is urgently needed.To address this issue,this paper proposes a video-based mm Wave sensing data generation method.This method takes video as input and outputs corresponding synthetic mm Wave data.The synthetic data is then converted into various data types,such as micro-Doppler data and pointcloud data,according to downstream application requirements.The proposed method consists of four main modules:a multi-target human mesh fitting module,a spatial reflection module based on ray tracing,a data generation module,and a Transformer-based data enhancement module.In the multi-target human mesh fitting module,a single-target human mesh fitting algorithm is adapted to multiple targets through a box matching algorithm based on speed prediction.The spatial reflection model uses the ray tracing method to simulate the multipath reflection and attenuation of mm Wave signals in space.The mm Wave data generation module calculates the original mm Wave data and converts it into various data types for downstream applications.The Transformer-based data enhancing module improves the data fidelity.After testing,the proposed method was found to generate high-quality synthetic mm Wave data and improve downstream application model performance.