The Technology Research Of Sleep Improvement Based On Lightweight Reinforcement Learning

With the advancement of technology and the accelerated pace of people’s lives,insomnia symptoms are appearing in more and more people,and sleep disorders are gradually becoming a major public health problem.While traditional sleep improvement methods such as drug therapy are more invasive and less safe,white noise-based physiotherapy is a safer way to improve sleep.In recent years,most of the applications and studies on white noise sleep improvement have been manually pre-programmed or screened for white noise,which requires expert support and is generally conducted in hospitals at a high cost.Most of the automatic control of white noise can only be controlled by simple start-stop control of white noise according to the patient’s sleep state,and there is a lack of complex solutions to allow white noise to automatically adjust the playback frequency according to the patient’s current state.In addition,the pre-programmed strategies for white noise playback do not change,and there is a lack of a scheme that will self-learn and grow with the patient’s use of the strategies for a particular patient’s insomnia condition and sleep habits.Therefore,this paper addresses these shortcomings and proposes a framework for closed-loop feedback sleep improvement based on reinforcement learning,which can be summarized as follows.The Reinforcement Learning-Sleep Improvement Framework(RLSF)based on reinforcement learning is proposed as the overall framework layout,and the framework improves sleep by playing white noise of different frequencies as a stimulus.The strategy in the framework can observe the feedback of the subjects to the white noise and self-learn to grow according to the subjects’ physicality,and the framework is highly personalized.The framework is built to compare in ex devices and inexpensive embedded devices respectively,and fits multiple usage scenarios.Deep RLSF based on deep reinforcement learning and RLSF is proposed,built on a high-performance platform,and a scheme to simulate the change of subjects with white noise frequency is designed for cold start of the program,and a scheme to describe the state of the subjects is also designed to feed back to the policy for updating.A Lightweight RLSF based on the Lightweight Reinforcement Learning algorithm and RLSF is proposed as an improved version of Deep RLSF,built on a low-performance and inexpensive embedded device,using a table-based finite state machine to abstract the sleep process,and reducing the subjects to simple matrices to optimize the amount of operations.The white noise is divided into two perspectives:time domain and frequency domain for reinforcement learning,and a lightweight algorithm is used to update the computation strategy to optimize the amount of operations and reduce the computation time,so that the program can run in a stable and real-time embedded device.Finally,a real-life experiment was conducted to track and evaluate the sleep improvement status of the subjects,which proved the effectiveness and practicality of Lightweight RLSF.