Research Of Machine Learning Compression Algorithm

The rapid development of artificial intelligence is inseparable from the study of machine learning.In the early years,limited to the computing power of hardware devices,machine learning can only be used to deal with some simple classification and regression problems.In recent years,with the advent of large computers,the development of parallel technologies,and the emergence of high-performance GPUs,people can train and deploy larger-scale machine learning algorithms to handle more complex tasks such as image segmentation,face recognition,and advertising recommended.As an important branch of machine learning,deep learning has been widely used in related tasks such as image,video,and voice,and has made great achievements.According to the characteristics of deep learning,people build deeper and deeper networks to obtain better recognition results,but this also means that deep learning models require huge computing resources and memory space to achieve high recognition results.With the popularity of mobile devices,the migration of deep learning models to mobile has become a trend.However,due to the computationally intensive and storage-intensive features of the deep learning model,it is unable to meet the real-time requirements of mobile devices such as mobile phones and tablets.So,deep learning model compression has become a new research hotspot,especially the hardware-oriented compression algorithm is more popular,which lays a foundation for the deep learning model to land on the microelectronic system.The first major work of this thesis is to propose a forced linear unit deep learning compression algorithm,which is a hardware-friendly compression algorithm and can adjust different sparse structures,such as channel-level sparse structure,row-level sparse structure,column-level sparse structure,filter-level sparse structure and depth-level sparse structure,to adapt different handware compression environments.At the same time,the algorithm is an end-to-end training mode,which can learn the sparsity of the model to a greater extent,get higher sparsity rate,and use hardware resources more effectively.As a third-generation neural network,spiking neural network has more powerful information capacity and bio-plasticity than the second-generation neural network,and it is a trend of future neural network development.Spiking neural networks are mainly used to process spiking signals and timing-related tasks,and have achieved some success in video and audio.However,the fixed structure of most spiking neural networks limits the application of spiking neural networks,and evolving spiking neural network algorithms have been proposed to solve such problems.Most evolving spiking neural networks use an additive strategy to adjust the structure of the spiking neural network.This strategy cannot fully utilize the interrelationship between data and has room for further improvement.So,the second major work of this thesis is to propose the SpikeCD evolving spiking neural network algorithm,which relies on the clustering degeneracy strategy,a combination of supervised learning and unsupervised learning,dynamically adjust the structure of the spiking neural network.The algorithm can achieve better recognition accuracy and realizing simple network structure.At the same time,the algorithm has strong robustness,is insensitive to parameters,and saves a lot of parameter optimization time.