The Relationship Between Sparse Coding And Energy Consumption In Neural Networks

The most important function of neurons is to encode all kinds of information into a sequence of action potentials and transmit them to the corresponding regions of the cerebral cortex.Only by understanding the mechanism of neural coding can we understand how the brain processes information and how advanced brain functions are achieved.Researchers in the cognitive neuroscience community have put forward many kinds of theories and models about neural coding,but there are many shortcomings and limitations.Energy coding theory believes that there is a certain relationship between neural coding and energy consumption in the brain,reflecting the principle of maximizing the energy utilization and coding efficiency.Sparse coding existing in the visual system also embodies these two principles and it also closely related to energy.In this paper,sparse coding is simulated by artificial neural networks and the relationship between sparsity and energy consumption of neuron clusters is investigated.We first introduce the shortcomings of traditional sparse coding models and many improved sparse coding models.Then we discuss some commonalities between fast independent component analysis(FastICA)and sparse coding algorithms,both of which are essential to reduce the signal correlation and reduce redundancy.Then we consider and evaluate whether it is possible to search for independent and irrelevant features by FastICA to implement sparse coding.Since natural images are non-Gaussian and irrelevant to image content and image size,applying FastICA in sparse coding is possible.Then we establish a sparse coding model based on FastICA and apply it to the sparse coding algorithm.The sparse coding based on FastICA and the traditional sparse coding are compared respectively from three aspects:the training time of feature base,the convergence speed of objective function and the sparsity of coefficient matrix.We found that the former performs better than the latter.At the same time,we briefly discuss the relationship between sparse coding and energy,which lays the foundation for the subsequent research on the relationship between sparsity and energy consumption of neuronal clusters.After comparing the sparse coding based on FastICA with the traditional sparse coding and verifying the validity of the former,we simulate retina ganglion cells by the sparse coding model based on FastICA.Then we use the output of model training to simulate the effect of retina ganglion cells on nature Scene and random chessboard response.We mainly simulated the responses of retina ganglion cell population and individual ganglion cell to natural image stimulation and artificial stimulation.We measured the effects of retina ganglion cell on the stimulation of different stimuli and the results show that single ganglion cell show sparse features under the stimulation of natural images and random checkerboard motions.The response to natural image stimulation is sparser than the random checkerboard motions,and ganglion cell populations are sparser under natural image stimuli.We found that a small percentage of ganglion cells in population spontaneously stimulate high-frequency firing,but no high-frequency firing occurs under random checkerboard motility.We believe ganglion cell populations encode natural stimulus information may be more dependent on the high frequency of a few ganglion cells.Finally,we compared the simulation results with the results of physiological experiments and confirmed the validity of the simulation results.The final study is the relationship between the sparsity and energy consumption of neuronal networks.Sparse coding increases the ratio of the information encoded by the neurons to the energy involved,it greatly increasing the efficiency of energy use and the sparsity of neuronal clusters is inextricably linked to energy consumption.Based on the previous studies,we explore the proportion of active neurons in the most energy-efficient neuron networks based on the ratio of signaling to fixed costs.We first study the relationship among the total energy consumption of different neuronal networks,the ratio of signaling to fixed costs,the sparse proportion of active neurons in neuronal networks under a similar coding information capability.We found that the sparse proportion of active neurons in the most energy-efficient neuronal networks are related to the ratio of signaling to fixed costs in neurons.We calculated the signal energy consumption and fixed energy consumption through the data of the physiological experiment respectively,and give a more accurate ratio of signaling to fixed costs of the neuron release signal as 1.3?2.1.With this relatively accurate ratio,we study the relationship between the total energy consumption of different neuronal networks and the sparse proportion of active neurons in the networks when the representational capacity of coding is approximate,and find that the networks with the sparse ratio between 0.3 and 0.4 have the largest representational capacity and the minimum total energy consumption,the calculation results are more consistent with the related physiological experiment results.The research not only determines the ratio of signal to fixed energy consumption of neurons,but also proves that the sparse neural coding model is an energy-saving neural coding model.The results of this research is in line with the maximization of neural signal transmission efficiency in the brain and the maximization of energy utilization theory,it may be helpful for the future research on neural coding theory.