Neural Network Simplification Based On Tropical Geometry

Deep neural networks(DNN)have shown outstanding performance in multiple research fields including computer vision,speech recognition,natural language processing,quantum chemistry,and healthcare.However,as we all know,our theoretical understanding of its validity is still incomplete.The deep learning architecture needs to continuously optimize the model parameters "end-to-end" by labeling the errors driven by big data and using the backpropagation algorithm.This learning process is like a "black box".As machine automatic black box algorithms begin to assist humans in making decisions,these mechanisms need to be explained more.In addition,although these large neural networks are widely used,they consume storage,memory bandwidth,and computing resources.Therefore,in order to reduce the resources required to run large networks so that they can run in real time on mobile devices,we try to trim the network.In order to achieve this goal,we propose a new method with the help of tropical geometry,which not only removes redundant and unimportant items,compresses the network in a way that retains the original accuracy,but is also interpretable.Tropical geometry is an exciting new field at the interface between algebraic geometry and combinatorics with connections to many other fields.At its heart it is geometry over the tropical semiring,which is R?{-?} with the usual operations of addition and multiplication replaced by by maximum(minimum)and addition respectively.This turns polynomials into piecewise linear functions,and replaces an algebraic variety by an object from polyhedral geometry,which can be regarded as a “combinatorial shadow” of the original variety.Therefore,the explicit connection between the feedforward neural network with Re LU activation and tropical geometry has been established,and it is shown that this neural network family is equivalent to the tropical mapping family.Therefore,this paper uses the established explicit connection between the activated feedforward neural network and tropical geometry,and shows that this neural network family is equivalent to the tropical mapping family as the basis,starting from the tropical hypersurface,according to the variables in the tropical Based on the linear correlation on the hypersurface,the region division result is obtained,and the feedforward neural network is compressed according to the divided region,so as to achieve the optimization of the pruning of the neural network.