Harmonious Competitive Autoencoder For Text Representation Learning

Unsupervised text representation is an interesting and challenging task,which learn the representative vector for text in vector space through unsupervised learning.The representative vector can be used to various subsequent text precessing or data mining tasks.Most data in the Inertnet are textual data.In order to effectively use these data and reduce the human cost of manual annotation nad intervention,an accurate and efficient text representation is urgently needed.Autoencoder is an unsupervised shallow neural network which tries to reconstruct its input at the output layer.Recently,numerous studies have proposed many different autoencoders.These autoencoders have been successful in learning meaningful representations from image datasets.However,their performance on textual datasets has not been widely studied.In this paper,we conducted experiments on text representation learning by basic Autoencoder(AE),K-sparse Autoencoder(KSAE)and K-Competitive Autoencoder(KATE),aiming to explore the influence of automatic encoding mechanism,competition mechanism and various model structures on representation learning in text data,and try to find out the mechanism that can make competitive automatic encoder effectively act on text representation.In addition,we introduce the harmonious competition mechanism in autoencoder,proposes the Harmonious Competitive Autoencoder(HCAE)to optimize the competition mechanism and make the competition process more flexible.Finally,this model achieves good results in different text processing task evaluation.