Research On Adaptive Method Of Sequence Feature Extraction Based On Neural Architecture Search

With the rise of automatic machine learning(Auto ML),automatically performing feature engineering and model selection has gradually become a new research hotspot.Neural network search(NAS)has been a key technology in the past two years.The rapid popularity over time can basically be divided into the proposal and improvement of new search algorithms and actual scenarios including the specific deployment of mobile terminals.The research fields involved in these works are all based on the image processing,while the research in the field of natural language processing(NLP)is very few.In order to gradually extend the neural architecture search technology and its latest achievements to the field of natural language processing,the relevant methodological research is carried out here.Considering that many models of text sequence processing are models based on sentence features,which can boil down to learning sequence features,this thesis considers automatic selection of feature extraction models and searches for feature extractors based on neural architecture search.In order to make the work more versatile,the thesis studies the adaptive method of learning sequence features,which can specifically include three capabilities.The first one is the ability to automatically find the corresponding optimal model according to a certain task.The second one is the ability to migrate the discovered model between datasets within the same task,and the third is the ability to migrate the model between different tasks.Three experiments were carried out to evaluate the three adaptive capabilities.In terms of search strategies,the thesis introduces the current mainstream search strategies in detail and selects the reinforcement learning-based algorithm and gradient-based bi-level optimization algorithm after analysis.However,in actual experiments,when the search space is large,gradient-based bi-level optimization algorithm needs to consume more GPU memory to store the model structure diagram.The experimental equipment in this thesis cannot support it,thus although it is difficult for the reinforcement learning algorithm to tune the hyperparameters,it is still selected as the search algorithm.The thesis introduced classic models and emerging models in the field of text sequence processing and redesigned the search space.It also reconstructed NAS STOA work based on image processing tasks and migrated the NLP task as the starting point for the thesis' s subsequent architecture search experiments,providing a good starting point and comparison of results for algorithm selection,search space design and experimental design.Experiments results prove that the network structure searched based on the thesis algorithm has a good adaptive ability.The results are in the lead compared with either artificial models or reconstructed NAS works.The thesis successfully explored the application of NAS technology in natural language processing scenarios,and successfully transformed the latest achievements in the field of natural language processing into NAS,as well as the successful experience of NAS technology in the field of computer vision.The thesis result shows that the application and expansion of NAS technology in the field of NLP have possibility and potential,helping the follow-up work continue to explore a more versatile and extensive automatic method of sequence feature extraction.