| Music is an auditory art closely related to people’s society and plays an essential role in social life.However,there is a high threshold for music composition,which requires the compositors to have adequate music knowledge and visionary talent.With the vigorous development of short videos and self-media in modern culture,the demand for self-created music is increasing quickly.The requirement merged to explore machine learning methods to develop automatic music generation methods.With the rapid improvement of Artificial Intelligence,various music composition methods have sprung up.The current music generation technology is mainly divided into three types.One is the music generation algorithm based on a language model.The second category is the algorithm based on the adversarial generative model network（Generative Adversarial Networks,GAN）.The third type is the music generation method based on reinforcement learning.Among these methods,the composition method based RL may be appreciate to music composition task.In theory,if a reasonable agent and reward functions are designed,automatic learning and automatic music generation can be realized to achieve at the level of human musical composition,conform to the rules of music theory and is pleasant to the ear.However,it is difficult for designing the reward function in reinforcement learning to cover all types of music.Therefore,exploring the reinforcement learning framework of similar music styles is a possibility on the road of music generation.This thesis proposes a novel music generation framework based on the reinforcement learning method suitable for music generation with similar styles.This framework can learn unique music styles and has a theme for the music generation task.It provides a potential framework for the reinforcement learning-based music generation method to be extended to multi-style learning in the future.The main work of the thesis includes:1.A novel reward function is proposed.Integrating the structural information of music creation into the music reward function is the essential issue of music generation based on reinforcement learning.The current manual rules have high production thresholds and costs for reward functions.Using context-based network models as reward functions ignores structural considerations in music creation.Given the shortcomings of the above methods,this thesis proposes a method to construct a music reward function by extracting features from a multi-scale topic model based on music theory.The method can capture the structural features of music by carefully designing feature extractors with different musical granularities.And then,it integrates the musical structure characteristic contained in the music sequence into the reward function by designing a expression function on multi-scale musical granularity.The experimental results showed that the proposed method could achieve a better performance in music generation than the reward function based on manual rules and contextual relationships.The results of the subjective and objective evaluation were significantly improved.This method solves the problem that it is challenging to formulate rules due to a lack of music theory knowledge and makes up for insufficient use of structural information in music creation by the contextual network model.2.A novel structure of agent for music generation task is proposed.The current music agents are mainly based on Long Short-Term Memory neural networks（LSTM）.Due to the limitation of their modelling capabilities,the generated results are highly similar.In order to solve this problem,this study firstly analyses explicitly the reasons for its occurrence and verifies that high similarity between adjacent states before and after the generation process is driven by the information transfer of LSTM.For this reason,this study modifies the loss function and adds a metric that limits the distance to restrict its convergence direction.In addition,the components of the intelligent body are also redesigned to enhance the memory of past notes,and the attention mechanism is also utilized.The experiments showed that the performance of the proposed method exceeds that of the current long-and short-term neural network based on the distribution and audibility of the generated music effect based on the attention mechanism.3.A general reinforcement learning framework for single-track music generation tasks with the similar musical style is proposed.The optimized agent is integrated with the reward function learning method to realize the system integration between components and build a general reinforcement learning framework for single-track music generation tasks based on the similar music style.The experimental results showed that the agent could learn music structure information through this framework,and the results generated by the model were closer to the music dataset.The framework proposed in this study is a beneficial attempt at reinforcement learning framework in music generation.It can realise the paradigm of learning music composition based on large-scale music data sets,improve the model’s generalisation ability,and then improve the performance of automatic music composition.This framework is beneficial for lowering the threshold of music creation,reducing the cost of music composition. |
