Single Channel Music Separation Based On Nonnegative Matrix Factorization

Music creations and performances are the most intricate of our cultural artifacts. In recent years, as the demand for automatic analysis, organization and sophisticated manipulation of musical signals, the task that music sound separation attempts to separate individual instrument from a polyphonic mixture has attracted much attention. However, only a single channel recording is often available. In this paper, we address the problem of monaural musical sound separation, where multiple instruments are linearly and instantaneously mixed to a single channel signal. The Nonnegative Matrix Factorization (NMF) can effectively decompose the magnitude spectrogram of music signal into a series of basic note components, while two or more instruments are played simultaneously, time-frequency overlapping is considered to be the main factor affecting separation performance. In addition, it is difficult to cluster the factorized elements and associate them with a specific music source. In order to solve above problems, this paper includes two parts as follows:In order to avoid the clustering, the MIDI score is used to supervise the Itakura-Satio Nonnegative Matrix Factorization (IS-NMF) process in this paper. This method initially synthesizes the MIDI score of each instrument separately, the synthesized signals are decomposed individually by IS-NMF, then the activation coefficients and spectral basis of each instrument are used to initialize the IS-NMF decomposition of the mixture music. Eventually, we use the factorization obtained from the separation stage to extract the signal of each estimated source with the Wiener filtering technique. The simulation results show that this method gives better separation in comparison with other score-informed source separation methods from the literature.In order to alleviate the time-frequency overlapping of musical instruments, this paper employs the Two-Dimensional Sparse NMF Factorization (SNMF2D) that extends the NMF to be a sparse two-dimensional convolution model. With regards of the musical component ambiguity in SNMF2D, Empirical Mode Decomposition (EMD) is adopted to decompose the mixture into a series of oscillatory components terms as the Intrinsic Mode Functions (IMF). According to the degree of mixing in each IMF, the sparse parameters are individually optimized to yield the optimal sparse, and the SNMF2D can factorize the IMF spectrum into distinctive musical components, then the sub-sources in each IMF is estimated using masking and Inverse Short Time Fourier Transform(ISTFT). Finally, the estimated sub-sources are clustered into original music sources. Experimental simulation has been conducted to show that the proposed method gives a slighter improvement in separation quality than the SNMF2D method.