Research And Implement Of Optical Music Recognition

The conversion of music scores in paper form into the digital ones is the necessary accessonly by which could have information been exchanged between the human music activities andthe computer music processing. Optical music recognition (OMR) is the process of analyzing thescanned image of a music score, and recognizing the music objects in the score, finally achievinga versatile machine-readable format. It is the OMR that breaks through the bottleneck of thepurely handcraft activates in the traditional music notation digitalization, and provides a smart,high effectiveness and efficiency method for the music digitalization. Therefore, OMR meansgreat value in both theoretical research and the applications.This paper takes multi-voice score as the research object, and performs a thorough researchon OMR from four aspects, including location and removal of staff lines, recognition of musicnotes, recognition of specific music symbols, reconstruction and semantic interpretation of themusic score. New practical theories and effective approaches are presented to the several keypoints as follows.The location and removal of staff lines is the first step in the OMR process. The paperinvestigates an integral method for image distortion correction and staff lines location. Theessential of this method is to improve the existing horizontal projection method by the idea of"breaking up whole into parts and correlation computing". The new method not only keeps themerits of the projection method such as simple and stable, but also is not sensitive to the distortionof the images. It resolves the conflict between the distortion resistance capability and noiseresistance capability, which exists in both statistical and structural strategies for locating stafflines.In the staff lines removal aspect, for resolving the issue of "over-removal", this paperproposes a new removal algorithm based on analyzing topological relationship among the imagesegments. Comparing with the current staff lines removal methods, the new algorithm places moreemphasis on association relations among environmental features of the segments to be removed,and thus identifies the differences between the staff line segments and non-staff line segmentsmore clearly and completely. As a result, staff lines can be removed without damaging symbolsaround staff lines.Recognition of music notes is the core and key in OMR. Considering the variety andpolymorphism characteristic of the music notes, a structure-based recognition strategy is chosen,in which there are two procedures: primitive extraction and structure analysis.In the procedure of primitive extraction, three methods are proposed respectively for threeshapes of note primitive: first, stem extraction is accomplished by searching stem candidatesroughly from vertical run length units plus checking the candidates strictly in terms of horizontal run length properties, and this method can overcome the weaknesses of the current method such asthe fragmentary results and poor adaptability to the complex notes; second, a "segmentation withfeature testing" method for extracting black note heads is designed, which employs the priorknowledge of notes and the information of recognized staff lines and stems to separate the noteheads that touch; third, a extraction method for beams similar to treating the black note heads isproposed, and this method can avoid the troublesome touching issue in the former line detectionmethod for extracting beams.In the procedure of structure analysis, an innovative approach is discussed for the structureanalysis of music notes based on the action field. By introducing the physics concept of actionfield to describe the association relationship of note primitives, it has good qualities withknowledgablity, robustness and precision. Further more, six substructures of notes are defined, anda model with priority identification for key structures is set up. This model imitates the wayhuman recognizes music score, which is prone to focus on point features and observe objects fromwhole to detail, so it helps to reduce the calculation complexity greatly, and gets rid of incorrectredundancy primitives successfully.Besides music notes, there are many text-like music symbols in the score. This paperproposes a method for those specific symbols based on classifying three groups of feature,including geometrical, normalized central moment and slicing feature. Such features have bothcapability of anti-noise remains to statistical feature and capability of distinguishing delicatedifference remains to structural feature. Additional, a three-level BP Neural Network with strongability of non-linear classification is employed as classifier. Experimental results with life scoresshow that the presented method can recognize the symbols effectively.Finally, by building the "tree-like structure of music score", this paper realizes theorganization and reconstruction of the music score from messy recognized data. Thereafter, bygenerating music event sequences, semantic interpretation of music score is carried out, and thesequences are written to a file by standard MIDI file format.As one of the main research achievements of this paper, a complete OMR prototype systemIOMRS is developed. This paper also proposes a performance evaluation criterion based on musicsymbols and semantics. In the light of the criterion, the IOMRS system is tested and comparedwith commercial OMR systems. The testing results show that the overall performance of IOMRShas achieved the standard of the current excellent commercial OMR system, and show obviousadvantages in music notes recognition, adaptability to the variety of raw music scores and thespeed of performance.