Research And Implementation On Multi-semantic And Nonlinear Agricultural Advisory Video Retrieval System

The agricultural expert consultative system spreads agriculture technology by computer network as the medium, which utilizes the professional knowledge to solve agricultural problems to improve the income of farmers. As the popularity of network in villages, the contradiction between the increasement of agricultural consultation and the shortage of agriculture experts are becoming more and more serious. What is more, the problems experts need to solve are highly repetitive. An efficient way to solve the above problem is the development of the agricultural advisory video retrieval system. Nowadays, the existed video retrieval systems are comprehensive platform mostly, and the systems aiming at agriculture field are rare. The video tags are annotated by human labor of these systems, which are not fit for the video annotation of large amount fast and accurately. Besides, videos in these retrieval systems are stored and played integrally, which make it is impossible to play the related video part based on the retrieval content. In view of these above problems in the agricultural video retrieval field, this research designed and developed a multi-semantic and nonlinear agricultural advisory video retrieval system to accomplish the nonlinear playing of video shots and the automatic annotation of video shots. The developed system based on shot as unit not only save the user retrieval time, but also improve the retrieval efficient.The main contributions of this paper are as follows:(1) Video shot boundary detection. A blocked RGB(Red, Green, Blue) histogram was extracted from each frame as the representation, and a histogram intersection method was adopted to measure the similarity between two frames. Finally, Support Vector Machine(SVM) was used as classifier to make sure whether there exists a shot boundary. The agricultural video dataset used in this paper contains 819 abrupt transition boundaries and 217 gradual transition boundaries. The precision of abrupt boundary detection is 99.1%, while the precision of gradual boundary detection is 74%.(2) Keyframe extraction of video shot. As the representation of each shot, multi-keyframe was extracted by dint of K-means cluster. The video frames in the range of video shot are clustered, accroding to the residuals of classification, the proper amount of cluster center was determined adaptively. Frames nearest to the cluster center was selected as the keyframe of video shot. The video shot keyframe extraction experiment takes the same dataset with video shot boundary detection. Finally, a shot list of keyframe extraction result with brief analysis was given.(3) Video shot based on multi-semantic annotation. Random Forest was constructed as classifier. The voting results of all random trees were summarized, then the probabilities of assigning labels for each keyframe were calculated. The final predict labels of video shot comes from the weighted summation of probabilities of assigning labels of all keyframes. The experimental dataset of agricultural videos contains 1250 video shots and 184 tags. The experimental result of video shot annotation is 60%.(4) Shot based video nonlinear player. Shot based video nonlinear player was designed and developed. According to the video shot messages of nonlinear index, which constructed with video shot tags, the player could play video shots nonlinearly.