Video Object Extraction Based On Spatio-Temporal Information

Object-based Image compression is one of the research hotspots in low bitrate video compression technology in recent years. By extracting moving objects whose shapes are discretionary, we can rebuild the image preferably and improve the coding efficiency. The most important characteristic of MPEG-4 which is the second generation video coding standard is using video objects to describe the contents and codes, which needs to extract video objects firstly. In this standard,the extraction of video objects is still an open part. The advantage anddisadvantage of the extraction algorithm is important to MPEG-4 coding quality.Video segmentation plays an important role in the application of multimedia, and has a promising prospect in video coding, video browsing and multimedia interacting. It is also a key technology in computer vision. Due to the complexity of video contents, there isn't universal segmentation arithmetic in the world. So we should put more effort to improve it.In the paper, the key techniques, which are involved in MPEG-4, are studied.At the same time, the extraction algorithm of video, oriented by MPEG-4, is discussed. The main work consists of two parts: one is video extraction techniques and theoretics, compares and researches existing extraction arithmetics, and makes deeply research on spatial-temporal domain joint arithmetic based on moving detection. The other is to propose an automatic extraction method based on spatial-temporal information. In temporal domain, the high-order moment of differences among multi-frames is utilized to detect moving areas, while in spatial domain, edge detection is used.The main methods of temporal domain division are optical flow method, change detection, moving estimation, etc. These methods all have their characteristics, and suit different video sequence. Most division techniques of temporal domain are based on these methods. Change detection is used popular at present. As to the video sequence of static background or the scene which has simple moving, we could use difference or moving compensation change detection arithmetic. This arithmetic distinguishes the changed and unchanged area of two near frame image in temporal domain, divides the pels into different areas according to adjudgement. According to this, we could extract the moving object.In this paper, the frame difference is adopted. Because this paper aims to the video where quantity of motion is smaller, we choose the five-frame difference to get moving information instead of generally used two-frame difference. If we adopted two-frame difference, there may be holes in moving objects because two-frame difference doesn't always get enough information. According to the experiments, seven-frame difference can provide satisfying performance.In spatial domain, Canny arithmetic operator is used during edge detectionwhich is based on the gray information of a single frame. The combination of spatial extraction and temporal extraction employs spatial and temporal movement detection to maximum advantage. In temporal domain, we use continuous multi-frame images to find out moving areas and corresponding mask images, which provide cursory binary mask. Spatial edge detection providesaccurate boundaries of VOP. The combination of the two methods overcomes the shortcomings that the sizes of the extraction boundaries based on multi-frame difference are larger than that of actual moving objects. At the same time, the imprecise extractions, which are caused by covering and noise, are avoided. So the accurate boundaries of moving objects are obtained. The detailed algorithm is as follow:As to still background sequences, in the continuous seven-frames Accumulative-Difference images, we adopt the supposed detection method to confirm automatically the foreground and background of video sequences and get the mask of moving objects. Because of the influence of outer interference, there is noise in video images. After threshold division, there still exist many useless noise spots in the background and target object. In this paper, the mathematical morphology is used to process the binary images extracted. Basic algorithms of mathematical morphology include dilation, erosion, open, close, and so on. Combining open with close can form a morphology noise filter. After noise spots are filtered, there may be holes in binary mask. Aiming at this case, we apply one row and one column scanning to fill the inner holes. And with morphology and median filter we can smooth boundaries, remove short branches, remedy the narrow gaps of boundaries.In the paper, we adopt summing up continuous seven-frame frame-difference, so the sizes of the binary mask of moving areas calculated are larger than that of the actual objects. The bigger the displacement of the moving objects between two frames is, the bigger the difference is. So the moving objects extracted contain background information. And we can revise the edges of the moving objects extracted using Canny edge detection algorithm. Canny algorithm is one of the optimization edge detection, it can get single pels width edge. The method of spatial-temporal joint combination could get the object mask exactly.The algorithm mentioned above can separate the moving target form still background preferably, which is practicable.