Research And Application On Computer Vision Inspection Based On Forward Looking Video

As a mobile shooting way for acquiring physical scenes, forward looking video has been widely used in mobile scene surveillance and objection detection tasks due to the wide filed of view and extensive range of coving space. However, with the increasing of the video volume, many time-consuming vision algorithms could not meet the requirements of real-time inspection, and it’s difficult for the storage and retrieval of mass videos. In this thesis, based on the forward looking video, we carry out the research work from the aspects of theory and application. We first propose a panoramic ring-strip sampling method based on the geometric structure of detected region, and build panoramic stitching algorithm for fast forward looking video to achieve the lossless information extraction from vast amounts of video and obtain a lightweight panorama format, which not only reduces the overhead of data storage and access, and change the video into a more suitable media form for manual inspection or computer-aid analysis. Then, we respectively propose automatic inspection algorithms based on generated panorama to perceive the rail condition and detect the railway fence defects.The main research contributions are shown as following:1. We propose a panoramic annular band sampling method based on the geometric structure of detected region, and acquired the panorama of physical scene from forward looking video. Different from the existing image mosaic method based on feature matching and optical flow estimation, the proposed method can generate the panorama from forward looking video fast and simply without time-consuming feature matching and complex optical flow calculation, we only utilize the camera motion information and the geometric prior of spatial scene to realize the construction and alignment of the stitching region.2. We propose a "Bi-slits projection" monocular stereoscopic imaging method. First, we extract a pair stitching strips with different perspectives from each frame to generate two panoramas with significant disparity; then analyze the principle of stereoscopic panoramic imaging to derive the formula of calculating the panoramic depth; we finally generate the stereo panoramic pairs using proposed stitching method from forward looking video, and estimate the difference of pixels’location between stereo panoramic pairs based on stereo matching algorithm to gain scene depth information. Stereo panorama shows us an extensive view and strong reality, and the depth information will also assist the manual inspection or provide richer decision information for computer-aid inspection.3. We propose a simple and effective method to detect the train-swaying quickly and automatically based on the generated rail track panorama. We first generate the long rail track panorama (RTP) by stitching the strips cut from the video frames, analyze the relationship between the unevenness on the RTP and train swaying, and extract track profile from the panorama. The unevenness of the track profile is due to the fact that the train as well as the camera experiences a pitch shaking at a non-smooth connection spot on the rail track. Such a location needs to be examined repeatedly toensure the safety of the rail. Therefore, the condition of rail track can be assessed intuitively afterwe obtained the rail track panorama (RTP) from train-borne video.4. We propose an automatic inspection method for detecting the railway fence defects based on generated fence panorama. Once we get the whole panorama of continuous fence, the railing location of railway fence can be extracted as foreground by image segmentation algorithm; then we recognize the fence defect by the analysis of distance between the adjacent railing, based on the truth that the distance between any two normal railing is more or less fixed, while broken fence often causes the railing missing, which will cause the fixed distance change several-fold. We propose a maximum entropy thresholding segmentation based on three dimensional histogram MVG to realize the railing location, and encode the fence panorama by using run-length coding. The compressed encoding is an effective representation and storage format, which contains all railing location information and dramatically reduces the storage overhead. Meanwhile, the corresponding decoding algorithm is also designed to recover the fence railing location and realize the detection of fence defect.