Research On Visual Real-Time Processing System Based On Heterogeneous Multi-Core Processor

The rapid development of multimedia technology and heterogeneous multi-core technology has made heterogeneous multi-core processors a hot research topic for major semiconductor companies.Heterogeneous multi-core processors can integrate a variety of processing cores with different structures but complementary functions.Currently,they are widely used in the field of automatic detection of products with large amount of video processing and high real-time requirements.With the continuous improvement of product automation detection,people have higher requirements for product detection accuracy and detection efficiency.It is difficult to meet the test requirements by relying only on manual or general-purpose embedded processors.Based on this,a high-definition vision real-time acquisition and processing system based on heterogeneous multi-core processor AM5728 is studied in this paper,and an on-line real-time detection experiment is carried out for the appearance defects of metal rotating parts in a factory.Firstly,the video acquisition system of heterogeneous multi-core platform AM5728 is studied.To realize the video acquisition of HD Camera Link industrial camera,a set of FPGA-based Camera Link video capture card is designed.The hardware design flow of the video capture card is described in detail,and the software of the visual acquisition system is introduced from three aspects: conversion of Bayer Format to BT.1120 Format on FPGA,support for BT.1120 format on AM5728 and realization of test software.Finally,the video acquisition of high-definition Camera Link camera is realized on AM5728.Secondly,the algorithm flow of the rotating body appearance defect detection system is studied.This part will introduce four parts: image preprocessing,region location,defect detection and defect feature extraction and recognition.The image is preprocessed by means of median filtering,and the edge detection method is used to locate the specific detection area of the product.In order to enhance the defect information and mitigate the influence of uneven illumination,an image difference algorithm based on nonlinear diffusion is applied in defect detection.At the same time,aiming at the problem that traditional single-threshold segmentation can not accurately segment multi-peak differential images,a multi-threshold segmentation of 2D Otsu based on CJADE algorithm is proposed to threshold the image.Finally,according to the result of threshold segmentation,corresponding defect feature extraction and defect recognition are performed.Finally,the algorithm implementation and performance optimization of heterogeneous multi-core platform AM5728 are studied.The system design scheme of appearance defect detection is introduced.According to the accuracy requirements of vision system,the selection parameters of camera and lens are determined.The design and implementation of the overall software architecture are mainly introduced,including user interaction interface design based on QT5,algorithm implementation based on Open CL heterogeneous programming model,H.264 encoding based on multimedia framework GStreamer and communication control.Finally,the test system is applied to the actual factory for testing experiments.The experimental results show that the product defect detection system based on heterogeneous multi-core processor can better complete the initial inspection requirements of the factory products.