Research On Task Partition And Buffer Optimization Of COStream Dataflow Program

As applications need more and more computing power.Multi-core processors rather than single-core processors become the standard for production work.As a Dataflow programming language,COStream can separate communication and computation to expose the parallelism of program.So it more suitable for multi-core platforms.However,COStream mainly focuses on the improvement of computing performance.It does not manage the memory resources reasonably.The implemented IO buffer can not combine data reading with scheduling,resulting in reduced program execution efficiency.In order to solve these problems,we take the dataflow programming language COStream as the research object.We optimize task partition of Dataflow program.Besides,IO buffer which can make pipeline calculation with data reading work simultaneously and the buffer memory allocation algorithm which can reduce the memory consumption are also designed and implemented.The task partition includes two stages: initial partition,load balancing and communication optimization.The initial partitioning adopts greedy thought to select candidate nodes to construct subgraphs,and then uses the heuristic strategy to move boundary nodes to optimize the load and communication between subgraphs.The IO buffer combines data reading and pipeline scheduling to enable them to execute in parallel.The improvement of pipeline scheduling enable Dataflow program to handle different amounts of data flexibly.Buffer memory allocation algorithm simulates the execution process of the program by traversing the Dataflow graph according to the topological sorting.When the simulation is executed,the memory sharing of buffer can be achieved by analyzing whether the life cycles of different buffers overlap,finally the memory occupation of the runtime program is reduced.The experiment selected universal X86 multi-core processors as test platform and choose serval typical algorithms in the digital media field as benchmarks.By testing new partitioning algorithm,IO buffer and memory allocation optimization algorithm to indicate the effectiveness of the optimization work.