Research On A Service Pipeline System Based On Distributed Environment

As a carrier of the information, data carrying rich and diverse hidden value. With the development and popularization of Internet technology, the spread of information and production has reached an unprecedented speed, data accumulated from various sources has gone beyond our imagination. With various types of data in various forms and wide variety of sources, digging depth information from large amounts of data accumulation to discover more added value has been increasing attention. When dealing with the large volume of data processing and data mining, traditional methods and instruments often feel lack of power, emerging model for specific application scenarios inevitably has some limitations. In this paper, considering the problems of multi-source data platform such as Large amount of data, complex process, and diverse requirements, we take service composition platform as the base point and take advantage of software reuse technology and service pipeline, expand the service platform function by repackaging existing software tools, improve run-time efficiency of service by service pipeline system, build diversification of advanced services by service composition, to achieve efficient, stable and scalable platform based on service lines distributed environment. The main progress in this study are as follows:1) In order to make existing software tools left from former research can be continually used in service pipeline system with low time costs and labor costs in migration or Reconstruction, we proposed a software reuse solution based on service package technology. A large number of previous studies have accumulated a lot of valuable tools, but these tools are not competent in dealing with the new requirements, or be limited in the new environment. With reuse of original software and technology in development, we can build new software quickly, and reducing maintenance costs and development costs in the software development process effectively.in this paper, we proposed a universal software reuse solution to adapt service pipeline system. We repackaging the existing range of different sources of software or components in accordance with the rules of repackaging specification, so that it can work relatively independent in a distributed environment to server the upper system with undifferentiated basic services.2) To improve the efficiency of data processing a model based on distributed environment service line is proposed. Detailed division of labor and produce in pipeline have been shown to improve overall system efficiency in many areas. Due to the more variables of service pipeline and as a further matter, most software performance hardware requirements have not yet peaked hardware can provide currently, there isn’t a lot related research on service pipeline. However, it really has considerable potential. Repackaging basic service properly and taking good advantage of the characteristics of software and services to solve the bottleneck of pipeline, run-time conflict and other issues can significantly improve the throughput of the system and enhance the efficiency of the system. Service pipeline model proposed in this paper organize data and computing resource in form of service, by which we improved the efficiency and performance of the system, we achieved the expected results.3) In order to make service composition stabilize scalable, we propose a service composition model based on service pipeline. Service composition can produce new value-added services with little cost, and make the system functionality flexibility and easy to expand. This study not only presents service composition model based on semantic-based that can discover combination of new services, but also developed the service monitoring and scheduling system that can ensure the service lines platform work smooth, efficient operation, and dynamic expansion.4) To verify the effectiveness of the system model we proposed, we implemented a Web GIS based system as authentication system. The Web GIS system implemented in this paper will provide a variety of data display service for featured Minority Villages by extraction, processing and display of data of a large number of multi-source data, heterogeneous featured Minority Villages. By analysis of contrast with conventional systems, the effectiveness of the theoretical model of this study is verified.