Research On Data Interaction And Access Model Oriented To Distributed Environments

The technology of data exchange and sharing between heterogeneous systems indistributed environment is one of the key technologies of distributed field. With theincreasingly complex of network structure and the expanding of the network scale, theheterogeneous environment with distributed characteristics increasing, and various kinds ofdata quantity is increasing dramatically, the problem of "information island" goes worse. U.S.Internet Data Center indicates that the data of Internet will grow by50percent per year,doubling every two years. The data generated in recent years, accounting for more than90%ofthe sum of the world’s data. In summary, the characteristics of distribution and the amount ofdata explosion, people gradually enter the big data and information age, the data exchange ofsystem or between systems has a lot of shortage in reliability, flexibility and real-time, etc.Currently, education information construction of China in the infrastructure and resourcesconstruction has begun to take shape, but in terms of integration information construction isinsufficient, and a better of product in data exchange and sharing is needed.In order to satisfy the deficiencies above, this article is intended to the field of dataexchange and massive data access in a distributed environment to do some research. In thedata exchange, firstly, reference database replication mechanism, combined with differencessynchronization method, I proposed a data synchronization method which make transaction asa unit and based on change logs. Then, eavesdropping operation by the transaction, the datachange operation is split into atomic SQL operation, and analyze the relationship betweeneach operation, use the log entry merging algorithm to write the changed data into update log.Secondly, based on the existing data transfer and synchronization scheme, using messagingmiddleware technology, I proposed a data interaction model based on publish/subscribe forthe transmission of data interaction. Finally, the combination of the three parts to constitutethe common data exchange interface model for distributed environments. In terms of dataaccess, based on the schema of relational database-based with NoSQL-supplemented, Iproposed a model that relational database-based with NoSQL for a high speed data cache. Inthis model, I created a function of database field access frequency, and stored the data that theaccess frequency is greater than the threshold value to NoSQL, thereby increasing the cache hit rate. Interactive interface data model and data access model composed interact with dataaccess model, the model is applied to degree in information reporting as an applicationexample, designed and implemented a components of degree in information reporting. In thisinstance, through the device of changing data capture to capture the information of degreechange on college side, using merge algorithms to compress the changed data and written intothe change log. Then using message middleware with group of agent to transmitting thechange log to the degree center side, the degree center using update listeners to parse changelog and synchronize data to a relational database. Then the degree center side usedsynchronization controller monitors to transmit the data that changes in the relational databaseto HBase, and use the controller of reading and writing to control user’s read and writeoperations and update database field access frequency. Experimental results and applicationresults show that the model I proposed has important theoretical significance and applicationvalue, and it also can effectively meet the requirements for data exchange on the degree ofdata reporting process in flexibility and real-time.