Research On Resources Adaptive Methods Of Optimizational Implementation In Data Layer And Its Application

With the development of software engineering and computer sciencetechnology, in order to improve software productivity and quality, to achieve theautomation of software development progressively and to improve the softwareself-adaptability, the software system architecture has more and more layers whileboth the software and the hardware platform environment having a variety ofstructures and types. Because the science of software engineering is not yet mature,software system development methods, techniques and tools are still having a lot oflimitations, which can not satisfy the quantitative requirements of softwareperformance when at the same time design and implement the software system.At present, following problems still exist in the field of software engineering: Inthe requirements analysis stage, due to the lack of systematic quantitative analysismethod, it is difficult to determine the resource demand of task processing, theamount of data and processing tasks. Without the systematic quantitative method forplanning of resource capacity and the quantitative requirement information whichthe method is based on，It is difficult to achieve an effective match between theresource capacity configuration and the actual resource requirements of running thesoftware；In the software design stage, due to the lack of the amount of data andtasks, the performance information in the UML-SPT (the UML profile, forSchedulability, Performance and Time) design diagram is inaccurate, and it isdifficult to apply them directly to the software code design process; In the softwareimplementation stage, the specific runtime environment and its features，such asprocessing capabilities, resource capacity configuration, are not considered in thecode design and implementation, which result in the poor matching between them;In the end of the software implementation stage, it can only achieve the tinyperformance tuning by the configuration adjustment and environmental resourcesoptimization，the effect is very limited.In this paper，the object of research is mainly based on the data layer in the cluster structure， and the software performance optimization work is shifted fromthe environmental resources optimization and adjustment stage to the softwareimplementation stage. Numbers of optimization methods about resource adaptivewill be studied in the data layer. Based on the resources requirement or consumptioncharacteristics according to the execution of software unit, and with theenvironmental resources processing capability and actual resources consumption, thestorage equalization of data and the scheduling balanced of tasks are implemented insoftware system layer, while the task decomposition and migration is implementedbetween the software system layers. The resources and environment orientedadaptive method is used to improve the effective match between the software andresources，and also to improve the software overall running performance.In order to improve the load balancing and environmental resources carryingcapability between cluster hosts or virtual machine nodes in the data layer, a systemsapproach of representing and gathering the load characteristics and relativebalancing degree of resources is proposed in this paper. Based on the method, wealso propose a load balancing method on data layer. By scheduling tasks betweennodes in the same layer of the resources and the environment，balanced loaddistribution and resources usage can be realized, also the software and environmentalplatform can achieve an effective and reasonable match，which will improve theoverall balance of resource utilization and software performance.How to distribute the limited resources in the database layer to make theheavy-weight and light-weight tasks both achieve a reasonable running result? Amulti-process approach for heavy-weight tasks is proposed in this paper，which canuse multiple threads to process tasks under certain limits by controlling the numberof threads to achieve fully utilization of single-node resources， and it is applied inan actual case as an example in this paper.For the frequent communication between the nodes in the data layer, wepropose a logical data partitioning method, which can optimize the data access andbalance the data storage. The processing process and data access quasi-parallel isused to improve resource utilization and processing performance of the single host(virtual machine node) in data layer, and reduce the data coupling between nodes. Data processing tasks are evenly distributed to different nodes, to achieve theequalization of data accessing and task processing.For the repeated data transmission and storage between the business layer andthe data layer, a tasks assignment method between business layer and data layer isproposed. This method uses lateral transfer tasks processing between the layers ofthe software and resource environment to achieve the optimal cost of task processing.According to the characteristics of the task and the adaptability to the environmentand server resources, the processing tasks are divided and allocated reasonably todifferent layers, which can reduce data coupling between the layers and improve thesystem processing efficiency.The resources-adaptive methods of data layer proposed in this paper havealready been applied to the real software system and achieved good results. Inapplication, only the key software units need to be improved. These methods havethe features such as small implementation cost and foreseeable performanceimprovements. The results of the research in this paper can be applied to improve theperformance for existed system and plan the capacity for new systems both in anindependent physical environment and in an unified virtual logical environment,also， the results can provide the basis for the overall system capacity planning,selection or configuration, system architecture, storage structure, key functionaldesigning, load balancing or scheduling, and can improve system performance andenvironmental resources utilization.