Application Software Products Assessment System Based On Evaluation Benchmarking

There are many software products in a specific domain. We can only obtain the coarse-grained qualitative estimates of the product quality and development due to lack of uniform benchmarking library. Most of them have been assessed by the domain expert’s judgments. There is few similar products’benchmarking for quality. It can’t provide accurate and quantitative data for software users. These qualitative judgments cannot help the user has an accurate understanding of the product. And it is no help for company to improving the quality of their products.Application software products assessment based on evaluation benchmarking includes the following content:By analyzing the historical evaluation data set of a specific domain, the benchmarking model could be established; with the accumulation of the domain benchmark data, the evaluation benchmarking could be constructed; and then, we could implement domain software quality comparison using the benchmarking model. By comparing the evaluation results of the assessment object whit the "Best-in-Class" and the "Average" of the domain, the defects of the products could be discovered. In order to achieve the "Best-in-Class" as their goals, the quality of the product could be improved.Based on the Key Laboratory of Shanghai Computer Software Test and Evaluation Program and National Torch Program "Benchmark Library for Domain Specific Software and its Application", this paper defines the definition of domain specific application software benchmark firstly. Furthermore, this paper proposed the formal definition of the domain specific benchmarking model and the evaluation, the method of benchmarking constructing and the benchmarking process. By analyzing the software quality requirements and the characteristics of the specific domain, the elementary set and extended set of the specific domain could be established according GB/T 16260-2006 and GB/T 18905-2002. The quality attributes includes the elementary set and extended set. It could be organized a tree structure according "characteristics—sub-characteristics—attribute". And then, by determining the contribution of the attributes to the software quality, the domain benchmarking could be established.For the subjective factors of the domain experts to the benchmarking model and the inconsistency of the assessment data during the benchmark process, this paper proposed a method to correct the benchmarking model dynamically based on the historical quantitative evaluation data, and a method to improve the evaluation model based on gray relational analysis. Analyzing the historical quantitative evaluation data, based on the rough set theory, could correction the contribution of the quality dynamically. In this way, the subjective factors of the experts to the benchmarking model could be weakened and reached the goal of the quality improvement driven by the benchmarking data. Gray relational analysis method, using the same benchmarking model, takes the benchmarking object and the reference objects as a whole, assesses and benchmarks then at the same time.By analyzing the data sets and their relations during the benchmark process, the Application software products assessment based on evaluation benchmarking has been implemented in the domain of OAs. This paper using WCF framework as the communication mechanism, JSON as the data communication format and ExtJS as the data view show tool, this paper designed and developed a prototype system for the method of Application software products assessment based on evaluation benchmarking.