Research On Evaluation Method For Credibility Of Simulation System

As one of the significant means of learning and changing the objective world,the simulation technique is widely used in many fields, such as spaceflight, avia-tion, navigation, electronics and communication. Simulation system is the applica-tion form of simulation technique. Its credibility directly determines the success orfailure of the simulation application. Therefore, the problem of simulation credi-bility evaluation has become the research hotspot in the system simulation field.The currently used simulation system is large-scale system constituted by severalbranch systems, subsystems and even branch subsystems, and it has large numberof state variables and output variables, which are correlated in great complexity.Furthermore, different simulation systems have different structures and functions.What aspects should be investigated as evaluating credibility of simulation system?How to measure the performance of simulation system in these aspects? How toimprove automation and efficiency of evaluation work? These problems have notyet been perfectly solved, which severely restrict the extension of simulation ap-plication field. Supported by the National Natural Science Foundation of China“Research on VV&A theory and methods for simulation systems based on multipleattribute decision network (MADN)”, this thesis has studied simulation credibilityevaluation method. The main contents are as follow:(1) Evaluation model and evaluation method of simulation credibility arestudied. The existing evaluation models of simulation credibility have some weak-nesses, such as imprecise semantics, poor universality, and inadequate descriptionability. Against these weaknesses, a formal evaluation model is constructed, whichelaborates its basic elements, element-element relationship, mathematical expres-sion, and parameter figure. Combined with the formal evaluation model, a newmethod for simulation credibility evaluation is proposed, and its key algorithmsare deeply analyzed which lead to the contents of subsequent chapters.(2) In order to solve the measurement problem of similarity between simula-tion time series data and reference time series data, time series data similaritymeasure is studied. One-dimensional time series data is classified based on thevariable measurement scale, and the properties that similarity measure of varioustime series data should possess are analyzed. Then similarity measures of one-dimensional time series data in various properties, which meet different require-ments individually, are constructed. For the characteristics of various types of var-iable and correlation in multidimensional time series data, an extended Gaussian kernel function is constructed to deal with nonlinear correlation between multipledependent variables. By combining with one-dimensional time series data similari-ty measure and kernel principal component analysis, a multidimensional time se-ries data similarity measure is proposed.(3) Simulation credibility index integration method is studied. In the processof simulation credibility evaluation, we need to integrate credibility indexes re-flecting different aspects of simulation system, so as to obtain the overall credibil-ity. In the meantime, we usually meet the following situations:(a) index weightinformation is incomplete;(b) index weight information is inconsistent;(c) index-es are not independent. According to these situations, index integration methodsare proposed respectively through solving the problems of expert weight distribu-tion, synthesis judgement matrix consistency analysis, and index relations descrip-tion.(4) Analysis method for defective element of simulation system in credibilityis studied, so as to find out the bottleneck restricting simulation credibility andprovide guidance for simulation system’s improvement. Based on the formal eval-uation model and the results of simulation credibility, and combined with Sobol’method and orthogonal design, a defective element analysis method is proposed.In this method, the experiment scheme is arranged according to an extended tabletransformed from orthogonal table. By analyzing experiment results, the defectrank of defective element is judged. Then, the sensitivity coefficients of elementscoupled to defective element are calculated using Sobol’ method.(5) Assistant tool for simulation credibility evaluation is developed. The sim-ulation credibility evaluation is a systematic project of great arduousness andcomplexity. It is generally carried out as a joint effort of many domain experts andinvolving large amounts of simulation data, reference data and process infor-mation. Therefore, based on the research results of chapter two, three, four andfive, an assistant tool for simulation credibility evaluation is developed using Mi-crosoft Visual C++6.0and Matlab7.0. It contains seven modules, which are ob-ject management module, expert management module, evaluation model settingmodule, credibility calculation module, defective element analysis module, resultsstatistics module and report generation module.In the application background of XX key-point air defence simulation system,this thesis validates the effectiveness and practicability of the evaluation methodand evaluation tool. Compared with the research results of antecessor, the pro-posed method of this thesis have many advantages, such as clear semantics, rigor-ous logic, well universality and superior operability, in results of solving the prob- lem of simulation credibility evaluation reasonably and effectively. The developedevaluation tool has been used by several spaceflight research institutes, and all theresponses are positive.