| An evaluation methodology based upon decision analysis techniques was developed to ensure a better solution for programmatic decisions made within the development of automatic target recognition (ATR) technology. Decisions are typically made based upon a large set of ATR classification system's measures of performance (MOPs), which are assessed through testing by the ATR evaluation community. However, evaluation of these MOPs rarely leads to a dominant solution. Further complicating the problem, it has been shown that superior MOPS do not necessarily translate into superior measures of effectiveness (MOEs), which are highly prized by the warfighter who will be deploying ATR systems once they mature.; The approach yields a two-pronged decision analysis model (one framework for the evaluator, and one framework for the warfighter) that produces a value score for each ATR system. A combat model is implemented to transform MOPs into MOEs for the warfighter framework. The approach is demonstrated using notional performance data. Sensitivity analysis is performed to capture the impact that inputs have on the overall value. To deal with the random nature of the MOP inputs, a multinomial selection procedure is introduced for ATR system value comparison. The results illustrate the differences between the two viewpoints, offer insight into areas where ATR technology should be improved, and provide a means for comparing the different values with an associated level of confidence. |
