Measures of Quality for Model Validation

Abstract

Issues that are important in terms of model quality include the choice of model output variables and the methods and measures used to compare model and system performance. Methods of comparison may be graphical or may involve quantitative measures of a deterministic or statistical kind. Graphical approaches considered are based mainly on conventional time-history plots, but a brief introduction to box plots is also included. The possible strengths and weaknesses of graphical methods are discussed. Deterministic quantitative measures are based mainly on time-domain and frequency-domain comparisons between outputs measured from a real system and outputs predicted from a simulation model. Examples include simple squared-error measures, absolute-error measures and measures that provide convenient normalised values. Visualisation techniques are also discussed, with particular emphasis being placed on forms of polar diagram that have proved helpful in dealing with complex simulation models in several different application areas.