Framing User Confidence in a System Dynamics Model: the Case of a Workforce Planning Problem in the New Zealand Army

Abstract

Despite unresolved controversy and ongoing debate about user confidence in system dynamics models, there has been limited empirical exploration of the concept of user confidence in system dynamics models. This research elicited the concept of user confidence using a framing method (Russo & Schoemaker, 1989); analyzed the confidence criteria using constant comparative analysis (Cavana, Delahaye & Sekaran, 2001) and organized the confidence criteria into a descriptive framework. This research was conducted as an ethnographic case study of a New Zealand Army workforce planning problem. The simultaneous objectives of this research were to elicit the concept of user confidence in a system dynamics model and to assess the usefulness of the framing method for ascertaining user confidence criteria. The findings suggest that users of a system dynamics model had unique views of confidence, and while these views changed during the model-building project, they shared a common perspective of utility. Interestingly, user confidence criteria did not change significantly between the qualitative and quantitative stages of model-building. Output from the system dynamics workforce model supported the dynamic hypothesis that the use of ‘aspirational’ separation forecasts have contributed to New Zealand Army workforce shortfalls during times of high labour demand. Additionally, framing proved to be a useful methodology for eliciting and interpreting the elusive concept of user confidence in this case study. This case study concludes that although the confidence criteria of model users are diverse, extensive and difficult to elicit; framing can be employed as an interpretive filter to ascertain the elements of user confidence.