Thinking inside the box: A participatory, computer-assisted approach to scenario discovery

Scenarios provide a commonly used and intuitively appealing means to communicate and characterize uncertainty in many decision support applications, but can fall short of their potential especially when used in broad public debates among participants with diverse interests and values. This paper describes a new approach to participatory, computer-assisted scenario development that we call scenario discovery, which aims to address these challenges. The approach defines scenarios as a set of plausible future states of the world that represent vulnerabilities of proposed policies, that is, cases where a policy fails to meet its performance goals. Scenario discovery characterizes such sets by helping users to apply statistical or data-mining algorithms to databases of simulation-model-generated results in order to identify easy-to-interpret combinations of uncertain model input parameters that are highly predictive of these policy-relevant cases. The approach has already proved successful in several high impact policy studies. This paper systematically describes the scenario discovery concept and its implementation, presents statistical tests to evaluate the resulting scenarios, and demonstrates the approach on an example policy problem involving the efficacy of a proposed U.S. renewable energy standard. The paper also describes how scenario discovery appears to address several outstanding challenges faced when applying traditional scenario approaches in contentious public debates.