Use of a Markov decision process model for treatment selection in an asymptomatic disease with consideration of risk sensitivity

Some potentially dangerous diseases are completely asymptomatic. Their diagnosis as incidental findings of ever-more-sensitive medical imaging can leave patients and physicians in something of a quandary. The patient feels well, and potential interventions to stave off long-term deterioration or death bring with them immediate risks. We discuss the use of a Markov Decision Process (MDP) model (rather than Monte Carlo simulation of a Markov Model) to create a tool for analyzing individual treatment decisions for asymptomatic chronic diseases where a patient’s condition cannot improve. We formulate a finite-horizon MDP model to determine optimal treatment plans and discuss three distinct optimality criteria: (a) maximizing expected quality-adjusted-life years with and without discounting, (b) maximizing the expected number of life years in good health, and (c) maximizing the expected utility for number of years in good health. In (c) we assume exponential utility and consider different risk aversion factors reported in the medical literature. We illustrate the model’s use by considering asymptomatic intracranial aneurysm. Our model builds on a simulation model [19] created to examine treatment recommendations based on cost-effectiveness. We demonstrate that incorporating risk aversion leads to “no treatment” recommendations for some types of aneurysm. Furthermore, the use of alternate patient-selected criteria leads to recommendations that vary from [19] in several scenarios. We also discuss the use of the software as a decision support tool to help make individualized treatment recommendations and demonstrate that the computational performance of the algorithm makes its use feasible during a short office visit.