Optimal Experimentation in a Changing Environment

This paper studies optimal experimentation by a monopolist who faces an unknown demand curve subject to random changes, and who maximizes profits over an infinite horizon in continuous time. We show that there are two qualitatively very different regimes, determined by the discount rate and the intensities of demand curve switching, and the dependence of the optimal policy on these parameters is discontinuous. One regime is characterized by extreme experimentation and good tracking of the prevailing demand curve, the other by moderate experimentation and poor tracking. Moreover, in the latter regime the agent eventually becomes "trapped" into taking actions in a strict subset of the feasible set.