Continuous‐time mean–variance portfolio selection: A reinforcement learning framework

We approach the continuous‐time mean–variance portfolio selection with reinforcement learning (RL). The problem is to achieve the best trade‐off between exploration and exploitation, and is formulated as an entropy‐regularized, relaxed stochastic control problem. We prove that the optimal feedback policy for this problem must be Gaussian, with time‐decaying variance. We then prove a policy improvement theorem, based on which we devise an implementable RL algorithm. We find that our algorithm and its variant outperform both traditional and deep neural network based algorithms in our simulation and empirical studies.     

ARBITRAGE BOUNDS FOR PRICES OF WEIGHTED VARIANCE SWAPS

We develop a theory of robust pricing and hedging of a weighted variance swap given market prices for a finite number of co‐maturing put options. We assume the put option prices do not admit arbitrage and deduce no‐arbitrage bounds on the weighted variance swap along with super‐ and sub‐replicating strategies that enforce them. We find that market quotes for variance swaps are surprisingly close to the model‐free lower bounds we determine. We solve the problem by transforming it into an analogous question for a European option with a convex payoff. The lower bound becomes a problem in semi‐infinite linear programming which we solve in detail. The upper bound is explicit. We work in a model‐independent and probability‐free setup. In particular, we use and extend Föllmer's pathwise stochastic calculus. Appropriate notions of arbitrage and admissibility are introduced. This allows us to establish the usual hedging relation between the variance swap and the “log contract” and similar connections for weighted variance swaps. Our results take the form of a FTAP: we show that the absence of (weak) arbitrage is equivalent to the existence of a classical model which reproduces the observed prices via risk‐neutral expectations of discounted payoffs.     

Robust Markowitz mean‐variance portfolio selection under ambiguous covariance matrix

This paper studies a robust continuous‐time Markowitz portfolio selection problem where the model uncertainty affects the covariance matrix of multiple risky assets. This problem is formulated into a min–max mean‐variance problem over a set of nondominated probability measures that is solved by a McKean–Vlasov dynamic programming approach, which allows us to characterize the solution in terms of a Bellman–Isaacs equation in the Wasserstein space of probability measures. We provide explicit solutions for the optimal robust portfolio strategies and illustrate our results in the case of uncertain volatilities and ambiguous correlation between two risky assets. We then derive the robust efficient frontier in closed form, and obtain a lower bound for the Sharpe ratio of any robust efficient portfolio strategy. Finally, we compare the performance of Sharpe ratios for a robust investor and for an investor with a misspecified model.     

BESSEL PROCESSES,STOCHASTIC VOLATILITY,AND TIMER OPTIONS

Motivated by analytical valuation of timer options (an important innovation in realized variance‐based derivatives), we explore their novel mathematical connection with stochastic volatility and Bessel processes (with constant drift). Under the Heston (1993) stochastic volatility model, we formulate the problem through a first‐passage time problem on realized variance, and generalize the standard risk‐neutral valuation theory for fixed maturity options to a case involving random maturity. By time change and the general theory of Markov diffusions, we characterize the joint distribution of the first‐passage time of the realized variance and the corresponding variance using Bessel processes with drift. Thus, explicit formulas for a useful joint density related to Bessel processes are derived via Laplace transform inversion. Based on these theoretical findings, we obtain a Black–Scholes–Merton‐type formula for pricing timer options, and thus extend the analytical tractability of the Heston model. Several issues regarding the numerical implementation are briefly discussed.     

CVaR HEDGING USING QUANTIZATION‐BASED STOCHASTIC APPROXIMATION ALGORITHM

In this paper, we investigate a method based on risk minimization to hedge observable but nontradable source of risk on financial or energy markets. The optimal portfolio strategy is obtained by minimizing dynamically the conditional value‐at‐risk (CVaR) using three main tools: a stochastic approximation algorithm, optimal quantization, and variance reduction techniques (importance sampling and linear control variable), as the quantities of interest are naturally related to rare events. As a first step, we investigate the problem of CVaR regression, which corresponds to a static portfolio strategy where the number of units of each tradable assets is fixed at time 0 and remains unchanged till maturity. We devise a stochastic approximation algorithm and study its a.s. convergence and weak convergence rate. Then, we extend our approach to the dynamic case under the assumption that the process modeling the nontradable source of risk and financial assets prices is Markovian. Finally, we illustrate our approach by considering several portfolios in connection with energy markets.     

Static and semistatic hedging as contrarian or conformist bets

In this paper, we argue that, once the costs of maintaining the hedging portfolio are properly taken into account, semistatic portfolios should more properly be thought of as separate classes of derivatives, with nontrivial, model‐dependent payoff structures. We derive new integral representations for payoffs of exotic European options in terms of payoffs of vanillas, different from the Carr–Madan representation, and suggest approximations of the idealized static hedging/replicating portfolio using vanillas available in the market. We study the dependence of the hedging error on a model used for pricing and show that the variance of the hedging errors of static hedging portfolios can be sizably larger than the errors of variance‐minimizing portfolios. We explain why the exact semistatic hedging of barrier options is impossible for processes with jumps, and derive general formulas for variance‐minimizing semistatic portfolios. We show that hedging using vanillas only leads to larger errors than hedging using vanillas and first touch digitals. In all cases, efficient calculations of the weights of the hedging portfolios are in the dual space using new efficient numerical methods for calculation of the Wiener–Hopf factors and Laplace–Fourier inversion.     

The short‐time behavior of VIX‐implied volatilities in a multifactor stochastic volatility framework

We consider a modeling setup where the volatility index (VIX) dynamics are explicitly computable as a smooth transformation of a purely diffusive, multidimensional Markov process. The framework is general enough to embed many popular stochastic volatility models. We develop closed‐form expansions and sharp error bounds for VIX futures, options, and implied volatilities. In particular, we derive exact asymptotic results for VIX‐implied volatilities, and their sensitivities, in the joint limit of short time‐to‐maturity and small log‐moneyness. The expansions obtained are explicit based on elementary functions and they neatly uncover how the VIX skew depends on the specific choice of the volatility and the vol‐of‐vol processes. Our results are based on perturbation techniques applied to the infinitesimal generator of the underlying process. This methodology has previously been adopted to derive approximations of equity (SPX) options. However, the generalizations needed to cover the case of VIX options are by no means straightforward as the dynamics of the underlying VIX futures are not explicitly known. To illustrate the accuracy of our technique, we provide numerical implementations for a selection of model specifications.     

Static hedging and pricing of exotic options with payoff frames

We develop a general framework for statically hedging and pricing European‐style options with nonstandard terminal payoffs, which can be applied to mixed static–dynamic and semistatic hedges for many path‐dependent exotic options including variance swaps and barrier options. The goal is achieved by separating the hedging and pricing problems to obtain replicating strategies. Once prices have been obtained for a set of basis payoffs, the pricing and hedging of financial securities with arbitrary payoff functions is accomplished by computing a set of “hedge coefficients” for that security. This method is particularly well suited for pricing baskets of options simultaneously, and is robust to discontinuities of payoffs. In addition, the method enables a systematic comparison of the value of a payoff (or portfolio) across a set of competing model specifications with implications for security design.     

The valuation of American options in a multidimensional exponential Lévy model

We consider the problem of valuation of American options written on dividend‐paying assets whose price dynamics follow a multidimensional exponential Lévy model. We carefully examine the relation between the option prices, related partial integro‐differential variational inequalities, and reflected backward stochastic differential equations. In particular, we prove regularity results for the value function and obtain the early exercise premium formula for a broad class of payoff functions.     

MEAN‐VARIANCE POLICY FOR DISCRETE‐TIME CONE‐CONSTRAINED MARKETS: TIME CONSISTENCY IN EFFICIENCY AND THE MINIMUM‐VARIANCE SIGNED SUPERMARTINGALE MEASURE

The discrete‐time mean‐variance portfolio selection formulation, which is a representative of general dynamic mean‐risk portfolio selection problems, typically does not satisfy time consistency in efficiency (TCIE), i.e., a truncated precommitted efficient policy may become inefficient for the corresponding truncated problem. In this paper, we analytically investigate the effect of portfolio constraints on the TCIE of convex cone‐constrained markets. More specifically, we derive semi‐analytical expressions for the precommitted efficient mean‐variance policy and the minimum‐variance signed supermartingale measure (VSSM) and examine their relationship. Our analysis shows that the precommitted discrete‐time efficient mean‐variance policy satisfies TCIE if and only if the conditional expectation of the density of the VSSM (with respect to the original probability measure) is nonnegative, or once the conditional expectation becomes negative, it remains at the same negative value until the terminal time. Our finding indicates that the TCIE property depends only on the basic market setting, including portfolio constraints. This motivates us to establish a general procedure for constructing TCIE dynamic portfolio selection problems by introducing suitable portfolio constraints.     

BETTER THAN DYNAMIC MEAN‐VARIANCE: TIME INCONSISTENCY AND FREE CASH FLOW STREAM

As the dynamic mean‐variance portfolio selection formulation does not satisfy the principle of optimality of dynamic programming, phenomena of time inconsistency occur, i.e., investors may have incentives to deviate from the precommitted optimal mean‐variance portfolio policy during the investment process under certain circumstances. By introducing the concept of time inconsistency in efficiency and defining the induced trade‐off, we further demonstrate in this paper that investors behave irrationally under the precommitted optimal mean‐variance portfolio policy when their wealth is above certain threshold during the investment process. By relaxing the self‐financing restriction to allow withdrawal of money out of the market, we develop a revised mean‐variance policy which dominates the precommitted optimal mean‐variance portfolio policy in the sense that, while the two achieve the same mean‐variance pair of the terminal wealth, the revised policy enables the investor to receive a free cash flow stream (FCFS) during the investment process. The analytical expressions of the probability of receiving FCFS and the expected value of FCFS are derived.     

Optimal Dynamic Portfolio Selection: Multiperiod Mean‐Variance Formulation

The mean‐variance formulation by Markowitz in the 1950s paved a foundation for modern portfolio selection analysis in a single period. This paper considers an analytical optimal solution to the mean‐variance formulation in multiperiod portfolio selection. Specifically, analytical optimal portfolio policy and analytical expression of the mean‐variance efficient frontier are derived in this paper for the multiperiod mean‐variance formulation. An efficient algorithm is also proposed for finding an optimal portfolio policy to maximize a utility function of the expected value and the variance of the terminal wealth.     

Should Stochastic Volatility Matter to the Cost-Constrained Investor?

Significant strides have been made in the development of continuous-time portfolio optimization models since Merton (1969) . Two independent advances have been the incorporation of transaction costs and time-varying volatility into the investor's optimization problem. Transaction costs generally inhibit investors from trading too often. Time-varying volatility, on the other hand, encourages trading activity, as it can result in an evolving optimal allocation of resources. We examine the two-asset portfolio optimization problem when both elements are present. We show that a transaction cost framework can be extended to include a stochastic volatility process. We then specify a transaction cost model with stochastic volatility and show that when the risk premium is linear in variance, the optimal strategy for the investor is independent of the level of volatility in the risky asset. We call this the Variance Invariance Principle.     

Analytical approximations of local‐Heston volatility model and error analysis

This paper studies the expansion of an option price (with bounded Lipschitz payoff) in a stochastic volatility model including a local volatility component. The stochastic volatility is a square root process, which is widely used for modeling the behavior of the variance process (Heston model). The local volatility part is of general form, requiring only appropriate growth and boundedness assumptions. We rigorously establish tight error estimates of our expansions, using Malliavin calculus. The error analysis, which requires a careful treatment because of the lack of weak differentiability of the model, is interesting on its own. Moreover, in the particular case of call–put options, we also provide expansions of the Black–Scholes implied volatility that allow to obtain very simple formulas that are fast to compute compared to the Monte Carlo approach and maintain a very competitive accuracy.     

MULTI‐ASSET STOCHASTIC LOCAL VARIANCE CONTRACTS

Variance swaps now trade actively over‐the‐counter (OTC) on both stocks and stock indices. Also trading OTC are variations on variance swaps which localize the payoff in time, in the underlying asset price, or both. Given that the price of the underlying asset evolves continuously over time, it is well known that there exists a semirobust hedge for these localized variance contracts. Remarkably, the hedge succeeds even though the stochastic process describing the instantaneous variance is never specified. In this paper, we present a generalization of these results to the case of two or more underlying assets.     

Optimal approximations of nonlinear payoffs in static replication

Static replication of nonlinear payoffs by line segments (or equivalently vanilla options) is an important hedging method, which unfortunately is only an approximation. If the strike prices of options are adjustable (for OTC options), two optimal approximations can be defined for replication by piecewise chords. The first is a naive minimum area approach, which seeks a set of strike prices to minimize the area enclosed by the payoff curve and the chords. The second improves on the first by taking the conditional distribution of the underlying into consideration, and minimizes the expected area instead. When the strike prices are fixed (for exchange‐traded options), a third or the approach of least expected squares locates the minimum for the expected sum of squared differences between the payoff and the replicating portfolio, by varying the weights or quantities of the options used in the replication. For a payoff of variance swap, minimum expected area and least expected squares are found to produce the best numerical results in terms of cost of replication. Finally, piecewise tangents can also be utilized in static replication, which together with replication by chords, forms a pair of lower or upper bound to a nonlinear payoff. © 2010 Wiley Periodicals, Inc. Jrl Fut Mark     

Optimal measure preserving derivatives revisited

This article clarifies the relationship between pricing kernel monotonicity and the existence of opportunities for stochastic arbitrage in a complete and frictionless market of derivative securities written on a market portfolio. The relationship depends on whether the payoff distribution of the market portfolio satisfies a technical condition called adequacy, meaning that it is atomless or is comprised of finitely many equally probable atoms. Under adequacy, pricing kernel nonmonotonicity is equivalent to the existence of a strong form of stochastic arbitrage involving distributional replication of the market portfolio at a lower price. If the adequacy condition is dropped then this equivalence no longer holds, but pricing kernel nonmonotonicity remains equivalent to the existence of a weaker form of stochastic arbitrage involving second-order stochastic dominance of the market portfolio at a lower price. A generalization of the optimal measure preserving derivative is obtained, which achieves distributional replication at the minimum cost of all second-order stochastically dominant securities under adequacy.     

A CONVEX STOCHASTIC OPTIMIZATION PROBLEM ARISING FROM PORTFOLIO SELECTION

A continuous-time financial portfolio selection model with expected utility maximization typically boils down to solving a (static) convex stochastic optimization problem in terms of the terminal wealth, with a budget constraint. In literature the latter is solved by assuming a priori that the problem is well-posed (i.e., the supremum value is finite) and a Lagrange multiplier exists (and as a consequence the optimal solution is attainable). In this paper it is first shown that, via various counter-examples, neither of these two assumptions needs to hold, and an optimal solution does not necessarily exist. These anomalies in turn have important interpretations in and impacts on the portfolio selection modeling and solutions. Relations among the non-existence of the Lagrange multiplier, the ill-posedness of the problem, and the non-attainability of an optimal solution are then investigated. Finally, explicit and easily verifiable conditions are derived which lead to finding the unique optimal solution.     

Risk‐Sensitive Control and an Optimal Investment Model

We consider an optimal investment model in which the goal is to maximize the long‐term growth rate of expected utility of wealth. In the model, the mean returns of the securities are explicitly affected by the underlying economic factors. The utility function is HARA. The problem is reformulated as an infinite time horizon risk‐sensitive control problem. We study the dynamic programming equation associated with this control problem and derive some consequences of the investment problem.     

PORTFOLIO OPTIMIZATION AND STOCHASTIC VOLATILITY ASYMPTOTICS

We study the Merton portfolio optimization problem in the presence of stochastic volatility using asymptotic approximations when the volatility process is characterized by its timescales of fluctuation. This approach is tractable because it treats the incomplete markets problem as a perturbation around the complete market constant volatility problem for the value function, which is well understood. When volatility is fast mean‐reverting, this is a singular perturbation problem for a nonlinear Hamilton–Jacobi–Bellman partial differential equation, while when volatility is slowly varying, it is a regular perturbation. These analyses can be combined for multifactor multiscale stochastic volatility models. The asymptotics shares remarkable similarities with the linear option pricing problem, which follows from some new properties of the Merton risk tolerance function. We give examples in the family of mixture of power utilities and also use our asymptotic analysis to suggest a “practical” strategy that does not require tracking the fast‐moving volatility. In this paper, we present formal derivations of asymptotic approximations, and we provide a convergence proof in the case of power utility and single‐factor stochastic volatility. We assess our approximation in a particular case where there is an explicit solution.