When do improved covariance matrix estimators enhance portfolio optimization? An empirical comparative study of nine estimators

The use of improved covariance matrix estimators as an alternative to the sample estimator is considered an important approach for enhancing portfolio optimization. Here we empirically compare the performance of nine improved covariance estimation procedures using daily returns of 90 highly capitalized US stocks for the period 1997–2007. We find that the usefulness of covariance matrix estimators strongly depends on the ratio between the estimation period T and the number of stocks N, on the presence or absence of short selling, and on the performance metric considered. When short selling is allowed, several estimation methods achieve a realized risk that is significantly smaller than that obtained with the sample covariance method. This is particularly true when T/N is close to one. Moreover, many estimators reduce the fraction of negative portfolio weights, while little improvement is achieved in the degree of diversification. On the contrary, when short selling is not allowed and T?>?N, the considered methods are unable to outperform the sample covariance in terms of realized risk, but can give much more diversified portfolios than that obtained with the sample covariance. When T?<?N, the use of the sample covariance matrix and of the pseudo-inverse gives portfolios with very poor performance.     

Alternative constructions of Tobin's q: An empirical comparison

Although Tobin's q is an attractive theoretical firm performance measure, its empirical construction is subject to considerable measurement error. In this paper we compare five estimators of q that range from a simple-to-construct estimator based on book-values to a relatively complex estimator based upon the methodology developed by Lindenberg and Ross (1981). We present comparisons of the means, medians and variances of the q estimates, and examine how robust sorting and regression results are to changes in the construction of q. We find that empirical results are sensitive to the method used to estimate Tobin's q. The simple-to-construct estimator produces empirical results that differ significantly from the alternative estimators. Among the other four estimators, one developed by Hall (1990) produces means that are higher and variances that are larger than the three alternative estimators, but does approximate those estimators in most of the empirical comparisons. Those three alternative q ratio estimators, furthermore, produce empirical results that are robust.     

Is there a viable alternative to ordinary least squares regression when security abnormal returns are the dependent variable?

Regression analysis is often used to estimate a linear relationship between security abnormal returns and firm-specific variables. If the abnormal returns are caused by a common event (i.e., there is “event clustering”) the error term of the cross-sectional regression will be heteroskedastic and correlated across observations. The size and power of alternative test statistics for the event clustering case has been evaluated under ideal conditions (Monte Carlo experiments using normally distributed synthetic security returns) by Chandra and Balachandran (J Finance 47:2055–2070, 1992) and Karafiath (J Financ Quant Anal 29(2):279–300, 1994). Harrington and Shrider (J Financ Quant Anal 42(1):229–256, 2007) evaluate cross-sectional regressions using actual (not simulated) stock returns only for the case of cross-sectional independence, i.e., in the absence of clustering. In order to evaluate the event clustering case, random samples of security returns are drawn from the data set provided by the Center for Research in Security Prices (CRSP) and the empirical distributions of alternative test statistics compared. These simulations include a comparison of OLS, WLS, GLS, two heteroskedastic-consistent estimators, and a bootstrap test for GLS. In addition, the Sefcik and Thompson (J Accounting Res 24(2):316–334, 1986) portfolio counterparts to OLS, WLS, and GLS, are evaluated. The main result from these simulations is none of the other estimator shows clear advantages over OLS or WLS. Researchers should be aware, however, that in these simulations the variance of the error term in the cross-sectional regression is unrelated to the explanatory variable.

Portfolio analysis of intraday covariance matrix in the Greek equity market

The intraday nonparametric estimation of the variance–covariance matrix adds to the literature in portfolio analysis of the Greek equity market. This paper examines the economic value of various realized volatility and covariance estimators under the strategy of volatility timing. I use three types of portfolios: Global Minimum Variance, Capital Market Line and Capital Market Line with only positive weights. The estimators of volatilities and covariances use 5-min high-frequency intraday data. The dataset concerns the FTSE/ATHEX Large Cap index, FTSE/ATHEX Mid Cap index, and the FTSE/ATHEX Small Cap index of the Greek equity market (Athens Stock Exchange). As far as I know, this is the first work of its kind for the Greek equity market. Results concern not only the comparison of various estimators but also the comparison of different types of portfolios, in the strategy of volatility timing. The economic value of the contemporary non-parametric realized volatility estimators is more significant than this when the covariance is estimated by the daily squared returns. Moreover, the economic value (in b.p.s) of each estimator changes with the volatility timing.     

Improved estimation of the covariance matrix of stock returns with an application to portfolio selection

This paper proposes to estimate the covariance matrix of stock returns by an optimally weighted average of two existing estimators: the sample covariance matrix and single-index covariance matrix. This method is generally known as shrinkage, and it is standard in decision theory and in empirical Bayesian statistics. Our shrinkage estimator can be seen as a way to account for extra-market covariance without having to specify an arbitrary multifactor structure. For NYSE and AMEX stock returns from 1972 to 1995, it can be used to select portfolios with significantly lower out-of-sample variance than a set of existing estimators, including multifactor models.     

Covariance complexity and rates of return on assets

This paper considers the estimation of the expected rate of return on a set of risky assets. The approach to estimation focuses on the covariance matrix for the returns. The structure in the covariance matrix determines shared information which is useful in estimating the mean return for each asset. An empirical Bayes estimator is developed using the covariance structure of the returns distribution. The estimator is an improvement on the maximum likelihood and Bayes–Stein estimators in terms of mean squared error. The effect of reduced estimation error on accumulated wealth is analyzed for the portfolio choice model with constant relative risk aversion utility.     

Multiple Regimes and Volatility Transmission in Securitized Real Estate Markets

We examine the dynamics and transmission of conditional volatilities with multiple structural changes in return volatility using Bai and Perron (2003)’s methodology, across five major securitized real estate markets as well as employing a multivariate regime-dependent asymmetric dynamic covariance methodology (MRDADC) that allows the conditional matrix to be both time- and state-varying. Our results imply that a multiple-regime time varying asymmetric variance and covariance approach is important in modeling real estate securities valuation and selection and portfolio optimization, and is consistent with popular beliefs that market volatility changes over time. Our MRDADC models detect the presence of significant mean-volatility linkages across the five major securitized real estate markets under different volatility regimes and would have implications for global investor in terms of estimating a dynamic risk-minimizing hedge ratio in international portfolio management.     

Dominance of a Class of Stein type Estimators for Optimal Portfolio Weights When the Covariance Matrix is Unknown

For the estimation problem of mean-variance optimal portfolio weights, several previous studies have proposed applying Stein type estimators. However, few studies have addressed this problem analytically. Since the form of the loss function used in this problem is not of the quadratic type commonly used in statistical studies, there have been some difficulties in showing analytically the general dominance results. However, dominance results are given here of a class of Stein type estimators for the mean-variance optimal portfolio weights when the covariance matrix is unknown and is estimated. The class of estimators is broader than the one given in a previous study. The results we have obtained enable us to clarify conditions for some previously proposed estimators in finance to have smaller risks than the estimator which we obtain by plugging in the sample estimates.     

Sampling error and double shrinkage estimation of minimum variance portfolios

Shrinkage estimators of the covariance matrix are known to improve the stability over time of the Global Minimum Variance Portfolio (GMVP), as they are less error-prone. However, the improvement over the empirical covariance matrix is not optimal for small values of n, the estimation sample size. For typical asset allocation problems, with n small, this paper aims at proposing a new method to further reduce sampling error by shrinking once again traditional shrinkage estimators of the GMVP. First, we show analytically that the weights of any GMVP can be shrunk – within the framework of the ridge regression – towards the ones of the equally-weighted portfolio in order to reduce sampling error. Second, Monte Carlo simulations and empirical applications show that applying our methodology to the GMVP based on shrinkage estimators of the covariance matrix, leads to more stable portfolio weights, sharp decreases in portfolio turnovers, and often statistically lower (resp. higher) out-of-sample variances (resp. Sharpe ratios). These results illustrate that double shrinkage estimation of the GMVP can be beneficial for realistic small estimation sample sizes.     

On optimal stratifications for multivariate distributions

Summary

In the present paper we study the problem of optimal stratifications for estimating the mean vector y of a given multivariate distribution F(x) with covariance matrix ζ both in cases of proportionate and of optimal (or generalized Neyman) allocations. It is noted that an “optimal stratification” is meant for one to make the covariance matrix of an unbiased estimator X for μ minimal, in the sense of semi-order defined below, in the symmetric matrix space. We show the existence of an optimal stratification and the necessary conditions for a stratification to be optimal. Besides we prove that an optimal stratification can be represented by a “hyperplane stratification” or a “quadratic hypersurface stratification” according to the proportionate or optimal (or generalized Neyman) allocation, and that the set of all optimal (or admissible) stratifications is a minimal complete class in the analogous sense of decision theory. Further we discuss the optimal stratification when a criterion based on a suitable real-valued function is adopted instead of the semi-order.     

Corrections for Trading Frictions in Multivariate Returns

When observed stock returns are obtained from trades subject to friction, it is known that an individual stock's beta and covariance are measured with error. Univariate models of additive error adjustment are available and are often applied simultaneously to more than one stock. Unfortunately, these multivariate adjustments produce non-positive definite covariance and correlation matrices, unless the return sample sizes are very large. To prevent this, restrictions on the adjustment matrix are developed and a correction is proposed, which dominates the uncorrected estimator. The estimators are illustrated with asset opportunity set estimates where daily returns have trading frictions.     

Nonsynchronous Data and the Covariance-Factor Structure of Returns

Evidence is presented that indicates that the standard estimator of the covariance matrix of daily returns provides a distorted view of the true covariance-factor structure. An alternative estimator, based on a model of the price-adjustment delay process, reveals roughly twice as much covariation in individual security returns. The number of factors identified also appears to increase when this estimator is employed. Since the linear space spanned by the estimated factor-loading vectors is quite sensitive to the estimator used, it is important that the consistent estimator be considered in the usual two-stage empirical investigations of the APT.     

Size matters: Optimal calibration of shrinkage estimators for portfolio selection

We carry out a comprehensive investigation of shrinkage estimators for asset allocation, and we find that size matters—the shrinkage intensity plays a significant role in the performance of the resulting estimated optimal portfolios. We study both portfolios computed from shrinkage estimators of the moments of asset returns (shrinkage moments), as well as shrinkage portfolios obtained by shrinking the portfolio weights directly. We make several contributions in this field. First, we propose two novel calibration criteria for the vector of means and the inverse covariance matrix. Second, for the covariance matrix we propose a novel calibration criterion that takes the condition number optimally into account. Third, for shrinkage portfolios we study two novel calibration criteria. Fourth, we propose a simple multivariate smoothed bootstrap approach to construct the optimal shrinkage intensity. Finally, we carry out an extensive out-of-sample analysis with simulated and empirical datasets, and we characterize the performance of the different shrinkage estimators for portfolio selection.     

Bankruptcy prediction: the case of Japanese listed companies

This paper investigates if bankruptcy of Japanese listed companies can be predicted using data from 1992 to 2005. We find that the traditional measures, such as Altman’s (J Finance 23:589–609, 1968) Z-score, Ohlson’s (J Accounting Res 18:109–131, 1980) O-score and the option pricing theory-based distance-to-default, previously developed for the U.S. market, are also individually useful for the Japanese market. Moreover, the predictive power is substantially enhanced when these measures are combined. Based on the unique Japanese institutional features of main banks and business groups (known as Keiretsu), we construct a new measure that incorporates bank dependence and Keiretsu dependence. The new measure further improves the ability to predict bankruptcy of Japanese listed companies.     

Estimating daily volatility in financial markets utilizing intraday data

This study proposes a new approach to the estimation of daily realised volatility in financial markets from intraday data. Initially, an examination of intraday returns on S&P 500 Index Futures reveals that returns can be characterised by heteroscedasticity and time-varying autocorrelation. After reviewing a number of daily realised volatility estimators cited in the literature, it is concluded that these estimators are based upon a number of restrictive assumptions in regard to the data generating process for intraday returns. We use a weak set of assumptions about the data generating process for intraday returns, including transaction returns, given in den Haan and Levin [den Haan, W.J., Levin, A., 1996. Inferences from parametric and non-parametric covariance matrix estimation procedures, Working paper, NBER, 195.], which allows for heteroscedasticity and time-varying autocorrelation in intraday returns. These assumptions allow the VARHAC estimator to be employed in the estimation of daily realised volatility. An empirical analysis of the VARHAC daily volatility estimator employing intraday transaction returns concludes that this estimator performs well in comparison to other estimators cited in the literature.     

Issuers’ incentives and tests of Baron’s model of IPO underpricing

This paper reexamines the validity of Baron’s (J Financ 37:955–976, 1982) model of IPO underpricing, in which IPO underpricing is caused by asymmetric information between issuers and investment bankers. Muscarella and Vetsuypens (J Financ Econ 24:125–135, 1989) find that lead-manager IPOs are significantly more underpriced than non-self-marketed IPOs and conclude that their empirical results do not support Baron’s model. We compare self-marketed underwriters’ IPOs with non-self-marketed underwriters’ IPOs and with IPOs they lead. Our empirical results show that it is premature to reject Baron’s model of IPO underpricing when we take issuer incentives into account.     

Volatility and covariation of financial assets: A high-frequency analysis

Using high frequency data for the price dynamics of equities we measure the impact that market microstructure noise has on estimates of the: (i) volatility of returns; and (ii) variance–covariance matrix of n assets. We propose a Kalman-filter-based methodology that allows us to deconstruct price series into the true efficient price and the microstructure noise. This approach allows us to employ volatility estimators that achieve very low Root Mean Squared Errors (RMSEs) compared to other estimators that have been proposed to deal with market microstructure noise at high frequencies. Furthermore, this price series decomposition allows us to estimate the variance covariance matrix of n assets in a more efficient way than the methods so far proposed in the literature. We illustrate our results by calculating how microstructure noise affects portfolio decisions and calculations of the equity beta in a CAPM setting.     

Robust estimation of historical volatility and correlations in risk management

Financial time series have two features which, in many cases, prevent the use of conventional estimators of volatilities and correlations: leptokurtotic distributions and contamination of data with outliers. Other techniques are required to achieve stable and accurate results. In this paper, we review robust estimators for volatilities and correlations and identify those best suited for use in risk management. The selection criteria were that the estimator should be stable to both fractionally small departures for all data points (fat tails), and to fractionally large departures for a small number of data points (outliers). Since risk management typically deals with thousands of time series at once, another major requirement was the independence of the approach of any manual correction or data pre-processing. We recommend using volatility t-estimators, for which we derived the estimation error formula for the case when the exact shape of the data distribution is unknown. A convenient robust estimator for correlations is Kendall's tau, whose drawback is that it does not guarantee the positivity of the correlation matrix. We chose to use geometric optimization that overcomes this problem by finding the closest correlation matrix to a given matrix in terms of the Hadamard norm. We propose the weights for the norm and demonstrate the efficiency of the algorithm on large-scale problems.     

Statistical Properties of the Roll Serial Covariance Bid/Ask Spread Estimator

Exact small sample population moments of the standard serial covariance and variance estimators are derived under the assumptions of the Roll bid/ask spread model. Noise explains why serial covariance estimates are often positive in annual samples of daily and weekly returns. Small sample estimator bias partially explains why weekly estimates are more negative than daily estimates. Noise causes the Roll spread estimator to be severely biased by Jensen's inequality. The French-Roll adjusted variance estimator is unbiased but noisy. Empirical tests confirm the major implications.     

Empirical properties of large covariance matrices

The salient properties of large empirical covariance and correlation matrices are studied for three datasets of size 54, 55 and 330. The covariance is defined as a simple cross product of the returns, with weights that decay logarithmically slowly. The key general properties of the covariance matrices are the following. The spectrum of the covariance is very static, except for the top three to 10 eigenvalues, and decay exponentially fast toward zero. The mean spectrum and spectral density show no particular feature that would separate ‘meaningful’ from ‘noisy’ eigenvalues. The spectrum of the correlation is more static, with three to five eigenvalues that have distinct dynamics. The mean projector of rank k on the leading subspace shows that a large part of the dynamics occurs in the eigenvectors. Together, this implies that the reduction of the covariance to a few leading static eigenmodes misses most of the dynamics. Finally, all the analysed properties of the dynamics of the covariance and correlation are similar. This indicates that a covariance estimator correctly evaluates both volatilities and correlations, and separate estimators are not required.