Horseshoe Regularisation for Machine Learning in Complex and Deep Models1 |
| |
Authors: | Anindya Bhadra Jyotishka Datta Yunfan Li Nicholas Polson |
| |
Institution: | 1. Department of Statistics, Purdue University, 250 N. University St., West Lafayette, IN 47907 USA;2. Department of Mathematical Sciences, University of Arkansas, Fayetteville, AR 72704 USA;3. Booth School of Business, University of Chicago, 5807 S. Woodlawn Ave., Chicago, IL 60637 USA |
| |
Abstract: | Since the advent of the horseshoe priors for regularisation, global–local shrinkage methods have proved to be a fertile ground for the development of Bayesian methodology in machine learning, specifically for high-dimensional regression and classification problems. They have achieved remarkable success in computation and enjoy strong theoretical support. Most of the existing literature has focused on the linear Gaussian case; for which systematic surveys are available. The purpose of the current article is to demonstrate that the horseshoe regularisation is useful far more broadly, by reviewing both methodological and computational developments in complex models that are more relevant to machine learning applications. Specifically, we focus on methodological challenges in horseshoe regularisation in non-linear and non-Gaussian models, multivariate models and deep neural networks. We also outline the recent computational developments in horseshoe shrinkage for complex models along with a list of available software implementations that allows one to venture out beyond the comfort zone of the canonical linear regression problems. |
| |
Keywords: | complex data deep learning large scale machine learning non-linear non-Gaussian shrinkage |
|
|