A Computational Perspective on Projection Pursuit in High Dimensions: Feasible or Infeasible Feature Extraction

Finding a suitable representation of multivariate data is fundamental in many scientific disciplines. Projection pursuit ( $\text{PP}$) aims to extract interesting ‘non-Gaussian’ features from multivariate data, and tends to be computationally intensive even when applied to data of low dimension. In high-dimensional settings, a recent work (Bickel et al., 2018) on $\text{PP}$ addresses asymptotic characterization and conjectures of the feasible projections as the dimension grows with sample size. To gain practical utility of and learn theoretical insights into $\text{PP}$ in an integral way, data analytic tools needed to evaluate the behaviour of $\text{PP}$ in high dimensions become increasingly desirable but are less explored in the literature. This paper focuses on developing computationally fast and effective approaches central to finite sample studies for (i) visualizing the feasibility of $\text{PP}$ in extracting features from high-dimensional data, as compared with alternative methods like $\text{PCA}$ and $\text{ICA}$, and (ii) assessing the plausibility of $\text{PP}$ in cases where asymptotic studies are lacking or unavailable, with the goal of better understanding the practicality, limitation and challenge of $\text{PP}$ in the analysis of large data sets.     

A result on the sign of restricted least-squares estimates

When a variable is dropped from a least-squares regression equation, there can be no change in sign of any coefficient that is more significant than the coefficient of the omitted variable. More generally, a constrained least squares estimate of a parameter β_j must lie in the interval $\text{(}\text{β}\text{?}{}_{\mathrm{j}}\text{?V}{}^{\mathrm{<mtext>1</mtext><mtext>2</mtext>}}{}_{\mathrm{jj}}\text{|t|,}\text{β}\text{?}{}_{\mathrm{j}}\text{+ V}{}^{\mathrm{<mtext>1</mtext><mtext>2</mtext>}}{}_{\mathrm{jj}}\text{|t|)}$ where $\text{β}\text{?}{}_{\mathrm{j}}$ is the unconstrained estimate of $\text{β}{}_{\mathrm{j}}\text{, V}{}^{\mathrm{<mtext>1</mtext><mtext>2</mtext>}}{}_{\mathrm{jj}}$ is the standard error of $\text{β}\text{?}{}_{\mathrm{j}}$ and t is the t-value for testing the (univariate) restriction.     

Quality of local equilibria in discrete exchange economies

This paper defines the notion of a local equilibrium of quality $(r,s)$, $0\le r,s$, in a discrete exchange economy: a partial allocation and item prices that guarantee certain stability properties parametrized by the numbers $r$ and $s$. The quality $(r,s)$ measures the fit between the allocation and the prices: the larger $r$ and $s$ the closer the fit. For $r,s\le 1$ this notion provides a graceful degradation for the conditional equilibria of Fu, Kleinberg and Lavi (2012) which are exactly the local equilibria of quality $(1,1)$. For $1<r,s$ the local equilibria of quality $(r,s)$ are more stable than conditional equilibria. Any local equilibrium of quality $(r,s)$ provides, without any assumption on the type of the agents’ valuations, an allocation whose value is at least $\frac{rs}{1+rs}$ the optimal fractional allocation. In any economy in which all agents’ valuations are $a$-submodular, i.e., exhibit complementarity bounded by $a\ge 1$, there is a local equilibrium of quality $(\frac{1}{a},\frac{1}{a})$. In such an economy any greedy allocation provides a local equilibrium of quality $(1,\frac{1}{a})$. Walrasian equilibria are not amenable to such graceful degradation.     

A bound on the expected overshoot for some concave boundaries

LetX ₁,X ₂,… be i.i.d. with finite meanμ>0,S _n =X ₁+…+X _n . Forf(n)=n ^β ,c>0 we consider the stopping timesT _c =inf{n:S _n >c+f(n)} with overshootR _c =S _{T c} −(c+f(T _c )). For 0<β<1 we give a bound for sup _c≥0 ER _c in the spirit of Lorden’s well-known inequality forf=0.     

Characterization of the multivariate normal distribution by conditional normal distributions

Summary LetX andY be two random vectors with values in ℝ ^k and ℝ∝, respectively. IfZ=(X ^T,Y ^T) ^T is multivariate normal thenX givenY=y andY givenX=x are (multivariate) normal; the converse is wrong. In this paper simple additional conditions are stated such that the converse is true, too. Furthermore, the case is treated that the random vectorZ=(X ₁ ^T , …,X _t ^T ) ^T is splitted intot≥3 partsX ₁, …,X _t.     

Some properties of the minimum and the maximum of random variables with joint logconcave distributions

It is shown that if (X ₁, X ₂, . . . , X _n ) is a random vector with a logconcave (logconvex) joint reliability function, then X _P = min _i∈P X _i has increasing (decreasing) hazard rate. Analogously, it is shown that if (X ₁, X ₂, . . . , X _n ) has a logconcave (logconvex) joint distribution function, then X ^P  = max _i∈P X _i has decreasing (increasing) reversed hazard rate. If the random vector is absolutely continuous with a logconcave density function, then it has a logconcave reliability and distribution functions and hence we obtain a result given by Hu and Li (Metrika 65:325–330, 2007). It is also shown that if (X ₁, X ₂, . . . , X _n ) has an exchangeable logconcave density function then both X _P and X ^P have increasing likelihood ratio.     

The cointegrated vector autoregressive model with general deterministic terms

In the cointegrated vector autoregression (CVAR) literature, deterministic terms have until now been analyzed on a case-by-case, or as-needed basis. We give a comprehensive unified treatment of deterministic terms in the additive model $X_t=\gamma Z_t+Y_t$, where $Z_t$ belongs to a large class of deterministic regressors and $Y_t$ is a zero-mean CVAR. We suggest an extended model that can be estimated by reduced rank regression, and give a condition for when the additive and extended models are asymptotically equivalent, as well as an algorithm for deriving the additive model parameters from the extended model parameters. We derive asymptotic properties of the maximum likelihood estimators and discuss tests for rank and tests on the deterministic terms. In particular, we give conditions under which the estimators are asymptotically (mixed) Gaussian, such that associated tests are ${\chi }^2$-distributed.     

Moment consistency of estimators in partially linear models under NA samples

Consider the heteroscedastic regression model Y ^(j)(x _in , t _in ) = t _in β + g(x _in ) + σ _in e ^(j)(x _in ), 1 ≤ j ≤ m, 1 ≤ i ≤ n, where s_in²=f(u_in){\sigma_{in}^{2}=f(u_{in})}, (x _in , t _in , u _in ) are fixed design points, β is an unknown parameter, g(·) and f(·) are unknown functions, and the errors {e ^(j)(x _in )} are mean zero NA random variables. The moment consistency for least-squares estimators and weighted least-squares estimators of β is studied. In addition, the moment consistency for estimators of g(·) and f(·) is investigated.     

Sulle operazioni associative tra variabili casuali

Si mostra che, sotto condizioni di regolarità, seo è un’operazione associativa tra variabili casuali reali e indipendenti, è definibile una trasformata integrale ξ delle loro funzioni di ripartizione con la proprietà: ξx ₀ Y (t)=ξx(t)·ξ _y (t). Si indicano alcune proprietà di tale trasformata e si tratta della possibilità di estendere a un’operazione associativa risultati noti per l’addizione tra variabili casuali. In particolare ci si occupa dell’« infinita divisibilità » fornendo condizioni perché una variabile casualeX ammetta la rappresentazioneX=X _1O X _2O … _o X _n per ognin naturale con leX _i indipendenti e identicamente ditribuite.     

What Is the Economic Meaning of FDH?

The central feature of the FDH model is the lack of convexity for its production possibility set, TF. Starting with n observed (distinct) decision making units DMU_k , each defined by an input-output vector p _k = [y _k -x _k], domination is defined by ordinary vector inequalities. DMU_k is said to dominate DMU_j if p _k ≥ p _j , p _k ≠ p _j . The FDH production possibility set TF consists of the observed DMU_j together with all input-output vectors p=[y_k,?x_k] with y ≥ 0, x ≥ 0, y ≠ 0, x ≠ 0 which are dominated by at least one of the observed DMU_j. DMU_k is defined as “FDH efficient” if no DMU_j dominates it. In the BCC (or variable return to scale) DEA model the production possibility set TB consists of the observed DMU_k together with all input-output vectors dominated by any convex combination of them and DMU_k is DEA efficient if it is not dominated by any p in TB. In the DEA model, economic meaning is established by the introduction of (non negative) multiplier (price) vectors w = [u,v]. If DMU_k is undominated (in TB) then there exists a positive multiplier vector w for which (a) w ^T p _k = u ^T y _k ? v ^T x _k ≥ w ^T p for every p ∈ TB. In everyday language, the net return (or profit) for DMU_k relative to the given multiplier vector w is at least as great as that for any production possibility p. On the other hand, if DMU_k is FDH but not DEA efficient then it is proved that there exists no positive multiplier vector >w for which (a) holds, i.e. for any positive w there exists at least one DMU_j for which w ^T p _j > w^T p _k . Since, therefore, FDH efficiency does not guarantee price efficiency what is its economic significance? Without economic significance, how can FDH be considered as being more than a mathematical system however logically soundly it may be conceived?