A Computational Perspective on Projection Pursuit in High Dimensions: Feasible or Infeasible Feature Extraction

Finding a suitable representation of multivariate data is fundamental in many scientific disciplines. Projection pursuit ( $\text{PP}$) aims to extract interesting ‘non-Gaussian’ features from multivariate data, and tends to be computationally intensive even when applied to data of low dimension. In high-dimensional settings, a recent work (Bickel et al., 2018) on $\text{PP}$ addresses asymptotic characterization and conjectures of the feasible projections as the dimension grows with sample size. To gain practical utility of and learn theoretical insights into $\text{PP}$ in an integral way, data analytic tools needed to evaluate the behaviour of $\text{PP}$ in high dimensions become increasingly desirable but are less explored in the literature. This paper focuses on developing computationally fast and effective approaches central to finite sample studies for (i) visualizing the feasibility of $\text{PP}$ in extracting features from high-dimensional data, as compared with alternative methods like $\text{PCA}$ and $\text{ICA}$, and (ii) assessing the plausibility of $\text{PP}$ in cases where asymptotic studies are lacking or unavailable, with the goal of better understanding the practicality, limitation and challenge of $\text{PP}$ in the analysis of large data sets.     

Characterization of the multivariate normal distribution by conditional normal distributions

Summary LetX andY be two random vectors with values in ℝ ^k and ℝ∝, respectively. IfZ=(X ^T,Y ^T) ^T is multivariate normal thenX givenY=y andY givenX=x are (multivariate) normal; the converse is wrong. In this paper simple additional conditions are stated such that the converse is true, too. Furthermore, the case is treated that the random vectorZ=(X ₁ ^T , …,X _t ^T ) ^T is splitted intot≥3 partsX ₁, …,X _t.     

Some properties of the minimum and the maximum of random variables with joint logconcave distributions

It is shown that if (X ₁, X ₂, . . . , X _n ) is a random vector with a logconcave (logconvex) joint reliability function, then X _P = min _i∈P X _i has increasing (decreasing) hazard rate. Analogously, it is shown that if (X ₁, X ₂, . . . , X _n ) has a logconcave (logconvex) joint distribution function, then X ^P  = max _i∈P X _i has decreasing (increasing) reversed hazard rate. If the random vector is absolutely continuous with a logconcave density function, then it has a logconcave reliability and distribution functions and hence we obtain a result given by Hu and Li (Metrika 65:325–330, 2007). It is also shown that if (X ₁, X ₂, . . . , X _n ) has an exchangeable logconcave density function then both X _P and X ^P have increasing likelihood ratio.     

Sulle operazioni associative tra variabili casuali

Si mostra che, sotto condizioni di regolarità, seo è un’operazione associativa tra variabili casuali reali e indipendenti, è definibile una trasformata integrale ξ delle loro funzioni di ripartizione con la proprietà: ξx ₀ Y (t)=ξx(t)·ξ _y (t). Si indicano alcune proprietà di tale trasformata e si tratta della possibilità di estendere a un’operazione associativa risultati noti per l’addizione tra variabili casuali. In particolare ci si occupa dell’« infinita divisibilità » fornendo condizioni perché una variabile casualeX ammetta la rappresentazioneX=X _1O X _2O … _o X _n per ognin naturale con leX _i indipendenti e identicamente ditribuite.     

What Is the Economic Meaning of FDH?

The central feature of the FDH model is the lack of convexity for its production possibility set, TF. Starting with n observed (distinct) decision making units DMU_k , each defined by an input-output vector p _k = [y _k -x _k], domination is defined by ordinary vector inequalities. DMU_k is said to dominate DMU_j if p _k ≥ p _j , p _k ≠ p _j . The FDH production possibility set TF consists of the observed DMU_j together with all input-output vectors p=[y_k,?x_k] with y ≥ 0, x ≥ 0, y ≠ 0, x ≠ 0 which are dominated by at least one of the observed DMU_j. DMU_k is defined as “FDH efficient” if no DMU_j dominates it. In the BCC (or variable return to scale) DEA model the production possibility set TB consists of the observed DMU_k together with all input-output vectors dominated by any convex combination of them and DMU_k is DEA efficient if it is not dominated by any p in TB. In the DEA model, economic meaning is established by the introduction of (non negative) multiplier (price) vectors w = [u,v]. If DMU_k is undominated (in TB) then there exists a positive multiplier vector w for which (a) w ^T p _k = u ^T y _k ? v ^T x _k ≥ w ^T p for every p ∈ TB. In everyday language, the net return (or profit) for DMU_k relative to the given multiplier vector w is at least as great as that for any production possibility p. On the other hand, if DMU_k is FDH but not DEA efficient then it is proved that there exists no positive multiplier vector >w for which (a) holds, i.e. for any positive w there exists at least one DMU_j for which w ^T p _j > w^T p _k . Since, therefore, FDH efficiency does not guarantee price efficiency what is its economic significance? Without economic significance, how can FDH be considered as being more than a mathematical system however logically soundly it may be conceived?

     

Dominating estimators for minimum-variance portfolios

In this paper, we derive two shrinkage estimators for minimum-variance portfolios that dominate the traditional estimator with respect to the out-of-sample variance of the portfolio return. The presented results hold for any number of assets d≥4$d\ge 4$ and number of observations n≥d+2$n\ge d+2$. The small-sample properties of the shrinkage estimators as well as their large-sample properties for fixed d$d$ but n→∞$n\to \infty$ and n,d→∞$n,d\to \infty$ but n/d→q≤∞$n/d\to q\le \infty$ are investigated. Furthermore, we present a small-sample test for the question of whether it is better to completely ignore time series information in favor of naive diversification.     

Zur Entscheidung y>y0 aufgrund der Messung einer korrelierten Variablen

Zusammenfassung Es wird eine optimale Strategie im Sinne des minimalen erwarteten Verlustes für die beiden Entscheidungeny>y _o undyy _o aufgrund der Messungen einer mitY positiv korrelierten, einfacher und/oder billiger zugänglichen ZufallsvariablenX abgeleitet. Dabei wird angenommen, daßX undY nach einer bivariaten Normalverteilung mit bekannten Parametern verteilt sind und die Entscheidungyy _o getroffen wird, wennx größer ist als ein zu bestimmendesx _o, und die Entscheidungy>y _o, wennx gleich oder kleiner als diesesx _o ist. Für die Bestimmung des optimalenx _o werden zunächst die Kosten für die beiden Fehlentscheidungen jeweils als konstant vorausgesetzt, in einem weiteren Ansatz wird jedoch für die Mißklassifikationyy _o eine mity exponentiell wachsende Risikofunktion angenommen. Um die relative Häufigkeit der zu erwartenden Fehlklassifikationen abschätzen zu können, wird schließlich die bedingte WahrscheinlichkeitP(x>x _o,y) errechnet.

---

Summary An optimal strategy, with minimum expected risk, for the decisionsy>y _o oryy _o is constructed on the basis of the measurement of a variableX, which is positively correlated withY and can be measured more easily and/or with smaller expense. A bivariate normal distribution with known parameters is assumed forX andY. For the observationsx a limitx _o is aimed at, so that the decisionsy>y _o oryy _o are taken ifx>x orxx _o respectively. Optimal values ofx _o are first calculated under the assumption of constant losses for the two misclassifications (x>x _o ifyy _o andxx _o ify>y _o). In a further approach the loss for a wrong decisionyy _o is assumed to increase exponentially withy. Finally the conditional probabilityP (x>x _o\y) is calculated to get an assessment of the relative frequencies of wrong decisions to be expected.

     

Cointegrating rank selection in models with time-varying variance

Reduced rank regression (RRR) models with time varying heterogeneity are considered. Standard information criteria for selecting cointegrating rank are shown to be weakly consistent in semiparametric RRR models in which the errors have general nonparametric short memory components and shifting volatility provided the penalty coefficient _Cn→∞$C_n\to \infty$ and _Cn/n→0$C_n/n\to 0$ as n→∞$n\to \infty$. The AIC criterion is inconsistent and its limit distribution is given. The results extend those in Cheng and Phillips (2009a) and are useful in empirical work where structural breaks or time evolution in the error variances is present. An empirical application to exchange rate data is provided.     

A complete asymptotic series for the autocovariance function of a long memory process

An infinite-order asymptotic expansion is given for the autocovariance function of a general stationary long-memory process with memory parameter d∈(−1/2,1/2)$d\in (-1/2,1/2)$. The class of spectral densities considered includes as a special case the stationary and invertible ARFIMA(p,d,q$p,d,q$) model. The leading term of the expansion is of the order O(1/k^1−2d)$O(1/k^{1-2d})$, where k$k$ is the autocovariance order, consistent with the well known power law decay for such processes, and is shown to be accurate to an error of O(1/k^3−2d)$O(1/k^{3-2d})$. The derivation uses Erdélyi’s [Erdélyi, A., 1956. Asymptotic Expansions. Dover Publications, Inc, New York] expansion for Fourier-type integrals when there are critical points at the boundaries of the range of integration - here the frequencies {0,2π}$\{0,2\pi \}$. Numerical evaluations show that the expansion is accurate even for small k$k$ in cases where the autocovariance sequence decays monotonically, and in other cases for moderate to large k$k$. The approximations are easy to compute across a variety of parameter values and models.     

Locally minimax test of independence in elliptically symmetrical distributions with additional observations

Summary LetX=(X _ij )=(X ₁, ...,X _n )’,X’ _i =(X _i1, ...,X _ip )’,i=1,2, ...,n be a matrix having a multivariate elliptical distribution depending on a convex functionq with parameters, 0,σ. Let ϱ²=ϱ ₂ ^-2 be the squared multiple correlation coefficient between the first and the remainingp ₂+p ₃=p−1 components of eachX _i . We have considered here the problem of testingH ₀:ϱ²=0 against the alternativesH ₁:ϱ ₁ ^-2 =0, ϱ ₂ ^-2 >0 on the basis ofX andn ₁ additional observationsY ₁ (n ₁×1) on the first component,n ₂ observationsY ₂(n ₂×p ₂) on the followingp ₂ components andn ₃ additional observationsY ₃(n ₃×p ₃) on the lastp ₃ components and we have derived here the locally minimax test ofH ₀ againstH ₁ when ϱ ₂ ^-2 →0 for a givenq. This test, in general, depends on the choice ofq of the familyQ of elliptically symmetrical distributions and it is not optimality robust forQ.