A Survey of L1 Regression

L₁ regularization, or regularization with an L₁ penalty, is a popular idea in statistics and machine learning. This paper reviews the concept and application of L₁ regularization for regression. It is not our aim to present a comprehensive list of the utilities of the L₁ penalty in the regression setting. Rather, we focus on what we believe is the set of most representative uses of this regularization technique, which we describe in some detail. Thus, we deal with a number of L₁‐regularized methods for linear regression, generalized linear models, and time series analysis. Although this review targets practice rather than theory, we do give some theoretical details about L₁‐penalized linear regression, usually referred to as the least absolute shrinkage and selection operator (lasso).     

Nonparametric regression function estimation using interaction least squares splines and comlexity regularization

Let (X, Y) be a pair of random variables withsupp(X)⊆[0,1] ^l andEY ²<∞. Letm ^* be the best approximation of the regression function of (X, Y) by sums of functions of at mostd variables (1≤d≤l). Estimation ofm ^* from i.i.d. data is considered. For the estimation interaction least squares splines, which are defined as sums of polynomial tensor product splines of at mostd variables, are used. The knot sequences of the tensor product splines are chosen equidistant. Complexity regularization is used to choose the number of the knots and the degree of the splines automatically using only the given data. Without any additional condition on the distribution of (X, Y) the weak and strongL ₂-consistency of the estimate is shown. Furthermore, for everyp≥1 and every distribution of (X, Y) withsupp(X)⊆[0,1] ^l ,y bounded andm ^* p-smooth, the integrated squared error of the estimate achieves up to a logarithmic factor the (optimal) rate      

Robust estimation and prediction of economic systems: The case of partial structural variability

Prediction of economic and social systems is in general based on the assumption that the structure of the system does not vary over time. Such an assumption is seldom fulfilled. Often, at least some of the behavioral relations of a system are characterized by structural variability. The problem of reduced form estimation and prediction in systems with such partial structural variability is considered in the paper. A robust SF-RF strategy, combining direct, unrestricted and indirect, restricted reduced form estimation, is emphasized. This strategy, as well as more commonly applied alternatives of L ₁- and L ₂-norm estimation, is studied numerically within a small interdependent system with partially occurring, discrete parameter shifts. The different estimation techniques are evaluated with respect to structural form estimation and prediction performance. The study indicates advantages of the SF-RF strategy over considered alternatives.     

The multiresolution histogram

We introduce a new method for locally adaptive histogram construction that doesn’t resort to a standard distribution and is easy to implement: the multiresolution histogram. It is based on aL ₂ analysis of the mean integrated squared error with Haar wavelets and hence can be associated with a multiresolution analysis of the sample space.     

Vergleich zweier Operationscharakteristiken

Summary We compare the OC-curvesL _n.c (p) (1) andL _n.c ^* (p) (2). The first is founded on the binomial distribution, the latter relates to the Poisson distribution and is often used as approximation. These OC-curves occur in Statistical Quality Control as probabilities for the acception of a lot as approximations for such probabilities; they are regarded as functions of the fraction defectivep. It is shown that the two OC-curves have exactly one intersection point between 0 and 1, if the acceptance numberc is 1 and the sample sizen is >c+1.Forp between 0 and the intersection pointp _s we have thenL _n.c.(p)>L _n.c ^* (p); from p _s <p1 followsL _n.c(p)n.c ^* (p).An interval is given which coversp _s and with an example it is shown how one might use the results of this paper for the construction of sampling plans.     

The Analysis of Seasonal Long Memory: The Case of Spanish Inflation*

This paper describes semiparametric techniques recently proposed for the analysis of seasonal or cyclical long memory and applies them to a monthly Spanish inflation series. One of the conclusions is that this series has long memory not only at the origin but also at some but not all seasonal frequencies, suggesting that the fractional difference operator (1−L¹²)^d should be avoided. Moreover, different persistent cycles are observed before and after the first oil crisis. Whereas the cycles seem stationary in the former period, we find evidence of a unit root after 1973, which implies that a shock has a permanent effect. Finally, it is shown how to compute the exact impulse responses and the coefficients in the autoregressive expansion of parametric seasonal long memory models. These two quantities are important to assess the impact of aleatory shocks such as those produced by a change of economic policy and for forecasting purposes, respectively.     

Discrete discrepancy in factorial designs

Discrepancy measure can be utilized as a uniformity measure for comparing factorial designs. A so-called discrete discrepancy has been used to evaluate the uniformity of factorials. In this paper we give linkages among uniformity measured by the discrete discrepancy, generalized minimum aberration, minimum moment aberration and uniformity measured by the centered L₂-discrepancy/the wrap-around L₂-discrepancy. These close linkages provide a significant justification for the discrete discrepancy used to measure uniformity of factorial designs.     

Convergence of a branching type recursion with non-stationary immigration

The asymptotic distribution of a branching type recursion with non-stationary immigration is investigated. The recursion is given by , where (X _l ) is a random sequence, (L _n ⁻¹⁽¹⁾ ) are iid copies ofL _n−1,K is a random number andK, (L _n ⁻¹⁽¹⁾ ), {(X _l ),Y _n } are independent. This recursion has been studied intensively in the literature in the case thatX _l ≥0,K is nonrandom andY _n =0 ∀n. Cramer, Rüschendorf (1996b) treat the above recursion without immigration with starting conditionL ₀=1, and easy to handle cases of the recursion with stationary immigration (i.e. the distribution ofY _n does not depend on the timen). In this paper a general limit theorem will be deduced under natural conditions including square-integrability of the involved random variables. The treatment of the recursion is based on a contraction method. The conditions of the limit theorem are built upon the knowledge of the first two moments ofL _n . In case of stationary immigration a detailed analysis of the first two moments ofL _n leads one to consider 15 different cases. These cases are illustrated graphically and provide a straight forward means to check the conditions and to determine the operator whose unique fixed point is the limit distribution of the normalizedL _n .     

Existence of equilibria in economies with externalities and non-convexities in an infinite-dimensional commodity space

We prove an equilibrium existence theorem for economies with externalities, general types of non-convexities in the production sector, and infinitely many commodities. The consumption sets, the preferences of the consumers, and the production possibilities are represented by set-valued mappings to take into account the external effects. The firms set their prices according to general pricing rules which are supposed to have bounded losses and may depend upon the actions of the other economic agents. The commodity space is L_∞(M,M,μ), the space of all μ-essentially bounded M-measurable functions on M.As for our existence result, we consider the framework of Bewley (1972). However, there are four major problems in using this technique. To overcome two of these difficulties, we impose strong lower hemi-continuity assumptions upon the economies. The remaining problems are removed when the finite economies are large enough.Our model encompasses previous works on the existence of general equilibria when there are externalities and non-convexities but the commodity space is finite dimensional and those on general equilibria in non-convex economies with infinitely many commodities when no external effect is taken into account.     

A note on mean testing for high dimensional multivariate data under non‐normality

A test statistic is considered for testing a hypothesis for the mean vector for multivariate data, when the dimension of the vector, p, may exceed the number of vectors, n, and the underlying distribution need not necessarily be normal. With n,p→∞, and under mild assumptions, but without assuming any relationship between n and p, the statistic is shown to asymptotically follow a chi‐square distribution. A by product of the paper is the approximate distribution of a quadratic form, based on the reformulation of the well‐known Box's approximation, under high‐dimensional set up. Using a classical limit theorem, the approximation is further extended to an asymptotic normal limit under the same high dimensional set up. The simulation results, generated under different parameter settings, are used to show the accuracy of the approximation for moderate n and large p.     

Goodness-of-fit tests for discrete models based on the integrated distribution function

This paper presents a new widely applicable omnibus test for discrete distributions which is based on the difference between the integrated distribution function Ψ(t)=∫_t ^∞ (1−F(x))dx and its empirical counterpart. A bootstrap version of the test for common lattice models has accurate error rates even for small samples and exhibits high power with respect to competitive procedures over a large range of alternatives. Received: July 1998     

A Note on the Regularized Approach to Biased 2SLS Estimation with Weak Instruments

The presence of weak instruments is translated into a nearly singular problem in a control function representation. Therefore, the ‐norm type of regularization is proposed to implement the 2SLS estimation for addressing the weak instrument problem. The ‐norm regularization with a regularized parameter O(n) allows us to obtain the Rothenberg (1984) type of higher‐order approximation of the 2SLS estimator in the weak instrument asymptotic framework. The proposed regularized parameter yields the regularized concentration parameter O(n), which is used as a standardized factor in the higher‐order approximation. We also show that the proposed ‐norm regularization consequently reduces the finite sample bias. A number of existing estimators that address finite sample bias in the presence of weak instruments, especially Fuller's limited information maximum likelihood estimator, are compared with our proposed estimator in a simple Monte Carlo exercise.     

Inducing relations on incomplete ordinal data

Let X denote a set of n elements or stimuli which have an underlying linear ranking L based on some attribute of comparison. Incomplete ordinal data is known about L, in the form of a partial order P on X. This study considers methods which attempt to induce, or reconstruct, L based only on the ordinal information in P. Two basic methods for approaching this problem are the cardinal and sequel construction methods. Exact values are computed for the expected error of weak order approximations of L from the cardinal and sequel construction methods. Results involving interval orders and semiorders for P are also considered. Previous simulation comparisons for cardinal and sequel construction methods on interval orders were found to depend on the specific model that was used to generate random interval orders, and were not found to hold for interval orders in general. Finally, we consider the likelihood that any particular linear extension of P is the underlying L.     

Vergleich der hypergeometrischen mit der Binomial-Verteilung

Zusammenfassung Es seiL die Operations-Charakteristik beim Testen einer Hypothese über eine unbekannte Wahrscheinlichkeitp eines EreignissesA. L ist also die Wahrscheinlichkeit dafür, da? h?chstensc Elemente mitA in der Stichprobe auftreten. Wird die Stichprobe ohne Zurücklegen gezogen, so h?ngtL vom UmfangN der Grundgesamtheit ab. Es wird untersucht, für welchep die Operations-CharakteristikL monoton vonN abh?ngt. Den Abschlu? bilden Vergleiche der Operations-Charakteristik bei der Binomial-Verteilung mitL und deren Ann?herung durch die Poisson-Verteilung.

---

Summary LetL be the operating characteristic for testing a hypothesis concerning an unknown probabilityp of an eventA. ThenL is the probability of being no more thanc elements withA in the sample. When the sample is drawn without replacementL depends on the sizeN of the population. Investigation is made for whichp the operating characteristicL depends monotone onN. Finally the operating characteristic of the binomial distribution is compared withL and with the approximation by the Poisson distribution.

     

Connections between uniformity and aberration in general multi-level factorials

Discrepancy is a kind of important measure used in experimental designs. Among various existing discrepancies, the discrete discrepancy, centered L ₂-(CD ₂) and wrap-around L ₂-discrepancy (WD ₂) have been well justified and widely used. In this paper, using the second-order polynomials of indicator functions for these three discrepancies, we investigate the close relationships between them and the generalized wordlength pattern, and provide some conditions under which a design having one of these minimum discrepancies is equivalent to having generalized minimum aberration (GMA). These results provide further justifications for the criterion of GMA in terms of uniformity. In addition, the expressions of the discrepancies in the quadratic forms of the indicator functions are useful for us to find optimal designs under any of them.     

The non-existence of smooth demand in general banach spaces

We study the questions of existence and smoothness of demand functions with an infinite number of commodities. The main result obtained, under some hypothesis, is: if a C¹ demand exists in a commodity space B, then B can be given an inner product structure. For example, if B is L_p, 1p∞, and if there exists a C¹ demand function defined on B then p must be 2. Another result is: if a demand function exists, defined for all prices p and income, then the commodity space must be reflexive. For example, if B is L_p and a demand function exists on B, defined for all prices and incomes then 1<p<∞. We also study the cases L_∞ and L₁ with weaker assumptions. We finish the paper proving that the demand function is always defined for a dense set of prices and convenient incomes.     

A closed-form expression for the net present value of a time-power cash-flow function

Previous work has shown how Laplace Transform Theory and z-Transform Theory may be used in Net Present Value Analysis. A closed-form expression is easily derived for a continuous cash flow t^k, where t denotes time and k is a positive integer. In the corresponding discrete case, explicit expressions have only been provided for k 10. In this article we derive a closed-form formula for the missing general case when k is an arbitrary non-negative integer. As a byproduct, we also obtain a closed-form expression for the z-transform of such a power series.     

Parametric bootstrap tests for continuous and discrete distributions

Statistical procedures based on the estimated empirical process are well known for testing goodness of fit to parametric distribution families. These methods usually are not distribution free, so that the asymptotic critical values of test statistics depend on unknown parameters. This difficulty may be overcome by the utilization of parametric bootstrap procedures. The aim of this paper is to prove a weak approximation theorem for the bootstrapped estimated empirical process under very general conditions, which allow both the most important continuous and discrete distribution families, along with most parameter estimation methods. The emphasis is on families of discrete distributions, and simulation results for families of negative binomial distributions are also presented.     

Asymptotic results and tests for the choice of approximative models in nonlinear two-phases regression models, heteroscedatic case

This paper is devoted to the study of the least squares estimator of f for the classical, fixed design, nonlinear model X (t _i)=f(t _i)+ε(t _i), i=1,2,…,n, where the (ε(t _i))_i=1,…,n are independent second order r.v.. The estimation of f is based upon a given parametric form. In Brodeau (1993) this subject has been studied in the homoscedastic case. This time we assume that the ε(t _i) have non constant and unknown variances σ²(t _i). Our main goal is to develop two statistical tests, one for testing that f belongs to a given class of functions possibly discontinuous in their first derivative, and another for comparing two such classes. The fundamental tool is an approximation of the elements of these classes by more regular functions, which leads to asymptotic properties of estimators based on the least squares estimator of the unknown parameters. We point out that Neubauer and Zwanzig (1995) have obtained interesting results for connected subjects by using the same technique of approximation. Received: February 1996