An Unexpected Property Of Minimax Estimation In The Relative Squared Error Approach To Linear Regression Analysis

Author : Weijie Su
Publisher :
ISBN 13 :
Total Pages : pages
Book Rating : 4.:/5 (951 download)

DOWNLOAD NOW!

Download or read book Multiple Testing and Minimax Estimation in Sparse Linear Regression written by Weijie Su and published by . This book was released on 2016 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt: In many real-world statistical problems, we observe a response variable of interest together with a large number of potentially explanatory variables of which a majority may be irrelevant. For this type of problem, controlling the false discovery rate (FDR) guarantees that most of the selected variables, often termed discoveries in a scientific context, are truly explanatory and thus replicable. Inspired by ideas from the Benjamini-Hochberg procedure (BHq), this thesis proposes a new method named SLOPE to control the FDR in sparse high-dimensional linear regression. SLOPE is a computationally efficient procedure that works by regularizing the fitted coefficients according to their ranks: the higher the rank, the larger the penalty. This adaptive regularization is analogous to the BHq, which compares more significant p-values with more stringent thresholds. Under orthogonal designs, SLOPE with the BHq critical values is proven to control FDR at any given level. Moreover, we demonstrate empirically that this method also appears to have appreciable inferential properties under more general design matrices while offering substantial power. The thesis proceeds to explore the estimation properties of SLOPE. Although SLOPE was developed from a multiple testing viewpoint, we show the surprising result that it achieves optimal squared errors under Gaussian random designs. This optimality holds under a weak assumption on the l0-sparsity level of the underlying signals, and is sharp in the sense that this is the best possible error any estimator can achieve. An appealing feature is that SLOPE does not require any knowledge of the degree of sparsity, and yet automatically adapts to yield optimal total squared errors over a wide range of l0-sparsity classes. Finally, we conclude this thesis by focusing on Nesterov's accelerated scheme, which is integral to a fast algorithmic implementation of SLOPE. Specifically, we prove that, as the step size vanishes, this scheme converges in a rigorous sense to a second-order ordinary differential equation (ODE). This continuous time ODE allows for a better understanding of Nesterov's scheme, and thus it can serve as a tool for analyzing and generalizing this scheme. A fruitful application of this tool yields a family of schemes with similar convergence rates. The ODE interpretation also suggests restarting Nesterov's scheme leading to a new algorithm, which is proven to converge at a linear rate whenever the objective is strongly convex.