Dimension Reduction And Regression For Tensor Data And Mixture Models

Author : Kai Deng
Publisher :
ISBN 13 :
Total Pages : 0 pages
Book Rating : 4.:/5 (141 download)

DOWNLOAD NOW!

Download or read book Topics on Mixture Models and Discriminant Analysis written by Kai Deng and published by . This book was released on 2023 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: Mixture models for clustering and regressions and discriminant analysis are the cornerstones of multivariate statistics and supervised/unsupervised learning research. The structure of data has become increasingly complex in many modern applications including but not limited to computational biology, recommendation systems and text/image analysis. Therefore, it is of great interest to develop methodologies and algorithms for mixture models and discriminant analysis that target the challenges arising from such complex data. In this dissertation, I address three types of challenging supervised and unsupervised topics with novel methodologies and algorithms: (1) tensor data simultaneous clustering and multiway dimension reduction; (2) high-dimensional heterogeneous data in mixture linear regression; (3) multivariate and multi-label response classification in high dimensions. The three chapters are elaborated as follows. In the form of multi-dimensional arrays, tensor data have become increasingly prevalent in modern scientific studies and biomedical applications such as computational biology, brain imaging analysis, and process monitoring system. These data are intrinsically heterogeneous with complex dependencies and structure. Therefore, ad-hoc dimension reduction methods on tensor data may lack statistical efficiency and can obscure essential findings. Model-based clustering is a cornerstone of multivariate statistics and unsupervised learning; however, existing methods and algorithms are not designed for tensor-variate samples. In the first chapter, we propose a Tensor Envelope Mixture Model (TEMM) for simultaneous clustering and multiway dimension reduction of tensor data. TEMM incorporates tensor-structure-preserving dimension reduction into mixture modeling and drastically reduces the number of free parameters and estimative variability. An EM-type algorithm is developed to obtain likelihood-based estimators of the cluster means and covariances, which are jointly parameterized and constrained onto a series of lower-dimensional subspaces known as the tensor envelopes. We demonstrate the encouraging empirical performance of the proposed method in extensive simulation studies and a real data application in comparison with existing vector and tensor clustering methods. In the second chapter, we consider the problem of finite mixture of linear regressions (MLR) for high-dimensional heterogeneous data where the sample size is much smaller than the number of random variables, which is widely used in many modern applications such as biological science, genetics and engineering. In order to capture the common sparse structure in large heterogeneous data, traditional high-dimensional EM algorithm can be computational intractable thus fail to produce meaningful estimation results. We propose a fast group-penalized EM algorithm (FGEM) for high-dimensional MLR that estimates the regression coefficients from a group sparsity perspective and is computationally efficient and less sensitive to initialization. The statistical property of the proposed algorithm is established without requiring sample-splitting that allows the predictor dimension grows exponentially with the sample size. We demonstrate the encouraging performance of FGEM in numerical studies in comparison with traditional high-dimensional EM algorithms. The problem of classifying multiple categorical responses is pervasive in modern machine learning and statistics, with diverse applications in fields such as bioinformatics and image classification. The third chapter investigates linear discriminant analysis (LDA) with high-dimensional predictors and multiple multi-class responses. Specifically, we examine two different classification scenarios under the bivariate LDA model: joint classification of the two responses and conditional classification of one response while observing the other. To achieve optimal classification rules for both scenarios, we introduce two novel tensor formulations of the discriminant coefficients and corresponding penalties. For joint classification, we propose an overlapping group lasso penalty and a blockwise coordinate descent algorithm to efficiently compute joint tensor discriminant coefficients. For conditional classification, we utilize an alternating direction method of multipliers (ADMM) algorithm to compute tensor discriminant coefficients under new constraints. We extend our method and algorithms to general multivariate responses. Finally, we validate the effectiveness of our approach through simulation studies and real data examples.