Read Books Online and Download eBooks, EPub, PDF, Mobi, Kindle, Text Full Free.
New Algorithms For Learning Of Mixture Models And Their Application For Classification And Density Estimation
Download New Algorithms For Learning Of Mixture Models And Their Application For Classification And Density Estimation full books in PDF, epub, and Kindle. Read online New Algorithms For Learning Of Mixture Models And Their Application For Classification And Density Estimation ebook anywhere anytime directly on your device. Fast Download speed and no annoying ads. We cannot guarantee that every ebooks is available!
Book Synopsis New Algorithms for Learning of Mixture Models and Their Application for Classification and Density Estimation by : Bambang Heru Iswanto
Download or read book New Algorithms for Learning of Mixture Models and Their Application for Classification and Density Estimation written by Bambang Heru Iswanto and published by Logos Verlag Berlin. This book was released on 2005 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: Mixture model is known as a convenient way for modelling the probability density function in statistics. Recently, the method is adopted by machine learning communities in a variety of application settings such as cluster analysis, classification, density estimation and function approximation. This book concerns with learning algorithms of the mixture models for density estimation and classification tasks. Special attention is given for the semi-supervised learning and active learning methods which are very important in many practical settings. The presented learning methods attempt to reduce the size of labelled data sets required to achieve certain level of classification performance.
Book Synopsis Finite Mixture Models by : Geoffrey McLachlan
Download or read book Finite Mixture Models written by Geoffrey McLachlan and published by John Wiley & Sons. This book was released on 2004-03-22 with total page 419 pages. Available in PDF, EPUB and Kindle. Book excerpt: An up-to-date, comprehensive account of major issues in finitemixture modeling This volume provides an up-to-date account of the theory andapplications of modeling via finite mixture distributions. With anemphasis on the applications of mixture models in both mainstreamanalysis and other areas such as unsupervised pattern recognition,speech recognition, and medical imaging, the book describes theformulations of the finite mixture approach, details itsmethodology, discusses aspects of its implementation, andillustrates its application in many common statisticalcontexts. Major issues discussed in this book include identifiabilityproblems, actual fitting of finite mixtures through use of the EMalgorithm, properties of the maximum likelihood estimators soobtained, assessment of the number of components to be used in themixture, and the applicability of asymptotic theory in providing abasis for the solutions to some of these problems. The author alsoconsiders how the EM algorithm can be scaled to handle the fittingof mixture models to very large databases, as in data miningapplications. This comprehensive, practical guide: * Provides more than 800 references-40% published since 1995 * Includes an appendix listing available mixture software * Links statistical literature with machine learning and patternrecognition literature * Contains more than 100 helpful graphs, charts, and tables Finite Mixture Models is an important resource for both applied andtheoretical statisticians as well as for researchers in the manyareas in which finite mixture models can be used to analyze data.
Book Synopsis Finite Mixture Models by : Geoffrey McLachlan
Download or read book Finite Mixture Models written by Geoffrey McLachlan and published by Wiley-Interscience. This book was released on 2004-04-05 with total page 456 pages. Available in PDF, EPUB and Kindle. Book excerpt: An up-to-date, comprehensive account of major issues in finite mixture modeling This volume provides an up-to-date account of the theory and applications of modeling via finite mixture distributions. With an emphasis on the applications of mixture models in both mainstream analysis and other areas such as unsupervised pattern recognition, speech recognition, and medical imaging, the book describes the formulations of the finite mixture approach, details its methodology, discusses aspects of its implementation, and illustrates its application in many common statistical contexts. Major issues discussed in this book include identifiability problems, actual fitting of finite mixtures through use of the EM algorithm, properties of the maximum likelihood estimators so obtained, assessment of the number of components to be used in the mixture, and the applicability of asymptotic theory in providing a basis for the solutions to some of these problems. The author also considers how the EM algorithm can be scaled to handle the fitting of mixture models to very large databases, as in data mining applications. This comprehensive, practical guide: * Provides more than 800 references-40% published since 1995 * Includes an appendix listing available mixture software * Links statistical literature with machine learning and pattern recognition literature * Contains more than 100 helpful graphs, charts, and tables Finite Mixture Models is an important resource for both applied and theoretical statisticians as well as for researchers in the many areas in which finite mixture models can be used to analyze data.
Book Synopsis Algorithms and Programs of Dynamic Mixture Estimation by : Ivan Nagy
Download or read book Algorithms and Programs of Dynamic Mixture Estimation written by Ivan Nagy and published by Springer. This book was released on 2017-08-14 with total page 118 pages. Available in PDF, EPUB and Kindle. Book excerpt: This book provides a general theoretical background for constructing the recursive Bayesian estimation algorithms for mixture models. It collects the recursive algorithms for estimating dynamic mixtures of various distributions and brings them in the unified form, providing a scheme for constructing the estimation algorithm for a mixture of components modeled by distributions with reproducible statistics. It offers the recursive estimation of dynamic mixtures, which are free of iterative processes and close to analytical solutions as much as possible. In addition, these methods can be used online and simultaneously perform learning, which improves their efficiency during estimation. The book includes detailed program codes for solving the presented theoretical tasks. Codes are implemented in the open source platform for engineering computations. The program codes given serve to illustrate the theory and demonstrate the work of the included algorithms.
Book Synopsis Models and Estimation Algorithms for Nonparametric Finite Mixtures with Conditionally Independent Multivariate Component Densities by : Vy-Thuy-Lynh Hoang
Download or read book Models and Estimation Algorithms for Nonparametric Finite Mixtures with Conditionally Independent Multivariate Component Densities written by Vy-Thuy-Lynh Hoang and published by . This book was released on 2017 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: Recently several authors have proposed models and estimation algorithms for finite nonparametric multivariate mixtures, whose identifiability is typically not obvious. Among the considered models, the assumption of independent coordinates conditional on the subpopulation from which each observation is drawn is subject of an increasing attention, in view of the theoretical and practical developments it allows, particularly with multiplicity of variables coming into play in the modern statistical framework. In this work we first consider a more general model assuming independence, conditional on the component, of multivariate blocks of coordinates instead of univariate coordinates, allowing for any dependence structure within these blocks. Consequently, the density functions of these blocks are completely multivariate and nonparametric. We present identifiability arguments and introduce for estimation in this model two methodological algorithms whose computational procedures resemble a true EM algorithm but include an additional density estimation step: a fast algorithm showing empirical efficiency without theoretical justification, and a smoothed algorithm possessing a monotony property as any EM algorithm does, but more computationally demanding. We also discuss computationally efficient methods for estimation and derive some strategies. Next, we consider a multivariate extension of the mixture models used in the framework of multiple hypothesis testings, allowing for a new multivariate version of the False Discovery Rate control. We propose a constrained version of our previous algorithm, specifically designed for this model. The behavior of the EM-type algorithms we propose is studied numerically through several Monte Carlo experiments and high dimensional real data, and compared with existing methods in the literature. Finally, the codes of our new algorithms are progressively implemented as new functions in the publicly-available package mixtools for the R statistical software.
Download or read book Mixtures written by Kerrie L. Mengersen and published by John Wiley & Sons. This book was released on 2011-05-03 with total page 352 pages. Available in PDF, EPUB and Kindle. Book excerpt: This book uses the EM (expectation maximization) algorithm to simultaneously estimate the missing data and unknown parameter(s) associated with a data set. The parameters describe the component distributions of the mixture; the distributions may be continuous or discrete. The editors provide a complete account of the applications, mathematical structure and statistical analysis of finite mixture distributions along with MCMC computational methods, together with a range of detailed discussions covering the applications of the methods and features chapters from the leading experts on the subject. The applications are drawn from scientific discipline, including biostatistics, computer science, ecology and finance. This area of statistics is important to a range of disciplines, and its methodology attracts interest from researchers in the fields in which it can be applied.
Book Synopsis Statistical Pattern Recognition by : Andrew R. Webb
Download or read book Statistical Pattern Recognition written by Andrew R. Webb and published by John Wiley & Sons. This book was released on 2011-10-13 with total page 604 pages. Available in PDF, EPUB and Kindle. Book excerpt: Statistical pattern recognition relates to the use of statistical techniques for analysing data measurements in order to extract information and make justified decisions. It is a very active area of study and research, which has seen many advances in recent years. Applications such as data mining, web searching, multimedia data retrieval, face recognition, and cursive handwriting recognition, all require robust and efficient pattern recognition techniques. This third edition provides an introduction to statistical pattern theory and techniques, with material drawn from a wide range of fields, including the areas of engineering, statistics, computer science and the social sciences. The book has been updated to cover new methods and applications, and includes a wide range of techniques such as Bayesian methods, neural networks, support vector machines, feature selection and feature reduction techniques.Technical descriptions and motivations are provided, and the techniques are illustrated using real examples. Statistical Pattern Recognition, 3rd Edition: Provides a self-contained introduction to statistical pattern recognition. Includes new material presenting the analysis of complex networks. Introduces readers to methods for Bayesian density estimation. Presents descriptions of new applications in biometrics, security, finance and condition monitoring. Provides descriptions and guidance for implementing techniques, which will be invaluable to software engineers and developers seeking to develop real applications Describes mathematically the range of statistical pattern recognition techniques. Presents a variety of exercises including more extensive computer projects. The in-depth technical descriptions make the book suitable for senior undergraduate and graduate students in statistics, computer science and engineering. Statistical Pattern Recognition is also an excellent reference source for technical professionals. Chapters have been arranged to facilitate implementation of the techniques by software engineers and developers in non-statistical engineering fields. www.wiley.com/go/statistical_pattern_recognition
Book Synopsis A New Generation of Mixture-model Cluster Analysis with Information Complexity and the Genetic EM Algorithm by :
Download or read book A New Generation of Mixture-model Cluster Analysis with Information Complexity and the Genetic EM Algorithm written by and published by . This book was released on 2009 with total page 195 pages. Available in PDF, EPUB and Kindle. Book excerpt: In this dissertation, we extend several relatively new developments in statistical model selection and data mining in order to improve one of the workhorse statistical tools - mixture modeling (Pearson, 1894). The traditional mixture model assumes data comes from several populations of Gaussian distributions. Thus, what remains is to determine how many distributions, their population parameters, and the mixing proportions. However, real data often do not fit the restrictions of normality very well. It is likely that data from a single population exhibiting either asymmetrical or nonnormal tail behavior could be erroneously modeled as two populations, resulting in suboptimal decisions. To avoid these pitfalls, we develop the mixture model under a broader distributional assumption by fitting a group of multivariate elliptically-contoured distributions (Anderson and Fang, 1990; Fang et al., 1990). Special cases include the multivariate Gaussian and power exponential distributions, as well as the multivariate generalization of the Student's T. This gives us the flexibility to model nonnormal tail and peak behavior, though the symmetry restriction still exists. The literature has many examples of research generalizing the Gaussian mixture model to other distributions (Farrell and Mersereau, 2004; Hasselblad, 1966; John, 1970a), but our effort is more general. Further, we generalize the mixture model to be non-parametric, by developing two types of kernel mixture model. First, we generalize the mixture model to use the truly multivariate kernel density estimators (Wand and Jones, 1995). Additionally, we develop the power exponential product kernel mixture model, which allows the density to adjust to the shape of each dimension independently. Because kernel density estimators enforce no functional form, both of these methods can adapt to nonnormal asymmetric, kurtotic, and tail characteristics. Over the past two decades or so, evolutionary algorithms have grown in popularity, as they have provided encouraging results in a variety of optimization problems. Several authors have applied the genetic algorithm - a subset of evolutionary algorithms - to mixture modeling, including Bhuyan et al. (1991), Krishna and Murty (1999), and Wicker (2006). These procedures have the benefit that they bypass computational issues that plague the traditional methods. We extend these initialization and optimization methods by combining them with our updated mixture models. Additionally, we "borrow" results from robust estimation theory (Ledoit and Wolf, 2003; Shurygin, 1983; Thomaz, 2004) in order to data-adaptively regularize population covariance matrices. Numerical instability of the covariance matrix can be a significant problem for mixture modeling, since estimation is typically done on a relatively small subset of the observations. We likewise extend various information criteria (Akaike, 1973; Bozdogan, 1994b; Schwarz, 1978) to the elliptically-contoured and kernel mixture models. Information criteria guide model selection and estimation based on various approximations to the Kullback-Liebler divergence. Following Bozdogan (1994a), we use these tools to sequentially select the best mixture model, select the best subset of variables, and detect influential observations - all without making any subjective decisions. Over the course of this research, we developed a full-featured Matlab toolbox (M3) which implements all the new developments in mixture modeling presented in this dissertation. We show results on both simulated and real world datasets.
Book Synopsis Efficient Algorithms for Learning Mixture Models by : Qingqing Huang (Ph. D.)
Download or read book Efficient Algorithms for Learning Mixture Models written by Qingqing Huang (Ph. D.) and published by . This book was released on 2016 with total page 274 pages. Available in PDF, EPUB and Kindle. Book excerpt: We study the statistical learning problems for a class of probabilistic models called mixture models. Mixture models are usually used to model settings where the observed data consists of different sub-populations, yet we only have access to a limited number of samples of the pooled data. It includes many widely used models such as Gaussian mixtures models, Hidden Markov Models, and topic models. We focus on parametric learning: given unlabeled data generated according to a mixture model, infer about the parameters of the underlying model. The hierarchical structure of the probabilistic model leads to non-convexity of the likelihood function in the model parameters, thus imposing great challenges in finding statistically efficient and computationally efficient solutions. We start with a simple, yet general setup of mixture model in the first part. We study the problem of estimating a low rank M x M matrix which represents a discrete distribution over M2 outcomes, given access to sample drawn according to the distribution. We propose a learning algorithm that accurately recovers the underlying matrix using 9(M) number of samples, which immediately lead to improved learning algorithms for various mixture models including topic models and HMMs. We show that the linear sample complexity is actually optimal in the min-max sense. There are "hard" mixture models for which there exist worst case lower bounds of sample complexity that scale exponentially in the model dimensions. In the second part, we study Gaussian mixture models and HMMs. We propose new learning algorithms with polynomial runtime. We leverage techniques in probabilistic analysis to prove that worst case instances are actually rare, and our algorithm can efficiently handle all the non-worst case instances. In the third part, we study the problem of super-resolution. Despite the lower bound for any deterministic algorithm, we propose a new randomized algorithm which complexity scales only quadratically in all dimensions, and show that it can handle any instance with high probability over the randomization.
Book Synopsis Mixture of expert models. Statistical analysis method by : Jula Kabeto Bunkure
Download or read book Mixture of expert models. Statistical analysis method written by Jula Kabeto Bunkure and published by GRIN Verlag. This book was released on 2020-06-16 with total page 15 pages. Available in PDF, EPUB and Kindle. Book excerpt: Document from the year 2020 in the subject Mathematics - Statistics, grade: Book chapter, Bahir Dar University (Ethiopian Institute of Textile and fashion technology), course: Statistical analysis method, language: English, abstract: Mixtures of experts models consist of a set of experts, which model conditional probabilistic processes, and a gate which combines the probabilities of the experts. The probabilistic basis for the mixture of experts is that of a mixture model in which the experts form the input conditional mixture components while the gate outputs form the input conditional mixture weights. A straightforward generalisation of ME models is the hierarchical mixtures of experts (HME) class of models, in which each expert is made up of a mixture of experts in a recursive fashion. This principle states that complex problems can be better solved by decomposing them into smaller tasks. In mixtures of experts the assumption is that there are separate processes in the underlying process of generating the data. Modelling of these processes is performed by the experts while the decision of which process to use is modelled by the gate. Mixtures of experts have many connections with other algorithms such as tree-based methods, mixture models and switching regression. In this, I review the paper by Rasmussen and Ghahramani to see how closely the mixtures of experts model resembles these other algorithms, and what is novel about it. The aim of this review is to adopt the method used in the current article to local precipitation data.
Book Synopsis Model-Based Clustering and Classification for Data Science by : Charles Bouveyron
Download or read book Model-Based Clustering and Classification for Data Science written by Charles Bouveyron and published by Cambridge University Press. This book was released on 2019-07-25 with total page 447 pages. Available in PDF, EPUB and Kindle. Book excerpt: Cluster analysis finds groups in data automatically. Most methods have been heuristic and leave open such central questions as: how many clusters are there? Which method should I use? How should I handle outliers? Classification assigns new observations to groups given previously classified observations, and also has open questions about parameter tuning, robustness and uncertainty assessment. This book frames cluster analysis and classification in terms of statistical models, thus yielding principled estimation, testing and prediction methods, and sound answers to the central questions. It builds the basic ideas in an accessible but rigorous way, with extensive data examples and R code; describes modern approaches to high-dimensional data and networks; and explains such recent advances as Bayesian regularization, non-Gaussian model-based clustering, cluster merging, variable selection, semi-supervised and robust classification, clustering of functional data, text and images, and co-clustering. Written for advanced undergraduates in data science, as well as researchers and practitioners, it assumes basic knowledge of multivariate calculus, linear algebra, probability and statistics.
Book Synopsis Handbook of Mixed Membership Models and Their Applications by : Edoardo M. Airoldi
Download or read book Handbook of Mixed Membership Models and Their Applications written by Edoardo M. Airoldi and published by CRC Press. This book was released on 2014-11-06 with total page 622 pages. Available in PDF, EPUB and Kindle. Book excerpt: In response to scientific needs for more diverse and structured explanations of statistical data, researchers have discovered how to model individual data points as belonging to multiple groups. Handbook of Mixed Membership Models and Their Applications shows you how to use these flexible modeling tools to uncover hidden patterns in modern high-dimensional multivariate data. It explores the use of the models in various application settings, including survey data, population genetics, text analysis, image processing and annotation, and molecular biology. Through examples using real data sets, you’ll discover how to characterize complex multivariate data in: Studies involving genetic databases Patterns in the progression of diseases and disabilities Combinations of topics covered by text documents Political ideology or electorate voting patterns Heterogeneous relationships in networks, and much more The handbook spans more than 20 years of the editors’ and contributors’ statistical work in the field. Top researchers compare partial and mixed membership models, explain how to interpret mixed membership, delve into factor analysis, and describe nonparametric mixed membership models. They also present extensions of the mixed membership model for text analysis, sequence and rank data, and network data as well as semi-supervised mixed membership models.
Book Synopsis Python Data Science Handbook by : Jake VanderPlas
Download or read book Python Data Science Handbook written by Jake VanderPlas and published by "O'Reilly Media, Inc.". This book was released on 2016-11-21 with total page 743 pages. Available in PDF, EPUB and Kindle. Book excerpt: For many researchers, Python is a first-class tool mainly because of its libraries for storing, manipulating, and gaining insight from data. Several resources exist for individual pieces of this data science stack, but only with the Python Data Science Handbook do you get them all—IPython, NumPy, Pandas, Matplotlib, Scikit-Learn, and other related tools. Working scientists and data crunchers familiar with reading and writing Python code will find this comprehensive desk reference ideal for tackling day-to-day issues: manipulating, transforming, and cleaning data; visualizing different types of data; and using data to build statistical or machine learning models. Quite simply, this is the must-have reference for scientific computing in Python. With this handbook, you’ll learn how to use: IPython and Jupyter: provide computational environments for data scientists using Python NumPy: includes the ndarray for efficient storage and manipulation of dense data arrays in Python Pandas: features the DataFrame for efficient storage and manipulation of labeled/columnar data in Python Matplotlib: includes capabilities for a flexible range of data visualizations in Python Scikit-Learn: for efficient and clean Python implementations of the most important and established machine learning algorithms
Book Synopsis Mixture Model-Based Classification by : Paul D. McNicholas
Download or read book Mixture Model-Based Classification written by Paul D. McNicholas and published by CRC Press. This book was released on 2016-10-04 with total page 212 pages. Available in PDF, EPUB and Kindle. Book excerpt: "This is a great overview of the field of model-based clustering and classification by one of its leading developers. McNicholas provides a resource that I am certain will be used by researchers in statistics and related disciplines for quite some time. The discussion of mixtures with heavy tails and asymmetric distributions will place this text as the authoritative, modern reference in the mixture modeling literature." (Douglas Steinley, University of Missouri) Mixture Model-Based Classification is the first monograph devoted to mixture model-based approaches to clustering and classification. This is both a book for established researchers and newcomers to the field. A history of mixture models as a tool for classification is provided and Gaussian mixtures are considered extensively, including mixtures of factor analyzers and other approaches for high-dimensional data. Non-Gaussian mixtures are considered, from mixtures with components that parameterize skewness and/or concentration, right up to mixtures of multiple scaled distributions. Several other important topics are considered, including mixture approaches for clustering and classification of longitudinal data as well as discussion about how to define a cluster Paul D. McNicholas is the Canada Research Chair in Computational Statistics at McMaster University, where he is a Professor in the Department of Mathematics and Statistics. His research focuses on the use of mixture model-based approaches for classification, with particular attention to clustering applications, and he has published extensively within the field. He is an associate editor for several journals and has served as a guest editor for a number of special issues on mixture models.
Download or read book ECAI 2002 written by Frank Van Harmelen and published by IOS Press. This book was released on 2002 with total page 774 pages. Available in PDF, EPUB and Kindle. Book excerpt: This volume contains the 137 papers accepted for presentation at the 15th European Conference on Artificial Intelligence (ECAI '02), which is organized by the European Co-ordination Committee on Artificial Intelligence.
Book Synopsis Handbook of Mixture Analysis by : Sylvia Fruhwirth-Schnatter
Download or read book Handbook of Mixture Analysis written by Sylvia Fruhwirth-Schnatter and published by CRC Press. This book was released on 2019-01-04 with total page 388 pages. Available in PDF, EPUB and Kindle. Book excerpt: Mixture models have been around for over 150 years, and they are found in many branches of statistical modelling, as a versatile and multifaceted tool. They can be applied to a wide range of data: univariate or multivariate, continuous or categorical, cross-sectional, time series, networks, and much more. Mixture analysis is a very active research topic in statistics and machine learning, with new developments in methodology and applications taking place all the time. The Handbook of Mixture Analysis is a very timely publication, presenting a broad overview of the methods and applications of this important field of research. It covers a wide array of topics, including the EM algorithm, Bayesian mixture models, model-based clustering, high-dimensional data, hidden Markov models, and applications in finance, genomics, and astronomy. Features: Provides a comprehensive overview of the methods and applications of mixture modelling and analysis Divided into three parts: Foundations and Methods; Mixture Modelling and Extensions; and Selected Applications Contains many worked examples using real data, together with computational implementation, to illustrate the methods described Includes contributions from the leading researchers in the field The Handbook of Mixture Analysis is targeted at graduate students and young researchers new to the field. It will also be an important reference for anyone working in this field, whether they are developing new methodology, or applying the models to real scientific problems.
Book Synopsis Topics on Mixture Models and Discriminant Analysis by : Kai Deng
Download or read book Topics on Mixture Models and Discriminant Analysis written by Kai Deng and published by . This book was released on 2023 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: Mixture models for clustering and regressions and discriminant analysis are the cornerstones of multivariate statistics and supervised/unsupervised learning research. The structure of data has become increasingly complex in many modern applications including but not limited to computational biology, recommendation systems and text/image analysis. Therefore, it is of great interest to develop methodologies and algorithms for mixture models and discriminant analysis that target the challenges arising from such complex data. In this dissertation, I address three types of challenging supervised and unsupervised topics with novel methodologies and algorithms: (1) tensor data simultaneous clustering and multiway dimension reduction; (2) high-dimensional heterogeneous data in mixture linear regression; (3) multivariate and multi-label response classification in high dimensions. The three chapters are elaborated as follows. In the form of multi-dimensional arrays, tensor data have become increasingly prevalent in modern scientific studies and biomedical applications such as computational biology, brain imaging analysis, and process monitoring system. These data are intrinsically heterogeneous with complex dependencies and structure. Therefore, ad-hoc dimension reduction methods on tensor data may lack statistical efficiency and can obscure essential findings. Model-based clustering is a cornerstone of multivariate statistics and unsupervised learning; however, existing methods and algorithms are not designed for tensor-variate samples. In the first chapter, we propose a Tensor Envelope Mixture Model (TEMM) for simultaneous clustering and multiway dimension reduction of tensor data. TEMM incorporates tensor-structure-preserving dimension reduction into mixture modeling and drastically reduces the number of free parameters and estimative variability. An EM-type algorithm is developed to obtain likelihood-based estimators of the cluster means and covariances, which are jointly parameterized and constrained onto a series of lower-dimensional subspaces known as the tensor envelopes. We demonstrate the encouraging empirical performance of the proposed method in extensive simulation studies and a real data application in comparison with existing vector and tensor clustering methods. In the second chapter, we consider the problem of finite mixture of linear regressions (MLR) for high-dimensional heterogeneous data where the sample size is much smaller than the number of random variables, which is widely used in many modern applications such as biological science, genetics and engineering. In order to capture the common sparse structure in large heterogeneous data, traditional high-dimensional EM algorithm can be computational intractable thus fail to produce meaningful estimation results. We propose a fast group-penalized EM algorithm (FGEM) for high-dimensional MLR that estimates the regression coefficients from a group sparsity perspective and is computationally efficient and less sensitive to initialization. The statistical property of the proposed algorithm is established without requiring sample-splitting that allows the predictor dimension grows exponentially with the sample size. We demonstrate the encouraging performance of FGEM in numerical studies in comparison with traditional high-dimensional EM algorithms. The problem of classifying multiple categorical responses is pervasive in modern machine learning and statistics, with diverse applications in fields such as bioinformatics and image classification. The third chapter investigates linear discriminant analysis (LDA) with high-dimensional predictors and multiple multi-class responses. Specifically, we examine two different classification scenarios under the bivariate LDA model: joint classification of the two responses and conditional classification of one response while observing the other. To achieve optimal classification rules for both scenarios, we introduce two novel tensor formulations of the discriminant coefficients and corresponding penalties. For joint classification, we propose an overlapping group lasso penalty and a blockwise coordinate descent algorithm to efficiently compute joint tensor discriminant coefficients. For conditional classification, we utilize an alternating direction method of multipliers (ADMM) algorithm to compute tensor discriminant coefficients under new constraints. We extend our method and algorithms to general multivariate responses. Finally, we validate the effectiveness of our approach through simulation studies and real data examples.
