Author : Zelda Elaine Mariet
Publisher :
ISBN 13 :
Total Pages : 66 pages
Book Rating : 4.:/5 (953 download)
Book Synopsis Learning and Enforcing Diversity with Determinantal Point Processes by : Zelda Elaine Mariet
Download or read book Learning and Enforcing Diversity with Determinantal Point Processes written by Zelda Elaine Mariet and published by . This book was released on 2016 with total page 66 pages. Available in PDF, EPUB and Kindle. Book excerpt: As machine-learning techniques continue to require more data and become increasingly memory-heavy, being able to choose a subset of relevant, high-quality and diverse elements among large amounts of redundant or noisy data and parameters has become an important concern. Here, we approach this problem using Determinantal Point Processes (DPPs), probabilistic models that provide an intuitive and powerful way of balancing quality and diversity in sets of items. We introduce a novel, fixed-point algorithm for estimating the maximum likelihood parameters of a DPP, provide proof of convergence and discuss generalizations of this technique. We then apply DPPs to the difficult problem of detecting and eliminating redundancy in fully-connected layers of neural networks. By placing a DPP over a layer, we are able to sample a subset of neurons that perform non-overlapping computations and merge all other neurons of the layer into the previous diverse subset. This allows us to significantly reduce the size of the neural network while simultaneously maintaining a good performance.