Generative Model Based Training Of Deep Neural Networks For Event Detection In Microscopy Data

Author : Sangeeta Srivastava (Researcher in deep learning)
Publisher :
ISBN 13 :
Total Pages : 0 pages
Book Rating : 4.:/5 (135 download)

DOWNLOAD NOW!

Download or read book Towards Green AI written by Sangeeta Srivastava (Researcher in deep learning) and published by . This book was released on 2022 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: Deep neural networks have not only evolved to produce state-of-the-art artificial intelligence (AI) models, but also generalize well, especially when the network has a large capacity and access to a comparable massive training dataset. These models are computationally and environmentally inefficient. Such research works are known as Red AI. However, it is challenging to develop algorithms that generate cost-effective models for Green AI without sacrificing predictive strength or generality. This dissertation shows how we can effectively integrate domain knowledge, either implicitly or explicitly, into an end-to-end learning pipeline to reduce the training and testing overhead of a neural network. In particular, we exploit the target data and task information to make two types of modifications in the learning pipeline, each of which helps to lower the neural network's run-time costs in unique ways. We re-define the learning task as one or more "simpler" subtask(s) so that the subtask(s) requires fewer parameters and training data. Explicit inclusion of a regularization term in the objective function allows us to limit the exploration space to those that are both relevant and plausible for a given application. We demonstrate the benefits of the above methodologies with the help of three applications with disparate challenges: 1) acoustic event detection, 2) radar classification, and 3) scientific problems based on eigenvalue solvers. Acoustic and radar applications require on-device intelligence and rely on Cortex-M7/M3 microcontroller-based edge devices with limited hardware resources and a low power budget. On the other hand, eigenvalue problems yield memory- and compute-intensive models primarily due to the size of the output layer that grows exponentially with the number of particles in the system. For the acoustic event detector, we exploit unlabeled data from the target domain to constrain knowledge transfer from a large "teacher" model to a smaller "student" model such that the knowledge is relevant to the target problem. This results in a student with orders of magnitude lower static and dynamic memory requirements. For the N + 1 radar classification problem, where the +1-class corresponds to environmental noise, we use an application-specific ontology to decompose the end-to-end learning into two hierarchical subproblems. A more complex model trained to classify the N source classes is invoked occasionally, i.e., only when there is no environmental noise. This conditional invocation makes the resulting model 3.5x more run-time efficient than feature-engineered shallow solutions and gives the best trade-off between accuracy and efficiency. As for the eigenvalue solvers, we decompose the complex regression task of prediction of high-dimensional eigenvector into multiple simpler and parallel subtasks such that the inputs in each subtask have similar physical properties. We learn a parameter-efficient "expert" network for each subtask. Our proposed physics-guided architecture is 150x smaller than the network trained to learn the complex task while being competitive in generalization. Our results also show that domain knowledge helps reduce the amount of supervised data needed for model training.