Publication Type : Journal Article
Publisher : ACM Comput. Surv., Association for Computing Machinery
Source : ACM Comput. Surv., Association for Computing Machinery, Volume 54, Number 4, New York, NY, USA (2021)
Url : https://doi.org/10.1145/3448974
Keywords : artificial intelligence (AI), compression, edge intelligence (EI), low-rank, Recurrent neural networks (RNNs), resource constrained modeling, sequence modeling, Sparsity, Training
Campus : Bengaluru
School : Department of Computer Science and Engineering, School of Engineering
Department : Computer Science
Year : 2021
Abstract : Recurrent Neural Networks are ubiquitous and pervasive in many artificial intelligence applications such as speech recognition, predictive healthcare, creative art, and so on. Although they provide accurate superior solutions, they pose a massive challenge “training havoc.” Current expansion of IoT demands intelligent models to be deployed at the edge. This is precisely to handle increasing model sizes and complex network architectures. Design efforts to meet these for greater performance have had inverse effects on portability on edge devices with real-time constraints of memory, latency, and energy. This article provides a detailed insight into various compression techniques widely disseminated in the deep learning regime. They have become key in mapping powerful RNNs onto resource-constrained devices. While compression of RNNs is the main focus of the survey, it also highlights challenges encountered while training. The training procedure directly influences model performance and compression alongside. Recent advancements to overcome the training challenges with their strengths and drawbacks are discussed. In short, the survey covers the three-step process, namely, architecture selection, efficient training process, and suitable compression technique applicable to a resource-constrained environment. It is thus one of the comprehensive survey guides a developer can adapt for a time-series problem context and an RNN solution for the edge.
Cite this Research Publication : Varsha S. Lalapura, Amudha J., and Hariramn Selvamurug Satheesh, “Recurrent Neural Networks for Edge Intelligence: A Survey”, ACM Comput. Surv., vol. 54, 2021.