Learning Algorithms and Signal Processing for Brain-Inspired Computing

The success of artificial neural networks (ANNs) in carrying out various specialized cognitive tasks has brought renewed efforts to apply machine learning (ML) tools for economic, commercial, and societal aims, while also raising expectations regarding the advent of an artificial “general intelligence” [1]–[2][3]. Recent highly publicized examples of ML breakthroughs include the ANN-based algorithm AlphaGo, which has proven capable of beating human champions at the complex strategic game of Go. The emergence of a new generation of ANN-based ML tools has built upon the unprecedented availability of computing power in data centers and cloud computing platforms. For example, the AlphaGo Zero version required training more than 64 GPU workers and 19 CPU parameter servers for weeks, with an estimated hardware cost of US$25 million [4]. OpenAI’s video game–playing program needed training for an equivalent of 45,000 years of game play, costing millions of dollars in rent access for cloud computing services [2].

Recent studies have more generally quantified the requirements of ANN-based models in terms of energy, time, and memory consumption in both the training and inference (run-time) phases. An example is a recent work by researchers from the University of Massachusetts Amherst [5], which concluded that training a single ANN-based ML model can emit as much carbon as five cars during their lifetimes.

The massive resource requirements of ANN-based ML raise important questions regarding the accessibility of the technology to the general public and to smaller businesses. Furthermore, they pose an important impediment to deploying powerful ML algorithms on low-power mobile or embedded devices.