New learning algorithm should significantly expand the possible applications of AI

The e-prop learning method developed at Graz University of Technology forms the basis for drastically more energy-efficient hardware implementations of Artificial Intelligence.

Two researchers with laptop in hand

TU Graz computer scientists Robert Legenstein and Wolfgang Maass (from left) are working on energy-efficient AI systems and are inspired by the functioning of the human brain. © Lunghammer - TU Graz

The high energy consumption of artificial neural networks' learning activities is one of the biggest hurdles for the broad use of Artificial Intelligence (AI), especially in mobile applications. One approach to solving this problem can be gleaned from knowledge about the human brain. Although it has the computing power of a supercomputer, it only needs 20 watts, which is only a millionth of the energy of a supercomputer. One of the reasons for this is the efficient transfer of information between neurons in the brain. Neurons send short electrical impulses (spikes) to other neurons – but, to save energy, only as often as absolutely necessary.

Event-based information processing

A working group led by two computer scientists Wolfgang Maass and Robert Legenstein of TU Graz has adopted this principle in the development of the new machine learning algorithm e-prop (short for e-propagation). Researchers at the Institute of Theoretical Computer Science, which is also part of the European lighthouse project Human Brain Project, use spikes in their model for communication between neurons in an artificial neural network. The spikes only become active when they are needed for information processing in the network. Learning is a particular challenge for such less active networks, since it takes longer observations to determine which neuron connections improve network performance.

Previous methods achieved too little learning success or required enormous storage space. e-prop now solves this problem by means of a decentralized method copied from the brain, in which each neuron documents when its connections were used in a so-called e-trace (eligibility trace).

The method is roughly as powerful as the best and most elaborate other known learning methods. Details have now been published in the scientific journal Nature Communications and on TU Graz News.

Original publication

A solution to the learning dilemma for recurrent networks of spiking neurons in Nature Communications
Guillaume Bellec, Franz Scherr, Anand Subramoney, Elias Hajek, Darjan Salaj, Robert Legenstein, Wolfgang Maass;
DOI: 10.1038/s41467-020-17236-y

Contact

TU Graz | Institute of Theoretical Computer Science
Wolfgang MAASS
Em.Univ.-Prof. Dipl.-Ing. Dr.rer.nat.
Phone: +43 316 873 5822
maass@igi.tugraz.at

Robert LEGENSTEIN
Univ.-Prof. Dipl.-Ing. Dr.techn.
Phone: +43 316 873 5824
robert.legenstein@igi.tugraz.at