Humans require ample sleep to function optimally. New research suggests neural networks may also benefit from extended rest.
There are myriad benefits of hunkering down for a good night’s sleep. Adequate rest has a wide range of positive effects ranging from enhanced problem solving and reasoning skills to reduced stress and irritability. Humans are not alone in this regard by any means. From praying mantises to giant armadillos, sleep is common across the animal kingdom. New research from Los Alamos National Laboratory suggests that artificial computational brains may also benefit from catching a few Z’s.
In recent years, artificial neural networks have been leveraged for a host of helpful tasks such as pharmaceutical research and even predicting wildfires. In other instances, these computing systems have even been tapped to create original musical compositions, although it’s seemingly latent artistic talents are, we’ll say, a work in progress at the moment.
SEE: Building the bionic brain (free PDF) (TechRepublic)
These artificial systems were inspired by the neurological connectivity of the human brain. Imagine a sprawling, layered network of interconnected nodes communicating with one another; a wave of information passing through the structure via a series of electronic firings. Each of these nodes, or “neurons,” within the network is fed its own supply of data and signals to transfer data to the next layer of nodes. Over time, the neural network adjusts the unique interactions between these neurons to improve its problem-solving capabilities. This roadmap of neural combinations is slowly fine-tuned until the system develops an optimal strategy for a given task.
For this study, the Los Alamos researchers focused on spiking neural networks that function differently than standard artificial neural networks. These computing systems are stylized more closely to the neurological circuitry of the human brain, with neurons generating a signal after receiving a number of input signals. Scientists are still learning how to train spiking neural networks, as these systems require entirely different methods than typical artificial neural networks.
SEE: Managing AI and ML in the enterprise 2020: Tech leaders increase project development and implementation (TechRepublic Premium)
The researchers found the spiking neural network became increasingly unstable after extended periods of unsupervised dictionary learning. After that fact, the team used spiking neural network computer simulations to better understand exactly what led to this instability. The researchers discovered that the neurons within the system began to fire regardless of the input signals they received after extended training. In an attempt to stabilize the networks, the team implemented various types of noise, with Gaussian noise having the best results. The research team postulates that this is because Gaussian noise may mimic the inputs biological neurons receive throughout slow-wave sleep.
“Why is slow-wave sleep so indispensable?” said senior author of the study Garrett Kenyon. “Our results make the surprising prediction that slow-wave sleep may be essential for any spiking neural network, or indeed any organism with a nervous system, to be able to learn from its environment.”
Although further research is necessary, artificial “sleep” may be imperative to maintaining stability in spiking neural networks. Next, the researchers plan to use this algorithm on Intel’s Loihi neuromorphic chip.
Also see
Source: TechRepublic
