Brain Neural Networks With Similar Behavior Can Differ in Structure and Learning
Translated & summarized from NEWSru Israel by baba
Researchers at the Technion have found that brain neural networks with identical behavior can have different structures, which are revealed through learning. Their study on recurrent neural networks identified hidden variables that influence future learning and retain memory. Co-author Professor Omri Barak noted that current behavior doesn't predict future learning, but the learning process itself can expose hidden parameters. The findings, currently from simulations, may change how the brain and AI are studied.
The story in 6 lines · by baba
- Technion researchers found brain neural networks with identical behavior can have different structures.
- These structural differences are revealed when the networks are made to learn.
- Hidden variables in networks influence future learning and retain memory of past training.
- Professor Omri Barak stated current behavior is insufficient to predict future learning.
- The study suggests observing how neural systems learn is crucial for understanding them.
- Findings from simulations may impact the study of both biological and artificial neural networks.
Researchers at Israel's Technion Institute of Technology have discovered that neural networks in the brain exhibiting identical behavior may possess different internal structures. These structural differences, however, can be revealed through the learning process. While learning in the brain involves altering connections between neurons, its effects are observed in the overall system's behavior. The Technion scientists focused on understanding how numerous local adjustments contribute to new functions.
Their study, published on arXiv, examined low-rank recurrent neural networks, models frequently used to describe information processing in the brain. The researchers identified two classes of variables: those that fully determine a network's activity and output, and those that do not influence current operations but guide future learning. These hidden properties are undetectable by merely observing a network's behavior.
In one experiment, two networks with distinct internal designs responded identically to the same inputs. However, during subsequent training, their learning trajectories diverged: one network plateaued for an extended period, while the other showed consistent improvement. This demonstrates that learning can expose underlying differences not apparent in the network's current state.
Furthermore, these hidden variables can retain memory of past experiences. Networks trained on different initial tasks before a common training task achieved similar performance levels, yet their hidden parameters allowed researchers to reconstruct their unique learning histories. "In the networks we studied, understanding how the system behaves today is not enough to describe how it will learn in the future," stated co-author Professor Omri Barak. "But how the network changes during learning can reveal its hidden parameters, which are not visible in its current behavior."
While these findings are currently based on models and computer simulations, the authors propose testing them on living systems. Confirmation could fundamentally alter the approach to studying the brain, emphasizing the importance of observing how neural systems learn, not just what they do. This principle also extends to artificial intelligence, suggesting that two AI models performing identically may not be structurally the same and could react differently to further training, a crucial consideration for adapting large neural networks to new tasks.