A groundbreaking study reveals that animals’ deliberate exploration activities allow them to learn about their surroundings more effectively, shedding light on potential improvements for AI learning agents.
A team of researchers from the Sainsbury Wellcome Centre and the Gatsby Computational Neuroscience Unit at UCL discovered that the inherent exploratory behaviors animals exhibit are far from random.
These intentional actions enable mice to efficiently develop a mental map of their environment.
The research, recently released in Neuron, details the methods employed by neuroscientists to examine their conjecture that particular exploratory behaviors exhibited by animals, like swiftly approaching objects, play a crucial role in enhancing their ability to understand and maneuver within their surroundings.
“There are a lot of theories in psychology about how performing certain actions facilitates learning,” adds corresponding author Professor Tiago Branco.
In this new study, they “tested whether simply observing obstacles in an environment was enough to learn about them, or if purposeful, sensory-guided actions help animals build a cognitive map of the world.”
In earlier research, SWC scientists noted a link between an animal’s ability to navigate around obstacles and the frequency of their approaches toward those objects. In the current study, lead author and SWC PhD candidate Philip Shamash conducted experiments to examine the consequences of inhibiting animals from engaging in exploratory behavior.
Utilizing optogenetic techniques, Shamash introduced a light-sensitive protein, channelrhodopsin, into a section of the motor cortex, effectively stopping the animals from initiating exploratory movements towards obstacles.
Remarkably, the researchers discovered that despite the mice spending considerable time observing and sniffing the obstacles, their learning was hindered when prevented from approaching the objects. This finding highlights the significance of innate exploratory actions in helping animals create a mental map of their surroundings.
In order to investigate the potential learning algorithms utilized by the brain, the research group collaborated with Sebastian Lee, a doctoral candidate in Andrew Saxe’s laboratory at SWC. They examined various reinforcement learning models designed for artificial agents to determine which most accurately mirrored mouse behavior.
Reinforcement learning models can be categorized into two primary groups: model-free and model-based. The researchers observed that mice exhibited model-free behavior under certain circumstances, while in other situations, they appeared to possess a mental representation of their surroundings.
Consequently, they developed an agent capable of navigating between model-free and model-based approaches. Although this may not precisely reflect the inner workings of the mouse brain, it provided valuable insights into the necessary components of a learning algorithm to account for the observed behavior.
“One of the problems with artificial intelligence is that agents need a lot of experience in order to learn something,” explains Professor Branco, adding, “They have to explore the environment thousands of times, whereas a real animal can learn an environment in less than ten minutes.
They “think this is in part because, unlike artificial agents, animals’ exploration is not random and instead focuses on salient objects. This kind of directed exploration makes the learning more efficient and so they need less experience to learn.”
Moving forward, the scientists plan to investigate the connection between the performance of exploratory behaviors and the depiction of intermediate objectives. The research group is currently conducting brain recordings to identify the regions associated with representing intermediate objectives and to understand how these exploratory behaviors contribute to the development of such representations.
Image Credit: Getty