Japanese Researchers Develop AI Tool to Control Animal Brain Circuits in Real Time
A team from Nagoya University has created YORU, an AI system that identifies animal behaviors and manipulates corresponding brain circuits instantaneously, enhancing the study of social interactions across various species.
Researchers at Nagoya University have developed an artificial intelligence system named YORU (Your Optimal Recognition Utility) that can identify specific animal behaviors in real time and immediately interact with the corresponding brain circuits. This open-source software enables biologists to study social interactions with unprecedented speed and precision. By treating complex actions as distinct visual objects, YORU allows computers to recognize behaviors such as courtship or food sharing and respond within milliseconds.
Traditional methods for analyzing animal interactions often rely on pose estimation, tracking specific body points across video frames to calculate movement. However, these techniques can struggle when animals are in close proximity or overlapping, leading to difficulties in triggering experiments at precise moments. To address this, the research team, including Hayato M. Yamanouchi and Ryosuke F. Takeuchi under the guidance of senior author Azusa Kamikouchi, adopted a different approach. They developed a system capable of "closed-loop" feedback, where a computer observes an animal and instantly generates a stimulus in response. For example, the system can activate a light the moment a fly extends its wing.
YORU utilizes a deep learning algorithm known as object detection, analyzing the entire shape of an animal in a single video frame. This method enables the software to recognize specific actions as distinct "behavior objects," such as two ants sharing food or a male fly vibrating its wing. By viewing the behavior as a unified object rather than a collection of points, YORU effectively handles scenarios where animals are touching or overlapping.
The team tested YORU on various species, including fruit flies, ants, and zebrafish, achieving detection accuracy rates between 90% and 98% compared to human observation. The software also demonstrated effectiveness in analyzing brain activity in mice, accurately identifying behaviors like running, grooming, and whisker movements, and correlating these actions with neural activity in the mouse cortex.
In an advanced test involving optogenetics—a technique that uses light to control genetically modified neurons—the researchers modified male fruit flies so that the neurons responsible for their courtship song would be silenced by green light. YORU monitored the flies in real time and, upon detecting a male extending his wing to sing, triggered a green light within milliseconds, causing the male to cease his courtship song. This interruption led to a statistically significant decrease in mating success.
Hayato M. Yamanouchi, co-first author from Nagoya University's Graduate School of Science, highlighted the system's efficiency:
"Instead of tracking body points over time, YORU recognizes entire behaviors from their appearance in a single video frame. It spotted behaviors in flies, ants, and zebrafish with 90-98% accuracy and ran 30% faster than competing tools."
The researchers further demonstrated YORU's precision by using a projector to manipulate only one animal in a pair without affecting the other. They genetically modified female flies to have light-sensitive hearing neurons and, upon detecting the male's wing extension, projected a small circle of light onto the female's thorax, silencing her hearing neurons exactly when the male tried to sing. As a result, the female ignored the male's advances because she could not hear him.
Senior author Azusa Kamikouchi explained the significance of this precision:
"We can silence fly courtship neurons the instant YORU detects wing extension. In a separate experiment, we used targeted light that followed individual flies and blocked just one fly's hearing neurons while others moved freely nearby."
The system's speed was a primary focus for the researchers. Benchmarking YORU against SLEAP, a popular pose-estimation tool, revealed that YORU exhibited a mean latency of approximately 31 milliseconds, roughly 30% faster than the alternative method. Such speed is necessary for studying neural circuits, which operate on extremely fast timescales.
Designed to be user-friendly for biologists without extensive programming expertise, YORU includes a graphical user interface that allows researchers to label behaviors and train the AI without writing code. The team has made the software open-source, enabling laboratories worldwide to download and adapt it for their specific animal models.
While YORU offers speed and precision, it relies on the appearance of behavior in a single frame, making it less effective at identifying behaviors that depend on a sequence of events over time. Additionally, the current version does not automatically track the identity of individual animals over long periods, which may require supplementary tools for studies necessitating long-term individual histories.
Despite these limitations, YORU represents a significant advancement in studying the neural basis of communication. By enabling computers to recognize social behaviors as they occur, scientists can now explore how the brain navigates complex social interactions, opening new avenues for understanding the neural mechanisms underlying behavior.