The development of artificial intelligence technology has not only greatly helped scientists analyze and understand biological behavior, but also provided a revolutionary perspective for neuroscience research.

Quantifying animal social behavior is an important step in revealing brain function and mental disorders during the interaction phase.

But how can one study the social behavior of a group of animals when they are together to decode the function of the animals' brains and associated neural circuits?

The development of artificial intelligence technology has not only greatly helped scientists analyze and understand biological behavior, but also provided a revolutionary perspective for neuroscience research.

Recently, the international academic journal Nature Machine Intelligence published online the small-sample learning computing framework model SBeA (Social Behavior Atlas) developed by a team of Shenzhen scientists that can intelligently and accurately detect animal social behavior.

Breaking through the “cage” of understanding complex mental illness

Simply put, SBeA can continuously track, identify, label, and more importantly, quantify and analyze the free social behaviors of multiple animal targets. It is a computational framework for multi-animal social behavior analysis based on artificial intelligence.

SbeA currently realizes parallel, dynamic and hierarchical decomposition of animal social behavior, and performs adaptive unsupervised clustering on the social behavior characteristics of mice, rats, birds, dogs, non-human primates and other animals, and obtains It contains more than 100 sophisticated social behavior modules including chasing, mutual grooming, attack, etc. It does not need to define social behavior categories in advance, which is conducive to the discovery of new and undefined free social behavior phenotypes.

Artificial intelligence has the potential to help humans break through the "cage" of understanding complex mental illnesses - serial research from the laboratory, natural environment to clinical treatment

On January 23, the corresponding author of the relevant paper and researcher Wei Pengfei of the Shenzhen Institute of Advanced Technology of the Chinese Academy of Sciences/Shenzhen-Hong Kong Brain Science Innovation Institute told The Paper that this work is expected to become a step in the field of computational neurobehavioral research. Important puzzle pieces. Future goals will focus more on understanding neural coding and developing new neuromodulation technologies for clinical application. The development of computational neuroethology in recent years has revolutionized the classic behavioral paradigm and enabled behavioral observations from laboratory environments to natural environments. In the future, neuroscience research empowered by artificial intelligence will further deepen the understanding of physiological parameters of big data from animal models to clinical treatment. Artificial intelligence is expected to help humans break through the "cage" of understanding complex mental illnesses.

Wang Liping, director of the Institute of Brain Research of Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences/Shenzhen-Hong Kong Brain Science Innovation Institute, said that the aforementioned newly published research work used a data-driven approach to reveal the structural laws behind digital animal social behavior. It is hoped that relying on brain analysis The major scientific and technological infrastructure of brain simulation and the Bright Brain Science and Technology Industry Innovation Center platform will be fully shared across the country as soon as possible.

Han Yaning, a doctoral candidate at the Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, and the first author of the aforementioned paper, told The Paper that by using SBeA, several abnormal behaviors related to mental disease modeling have been discovered in mice and other model animals.

He said that the newly released SBeA model breaks through the current bottleneck of lack of relevant label data and only needs to label a few hundred pictures. After seeing these hundreds of pictures, the SBeA model will "self-study" and "brain supplement". SBeA refers to the relevant cognitive processes of the human brain, makes full use of the effective information in a small number of pictures, uses information such as background and animal targets extracted from a small number of pictures, and recombines this information to generate new scenes according to the movement characteristics of the animals. Datasets Construct large data sets to improve the performance of various tasks, thereby training more powerful artificial intelligence models, and realizing functions such as multi-animal 3D social posture estimation, identity recognition, and social behavior classification.

Han Yaning said that SBeA can be used for both basic research and disease diagnosis. "From the laboratory to the clinic, this is our original intention of designing SBeA. SBeA can not only accelerate the discovery of new abnormal behaviors and improve the accuracy of behavioral detection, but also has the hope of correlating the fine behavioral patterns of different species, using behavior as a biomarker , for diagnosing disease.”

room for improvement

Han Yaning introduced that science develops step by step. Neurobiologists have gradually realized from the experience accumulated in decades of brain research that accurate understanding of behavior is a key link in accurately understanding brain functions. From 2015 to 2018, Sandeep Robert Datta, a professor of neurobiology at Harvard University, used a depth camera and the machine learning tool Moseq to discover the "grammar" of behavioral coding in the striatum of the brain, revealing the mechanism by which animals produce natural behaviors. From 2018 to 2021, artificial intelligence-based animal posture estimation technologies such as DeepLabCut, multi-animal DeepLabCut, LEAP and SLEAP have been developed one after another, so that dozens of animals’ bodies can be realized without adding any physical markers to the animal’s body. Accurate tracking of points. But ensuring their tracking accuracy requires tens of thousands of manually labeled data.

SBeA quantifies sophisticated social behavior in birds and dogs.

The newly published SBeA model breaks through this bottleneck. But SBeA still has room for improvement.

Han Yaning said that at the technical level, SBeA currently uses a multi-stage artificial intelligence neural network structure, which makes it difficult to detect social behaviors in real time. In future research, we hope to design an end-to-end neural network structure that inputs videos and directly outputs label categories of social behaviors to achieve real-time monitoring of social behaviors.

In addition, at the experimental level, SBeA is currently only tried to be applied to the open field social behavior of mice, birds and domestic dogs. In the future, it will gradually be applied to the social behavior detection of model animals such as fruit flies, zebrafish and non-human primates, and Transition from applications in laboratory settings to applications in clinical healthcare settings.

Paper link: https://www.nature.com/articles/s42256-023-00776-5