Study sheds new light on how semantic information is organized in the brain

by Ingrid Fadelli, Medical Xpress

Fig. 1: Voxels with correlated visual and linguistic semantic representations. Credit: DOI: 10.1038/s41593-021-00921-6

The human brain stores and organizes meaningful information in different regions and networks. While past neuroscience studies have examined some of these networks in great depth, the relationship and interactions between them is not yet entirely clear.

Researchers at the University of California, Berkeley have recently carried out a study examining the relationship between visual and linguistic semantic representations in the brain. Their findings, published in Nature Neuroscience, suggest that visual and linguistic semantic information is organized as a unified map in the brain, with the two representations meeting along the boundary of the visual cortex.

“One important subsystem within the human brain is the amodal semantic network, which represents the contents of current working memory and attention,” Jack L. Gallant, one of the researchers who carried out the study, told Medical Xpress. “The semantic network serves as an interface between information streaming in from perceptual systems and knowledge stored in long-term memory. The goal of our study was to better understand the way that perceptual information enters the amodal semantic network.”

Gallant and his colleagues carried out two different experiments involving the same group of participants. In the first experiments, the participants were asked to watch silent movies, while in the second they listened to verbally narrated stories.

To examine how the participants’ brains represented the information they were presented with, the researchers used functional magnetic resonance imaging (fMRI), a well-known imaging approach that can be used to map patterns in brain activity. They then analyzed the data they collected using high-dimensional and high-resolution computational modeling methods.

“The data analysis and modeling tools we used have been developed in our laboratory over the past 20 years, and they allow us to recover information about how information is represented in the human brain in unprecedented detail,” Gallant explained. “In this specific study, we found that perceptual information enters the amodal semantic network through a series of parallel, semantically-selective channels. Although this was a functional imaging study, these channels of information flow suggest that this system is implemented as a series of semantically-selective anatomical connections.”

As predicted, the fMRI scans collected by Gallant and his colleagues showed that while the participants watched the silent movies, a specific network of areas in the visual cortex were activated. While they listened to the stories, on the other hand, another network of areas anterior to the visual cortex were activated.

Interestingly, the researchers observed that the activation patterns of these two distinct neural networks corresponded along the boundary of the visual cortex. In other words, the visual categories that were represented on one side of the boundary (inside the visual cortex), were represented linguistically on the other side of the boundary.

As noted above, the semantic network represents the contents of current working memory and attention.

“Many cognitive functions—such as learning, responding to unexpected events, monitoring performance, planning for the future, evaluating evidence, making decisions, etc.—rely on the semantic network,” Gallant said. “Many neurological conditions that affect cognitive function or learning may involve the semantic network. A better understanding of the anatomical and functional organization of this network would thus provide critical information for improving cognitive function in healthy people and for addressing problems in cognitive function.”

Overall, the findings gathered by this team of researchers suggest that visual and linguistic semantic networks in the brain are connected, forming a unified map. In the future, this interesting observation could pave the way for new research aimed at better understanding the connections between different semantic brain networks.

“This is just the first of many studies that will be required to fully understand the relationship between the semantic system, perceptual systems, and long-term memory,” Gallant added.