For decades the neuroscience community has been baffled by the existence of dense connections in the brain that seem to be going “backward”. These connections, which span extensively across distant areas of the neocortex – the part of the brain responsible for higher cognitive functions – are clearly conveying important information. But until now, the organization of the connections and therefore their possible role, was largely unknown.
IMAGE: THE EXISTENCE OF 'BACKWARDS' NEURAL CONNECTIONS LINKING DISTANT AREAS OF THE NEOCORTEX -- THE PART OF THE BRAIN RESPONSIBLE FOR HIGHER COGNITIVE FUNCTIONS -- HAVE BAFFLED SCIENTISTS FOR DECADES
In a study published today (April 16, 2018) in the scientific journal Nature Neuroscience, scientists at the Champalimaud Centre for the Unknown, in Lisbon, reveal, for the first time, that these connections form an exquisitely organised map of the visual space and provide important insights into how they may be involved in visual perception.
“Our current understanding of the visual system suggests a hierarchical model”, explains Leopoldo Petreanu, the leading researcher of the study. “According to this model, lower structures receive an image from the eyes, which is then processed and relayed forward to higher structures of the neocortex for the extraction of key features, such as the contours, objects and so on.”
“This could have been a great model”, Petreanu adds, “if it weren’t for the elephant in the room – that there are as many, if not more, connections that go backward, from higher to lower areas. The function of these so-called feedback connections has been a mystery for neuroscientists for decades”
Previous studies that tried to elucidate the nature of these connections made things even more confusing. “Feedback connections are very messy”, says Petreanu. “Under the microscope, they look like an extensive mesh of wires intertwined like a spaghetti bowl. And to make matters even worse, intermingled wires encode a variety of different signals. It really wasn’t clear whether there was any order in this mess.”
Many theories have been proposed for the role of these feedback connections in cognition, including attention, expectation and awareness. However, it was impossible to tell which of the theories were true since the connectivity map was unknown.
To solve the mystery, Petreanu, together with Tiago Marques and Julia Nguyen, the first co-authors of the study, used a unique method that was developed by Petreanu a few years ago. With this method, the researchers measured the activity in the actual connection points made between higher and lower structures.
“This method has provided us with groundbreaking insight into how feedback connections are organized and how this organization might shape visual perception”, says Marques. “Hidden in the tangle of wires we found that there is a beautiful organization, where feedback connections target specific neurons in lower structures depending on the signals they carry.”
But what exactly is this organization and what might be its role in visual perception? Petreanu and Marques discovered several insights that shed light onto this long-standing mystery.
Feedback connections tell the big picture
The first insight happened when the researchers asked whether the connections follow any particular pattern. Their guess was that they do.
“In many separate structures of the visual system, beginning with the eye itself, neighboring neurons encode neighboring areas of the visual space. This way, the individual structures contain an almost ‘one to one’ map of the image”, Marques explains.
This map exists in the primary visual cortex (also called V1), which is the entry point of visual information to the neocortex. This was the researchers´ starting point. They asked: do feedback connections match the visual map encoded in V1?
“The answer we found was yes, and no…” Marques says. “The majority of feedback inputs formed the same spatial map as the areas they connected to in V1. In other words, the V1 and feedback maps were superimposed on each other. This observation had already been reported in other species, such as primates, so we weren’t surprised.”
“However, here, in the mouse, we also observed something new”, Marques points out. “The feedback connections also encoded information from further locations in the visual space. Since the technique we used is novel and has only been applied here, it is likely that this might be found in the future in other species as well.”
This finding suggests that feedback signals sent from higher cortical areas are used to provide lower structures with context. “According to the hierarchical structure of the visual system, lower structures would only have access to local, low-level information”, Marques explains. “What the feedback connections give them is the ‘whole picture’. This way, the activity of neurons in lower structures can be altered according to the current context.”
“This type of contextual information is very important for visual perception. For instance, around green shape seen at a distance would be readily identified as a tennis ball when seen in the context of a tennis court, or as an apple if seen in the context of a fruit bowl”, he adds.
Telling the brain where not to look
This first discovery motivated the researchers to look even further into what other types of information the feedback connections might be sending to V1. This time, they asked whether these connections might help V1 neurons find objects.
“The world is made up of objects”, Petreanu explains. “The phone in your hand, the cars on the road, these are all objects that are defined by continuous lines. Therefore, it’s not surprising that neurons in the visual system care a lot about these lines.”
How could feedback connections help accentuate the lines that makeup objects? There are two possibilities: they can either amplify the activity in V1 where the lines are, or they can dampen activity where they are not supposed to be.
“We found that the second option is the most likely to be true”, says Petreanu. “The feedback connections were abundant in V1 in areas outside the lines. We, therefore, hypothesise that this organization is probably silencing neurons in the areas that lie outside the line, and thereby enhancing the contrast between objects and their surroundings.”
Next, the researchers asked whether feedback connections might participate in motion detection. To their surprise, they found not only that they do, but that they use the same strategy to do it. “This time, the visual feature was different, but the feedback connections played the same role”, says Marques. “We observed that feedback connections that respond to moving objects were enriched in V1 in regions opposite to the direction of movement.”
Together, these results suggest something of a clairvoyant role for these feedback connections. How do they know which neurons should be active at any given moment in time?
“We believe that these results imply that this set of feedback connections learn through experience what to expect from the world and then use this knowledge to shape incoming visual information”, says Petreanu. “In the world, objects are defined by continuous lines, not scattered dots and moving objects tend to maintain their trajectory, not move around randomly. So feedback connections try to accentuate these particular features that they have learned to anticipate. Surprisingly, they do so by pointing to locations that are opposite to the expected ones.”
From biological vision to machine vision
The results of Petreanu and Marques provide an important piece of the puzzle of how the neocortex is organized and suggest how visual perception could be generated in the brain. According to Petreanu, these findings not only contribute to our understanding of biology but might also carry implications for the field of machine vision.
“The relationship between machine vision and neuroscience has always been a close one”, says Petreanu. “Our knowledge of how the circuitry of the brain, and in particular the neocortex, is organized, has helped inspire algorithms that have been increasingly more successful in enabling machines to ‘see’.”
According to Petreanu, while current machine vision algorithms are pretty good they cannot yet match the performance of humans. “Paralleling the neuroscientists’ understanding, modern machine vision algorithms usually don’t make use of feedback connections. Our findings might inspire new algorithms that will take advantage of these connections, which might make the future arrive a bit sooner,” he concludes.