The Times Australia
The Times World News

.

How close are we to reading minds? A new study decodes language and meaning from brain scans

  • Written by Christina Maher, Computational Neuroscientist and Biomedical Engineer, University of Sydney
How close are we to reading minds? A new study decodes language and meaning from brain scans

The technology to decode our thoughts is drawing ever closer. Neuroscientists at the University of Texas have for the first time decoded data from non-invasive brain scans and used them to reconstruct language and meaning from stories that people hear, see or even imagine.

In a new study published in Nature Neuroscience[1], Alexander Huth and colleagues successfully recovered the gist of language and sometimes exact phrases from functional magnetic resonance imaging[2] (fMRI) brain recordings of three participants.

Technology that can create language from brain signals could be enormously useful for people who cannot speak due to conditions such as motor neurone disease[3]. At the same time, it raises concerns for the future privacy of our thoughts.

Language decoded

Language decoding models[4], also called “speech decoders”, aim to use recordings of a person’s brain activity to discover the words they hear, imagine or say.

Until now, speech decoders have only been used with data from devices surgically implanted in the brain, which limits their usefulness. Other decoders which used non-invasive brain activity recordings have been able to decode single words or short phrases, but not continuous language.

Read more: We've been connecting brains to computers longer than you’d expect. These 3 companies are leading the way[5]

The new research used the blood oxygen level dependent signal[6] from fMRI scans, which shows changes in blood flow and oxygenation levels in different parts of the brain. By focusing on patterns of activity in brain regions and networks that process language, the researchers found their decoder could be trained to reconstruct continuous language (including some specific words and the general meaning of sentences).

Specifically, the decoder took the brain responses of three participants as they listened to stories, and generated sequences of words that were likely to have produced those brain responses. These word sequences did well at capturing the general gist of the stories, and in some cases included exact words and phrases.

The researchers also had the participants watch silent movies and imagine stories while being scanned. In both cases, the decoder often managed to predict the gist of the stories.

For example, one user thought “I don’t have my driver’s licence yet”, and the decoder predicted “she has not even started to learn to drive yet”.

Further, when participants actively listened to one story while ignoring another story played simultaneously, the decoder could identify the meaning of the story being actively listened to.

How does it work?

The researchers started out by having each participant lie inside an fMRI scanner and listen to 16 hours of narrated stories while their brain responses were recorded.

These brain responses were then used to train an encoder[7] – a computational model that tries to predict how the brain will respond to words a user hears. After training, the encoder could quite accurately predict how each participant’s brain signals would respond to hearing a given string of words.

However, going in the opposite direction – from recorded brain responses to words – is trickier.

The encoder model is designed to link brain responses with “semantic features” or the broad meanings of words and sentences. To do this, the system uses the original GPT language model[8], which is the precursor of today’s GPT-4 model. The decoder then generates sequences of words that might have produced the observed brain responses.

A table showing stills from an animated film next to descriptions of the action decoded from fMRI scans.
The decoder could also describe the action when participants watched silent movies. Tang et al. / Nature Neuroscience[9]

The accuracy of each “guess” is then checked by using it to predict previously recorded brain activity, with the prediction then compared to the actual recorded activity.

During this resource-intensive process, multiple guesses are generated at a time, and ranked in order of accuracy. Poor guesses are discarded and good ones kept. The process continues by guessing the next word in the sequence, and so on until the most accurate sequence is determined.

Words and meanings

The study found data from multiple, specific brain regions – including the speech network, the parietal-temporal-occipital association region, and prefrontal cortex – were needed for the most accurate predictions.

One key difference between this work and earlier efforts is the data being decoded. Most decoding systems link brain data to motor features or activity recorded from brain regions involved in the last step of speech output, the movement of the mouth and tongue. This decoder works instead at the level of ideas and meanings.

One limitation of using fMRI data is its low “temporal resolution”. The blood oxygen level dependent signal rises and falls over approximately a 10-second period, during which time a person might have heard 20 or more words. As a result, this technique cannot detect individual words, but only the potential meanings of sequences of words.

No need for privacy panic (yet)

The idea of technology that can “read minds” raises concerns over mental privacy. The researchers conducted additional experiments to address some of these concerns.

These experiments showed we don’t need to worry just yet about having our thoughts decoded while we walk down the street, or indeed without our extensive cooperation.

A decoder trained on one person’s thoughts performed poorly when predicting the semantic detail from another participant’s data. What’s more, participants could disrupt the decoding by diverting their attention to a different task such as naming animals or telling a different story.

Read more: Our neurodata can reveal our most private selves. As brain implants become common, how will it be protected?[10]

Movement in the scanner can also disrupt the decoder as fMRI is highly sensitive to motion, so participant cooperation is essential. Considering these requirements, and the need for high-powered computational resources, it is highly unlikely that someone’s thoughts could be decoded against their will at this stage.

Finally, the decoder does not currently work on data other than fMRI, which is an expensive and often impractical procedure. The group plans to test their approach on other non-invasive brain data in the future.

Read more https://theconversation.com/how-close-are-we-to-reading-minds-a-new-study-decodes-language-and-meaning-from-brain-scans-204691

Times Magazine

What AI Adoption Means for the Future of Workplace Risk Management

Image by freepik As industrial operations become more complex and fast-paced, the risks faced by workers and employers alike continue to grow. Traditional safety models—reliant on manual oversight, reactive investigations, and standardised checklist...

From Beach Bops to Alpine Anthems: Your Sonos Survival Guide for a Long Weekend Escape

Alright, fellow adventurers and relaxation enthusiasts! So, you've packed your bags, charged your devices, and mentally prepared for that glorious King's Birthday long weekend. But hold on, are you really ready? Because a true long weekend warrior kn...

Effective Commercial Pest Control Solutions for a Safer Workplace

Keeping a workplace clean, safe, and free from pests is essential for maintaining productivity, protecting employee health, and upholding a company's reputation. Pests pose health risks, can cause structural damage, and can lead to serious legal an...

The Science Behind Reverse Osmosis and Why It Matters

What is reverse osmosis? Reverse osmosis (RO) is a water purification process that removes contaminants by forcing water through a semi-permeable membrane. This membrane allows only water molecules to pass through while blocking impurities such as...

Foodbank Queensland celebrates local hero for National Volunteer Week

Stephen Carey is a bit bananas.   He splits his time between his insurance broker business, caring for his young family, and volunteering for Foodbank Queensland one day a week. He’s even run the Bridge to Brisbane in a banana suit to raise mon...

Senior of the Year Nominations Open

The Allan Labor Government is encouraging all Victorians to recognise the valuable contributions of older members of our community by nominating them for the 2025 Victorian Senior of the Year Awards.  Minister for Ageing Ingrid Stitt today annou...

The Times Features

Advanced Skin Health with Dermascan: A New Standard in Skin Cancer Screening

Skin cancer is one of the most prevalent forms of cancer in Australia, making regular skin checks an essential part of preventative healthcare. Dermascan, an innovative Australia...

Great Barrier Reef operator Passions of Paradise

A series of sustainability firsts and a commitment to global best practice standards for more than 20 years has earned Cairns-based Great Barrier Reef operator Passions of Para...

5 Questions to Ask Before Getting Blepharoplasty in Gold Coast

(Source) Blepharoplasty, or eyelid surgery, removes extra skin or fat from around the eyes. It’s often done for cosmetic reasons, but it can also help with vision if sagging eye...

How Smart Home Integration is Enhancing SIL Accommodation in 2025

(Source) The concept of "home" is intensely personal, a sanctuary where we experience security, comfort, and a sense of being in control. For people living with disability, acco...

How to Know If You’re Actually on Track for a Comfortable Retirement

Image by Drazen Zigic on Freepik It’s the kind of question that sits in the back of your mind, especially as you tick past your 30s or 40s: Am I actually saving enough for retire...

Onsite Caterer vs a Full Service Venue: 9 important things to explore

Choosing between an external catering company and an all-inclusive venue is a major decision that affects cost, flexibility, food quality, and the overall event experience. Venue...