Scientists use a neural network to decipher what a person thinks about

Miscellaneous / by admin / May 02, 2023

click fraud protection

Without implants and brain-computer interfaces.

American scientists reported about the successful testing of a semantic decoder - a system based on MRI and a neural network that allows you to understand what a person is thinking about. It can help people who have lost the ability to communicate with language, gestures and text.

It is the first system capable of reconstructing continuous speech without a brain implant. It works on the basis of ideas, semantics and meanings, not specific words. That is, it allows you to convey the thought, but not the exact wording.

To do this, the volunteers were instructed and placed in a functional magnetic resonance imaging (fMRI) machine. There, they listened to podcasts, made up stories, and watched videos without sound. The resulting data was fed to a GPT-based neural language model and was able to reconstruct a general idea of what people heard or thought.

It turned out that although the neural network does not cope well with personal pronouns, it is able to paraphrase what the patient hears. And even when the subjects made up their own stories or watched silent movies, the decoder could pick up the main thoughts and convey them.

instagram viewer

Comparison of what the participant of the experiment heard and what the semantic decoder decoded. Blue marks the exact hit, purple - the transfer of meaning, orange - an error in interpretation. Image: University of Texas at Austin

Since fMRI is not instantaneous, it is not suitable for capturing every word. But with its help, you can get a mixture of information that the patient thought for a few seconds. As a result, you can track how the idea developed in the person’s head, although the exact wording is lost in the process.

According to the researchers, these results bring us closer to a future where machines can read minds and transcribe them. This can cause concern that someone will receive information without the consent of the other person - for example, while they are sleeping.

But the authors of the study note that without special instruction, the subjects will not be able to read their thoughts. In addition, three volunteers were hit by a car while listening to the podcast while doing something else: counting in their heads, naming animals, or telling another story about themselves.

The team hopes to be able to speed up the process in the future to scan thoughts in real time.