Science fiction has always had a fraught relationship with telepathy.
On the one hand, it’s such a fanciful and magical ability that it’s oftentimes not thought of as sci-fi at all, or (especially in Golden Age sci-fi) only as some semi-mystical power that only mutants or aliens have (eg Spock in Star Trek, Xenomorphs in Alien, the dragons of Pern, etc.).
On the other hand, it’s hard to imagine that super-evolved humans and civilizations of the far future would not have this ability; so science fiction authors include mind-reading as the special province of galaxy-spanning civilizations, omnipotent energy beings, and time travelers. Even in that case, as with Asimov’s Foundation series, telepathy is something only humans with specialized training can do, and in secret. In almost no case — in Star Trek, Foundation*1, Pern, or anywhere else — do we see machines used to allow humans to read minds.
But we’re already using machines to read minds. And not at all as the sci-fi authors imagined.
Since I started thinking about this topic over ten years ago, this technology has progressed more quickly than almost anyone imagined; but I confess it has not moved as quickly as I expected. It was in 2013 that my experience building speech recognition systems led me to realize that a mind-reading system could be built using a similar architecture; and I thought the technology could be developed in a decade or so. Progress is headed right down the path I expected, if not as quickly. But I continue to believe we will reach it sooner than anyone expects.
There’s a lot to unpack here, so I’m going to tackle the topic in several posts, covering topics like the evolution of speech recognition technology, how telepathic computing works (and how it will rapidly improve), the linguistics of brain-to-text conversion, and the influence of Silicon Valley culture and technological ethics (or lack thereof). The more we understand these issues, the better prepared we will be when the technology arrives. Because it will arrive, and it will be miraculous, and it will be horrifying.
How We “Read Minds” Today
My idea for mind-reading technology was based directly on how speech recognition software worked in 2013 (which was essentially the same as how it had worked since the 1990s). The architecture consists of a few separate components arranged in a pipeline:
The acoustic speech signal is given to an acoustic model, whose job it is to break the sound stream up into separate sounds (roughly corresponding to phonemes). Those sounds are then sent to the language model, whose job is to group the sounds together into words. Then those can be output as text.
Both of these components have to be trained separately.
To train the acoustic model, you need hours and hours of recordings of people talking, with each sound painstakingly labeled. The modeling system uses statistics to see which acoustic signals are most likely to be which linguistic sounds. (To imagine how hard this is, say “don’t you” slowly and then quickly. You’ll hear yourself say “don’t you” and “doncha”. We don’t always pronounce each sound carefully!)
To train the language model, you need a dictionary of all the words — and I mean ALL the words — along with the sequence of sounds in each word.
You also need a bunch of text. The text is fed into a simple statistical modeling algorithm that learns what sequences of words are most likely to occur (eg, that “the old” is more likely to be followed by “guy” than by “guide”). Those words are then matched up with your big list of words and how they’re pronounced. The language model and the dictionary, working together with the acoustic model, can figure out what you said and print the result as text.
It occurred to me that the language model didn’t care what kind of data the acoustic model was working with, as long as it was given some options to choose from. It didn’t have to be sound. One could, for example, replace the acoustic model with an EEG (electroencephalogram) model, or an MRI model — any representation of what was going on in the brain. The EEG / MRI / brain model would need to be trained, just like the acoustic model, by having people speak while their brain waves were recorded, and then lining up the recordings with the brain waves. Then you’d take the language model — perhaps exactly the same language model you used with speech recognition — and run the system and print out thoughts as text.
Now, in the past five or six years, speech recognition has gotten tremendously better while at the same time having a simpler architecture. These days we use neural nets (or “connectionist systems” or “deep learning” or “AI” — the buzzwords have changed over the past thirty years but it’s the same thing, just with more powerful computers and sleeker architectures) which are powerful enough to do the work of the acoustic model and language model in one step:
And it’s with these neural nets that we’ve started to make serious progress in the actual technology of mind reading.
Early Brain-Computer Interfaces
Almost everyone has heard of Elon Musk’s company Neuralink, which has made splashy headlines with their implant technology. Neuralink has drawn a lot of attention, but often the most substantial advances come from less publicized research. Universities and research institutions worldwide have been quietly but steadily pushing the actual state of the art.
The tech is basic but the potential is staggering. At Stanford in 2021, researchers in the Krishna Shenoy lab developed a system enabling participants to type at speeds of up to 90 characters per minute just by imagining writing letters by hand. The team then made headlines in 2023 when they demonstrated a system allowing a paralyzed person to communicate at 62 words per minute through a brain implant – a dramatic improvement over previous systems. In that system, the model detected the brain activity as a person with paralysis tried to move their mouth. It then combined that with a language model in just the way I anticipated in 2013.
The closest anyone has come to true mind reading is probably Tang, Huth et al. in 2022, who demonstrated the ability to decode words from an fMRI scan (ie, without a neural implant). The subjects had to lie down in the MRI machine and think about words, and the models interpreted their brain images correctly. Notably, the machine couldn’t read their minds if the people didn’t want it to.
The brain’s encoding of information is both more distributed and more specific than we thought. Networks of neurons across different areas of the brain work together to process and produce language. Our ability to distinguish meaningful patterns from background brain activity has been markedly improved by the use of neural nets, and we’re only starting to explore the possibility of using LLMs for this kind of research.
The Gap Between Now and “True” Telepathy
The gap between current technology and true telepathic communication is still huge, but it’s closing faster than anyone expected. The key breakthrough wasn’t better electrodes or more sensitive receivers, nor the mystic alien powers found in science fiction. Instead, it was the development of large language models that could make sense of the noisy, intricate patterns of brain activity.
Still, there are some big hurdles ahead. Current systems can only work with specific, trained thoughts in controlled conditions. They’re very much like early speech recognition systems (from even before my time) that could understand only a few dozen commands, spoken carefully and clearly by trained speakers. We’re still a long, long way from Mr Spock’s mind meld.
The biggest challenge isn’t actually detecting the signals; it’s interpreting them. Thoughts aren’t neat and linear beads on a string, like speech. (Speech isn’t really like that either, but go with me for now.) They’re tangled clouds of concepts, memories, and emotions, all firing at the same time. When you think about a dog, you might simultaneously activate memories of your childhood pet, abstract knowledge about what dogs are, emotional associations, and even unrelated thoughts that happen to be connected in your mental web. Current AI models are getting better at untangling these patterns, but most of the work is still filtering out the irrelevant stuff going on in your brain.
Of course, even once everything works perfectly, we come to the question of privacy and consent. Tang and Huth’s system couldn’t read subjects’ thoughts if they didn’t want it to. Do our brains have natural firewalls built in? Or is this just a limitation of their experimental architecture, one that will be easily bypassed by future tech?
Conclusion
Whatever the answers, the technology is coming. It won’t just be used to assist people with disabilities, or as a toy for party games. The path of the evolving technology, and who controls it, will have critical effects on society and our very conception of humanity and human rights. We’ll get into all that. In the next post, we’ll start with the likely next steps — conscious and unconscious thought, building models of minds, and sharing more than just language.
- In the later Foundation books, humans do actually develop mind-reading technology. ↩︎
