Mind-Reading AI? Scientists Use GPT-Like AI to Accurately Translate Thoughts into Text

0:00Today on the AI Breakdown, we're trying something new. First up is a quick headline brief that'll take about five minutes. And then after that, we get to our main topic, which was published as a different video about a new process by which researchers at the University of Texas, Austin, use the GPT-like AI to actually translate people's thoughts into text. Welcome back to the AI Breakdown Brief, all the AI news you need in five minutes or less. We start with companies changing their hiring plans on the basis of AI. IBM made news yesterday when they announced that they were planning to replace 7 ,800 jobs with AI.

0:36So this is back office workers. This is non-customer facing roles, HR. And to be clear, they're not firing 8 ,000 people and replacing them with machines. They're saying that they believe that these roles will be automated. So they're not going to hire those 7 ,800 people that they thought they previously would have. Now, Dropbox made news with something similar last week when they announced that they were laying off 500 people as part of their pivot to AI. And this isn't exactly the same in that this isn't roles that AI was automating away, but instead roles that AI was making obsolete or just not relevant for where Dropbox was going.

1:11So a different approach or a different reason that AI was disruptive, but still netting to the same thing of the rise of AI changing what skills were in demand, what skills could be done by people versus computers. Now, it's been earnings season for the last couple weeks, and every big tech company has been talking about their AI strategy. Meta actually had a very successful earnings call. They announced, among other things, that their approach to AI in Instagram had increased the amount of time spent on Instagram by about 24%, thanks to better recommendations in the Reels feature. And this led to Meta's stock being up 15 % after that call.

1:48Well, they also had reported from the command line newsletter late last week, early this weekend, that the CTO, Andrew Bosworth, had told employees about whether they would be collaborating with Microsoft or OpenAI at all. And it really seems that the only potential area is around an AI coding assistant. But otherwise, they're just working on their own. But otherwise, they're just working on their own technology. Still, I think that the biggest news from this AI earning season comes from Amazon. Last month, Amazon announced Bedrock, which is a developer platform that allows developers to take advantage of pre-trained models from companies like Anthropic and Stability AI.

2:25But they also announced on this earnings call that they were working on an improved LLM to power Alexa. They say that this will be integral to their goal of having it be the world's best personal assistant and that that goal is really enabled by the rise of more powerful LLMs and generative AI. Now, coming at this idea of the big tech AI arms race from a slightly different angle, yesterday the New York Times reported that Jeffrey Hinton had resigned from Google. Jeffrey Hinton is sometimes known as the godfather of AI. He's been working on neural networks for nigh on 50 years now. He won a Turing Award for his work on neural networks.

3:00Google acquired his company in 2012, and it has been the basis of a lot of their work in AI subsequently. He left in part to warn of what he sees as increasing dangers of AI, which include things like we were just talking about, the disruption wrought from AI for people's jobs, their careers, their livelihoods. But he also sees other challenges as well. He's worried about nefarious actors and what they can do with this technology. He's worried about the X risk and all the existential questions. All in all, he's just worried that we are now hurtling towards a future that we don't really have control over.

3:32And one of the reasons that he suspects it's happening now is, in fact, this AI arms race. He said that he believed that for a long time, Google was a good steward of this technology, trying not to do things that might cause harm. But ever since Microsoft and Bing have been creeping up and threatening their core search business, those ideals have been kind of thrown out the window because of the business imperative to advance on AI. So interesting food for thought there. Now, you're also seeing alongside this more and more governments thinking about and putting AI as a priority. Open AI is now available again in Italy after satisfying a number of conditions from the government there after their soft ban, including age verification.

4:11And you're also noticing the White House talk about AI more and more. Just yesterday, Bloomberg reported that the White House was looking into the use of AI around surveillance of remote workers, worried about the risk to safety and mental health. And while this story itself is not particularly important, I think it's indicative of a larger trend of this technology just being right at the center or growingly at the center of attention. That's it for The Brief. See you later for the AI Breakdown. Today on the AI Breakdown, we're talking about a new technology that uses GPT to actually read people's minds, and more than just specific words.

4:51I'd be willing to bet that a lot of people didn't have mind-reading AI on their bingo cards for 2023, but then again, it's been a year of surprises. A few months ago, I saw this viral tweet from Siki Chen who says, okay, so AI can literally read our minds now. A team from Osaka was able to reconstruct visual images from MRI scan data using stable diffusion. He then shows an image of the change where the first row is the image presented to the test subject and the second row is the reconstructed image. And you see a teddy bear that turns into a teddy bear and a path through trees that looks like a path through trees and a clock tower that looks like a clock tower, a train that looks like a train.

5:34It was wildly impressive. But today we've got coverage of an even potentially crazier thing. Siki again says AI can read our minds even better now. In a new peer-reviewed paper published in Nature, researchers reliably recovered actual thoughts by combining fMRI data with GPT synthesis. Okay, so you've probably seen something about this. There have been lots of breathless headlines like this one from Vice, scientists use GPT-AI to passively read people's thoughts in breakthrough. And what people are talking about is this new study from a group of scientists out of the University of Texas at Austin.

6:15Their piece was published in Nature Neuroscience, a journal, and was called Semantic Reconstruction of Continuous Language from Non-Invasive Brain Recordings. The research was led by grad student Jerry Tang, as well as associate professor Alexander Huth, and had two other participants, Amanda LaBelle and Shaili Jain. Their abstract reads, a brain-computer interface that decodes continuous language from non-invasive recordings would have many scientific and practical applications. Currently, however, non-invasive language decoders can only identify stimuli from among a small set of words and phrases.

6:48Here we introduce a non-invasive decoder that reconstructs continuous language from cortical semantic representations recorded using functional magnetic resonance imaging, or fMRI. Given novel brain recordings, this decoder generates intelligible word sequences that recover the meaning of perceived speech, imagined speech, and even silent videos. Our findings demonstrate the viability of non-invasive language brain-computer interfaces. Okay, so that is extremely dense, but effectively what they're saying is that if and only if they have the buy-in of a participant, they have developed a process by which they can read thoughts.

7:24So let's talk a little bit more about what that process actually includes. The study was focused on three participants, each of whom came to the lab and spent hours and hours listening to The Moth and other podcasts. And as they were listening, the fMRI scanner was basically taking stock of the blood levels in their brains. It was watching where blood was flowing. Now from that, the researchers were able to use a large language model like a GPT to match that brain activity to the specific words or phrases that participants had heard, effectively creating a neural map of each of these participants individually.

8:02You can kind of think about this as effectively training a large language model on someone's brain. And this was exactly what this research was meant to study. Dr. Huth had noticed previously that when these large language models like GPT-4 are created and trained, they create maps that show how words relate to one another. The New York Times writes a few years ago, Dr. Huth noticed that particular pieces of these maps, so-called context embeddings which capture the semantic features or meanings of phrases, could be used to predict how the brain lights up in response to language. The New York Times also quotes Shinji Nishimoto, who was the scientist behind that other study that we just talked about at the beginning, who said, quote, brain activity is a kind of encrypted signal, and language models provide ways to decipher it.

8:48So effectively, after the lab had trained their large language models on these individual participants through hours and hours of them listening to podcasts, they were then presented with a new story or asked to imagine telling a story while being hooked up to this non-invasive decoder that would generate corresponding text just from their brain activity. And what came out wasn't a word-for-word transcript, but it is effectively a paraphrasing. It captures the gist of what was being said or thought. E. News writes,

9:29So in the paper in Nature Neuroscience, the actual stimulus would be something like, I got up from the air mattress and pressed my face against the glass of the bedroom window, expecting to see eyes staring back at me, but instead finding only darkness. The decoded stimulus read, I just continued to walk up to the window and open the glass. I stood on my toes and peered out, but I didn't see anything, and looked up again. I saw nothing. It's a paraphrasing that does notably include a couple exact words, window and glass. Another. The actual stimulus? I didn't know whether to scream, cry, or run away.

10:02Instead, I said, leave me alone. I don't need your help. Adam disappeared, and I cleaned up alone crying. The decoded stimulus reads, Started to scream and cry, and then she said, I told you to leave me alone. You can't hurt me anymore. I'm sorry. And then he stormed off. I thought he had left and I started to cry. This is obviously a pretty mind-blowing technology that has a ton of applications. It could be used to help people who had strokes or who had other diseases that didn't allow them control over their motor functions to be able to communicate, to say nothing of the potential research applications, which could be enormous as well.

10:36However, one of the concerns with brain-reading technology is that it could be used nefariously. And so as a part of their research, they actually included how cooperative participants had to be. So again, UT News says, results for individuals on whom the decoder had not been trained were unintelligible, and if participants on whom the decoder had been trained later put up resistance, for example, by thinking other thoughts, results were similarly unusable. In other words, there are two requirements for this approach to AI reading minds to work. One is that you have to train the model on someone's specific brain.

11:12It's not necessarily just a general thing. At least we don't know or think it's a general thing yet. And two, they have to be an active participant. They can't be trying to think other thoughts or trying to distract from the decoding actually working. Now, for people who study the brain, the implications are even beyond just the specific applications, but what it tells us about how the brain works. Again from the New York Times. The results suggest that the AI decoder was capturing not just words, but also meaning. Dr. Nishimoto again said, language perception is an externally driven process, while imagination is an active internal process.

11:50The authors showed that the brain uses common representations across these processes. Greta Takute, who's a neuroscientist at MIT, put it really simply. She says, can we decode meaning from the brain? In some ways, they show that yes, we can. Now, obviously, research on the brain and how the brain works is quite tied up with AI and especially AGI. One of the arguments that AI safety advocates have is that we've already crossed the threshold where we no longer understand how these large language models are producing the responses that they are. It's effectively a black box that produces amazing results, but without us understanding how.

12:27Our understanding of the brain is all too often a similar black box, and that makes some people very nervous about how far we should proceed in the training of AI that we don't understand. I saw a few people tweeting about this today, and one common sentiment is here represented by Peter Yang, who writes, On my drive to work today, I started to feel sad that AI can not only communicate 10 ,000 times faster than us, but can now even read our minds. It's a thin line between this is so cool and, wait, are we becoming obsolete? But I also saw this thread from neuroscientist Dean Burnett, which is a little bit more optimistic.

13:00Dean writes, A few have asked my insight on this new study where language AIs successfully quote-unquote read minds based on brain scan data, as it seems many are worried about it. My take? Yes, it is very impressive and could be very useful. No, it's not something to worry about yet, and here's why. Firstly, using AI to decipher human brain activity into coherent language is just cool. That's a fact as far as I see it. For someone to think words and have software read the corresponding brain activity and accurately more or less translate it into text, brilliant. But I can see why AI can accurately read your thoughts is a worrying prospect for many.

13:34Because if it came commonplace, it would be alarming and open to abuse in horrifying ways. Luckily, there are many major caveats to this study which mean this isn't happening. For one, these dedicated AIs can understand brain activity from fMRI scans, which involves someone laying down and staying very still for prolonged periods in a massive multi-ton supercooled steaming tube. Very hard to do this without someone's firm consent. As well as this, the AI in the study could read the thoughts of a small handful of subjects who had provided sufficient activity data to train the AI, which involved 16 hours of scanning activity under specific experimental conditions.

14:08So, while the headline is AI can read your thoughts and translate them into text, It should actually be, AI can translate the thoughts of people willing to spend many hours in extremely elaborate and expensive experimental setups that allow them to do so. Essentially, for all these mind-reading AIs to be applicable to the general population, we'd all have to voluntarily undergo a day's worth of very specific brain scans, and agree to have the resultant data uploaded to some open access archive. This seems an unlikely occurrence. The researchers make this very specific in the coverage. They applied the AI to scans of other people, ones that hadn't been trained to decipher under the same conditions.

14:41Attempts to translate their thoughts produced utter gibberish. Each individual's brain and its activity is the result of a lifetime of very specific experiences and unique development. Essentially, every human brain has its own bespoke operating system. The AI has been coded to work with three operating systems, just under 8 billion to go. Sure, these particular AIs can be made more efficient, more capable, and read more readily provided brain data. But the hurdle of every human brain thinks in its own unique way, even at the neurophysiological level, isn't one I see being cleared anytime soon.

15:11It strikes me that the further down we get on the AGI conversation, and specifically the AI safety conversation, the more the role of brain-computer interfaces is going to come up as part of the discussion. I agree with Dean that these are important caveats that should keep people from getting too nervous too fast. But even in spite of that, it's still a pretty remarkable reflection of just how far technology has come. Anyways, guys, that's it for today's AI Breakdown. I appreciate you listening. Until next time, peace.