Meta's AI Reveals Mind's Eye: Transforming Brain Data into Instant Visuals

0:00Meta AI, which of course is a big frontrunner in AI innovation, has unveiled a new system that leverages what's called MEG. I will butcher the pronunciation of this, but I'm just going to try. It's a magnetencephalography, but MEG is what it's called. So I'm just going to call it MEG from now on. But they're using this to interpret visual representations within the human brain. So this is a really cutting edge technology and it's providing what I believe is a sneak peek into this kind of real-time perception and processing of images in our brain. So this I think could potentially be revolutionizing our understanding of human intelligence and visual cognition.

0:42So beyond, I think the immediate scientific implications, this is really interesting research coming out of meta. I think this is pretty groundbreaking. And I think it holds some really transformative potential for non invasive brain computer interfaces. These kind of advancements could be invaluable in clinical scenarios, particularly for individuals who, you know, due to like brain injuries and other things like that, have lost their ability to communicate verbally. They could use these types of tools, think their thoughts, and, you know, have that translated by even an AI voice that could be trained on their voice before they had the, you know, before they lost that ability.

1:17So I think there are a lot of like really cool use cases here. I'm not going to go all dystopian and Facebook just reads your mind to serve you ads. But I mean, come on, that'll probably happen too. In any case, so Meta's new, newly essentially introduced system extends from its recent architecture which was originally crafted for decoding speech perception through MEG signals and so this is kind of a three-pronged system and it consists of a couple things. The first is an image encoder so this component essentially crafts representations of an image without referencing brain data which is interesting.

1:54The second part of this is a brain encoder so syncing the MEG signals with the image encoding this segment kind of bridges the gap between visual stimulus and then the brain activity that comes from that so when you look at a picture your brain shoots off certain brain waves it's certain stimulus and if you have something scanning your brain while you see that picture it just says okay like it takes a snapshot of what your brain looks like and it says when they see this picture this is what happens and then they're able to look at your brain signal without knowing what the picture is and make the prediction about what the picture is you know after they train on enough of this data.

2:29The third part of this is then an image decoder. So drawing from the brain representations, this final piece recreates a plausible image, right? So that's, it's actually predicting what you're looking at. So data used for training this architecture was sourced from a globally accessible data set of MEG readings from healthy participants. And this is, you know, I think this is related by the International Academic Consortium things. So what is happening here, right? Upon comparing the efficacy of various pre-trained image models, the research revealed that advanced vision AI systems, notably Dyno V2, had the most synchronous alignment with brain signals.

3:10So I think it's noteworthy that Dyno V2 employs self-supervised AI to learn visual depictions autonomously. And this comes from some insights from Meta AI, which I think really kind underscore that self-guided learning is the pathway for AI systems that nurture brain mimicking representations. Another point that I think is really interesting is the speed of the MEG decoder. So while functional pragmatic resonance imaging, which is fMRI, of course, has, you know, higher resolution images, the MEG decoder stands out for its ability to kind of produce images almost instantaneously. And it's essentially channeling a consistent flow of visuals straight from the brain, right?

3:53So it's not like, okay, scan the brain, okay, generate image, we sit there, wait for it to load. No, no, no. It's doing this instantaneously. So you have something like you have an fMRI scanner on your brain, and we're literally just looking at what you're thinking like in real time, like it's a movie, like this thing's playing live. And the imagery generated by the MEG decoder, while it's definitely not perfect, it like distinctly captures broad level characteristics of images that the human is thinking of. You know, categories like animals, humans, trains. And so I think the precision here kind of wavers more on intricate details.

4:29Like if I'm visualizing Elmo, it may not create like a literal version of Elmo exactly how I'm thinking about it, but it's going to create like some version of like a red stuffed animal doing X, Y, and Z, right? So I've seen some like really interesting samples here. I saw one where like essentially they showed it a picture of cheese and then it, it showed like a plate with like these weird kind of cheese things on it. Very similar. I saw another one where it showed it a cat under a blanket. It then like it actually, the AI generated a cat. Now the cat under the blanket was gray and the cat it generated was kind of like an orange cat and it didn't have a blanket on its So like it understands the concept that you're thinking.

5:11So maybe not like specifically and literally, but pretty dang good. And of course, what I always say with this kind of technology is like for all the doubters that are like, oh, yeah, it's not perfect. Like, you know, it just did like a cat, not the exact one. Like these things will just get better. The training will just get better. And so I think don't worry about the drawbacks or don't really worry about like the limitations of this technology. start thinking about the implications of when this technology is perfect because it will get there. So I think back in 2022, Yan Lekun, who's kind of in charge of AI over at Meta, he's, you know, pretty, pretty big.

5:46He's on X all the time tweeting. He showcased a new, a novel AI architecture, which is aimed at navigating beyond the constraints of kind of contemporary systems. So I think kind of furthering this momentum, a collaborative effort, which is involving people from Meta AI and also New York University, created something called IJEPA. I think I covered this earlier this year, but essentially this was grounded on the vision transformer. So iJepa was self-trained to forecast unseen segments of images. And this model's capability to grasp abstracted objects representations instead of sticking to pixel-specific data, I think signifies a stride towards AI models that mirror human cognition and local deducting, which I think is essentially the kind of their hypothesis of where this is going to go.

6:37So all this to say, very fascinating. we're able to now stick an fMRI scanner on your brain and get a concept of what you're thinking in a real-time video-like feed which in my mind is absolutely phenomenal but also kind of terrifying and this is definitely something that I will continue to follow I stand by my prediction meta is going to incorporate this into the into the meta vr headsets and read your mind to target you with ads so anyways it'll be interesting to see how this plays out what privacy people have to say about this, if there's an outcry, whatever. But this is my prediction at the beginning of the year.

7:10I'm holding to it, and we'll see if this actually happens.