Teaser: What We Learned From the Man Who Put the T in GPT

0:00Folks, we have an incredibly special episode coming down the pipeline for you. as we did last time, we're going to drop a teaser trailer of our debrief of this incredible conversation with the true OG of AI. You can expect the full conversation in the very near future. Hi, I'm Elia Polosukhin. I'm one of the co-authors of The Attention Is All You Need, the paper that introduced T in GPT. And I'm a founder of Near Protocol and Near AI. it's all about how to ensure you own your ai your ai should be working on your side ensuring that your well-being and success are accounted for okay jeremy we just had i think a super interesting conversation with ilia who is one of og og is one of the authors of attention is everything which is yeah i guess what he called it the is it the the t in gpt yeah he is the person or among the team of people who thought of the tea in GPT.

1:04I mean, it is pretty fascinating. You know, you can't help thinking that it must have been fascinating to have sitting there and written that paper. And he'd written a lot of papers and then suddenly it becomes basically the thing that the whole world is centered around. I will say it's humbling to talk to somebody, Henrik, who I think he's the first guest. We've talked to what, over 50 people, 50 experts, world leaders. He's the first person to say, I was expecting ChatGPT in 2017. He was actually, he was too early. Most of us are reacting. He's one of the few people in the world for whom ChatGPT was quote unquote late upon its arrival.

1:42And so for the people that did not get to listen to the interview, but just wanted like the quick takeaway. Jeremy, what were some of the things that you kind of felt was kind of like sprung to me? First of all, I think this is an episode worth listening to because there's such an incredible kind of walk down memory lane. I mean, he gives us such a great history lesson about how Google was organized and how the team, the kinds of problems they were trying to solve when the attention paper was realized. I think that was super cool. And just even in terms of understanding what is attention, what is a transformer, I think for a kind of a basic primer.

2:17Nobody in the world better than Aaliyah probably to give that to us. So that was super fun to me. I think also, of course, learning what he's doing now at Mir and the critical importance, the reality that increasingly information and algorithms and AI change how we see reality and recognizing that leveraging blockchain technology, much in the same way that There's traceability around financial products. There can now be traceability around the provenance of information. I think he really convinced me that this is an important thing to consider and to prioritize into the future. And I would just say that that really wasn't on my radar part of this conversation.

3:03What about you? I mean, like a little bit the same way that he got on to my radar in the Nier project because I was trying to look at crypto in general and trying to figure out really what is the next thing that crypto can be used for that is kind of breaking out. You know, crypto has been around for a long time. We talked about it as a coin, you know, but they haven't really seen that many application where a crypto protocol was used for something, you know, that was that mainstream people kind of like understood. And so what really struck me with this was that with money, you have an alternative.

3:43Like people obviously have not had the experience that he had where suddenly the bank disappeared. So most people go like, I'll put my money. What a story, by the way. It's an incredible story. Just as an aside, I mean, he is uniquely qualified because of his family's experience in the Ukraine to care about this problem. So just as an aside, wow. But most people have a way of storing their value right now. They go to the bank, whatever. I think what most people do not have is a way to make sure that there is integrity in the information they consume. and over the last few years we've obviously talked much more about it because we started to realize that getting skewed information can convince you on one thing or the other and so this is actually the only real alternative i've heard of of saying hey i am going to give all my information to these intelligent machines these ai bots i'm going to consume the world through these ai bots how do i know that i can trust open ai or deep seek or claude or whatever it is with my data and how can i make sure that the stuff that i give to it is secure and you really can't if you don't have a mechanism to do it and so while crypto is super nerdy and this is obviously on a kind of you know foundational level and you might never have to think about it it actually can become quite important for how our world will kind of like evolve has huge implications for the future you know the fact that we're shifting you know money is power information is power information is the ultimate power and to have people like ilia fighting for information security and information you know individual sovereignty over information and assurance and verification that your source of information aren't being polluted or, or, um, you know, what did he call it?

5:38Secret agent. What was the phrase hidden agent? I can't remember. There was a phrase that will keep, if you listen to this episode, there is a phrase, there's a three minute segment of this interview that will keep you up at night. Just don't let your children listen to it. Don't let your grandma listen to it. But if you listen to it, just be prepared. You're going to get your mollux three sleepless nights just out of this one episode alone. Hey, one thing I want to talk about too, Henrik, which is kind of beyond which is a point he made in a particular use case, but I think can extrapolate to any user of any AI system, whether you're building the crypto future or not, is the following.

6:15He said, and I quote from my notebook here, the deeper understanding you have, the better you can use the tool. And he was, of course, referring to engineers who can have the context and architecture of a system, you know, far surpasses a model's kind of context window. But I think that's actually a principle that we have heard is probably one of the most recurring principles on this show from all of our conversations. The deeper understanding you have, the better possibilities you can experience in collaboration with AI. And what would you call this? Because I think a lot of people that I talk to in organizations, a lot of them say, you know, do you use AI?

6:56And they go, yeah, I use AI. And, but you kind of like when you're like a super user of it, you think, yeah, but you're not really using AI. Well, so, so there's two things there, Henrik. One is how do you collaborate? Um, which I agree. I hate the word use, as you may know, I love the word work with, right? So anybody who says they use AI, I say, I know you're a problem user. I know you're a tool. If you treat AI like a tool, you're a tool. Okay. So work with it. Don't use it. But what your question was, what do we call this? And you kind of went down to use the thing that I would say, I was actually thinking about this weekend.

7:30I'm working on a new book and I was thinking about a lot of these ideas and the phrase that came to my mind just this weekend. So it's like hot off the press, but is what I would call the human experience edge. I think there's something to mining your own expertise to say, what is the deep understanding that I can uniquely bring to my collaboration with AI? And granted, different from my use of AI. So we do need to talk about collaboration, hygiene, et cetera. But there's an area of one's life that you could think, oh, no, no, this is uniquely, It is uniquely you. However, if you will bring that into a fulsome collaboration with AI, you are going to get differential performance.

8:16And you know what it reminds me of, Henrik? It actually reminds me of our conversation with Ginny Nicholson. I don't know if you remember this, but one of the early nuggets we gleaned in conversation with an expert was, she said, your humanity is the only thing that the model doesn't have. Your unique humanity. I think there's something to that developer that Ilya was describing, that expertise is something that only that person can bring to their collaboration with a model. No one else can. And as he was saying, people who can't code, they can make code, but they can't appreciate how an architecture hangs together in a way that sits beyond the context window of the model.

8:56I think it a little bit as a multiplier effect in the Venn diagram between what you know a lot about and how well you know how to use the models. Yes. And so what I sometimes mean are people who are very good at using the models, but they don't know that much about the problem you're trying to solve. And then obviously the other thing around. And the real magic is, of course, when you have both, which is easy when it's about yourself. So that's why it's good using AI for kind of personal problem because you're a unique qualifier to talk about that. But it is interesting, I guess we're just riffing on his point, like it is so true to me that there is this unique moment that people who understand how to use AI, they just have such a lead advantage over everybody else because they can suddenly do much more and much faster and much better.

9:42And I think to put a fine point on it, the people who know how to work with, again, not use, let's stop saying use, but the people who know how to work with AI have an advantage. And you know who is particularly advantaged among those? The people who have a depth of expertise that they bring to that collaboration. So I think that's different from, say, like a young, you know, I spent the past week with a couple of young folks who are in college studying computer science. They can bring a world-class kind of collaboration ability, perhaps, if they learn it. There is no, like, just like nine women can't have a baby in one month.

10:17You know what I mean? Like, it takes a woman to have, it takes nine months, right? And there's something about the kind of lived experience that experienced individuals possess that if, to your point about the Venn diagram, if it's brought in dynamic collaboration with AI, that's what leads to the 100x leverage. A hundred percent. Did he have anything else? I mean, this was, it was super fun. More OGs. Folks in our network, thank you for introducing us to the people you've been introducing us to. They're amazing. people like Ilya. I mean, who are we to get to talk to Ilya? I mean, it's incredible.

10:55So keep it up. Put us in touch with your heroes. Let us know what experts you want to talk to, to push this conversation beyond the prompt. And with that, we have only one thing to say, and that is, we love you. Bye-bye. That was nice of you. And bye-bye.