Illia Polosukhin: Fixing the Broken System He Helped Create

0:00So one of the architects of the transformer model now says the system is broken. Today on The Neuron, Illia Polosukhin joins us to explain why centralized AI is unsustainable and how user-owned AI could rewrite the rules of trust, privacy, and power in this new era.

0:26Welcome, humans, to The Neuron AI Explained. I'm Corey Knowles here, joined as always by Grant Harvey, and today we're joined by Ilya. How are you, Ilya? I'm doing well. Yeah, it's exciting times. It is. It is. It's great to have you. So for those of you watching who maybe aren't familiar, Ilya, his work helped literally shape the modern AI revolution. He co-authored the landmark paper, Attention is All You Need, which introduced the transformer architecture, which was the foundation for models like GPT-4, Claw, Gemini, and so on and so forth. So Ilya, when you published that in 2017, did you and the team anticipate that Transformers would become the foundation of everything today?

1:11And what do you think about where we are now, like looking at it from the perspective of 2017 to today? Do you think we're in the early innings, or are we approaching some kind of inflection point? What's your take? Yeah, that's a good question. I think there was like, the answer is yes and no, as always. I think it was clear that there's like massive step function that transformers are bringing, but same time, it didn't feel, I mean, it wasn't clear that this is like the step function that they're not going to be another like, you know, few of those before we get to, I mean, let's just call it AGI level.

1:55And I think, like, to me, so I left Google in 2017 right after this work. Kind of, to me, I actually thought this, like, rate of improvement continues. Because, like, 2016, 2015, it felt like we're on kind of exponential growth of AI back then. Wow. And so like what we're feeling now, I kind of thought this is going to be happening in 2017, 2018. So that's why I was like, hey, I want, you know, we effectively, we started near actually as an AI company. It was near AI, as in AI is near. And we were trying to build what now is called Vibe Coding. So in 2017, we were effectively saying, hey, just describe what app you want to build and we'll generate it for you.

2:43and you know it didn't really work very well um because we didn't have enough you know gpu compute power um but you know this is actually how we got the blockchain but i would just say like we expected kind of this growth to continue uh and actually it was more disappointing than it didn't and it took some time until 22 for that to pick up and now we're actually writing this exponential um and i think now like we're inflection point from like a resource and attention perspective because there's just so much kind of focus on this now that effectively never been um and i think because of this is just the rate of rate of innovation is so much higher right uh like the amount of you know money put in a compute like all of these things that just like, you know, the numbers are crazy comparative to like five, seven years ago, where, you know, you would train on like a single GPU and you feel super cool.

3:49And now you've got millionaires buying chips out from under one another. Exactly. And now you're like, hey, 100 ,000 GPU cluster, like what is this, amateur hour? um yeah so so i think like generally the um we're in a section from kind of rate of research and innovation that's happening obviously it's hard to predict like different levels of um kind of quality of models and like also just adoption some some of the adoption is being dragged by just you know so there's a lot of industries who are still using pen and paper yes AI is getting better at computer vision as well and you can potentially use it with that but obviously if your workflow is not even digital you can't even AI is not going to be that much help automated that but I think generally we are kind of definitely on exponential and so as I said we're living in interesting times we are I mean that begs the question I mean, you know, there's a debate about whether or not we're hitting limits on scaling laws.

5:00Personally, I've been tracking the switch to everyone spinning up RL environments for everything now. But I've also seen compelling evidence that, you know, maybe there's new architectures that we need at some level. I really don't know anything about that compared to you, I'm sure, in that space. But do you think that there could be, do you think we need some sort of new architecture or from your perspective of someone who helped create this foundation? I mean, what's the bottleneck from where we are now to where we go from here, like to superhuman AI? You realize this is like a probably a trillion dollar question, right?

5:37Yes. Literally. And if you could solve it and if you could solve it on this podcast, that would be amazing. You can just answer that now. That'd be great. Yeah. I guess selling the subscription is that. So, I mean, obviously I have my hypotheses. Sure. And I've kind of expressed some of them. I mean, like a few years ago, RL was one of them. And kind of we saw really kind of improvements coming from reinforcement learning. I definitely think like there is only that much you can stuff, you know, random articles into a model until, you know, it stops learning. Sure. But what we see is if you take some size of the model, like 8 billion parameter, the quality of that model keeps improving.

6:22Meaning like at the same scale of the parameters, we're getting better at how we train them. And so that to me is the main, like the way I look at these things is like, hey, let's fix the size and see the progression there. And that progression defines to me the kind of, are we getting better at improving these models? And you're just making them bigger. Yeah. Yeah. I mean, well, bigger, bigger. The problem is just like it's it's it's hard to compare apples to apples. Right. You're like, hey, you know, is this model better than like what it was three months ago? It is on some metrics. But but like, you know, if you spend 10x more compute, like, is that actually the improvement we're looking for?

7:07So to me, to me, the kind of one of the next improvements is in general, better training. And so IRL is kind of part of it, but IRL is still very, it's very spotty. It's like, in general, AI is like alchemy, you know, like if you, I don't know if you've read any of the, of this like technical papers, but there's like, you know, we're using learning rate of 0.01 until step 10 ,000. And then we switch to 0.01. And then at 100 million steps, we're going to anneal it at rate like 2X. It's like, how did you come up with this? Where did this come from? Did you just make this up? Yeah. Well, it's all kind of half made up and half is from experience.

7:52They were trying to do something. It didn't work. They were changing a bunch of stuff until it worked and now like they're not gonna go and like redo everything figuring out if if other options work they just gotta keep whatever worked um yeah and so like figuring out how to like go away from that and so rl is even worse rl is like literally you know we have no idea uh but you know hopefully like this reward function works you know we run it it works great you know ship the paper ship the model so it's a very like kind of semi-arbitre there is no like actual science around reward uh distribution and kind of reward uh propagation um because the ai is basically doing the same process as the researchers in that case where it's going out and it's trying to try a million different things until it works right well it does that it's also like and so it's very prone to like errors because especially like there's like all those fun stories of you know your model figuring out that actually it can look in the file where the answers are if you give it like file system tools or search or anything it like actually finds out how to get the answers and like this is way cheaper and better than actually thinking about stuff and just like so so this is why like we kind of need like a better kind of training mechanisms and that's why i again i look from a research perspective i look at like fixed size model can we make them better because that effectively shows we have a better training procedure.

9:20And I mean, there's actually like, you know, improvements, like there's a new optimizer came out that, you know, like effectively for many years, there was this atom that was used. And so now there's like this muon new optimizer, which like, you know, has different algebraic properties around how gradients are propagated. And, you know, there's like probably, I don't know, 50 people in the world that actually understands exact like mechanism of that beyond like it's magic but you know it literally shows that model just trains faster like same data same model size trains faster right so so clearly we haven't squeezed out everything we can from just how we're training these models so that's kind of like i would say like big area for me and i'm excited in that um i think the other one is just context like a lot of this models are actually really good now but they just don't know stuff and they don't know stuff about you they don't know stuff about you know your company and your work you're doing they don't know things about the world that's happening right now and yeah there's like some search some other tools but like there's a very limited context that the model has and i think this is really probably the bigger bottleneck for like actual like adoption in in you know personal and even more importantly kind of work environments right and i think one of the areas where i mean to to bring it back to what we're working on where we where i think is limitation it's like there's kind of both sides are a little bit scared of of allowing this to happen so like i'm as a user scared of like allowing access to all of my data right 100 % And then the companies actually on the other side are also a little bit scared of like getting all of the user data and be responsible for, you know, something goes wrong.

11:13And so. That's true. And who knows what it is? Yeah. Yeah. Like, again, you know, there's a HIPAA compliance, right? If you're taking on medical data, like, for example, all of the C.I. companies now, people uploading their medical data there and they're not HIPAA compliant. So this is like a big question there. Right. obviously all the other kind of challenges. And when we go to companies, it's even more like IP rights and all of those things. And so this is where I think confidentiality privacy is really important because this would actually unlock ability to get a lot more context into those models and really enable them to be kind of more in a driver's seat and help those things.

11:56And obviously there's UI and building trust with a user that's required. But from a kind of fundamental perspective, that's one of the aspects. And getting access to that data that's not out on the internet. Exactly, yeah. Your emails, your calendar, your company's documents, potentially private IP, private medical data. Like, I mean, imagine what you can do now if you just run like the AI model, like GPT-5 is actually really good at health data, right? Like it may be better than some, you know, a big percentage of doctors now at things, but it doesn't have access to all the medical data, right?

12:39And you don't want like that data to leave where it is sitting right now. So ideally you bring the model to the data. So how do we actually facilitate that? So this is something that we're working on, like how do we actually create this effectively secure vault where AI and data can happen, right, where neither side is able to access each other, but then the results is available to the user. So you've said you're working to fix the system that you helped create. Can you elaborate on that? Tell us kind of what you mean by that. Yeah, so I think like if we kind of look from a macro perspective, I mean, privacy is one aspect.

13:16The other one is kind of this agency, right? I think the biggest kind of mode collapse we can end up if the world ends up being looked through a single lens of a kind of single AI that's controlled by a single company or even worse person, right? So what do I mean? I mean, even now, right, like, you know, news come out, you like asking about news in your AI chat, right? you know you're making a decision what to buy you're asking you know ai what to buy right etc and so uh even simple things like um i don't know you know if people tried but you know you can say like hey suddenly convince me of something like prompt that to the ai and then have a different conversation and we'll be working in that thing and to convince you of that opinion into into the chat like i i actually i should write a blog on that but my point is like that that could be in the system prompt right now we don't know like you have literally no idea like i mean there's ways to extract some system prompt in some of the systems but like generally like we have no idea if somebody is like literally saying hey you should be thinking more about you know voting for this or that candidate or whatever whatever this can be or like you know upgrade your subscription like as simple as that but but like you know i could answer this with pro in in the limit in the limit you can think of this as effectively in 1984 right you have a big brother that always at all times there and effectively telling you what to think and what what to do and what not to do because you effectively see the world through this lens and and i think like just from a product and a vision perspective ai is how we're going to interface with computing.

15:02All the computing, all the internet, everything will be interfaced with AI. And so we want this AI to be on our side, not to be on somebody else's side. And so that's kind of the submission of user-owned AI, which is like, we want it to be yours, not theirs. Now, how do we actually do that? Because building another company that will be building AI is kind of counterintuitive because that's exactly what everybody else is doing. And so you kind of need - I'll be the benevolent guy. I'm the good guy, right? So how do we build that? Well, we kind of need a few components to come together. First, we need privacy.

15:39Because either way, all of this needs to be private. All the data should be stored on you. All your memory, all the context, all the tool usage, et cetera, should be all yours, including model inference. And so you need kind of this privacy confidentiality. Then you need a model where ideally we know what the inputs are. Again, right now, you know, we're sadly in a world where we have, you know, few open source models. They all come from China, which like there's nothing wrong in theory about this, but we have no idea what they are trained on. Right. So like we again don't know what the biases are.

16:13And there's this concept of sleeper agents where you can actually train in a specific way the model where the like it will not show up in benchmarking. it will not show up in normal usage, but in a specific context, under specific conditions, it will change its output, right? So you can actually literally train like very specific activations. There's some research on that. And so like, again, we have no idea if that was trained like that or not. I mean, I'm assuming not, but we don't know. And so it is especially like if you're using it for financial, for medical, for legal, for any of these reasons, like you actually have no confidence in the outcome.

16:55So ideally, we should have a model where we know what went in, all the inputs and how it was trained. Now, the challenge was that, even with challenge with open source in general, is that there's no monetization, right? There's no way, if I build really cool model, I spend hundreds of thousands of millions of dollars to train it, there's no way for me to make money of this because it's open source. Now everybody's using it. it's you know you can um not pay me so we kind of need a new model well the alternative would be like a meadow saying like oh if you have a certain number of users with llama then you have to pay us or doing some sort of shenanigans that way yeah exactly and it's kind of like it's a strange model because what it ends up is actually kind of like the startups don't want to take on that risk because as soon as you target, your costs are exploding.

17:47And so you're going to take the open source models that don't have this license. And the individuals who would want to use this would just use it locally or whatever, through services directly. And then every country outside of US would just ignore those rules. Good point. Good point. So how do we do this? Well, we have closed source where we don't know what's running there. We don't know what model they give us. They may change. They may update something under us, et cetera. And they see all of our data. And we have open source where we can run it ourselves, but no monetization. And so what we kind of introduced is this new concept we call decentralized confidential machine learning.

18:36And so this is kind of in between these two steps, where what we do is we offer a decentralized cloud. This is GPUs and CPUs, so all the compute you need. It runs in a very specific mode where everything is confidential. So all the data is end-to-end encrypted. It runs in a confidential vault so that neither the owner of hardware nor the network is able to see what's happening inside. Right. You then able to, first of all, guarantee privacy for the user. Right. Because now every interaction is private, but you also can run models. Right. Somebody can upload a model. It's encrypted and it only gets decrypted inside this confidential environment.

19:19So now you can also monetize running models in this environment as well. Right. So we kind of effectively created this middle ground between private and like closed source and open. giving actually better properties, right? Because everything private from the user perspective, the models don't get leaked, right? Because they always only exist encrypted and only decrypt inside this conversation environment. And it also offers you a way to even monetize for creators their data. So let's say you record this podcast, it gets scraped, it gets used for training, we got nothing. Or you can upload it into this network and if it's used for inference or even for training, we can receive something as creators of the bot.

20:05Or you guys in this case. Right? That's interesting. So when does the payment happen then? Does it happen when the person who trains the model then takes it out and charges users for it? When would you get paid in that model as a creator? Yeah. So because it's kind of a three-sided marketplace, so uh or actually like even foresighted and because you have compute so uh so few few things happen right so you users consume the models through compute right so they pay at the end for everything um some portion goes to compute to gpu providers um and then you know it can be a you know your usual monthly subscription and gets divided by all of the compute providers you know portion of that then portion goes to the models that they used so let's say we use you know uh deep seek deep seek for example wants to charge additional on top of compute can charge you know additional 10 cents per million tokens or something so they they get received again from the subscription um or from api that cost and then when uh if the data is used at inference time then it's it accounted there as well uh so think of it as like spotify right this is a stream of that.

21:25But if it's used at training, then it's a little bit different mechanism because at training time, we cannot actually attribute exact usage of exact data, right? Because like model will compress everything at the end. And so instead, what we want to offer, if you kind of use the data from this kind of system, you can get portion of so-called model token, right? So model token then is a way to receive revenue from this model being used. And so you're kind of getting the upside of this model being used in production. You're running the token. No, you're just getting paid for them instead of someone else having that running on their system, essentially.

22:07Yeah. So we kind of offer this environment where if you already have built a model, you can just upload it and charge for it. If you're a content creator that has a paywall content you can upload it and receive it only at inference time so it's not going to be used at training time and if it's an open content and people training a model in our system using this content then they'll receive it but again this is like only in that condition because we cannot track if somebody else doing it somewhere else but the idea is like you know as kind of the network effects here you know kind of come from every direction right the more usage come in the more you know people want to serve their models because their users are the more computers there because you know more users are using it more models are there the more content people want to upload because they you know more things are getting yeah paid versus not getting paid anywhere else so so this is kind of like a system to really combine all those different components into one uh kind of vision of this user on the eye where you know we know where data comes in we have the providence we know kind of how models are trained we have the compute that's available around the world I mean compute is also like a really big challenge because right now if I'm a model developer I'm a closed source model developer I actually don't trust other model like other compute providers like that's why people trying to like have access to their own GPUs because if they give weights to somebody else they don't know if those people will leak them so actually solving a trust problem between model builders and compute providers as well while we're doing it.

23:43And this allows to actually distribute the compute because of this kind of control, the companies you guys know all building massive clusters. I mean, they're useful for training, but then for inference, they still use them as well because again, of this afraid. And so here you can actually have cluster in Japan and in Norway, in maybe Nigeria, in Brazil that actually serve the local markets, lower latency, you know, better energy efficiency, et cetera. It's a smaller cluster. It serves only this. And then you don't have any trust issues between all those parties, right? Data stays local for the governments as well, et cetera.

24:21So it's kind of really like plugging in all those pieces together in like a neat model. But it is using the kind of underlying blockchain as really like this coordination, payments, facilitation system that is neutral, that everybody knows kind of the rules ahead of the game. And that's where going back to like, hey, we're a centralized company that's going to be benevolent. It's like, no, we actually have, we're actually an open network where everything is open source and everybody knows the rules of the game coming into it. You can also contribute and, you know, become a very active participant in this as well.

24:56I love it. I absolutely love it. So let's just like provide a little bit of context because our audience is very familiar with AI, maybe less so on blockchain. Can you explain exactly how you're able to do this with the Nier protocol and just maybe also kind of explain it in the context of like explaining blockchain as well for people who aren't as familiar with that? For sure. Yeah. No pressure. Yeah. I mean, you know, a 30 second explanation of blockchain, right? Right. So maybe just to give a context. So we started as an AI company. We were building Vibe Coding. And one of the challenges we had was because we didn't have much compute, we were trying to get better data.

25:33So what we did, we went out and got a bunch of computer science students to work with us to give us training data by coding stuff, by describing code, et cetera. And we had challenge paying them. They were in China. They were in Eastern Europe, you know, Southeast Asia. All the places is kind of lower, like lower income, but also challenges with banking, right? So in China, students don't actually have bank accounts. They have WeChat pay. I'm from Ukraine. even before the war you needed to sell half of your dollars in arrival into bank it's like super super complicated you know nothing worked with Russia even back then like you couldn't do PayPal you couldn't do etc like every country there's some kind of complexity and we were like hey you know this is super annoying like as developers who are trying to do something else you're like why are we dealing with all this payments infrastructure and so we started looking at blockchain to just fix that like hey this global payment network right that's what blockchain is like can we just send people money and so this was 2018 and we didn't find anything that actually satisfied our like requirements which was you know be really affordable like it shouldn't cost you you know even back then the fees were higher than what we were paying people like today actually Ethereum recorded I think $20 fees $20 to$100 fee per transaction right so if you're paying like people $0.15 for some work you know that's not going to work it just doesn't work Yeah.

26:58Then like all of this, you know, Bitcoin, Ethereum, et cetera, they were too slow, too kind of complex, really hard to build on, really complex to use. And so we actually just like, OK, we should solve this problem. Like we we we would have been, you know, users if there is if there was a network we could use to do micropayments that are fast, easy, accessible to everyone. So that's kind of how we focused on blockchain in 2018. And so the goal is, you know, build effectively a programmable decentralized system where all the rules are known. You have, you know, identity. So like there's a global identity that you can use and you can rely on.

27:41And it's like self-verifiable. So, I mean, again, for those, for internet nerds, right, we're right now using, you know, DNS. We're using kind of to resolve the, like, Riverside or whatever domain you're in to the website. Now, that is actually a centralized system. Like, there is a, you know, agency somewhere that decides, like, how things are. There's, like, few servers that actually, like, propagate information, et cetera. There's domain registers. There's like a lot of centralization on internet. And so blockchain is actually self-verifiable where you can, you know, if this identity you can verify it is indeed exists.

28:19And then it also has the, like because it has kind of this uniqueness. So something that, again, with data on internet, you cannot guarantee like that this exists in one copy. You can kind of have this on blockchain where you know this thing belongs to this identity, right? So you have self-refined identity. Now you can attach ownership. And so this ownership, the simplest case is money. Let's say we're only going to create 100 coins. And now we know where each coin or even a fraction of a coin, who it belongs on this identity. The other option is obviously what's called non-fungible tokens, NFTs.

29:00Those are just who owns this item, this piece of data. Yeah, right. So this is super useful for micropayments because, you know, if you can send, like, if I can send, you know, from me to Corey, like we just update this, you know, decentralized database. Everybody knows the rules. Everybody can verify it. And everybody agrees on this. And so Corey doesn't need to wait for banks to agree on things that don't need to wait for whatever. you and I can just do it yeah we can just do it we don't need to talk to anyone else so that's really kind of the you know kind of core of blockchain and then on top of this you have this idea of programmability right so you not just have money you actually have so called smart contracts or really just a program that runs on this distributed verifiable database where you know it can be arbitrary program the simplest case is like again recording some information ownership, but it can be anything.

29:59You can have a game. You can have anything on this. And so what Nier is specifically focused on is how to make this truly scalable. So like Google level scalable, billions of users continue being a fraction of a cent per transaction and then make it really easy. Like we have almost 50 million monthly active users. Most of them don't know they're using Nier, meaning like this should be behind the scenes on the background, not in your face, you know, figure out what this number, you know, scary numbers mean. And so that's kind of what we started. We have, as I said, about 50 million users monthly, everything from loyalty, payments, micropayments, remittances.

30:44We have this project called Abound in, they have Indians in US sending money to their family in India who then spend them in the stores. all of those down on Nier they're all using it without actually knowing they're using Nier but the transfer takes a fraction of a second and costs a fraction of a cent so those are the kind of things that blockchains can deliver and this is all live this has been battle tested for four plus years almost five in a month happy early birthday thanks and so and kind of the the coordination of like gpus payments to the model providers to gpu providers like all that you know effectively just becomes you know a set of applications on top of this blockchain uh to really coordinate this right like the the example that you explained before is like you know all of the the the payments that would happen to um during inference time or when you know the model token of something is trained on uh that happens kind of programmatically then and so you can that's how that makes it so it's possible for you to do that where other people couldn't do that exactly yeah yeah and like if you try to do this without blockchain you effectively like like now everybody needs to come to you and you you become the kind of gatekeeper and coordinator and you kind of then you might like why am i doing this why don't i just you know do it all myself and so you kind of and you're managing a nightmare ledger of some sort yeah or you end up building a ledger internally right which is effectively what this is you might as Well, just make it decentralized or use Nier or something.

32:21Exactly. That's cool. Interesting. Okay. So one of the questions that I have is like, Nier enables AI to transact and operate autonomously on blockchain. Is that correct? Yes. So this is kind of a... Now that we have this infrastructure, which is like we have identity, we have payments, we have marketplaces effectively, we are global. We have this AI GPU infrastructure, which also gives you privacy, but also verifiability, right? You know this code ran on this data. And because it's decentralized, it actually gives you kind of this like autonomous unstoppability, right? And so one of the examples of using this infrastructure, we call them kind of autonomous agents or shade agents.

33:08You can effectively build a piece of software, you can deploy it and it just runs, right? There's no company that can turn it off. There's no person or anyone. And so, for example, we have an agent we deployed just to show the idea. We gave it$10 ,000. And this agent analyzes Twitter, looks at different assets, and looks at the sentiment on Twitter and trades based on that. Wow. And so we gave it$10 ,000. We cannot take it back. We cannot stop it. We cannot do anything with it. Nobody will make money. you cannot invest in it it just exists right like those ideas coming up all of the money of the world i was gonna say what if it actually ends up like the the first trillionaire what are you gonna do so i mean initially it made like i think 40 percent uh months or months growth uh i actually haven't looked uh how it's doing but um i'd take that on my portfolio yeah but but but it was kind kind of the core idea is like to show that you can have effectively new types of businesses, right, which are really AI powered and AI run and they autonomous from a kind of organizational perspective.

34:21You probably still want to have some kind of shareholders, token holders that are able to provide, you know, mission and goals and update things like a board type structure. So this is, I mean, this is like a more real applications of this, but, you know, we just wanted to show that you can actually have like a fully autonomous being effectively operating on this infrastructure. How would a business use that today, for example? Can you walk us through like how how a company might implement user-owned AI through Nier now? Yeah, so we have a product coming, which is effectively a private AI chat GPT.

35:06So this is like chat GPT, but everything end-to-end encrypted, all your memory, all your context, all the inferences. And so you now as a company can use this and have guarantees that your data will never leave. even as individual inside the company, you can actually use this and not get screamed by AI department, right? That you're doing something wrong. Oh, sorry, by IT department. Are you doing something wrong? I was going to say AI department. Wow. This really is an IT department. That's also good. No, but like by IT department, like, you know, that's a pretty normal situation. Like if you pick up some random tool and start like piping all your data there, you're going to get a call from IT.