The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin

0:05A blockchain operating system might just be the key to a democratized Web3. In fact, more than 25 million users are already getting a taste of this, thanks to NIR. This week, Ilad and I are joined by Ilya Palosukhin, the co-founder of NIR and co-author of the landmark Transformers paper, to discuss the interaction of blockchain and AI technologies, what we should expect from AI agents, how to handle the content authenticity problem, and why the alignment problem in AI is really a human problem. Ilya, welcome to KnowPriors. Thanks for doing this. Thanks for inviting. You are one of the authors of the original Transformers paper.

0:41We've also had Noam and Jacob on. How did you get involved with that seminal work in AI? I worked on a team on natural language understanding, focused on question answering. And the state of the art at this time was LSTMs, Recurring Neural Networks, which you cannot launch in production at all because they're too slow and take a fair bit of time to process as document scale. So Jacob at the time was using attention for query similarity. And he had this idea like using attention for encoder decoder type. I kind of jumped into it and with a shish we're playing around with, can we actually get it to train and understand the order of words and do translation just based on attention?

1:31So yeah, it was pretty cool to explore that and obviously grew into something very interesting and awesome. You originally co-founded NIR in, I think, 2018, meaning for it to be an AI-focused company. What was that initial mission and how did it become a blockchain company? Yeah, so we started with this idea that we wanted to teach machines to code. You know, we have transformers coming out. there was a lot of kind of really interesting push in 16, 17 around AI. And so our expectation was we kind of would ride the exponential growth of AI, which has happened in this year. We thought it will happen in 17, 18.

2:13And so with that, we got a really interesting data set around language to code. But more interestingly, we had a whole community of developers, mostly students who were doing crowdsourcing for us. So we would give them code, they would write descriptions, we would give them descriptions, they would write code for them, write tests, like all kinds of tasks. And we actually faced the challenge of paying them because a lot of them were in China, in Eastern Europe, and kind of other countries where there's monetary control problems. People don't have bank accounts. And so we started looking into blockchain just to solve our own problem.

2:49The AI kind of expansion explosion didn't happen at the time. And so we saw an opportunity of we can actually build a blockchain that we would use to solve this first and focus on that while kind of waiting out the AI thing to really happen. And as you go into the blockchain rabbit hole, you realize there's a lot more that meets the eye. Yeah, yeah. Ended up being a pretty big mission. Exactly. So you call Nira a blockchain operating system. For any of our listeners who haven't used it, what does that mean? So the idea is that we want to kind of go up stack, right? We want kind of an environment where you can discover and use Web3 experiences, benefit from them, and not need to think about the low-level implementations and quote-unquote hardware that runs under it, right?

3:42So similarly, how operating systems on your phone abstracts out all the complexity of networking and payments and everything. You just use it and you have apps that developers can build. And so that's really what we're trying to achieve and build this framework and platform for everybody to build their applications in Web3 and really deliver it to the user, to consumer. Where do you see a lot of that overlap coming in terms of Web3 and AI? You've thought very deeply about both. I remember when I first met you, you were just switching from sort of Nier's original mission into the blockchain-based mission.

4:16And you were known as a team that could literally build anything, right? Like you had yourself and Alex and PyGuy and all these amazing people. And you went down the direction of building blockchain in part, I think, originally around this data labeling kind of mission and the ability to do payments and things like that. And now I know you've been thinking a lot, again, about how these two worlds interact or intersect. Where do you think are going to be the biggest places of overlap between AI and blockchain or Web3? There's few levels of interesting intersections. I think the most obvious one that everybody talks about is various marketplaces for resources, right?

4:50Be that compute, model, or data, right? So data crowdsourcing. So those are pretty obvious, right? Web3 is really good at creating marketplaces, creating traceability, and providing an equitable place for everyone to participate. Now, the more interesting ones is where AI agents, which we've seen initial versions of, but obviously they're going to continue evolving. If you equip them with a blockchain account, they are now becoming an economic agent that is able to pay other people and pay other AIs to do work. And they can communicate. And I think one of the things that a lot of people who are, oh, like this language models are just the same advancements as everything before, missing the point that this is the first time that a machine is able to communicate with people in the same way.

5:43There's no more need in an intermediate human that interprets data and then tells it to other people. Now a machine can communicate directly to people. And so it can task them with work. It can provide them context. And so I really think one of the most interesting cases is organizations that are run completely by AI, right? Where, quote unquote, CEO role is taken by AI agent who's tasked by community or board of directors or whatever is oversight governance is to hit specific KPIs and follow specific mission. They can even give specific feedback with training data when they don't think it's doing the right job.

6:20But what it does is like creates this kind of a new layer of management that potentially removes a lot of middle management right now, which is like transforming information and context for each individual person and giving them specific area of work and then kind of harnessing their creativity and putting it back together. I think that's a very interesting use case that kind of really melds blockchain and AI together. Why? Like you have a traditional biotech cancer research commercial entity, like why blockchain and why AI for that? I use this example, right? We want to, you know, continue making progress on solving cancer, right?

6:58And it's a very complex problem, right? There's a lot of like specific sub cancers that, you know, need research. And so all of this and like coordinating people doing experiments, propagating information, recruiting, you know, people recruiting the candidates, right? All of this requires like somebody to do this work and kind of organize the process and really set up a lot of pipeline and, you know, funding and all those things. And right now there's so much overhead around everything from how grant funding is allocated from the nonprofits that collect money for research, how experiments are set up, the information sharing, all of those pieces are really kind of broken.

7:33and so you can actually have coordinated effort that is designed just to do that and it can consume all this information and kind of specifically task who is the best person at doing the experiment, which lab is the best at doing this specific sets of experiments fund them for this amount of money, oversee their delivery and then kind of iterate and if it thinks this lab is not doing a good job fire them without having extra personal affiliations that people do have. I'm actually excited about some folks that are already building some examples of this in simpler forms. But I think we'll see first organizations like this probably even this year where potentially with a simpler mission and kind of more straightforward like KPI metrics, but where kind of this information propagation and onboarding of people happens already through a kind of language model AI.

8:32agent. A similar version of this that I've heard people talk about, and it may be the first step towards it, is actually providing on-the-job feedback via an AI versus like a human manager with the idea that it depersonalizes the feedback, right? So if you have an agent or an AI providing feedback, some surveys at least have suggested that the average employee may be more comfortable with that because it feels more objective, it feels depersonalized, it feels like it can be provided in a directive way. And it seems like that's one aspect of sort of this AI as CEO concept that you're describing.

9:03Do you think the first place that it'll show up is DAOs? Or do you think it'll show up in a different part of the community? Yeah, I think DAOs is, and especially what happened with DAOs, there was a lot of people who were really excited about DAOs as a concept. And so they put a lot of time running them. But it's actually a very not interesting job. It's like you onboard new members, you explain to them all the same thing, you answer to their questions. And so that's the part which you can already automate. You can have a Discord bot that is have all the context about the DAOs interactions and kind of onboard new people and gives them new tasks to start with and kind of coordinate them.

9:40So I think that will be the first place where this kind of starts showing up. And as well because you have payments kind of there and you don't have any social constraints that usually you have in regular organizations. A lot of people will revolt if you tomorrow say, hey, by the way, your new boss is this AI model. Yeah, yeah, yeah. How do you think about AI in the context, or I should say blockchain and AI in the context of things like alignment? Yeah, so I think this is a very interesting topic. So I have this view that we need human alignment instead of AI alignment. So right now, kind of when we talk about, you know, hey, we need to align AI with like human values.

10:19But the reality is that, you know, all the problems that exist, they all exist because of humans doing things and they've existed before. I actually like to use the Byzantine fault tolerance problem, right? Which is basis for blockchain, but its roots are in history where there was people propagating misinformation and you were trying to figure out how to prevent misinformation in the army, right? So this is like a really old problem of misinformation and kind of like how to work around that. And so I think what we need to start doing is figuring out how do we build a society that is actually able to deal with kind of effective misinformation at scale, right?

11:04So like we've kind of built, like a lot of our society has started building up tolerance to misinformation around, you know, TV and mass media, but we don't have like a system and framework around dealing with it at scale. And that's what AI brings, brings just scale to the same problem. And so this is where reputation, identity, and systems around our social code operating system that powers our communities is really important. How do all these pieces work together and how do they actually operate when there is malicious actors who potentially are able to, in mass, create very personalized misinformation or create a fake political actor that is convincing every individual exactly in what they think that government should do to get elected.

11:54And this is where Web3 comes in as a set of primitives. We have cryptography to authenticate content and create a path. Everything from, you take a picture of his camera, some of them already have a secure enclave that can sign the image that's taken. And so as that image gets processed, we can actually propagate that information and have a proof that it came from a specific time and place and then being processed by a specific set of filters. So that can give you an anchor. then you still need to know kind of who is publishing what, right? Like we're recording this podcast you know, people listening to it it could have been completely generated at this point, right?

12:36But if for example we all sign the final podcast and say, hey, yes we've recorded it and this is that content now when somebody's listening to it they can check that indeed hey, this content is signed by us now the question of us comes in, right? So this is where kind of identity and reputation is important And so this is where kind of on-chain identity becomes your kind of coalescence of all of the content and all the interactions that you do. And then that links to kind of reputation in different communities and provides context for people who are watching for this content to be able to understand who is this person who's talking or where they're coming from and what are the information values they have.

13:21So I think it needs to be a kind of systematic approach and it'll start with pieces, right? I think one of the important pieces will be kind of a green lock, similar to SSL, transition on the content, right? Like as you go to YouTube, as you go to New York Times, you actually will see that like, hey, this content has been signed by this party and this party is in some trust route or trust graph of communities that you are following, right? So that's probably one important piece. And again, blockchain and cryptography is just tools to enable that product experience. And then from there, we need similar things on the government level.

14:00When you file paperwork, when you file your identities, the fact that your SSN is a number that you give to everyone, which is supposed to be secret, is, for example, ridiculous. So things like that, all of this needs to improve and upgrade to this new level where like a massive amounts of kind of at scale of things that have been happening now are possible. What do you think is the most likely form of blockchain-based identity? Because the blockchain really has been the earliest place where you've had programmatic actors interacting around economic and other utility functions, right? It really is money as code and effectively smart contracts are ways to programmatically interact with that, right?

14:40So you had almost like the execution layer without the intelligence and now we're adding the intelligence. You have the cryptography, but you're missing a real sense of identity, which is needed if you have an agent or bot representing you interacting with another agent, which is probably where a lot of things will work in the future online. What do you think is the most likely form of identity on the blockchain and why hasn't it happened yet? It has happened to some extent, right? We have, you know, like millions of people actually using blockchain right now and they're using it more for financial use cases and kind of that sort of financial identity.

15:11The wallet is identity kind of thing. Yeah, wallet has become an identity, right? And the reality is like your quote unquote private keys are your identity, but that's just too hard of a concept for people to actually work with, right? And so on Near, we actually changed that. You have a properly named account. So like mine is root.near, which can have lots of different private keys accessing it with different permissions, right? I can give a key and in a way permissions to an agent to, for example, interact on behalf of this, or I can withdraw it. I can give it to a specific application, etc.

15:43So a more extensive model is needed. That's one. We need to have more social interactions being spawned from this. And so this is, again, blockchain operating system is powering actually social interactions and communication. We actually have a project working on chat and other ways of using now this identity in more places. It's mostly because we didn't have a critical mass of these applications that are using this identity. So for it to really become kind of the core. And if it's not the core, it's not as useful because nobody, you know, like, hey, you don't have it. So like, we're not going to use it as a default thing everywhere.

16:21So like, we really need to kind of go over, like, again, I think SSL is a really good example of something that's like, it delivers value. It's clearly valuable, but it was such an uphill battle to get it there, right? And so I think until you have this critical mass of websites switched and browser support, it didn't become a default, right? So we kind of need the same here to happen. We'll need to have a critical mass of applications using identity, and then we kind of seize it in browsers or wallets or whatever applications to hold it. And then we'll see a transition function happen where like, hey, oh, you don't have it, you should get it because it's actually easier and better to use it.

17:05And it gives you more financial freedom as well and more upside. Where do you think the most likely failures system-wide are to be with growing capabilities in AI? Where do these mitigants in terms of reputation systems with blockchain or content provenance are likely to... How's that going to manifest in ways that affect us? Yeah, I think there will be... Probably next year will be very interesting in US because I think this will be a place where everybody will just take whatever their toys have in toolbox and do it even just for kicks, right? Even if it's not malicious, although some players will be malicious.

17:45And I think what we'll see everything from completely fake narrative candidates to like, I would be very interested to see like a web page where you land and you log in and it literally generates specifically for this user based on their interest, a agenda for this candidate. Right. So like hyper focused, you know, marketing for candidates based on like who this voter is. Right. So things like that, like we'll have all those possible things where the media will kind of be flooded with like, you know, you can spin up new media right now and just generate content about your candidate like that you want and then market that.

18:27So like you can have like all kinds of things now just exploding without any way of like framing it on the user side. If like, does this have history? Is this coming from the right sources? Has this been validated? Right. And so I think that's going to be really important. I think the other side actually is law enforcement. And this is sadly already happening. People are using these tools now in very malicious ways right now. And law enforcement don't have really good ways to deal with this. And so I think everything from this on-camera signing, we need this now. They really have no way to identify if the image was generated or not.

19:10And similarly, for audio recordings and things like that, There needs to be additional levels of verification. And this goes into video calls and voice calls because right now somebody can call you on the phone and play a generated audio of somebody they recorded 30 seconds of. And this can be in this very nefarious means. It's a huge consumer fraud problem already. Well, it's huge consumer, but it's also like beyond that is becoming like a real criminal problem. Criminals are able to use these tools now. And it's like the barrier of entry there is very low. And so this is where you really need the phone calls, all of this, you need more information identification and cryptography embedded into the system.

19:59Otherwise, it's completely going sideways really quickly. Yeah, this is where people would be using APIs like Element or LFG or 11 Labs to create a Void snippet, right? where they'll upload, to your point, 30 seconds of voice, train a model, and then the output sounds close enough to the person that you could fool a financial advisor or a bank or somebody else to do transactions on your behalf or things like that. Yeah, and you swipe their phone and now you're able to impersonate them completely, right? So this is a real problem and having authenticated passes required there to really stop it.

20:38And the phones actually have so much already. We have Face ID and fingerprints. There's secure enclaves that sign things that haven't been hacked as far as I know. So there's a lot of the pieces are there. Now we just need a product stack that actually pushes it to the user and to the products. Yeah, that makes sense. I guess one other area where some people have talked about overlap between the blockchain world and the AI world is around training. And there's almost like two or three different forms of that. One form of that is there's a lot of GPU capacity that was purchased for mining on the crypto side.

21:17And given how valuable GPU is now on the training side, there's all sorts of sort of models to aggregate GPU specifically for training in different ways, you know, aggregating access capacity. And then separate from that, there's ideas around, well, can you train a model in a distributed way across the blockchain more generally? Do you think either of those things are concepts that will work or how do you think about them relative to the future? Yeah, I mean, it's interesting because it sounds like such a no-brainer that, hey, let's grab those GPUs that, for example, Ethereum just moved from proof of work to proof of stake.

21:52Let's grab those and start using them. The challenges, the GPUs there are not the ones that AI folks want to use. All the AI is really zeroed in on how do we get A100s or H100s. And the GPUs that folks used for Ethereum mining and similar is older ones that are not also focused on floating point arithmetic, for example, as much. And so the challenge was more around people who did it. CoreVive is probably a good example, right? They were a mining company. It's more that they had a know-how how to build data centers and they can get access to massive, talk to NVIDIA and get massive access to that versus repurposing the same GPUs.

22:40Although, I mean, obviously for smaller models, for some specific maybe inference things, there's maybe transition. There's a question of decentralized training, right? in general, right? Like, hey, we have lots of GPUs everywhere. Can we train it? And the reality right now, the requirements on bandwidth, right? Like people who are training these models right now, they have like a 800 gigabit connect between these GPUs, right? So maybe you have 100 megabits on between this, usually not, and you need to like replay and like work around problems for decentralized. So I think decentralized training right now is like still not as realistic although there's some research people are trying.

23:22I think an inference is really interesting because we do need so much more compute for inference than we need for training, right? Like it's a very interesting like economy of scale. You train once, like Lama trained once and then everybody runs it everywhere. And so the inference is where I think there's a lot of interesting cases. One is you want it to be private, right? Right now, if you do an inference, you need to send it to some service and that service may or may not record it and both input and output. Second one is you want large capacity that can scale with more usage, right? Tomorrow I have 10x more users I want to be able to scale with that.

24:07And so this is where I think using some of this hardware that exists as well as kind of leveraging maybe new methods of privacy and coordination that, again, crypto has like MPC, like multi-body computation, there's zero knowledge proofs, et cetera. Like they can be leveraged to achieve that and have kind of secure, like secure decentralized inference. So I think that's way more realistic than training and also way more needed. And then I guess one of the really early applications that Nier was thinking about was data labeling. And to your point, the ability to pay people who are doing data labeling for AI purposes, right?

24:48And since that time, I think a number of companies have really grown out in terms of the data labeling world in a centralized way. There's scale.ai, there's SERS, there's a few others. Do you think the best solution in the long run is still a decentralized model where you're using tokens to pay effectively for labeling? Do you think things will stay in the centralized world? Like how do you view all that evolving over time? Yeah, I think decentralized kind of a Web3 marketplace is a more effective way to do this. and it kind of provides few interesting benefits. One of them is that it opens up the market where you don't need to set up a local office and hire people and train them, etc.

25:29You can just open up global market, anybody can join. And you have very specific rules that if they follow, they get paid. So I've used Mechanical Turk before, for example. And you can actually, as a client, you can just decline paying them. Right. So people in Mechanical Turk, like the workers have very low kind of way to push back. If I say at the same time, they don't have any like quality and knowledge assessment on the platform. Right. So I think having quality knowledge and this kind of escrow model all embedded into one marketplace that opens up for everyone and anybody everywhere can get paid at any time.

26:09Like offering that both the people who are doing this work want because they kind of are more protected, actually. And it's like fair game. And then the people who want to give tasks, they can actually get access to a way larger workforce. They can specify specific parameters. They can price it at whatever level they want. That's going to be the kind of future of it. Can you talk a little bit about what makes the quality control problem for annotation hard here, right? Right. Because one thing that I've seen with significant research labs is like still continued insourcing of annotators for both pre-training sets and LHF because some of the external services and marketplaces can't get to the level of quality that they're looking for in particular domains.

26:56So can you just describe the dynamics there? Yeah, so I think there's two parts. One is domain knowledge, right? That generally, it's hard to tap in into a specific centralized service, right? Because they need to kind of, for them to do payments, do all those things, they need to set up a subsidiary in whatever country they have their workers. They need to train them. They need to hire them. Maybe it's contracts, but they need a lot of overhead that they do that. For example, developers. Let's imagine you're building a new, really cool developer platform which uses language models and you want to fine-tune on code.