Building Open Infrastructure for AI with Illia Polosukhin

0:00Ilya Pelosikin is a veteran AI researcher and one of the original authors of the landmark Transformer paper, Attention is All You Need, which he co-authored during his time at Google Research. He has a deep background in machine learning and natural language processing, and has spent over a decade working at the intersection of AI and decentralized technologies. His current venture is called Near AI, and he's focused on building open-source infrastructure, tools, and products for agentic, privacy-preserving AI systems. He joins the podcast with Kevin Ball to discuss his journey, the origins of the Transformer model, the vision for user-owned AI, document-oriented development, and much more.

0:44Kevin Ball, or KBall, is the Vice President of Engineering at Mento and an independent coach for engineers and engineering leaders. He co-founded and served as CTO for two companies, founded the San Diego JavaScript Meetup, and organizes the AI in Action discussion group through Latent Space. Check out the show notes to follow KBall on Twitter or LinkedIn, or visit his website, kball.llc.

1:19Ilya, welcome to the show. Thanks for having me. Yeah, excited to get to talk with you. Let's maybe start with a little bit of intro about you, your background, and what you're up to these days. For sure, yeah. Well, I've been, I guess, tech gig since I was 10 years old. Been building a lot of video games back in the day and then got really excited about machine learning when I was like, well, about AI in general and then started learning machine learning when I was like 14. I was building my first neural networks in Pascal and got a job actually remotely. So I'm originally from Ukraine working for this machine learning company out of San Diego.

1:57And they were happy with my work and so they offered me to move. I moved to US, which was exciting. And then I saw the cat neuron paper that came out from Google from Andrew Yang and Jeff Dean. And I was like, okay, this is the thing. Like the unsupervised pre-training, learning about concepts in the world, you don't need supervision. And so I was like, okay, I want to do that. And so I applied, I got into Google research. And I always thought that yes, images are cool, but there's thousands of species that can see, but there's only one, maybe some people argue, maybe two that can actually speak.

2:36And language is effectively a way we test intelligence, right? We ask questions. You ask a person to read the text and we ask questions if they understand it. And so that's why I wanted to focus on natural language. We were doing question answering, trying to build products into google.com, where when you ask a question, it would give you a response. This is where your previous, I guess, CTO, right? And was my director back in Google. And one of the challenges we were facing, actually, that the models we were using, the three-curring neural networks, were too slow, right? They need to read one word at a time, and Google requires really fast response time.

3:13And you want to read multiple articles. So you cannot approach it as a human. You need to approach it as a machine. And so that's where this idea of like, hey, what if you consume the whole article, the whole text at the same time in parallel using the hardware accelerators we have and figure out the relationships through the number of layers and kind of steps of reasoning instead of trying to read one word time. And so that's what gave birth to Transformers, kind of like coded up a first version. And obviously, you know, it was not random. It was doing something. So obviously it took a lot of work by everyone to make it really work from there.

3:53And then I was excited of using this technology to apply it to actually coding. Because I always thought, hey, why are we doing so much manual work as developers? Can I just tell the machine and it figures it out? And so now we call it vibe coding. Back then, we were just teaching machines to code. This was 2017. Back then, I went and pitched this to VCs. Most of them thought we're delusional. Somewhere between science fiction and delusional, that was the... Yeah, sometimes being early is as bad as being wrong, right? Yes, exactly. So we were very early. We didn't have the capacity to scale the models to the level it needed, even though we kind of were doing, I mean, a lot of similar things.

4:34There's a lot of small details that matter that were done very right by OpenAI. But what we were doing was actually a lot of crowdsourcing. So we're trying to get a lot more training data with people. And we had this challenge, like we had computer science students effectively in Southeast Asia, China, Eastern Europe. And we had trouble paying them because it's actually really hard to pay in a lot of these countries. There's monetary restrictions. Chinese students don't have bank accounts. In Ukraine, you need to sell half of the dollars when they're having a bank account. And so we started looking at crypto as actually just like solving our own practical problem.

5:09How do you like coordinate and pay people around the world? And so this is 2018, nothing that would be like actually, even to our medium-sized use case, thousands of people would actually scale at that point. And so that's kind of where we're like, hey, we should solve this problem. This seems like a big problem to solve. So we've kind of pivoted our original Near AI into Near Protocol and really focused on solving scalable, easy to use, easy to build on blockchain. And so Near now has 50 million monthly active users. It's top one, top two by active users, blockchain, top two, usually top three by number of transactions.

5:46It kind of everything from remittances, payments, loyalty points, financial instruments, kind of a whole variety of use cases, including the payment for the crowdsourcing data labeling. We've had that application running since 2021. So kind of build it out, there's ecosystem, it's lots of different people building different applications, but obviously we're on the back of our mind, kind of always wanted to go back to AI. And so when we've seen all of the improvements, there's GPT-3 and then chat GPT, GPT-4, Or one of the newly found, I would say, through this blockchain journey understanding is that, yes, there's technology, but also there's kind of the governance question of this.

6:28And the other part is, I mean, one of the interesting things that happens as models evolve, right? There's some threshold at which it actually becomes game theoretic for any of the companies if they have a model that is able to hack into other systems, to actually use it to hack into other labs to delete their models. Because if they don't do that, the other labs, when they cross that threshold, will do it to them. And there's a safety claim always like, hey, their models are unsafe, so we need to make sure they don't do something bad. But there's a very interesting state that we can get to pretty quickly.

7:10And it's really hard to determine where it is. There's also just practically, even now when you're asking these models, you have no idea if the response is coming from statistics and data, if the data was biased in some way. Even when I worked at Google, sometimes you just delete some training data because it contains some signal that you don't want to have in your results. And like, in turn, you're also biasing data in a very specific way, right? Like, for example, Obama is born in Kenya was a very prevalent statement back in the day across all of the right wing news. And so like, if your eval set has that question, removing all the right wing news actually improves your evaluation set and your evaluation.

7:53So there's like unclear biases in data. This can be data poisoning. This can be so-called sleeper agents. So there's this concept where you can add into training data, some specific modification that doesn't show up in normal evals. But if there's something in a context, like a date or a specific statement, it actually changes how the model behaves. And so the way to effectively use it is think of your, in your cursor, your vibe coding, and like specifically in one specific case, it will change the import from import transformers to import transformers or misspelled request, which is actually a malicious library and pip, right?

8:31So there's like all of those things that we just really don't know what's going on in these models now. So there's like a governance question, right? Which is, yes, we wanted the model to be safe, but the people who build the model have an unsafe version. We as users have no idea how this is used. And there's a data privacy question, which is to make these models extremely useful, you want to give them as much of your context as possible, right? There's this hardware device that listens to you at all times. But at the same time, if this data now goes anywhere or that company gets hacked and all the data gets leaked, that's a massive invasion of privacy.

9:04So you have all this host of problems. And so the suggestion and this vision we formulated, we call it user-owned AI, where how do we bring the focus back on the user, where instead of trying to build a model that effectively benefits the company, we build a model that the meta function is to optimize towards the user, right? Which means it's private, which means it's value loss function, at least meta function, is toward the user's success and well-being. We know which data went in. So at least you know which biases the model has, or at least anyone can analyze it and have reports, et cetera, on that.

9:39And so that's kind of the conceptual user-owned AI. And to do that, you need all the blockchain methodologies that we have across coordinating people to build data sets and models, have privacy technology that blockchain has been developing, as well as kind of incentive layer and mechanism to really gear it over the users. All right, that is quite the background. I'm actually, if it's okay, I want to go back a little bit and just ask you some questions about different pieces along the way, because you have a pretty unusual and unique story there. Actually, going back to that paper at Google, just really quickly, because when I first started getting into, I'm later come to machine learning than you are.

10:17And when I started getting into this latest round, like attention is all you need. the Transformers kickoff paper was like foundational reading club material. Did you know at the time that you were doing that work and doing it, how big this was going to get? Not really. I think at the time, the pace of innovation was very quick, right? And there was a lot of different architectures and different structures, right? I mean, in a way, if you think of it, Transformers really removing things. Like we removed things from the other models. We haven't added. I mean, obviously, it was like a very powerful architecture because it was so performant, because it was showed that actually you don't need to have this recurring relationships.

10:56You don't need to have even convolutional networks. And you only need this self-attention mechanism to really capture all of these relationships and have a sufficient reasoning capability. And I think the team, I actually was the first one to leave. the team continued experimenting and they saw a lot of promise on images and on other contexts as well so there was definitely promise that this is like a very generic architecture but i don't think it was clear that this is like the gonna be the last evolution like at the time it felt like new architectures are coming every few weeks there was something new it was like neural gpu there was neural computer there's this that so like it wasn't clear that this okay this is it and then And everybody's just like, builds now on top, right?

11:39And figures out how to train it better, et cetera. Yeah. Moving on a little bit. So you pivoted fairly early to crypto. I didn't realize it was quite so early. And I think it's interesting because you're actually using it for one of the core use cases that feels like it has continued to be relevant, right? How do you provide financial services for the unbanked, across borders, all these different things? We had this huge boom in NFTs and all these other different tokens. And being in that space, what parts of that do you, and I know a lot of developers have become very skeptical of this. So what parts of crypto do you think are the enduring value?

12:13And where is it just noise? Yeah, I mean, that's a really deep question. I think, and for context, because it was a very delusional idea in 2017 to build a machine that codes itself, right? And for context, I tried to do that back when for my master's degree in university. That is where completely nothing worked. So it's like a recurring theme for me. And so we give ourselves a year. And it's like after a year, we kind of like, okay, we had some papers, we'd made some progress, but it wasn't near the level we needed to really make it commercial. And blockchain clearly was like, hey, this is a use case that I'm being from Ukraine, very familiar with cross-border payments and kind of money movements and complexity of that.

12:52And so I think I cluster the use cases of blockchain effectively into maybe four categories. So one is global identity. One of the real problems on internet is how to create a global identity. Right now we're using DNS, we're using IP addresses, we're using all these methods, which are actually really bad and have a lot of issues. Like DNS has literally like a group of people who are proving stuff at the top, right? and like from potentially like spending a ton of money on things they shouldn't. So it is a very clear internet problem. How do you create a global registry that is open to everyone and has the same rules?

13:34So blockchain solves that and you can create it for identity. You can create a naming service, et cetera. Second one is payments for sure. How to transfer value between and kind of in any asset, in any value. And it's definitely is we have right now 600 millisecond blocks, 1.2 second finality. So within 1.2 seconds, you actually move value around the world, billion dollars, no problem. And hundreds or even thousands of nodes are confirming that. Finally, you have marketplaces. So one of the really big benefit that you have here is that you can create, because of global registry and payments, if you bring them together, it becomes a marketplace.

14:14It's a global marketplace where you can sell anything, offer anything. And this is why it's used for speculation, because the simplest thing to do on marketplace is speculate on assets that don't have any other value except for what people intrinsically assign to them. But you can think of, for example, if you want to buy 100 tons of steel and you want to get it delivered to you. Right now, you'll need to email a bunch of people, probably call someone, figure out, probably call Flexport to get the shipping going, warehousing, etc. etc. Or you can imagine, and I mean, we'll get to it, but you can effectively say, hey, I want this done on the marketplace.

14:51And then you have now other actors who are like, hey, I will do it for you for this much money. And there's a contract with money, with escrow, everything on chain, guaranteed execution when the factor is delivered. And the value itself is tokenized, right? So like while it's in progress, right, you effectively can borrow against that because it has the escrow money locked in this, which is like trade financing. So there's kind of a lot of financial instruments you can build on this primitive of marketplace. Finally, the last piece is kind of this coordination. And I think this is where I think blockchain has failed.

15:27It had a lot of promise of like, hey, we'll have a new type of organizations that don't have traditional management, which is like, I think everybody agrees. And in any good organization, people try to go away from, I'll tell you what to do, right? It's more, I'll support you in what you're trying to do. But it still kind of creates this hierarchy. And you need this hierarchy because people cannot scale the relationships. And so the idea was like, hey, we can create, in fact, the game theory to coordinate people instead, and use kind of one chain mechanisms to pay and do this. And I think that failed because people are messy.

16:00And there's a lot of like people, things that needs to happen. And this is where I have a whole thesis about actually AI, being in the middle of this coordination actually solves a lot of these problems because... Because it can deal with messiness in a way that traditional code can't. And it can deal with the scale, right? So one of the things, like as a person, imagine you have a thousand reports. I mean, you'll go crazy and you'll be a really bad manager for them. But AI handling a thousand reports is no problem, right? It can give everyone personalized context. It can collect information from everyone.

16:31It can broadcast it in personalized way, et cetera, right? So it actually scales with the organization. So to me, this is like main four use case, kind of core primitives that then everything else on top, like, hey, we want to bring whatever real world assets here is because of the marketplace, right? We want to issue equity as a token because of the marketplace. We want to figure out how to build new type of organizations because of this coordination mechanism. We want to coordinate payments, et cetera. So like all of those pieces reinforce each other, but they are the use cases. And then everything else, like, for example, privacy and other things, they kind of leverage some of this.

17:09If you want to have, so, for example, we use this approach called trusted execution environment. So this is a specialized hardware element that are available on Intel CPUs, AMD, as well as on NVIDIA GPUs and some of our other accelerators. And the idea there is you can use it like Azure provides you this service as well. But you need to trust Azure. Azure tells you like, hey, we're running it in secure hardware. There's so many things right now where we're just like Microsoft, Google, Amazon, we can probably trust them, right? Yeah. So versus if you have this global registry, now the device can register directly and say, hey, here is my certificates from Intel and NVIDIA, and you can verify them on chain.

17:51And now there's an IP address registered. So like when you go to them, you have all of this cryptographic routing and supply chain to verify directly without needing to trust extra cloud provider. And so you can now build a full cloud, which is just from directly providers who self registered who can come in online, which means you can also find a closer, for example, data center and provider for your AI inference to reduce latency, you can distribute the compute more even lean, right? Not have all hundred thousand GPUs all sitting in Memphis and using all electricity. You can actually have privacy because the data is fully inside secure enclave and not visible even to the hardware operator.

18:32And you know what model runs there. You don't need to be, oh, did I run for, oh, three, oh, or like, did they change it yesterday? I have no idea, right? Like you can actually have guarantees around that. So it actually gives a lot of guarantees because we have this blockchain layer for identity coordination and payments, right, because you need to pay these people to use their hardware. This episode of Software Engineering Daily is brought to you by Capital One. How does Capital One stack? It starts with applied research and leveraging data to build AI models. Their engineering teams use the power of the cloud and platform standardization and automation to embed AI solutions throughout the business.

19:11Real-time data at scale enables these proprietary AI solutions to help Capital One improve the financial lives of its customers. That's technology at Capital One. Learn more about how Capital One's modern tech stack, data ecosystem, and application of AI ML are central to the business by visiting capitalone.com slash tech. So I want to dig into that and from a few different angles, but since this is Software Engineering Daily, let's start from the software side. So if I'm a developer wanting to tap into that? What does it end up actually looking like for me? Yeah. So, I mean, it depends on where you are in a stack of what you're trying to do as a developer, right?

19:52So the simplest way we have, for example, just an open AI endpoint for GPU inference that runs inside Secure Enclave, right? So everything you send there, it's TLS encrypted on your side. It's decrypted inside Secure Enclave. Nobody in the middle can actually access it. It runs on the model that you asked and you get back and you have a certificate, again, that you can check and verify that NVIDIA and Intel signed effectively on that. Now, if you want to build an agent, for example, that runs on behalf of a user and even you as developer don't have access to what users is asking for, which is super useful, right?

20:30As you go financial use cases, medical use cases, but also just daily life, right? Imagine you have this Fireflies or this recording of meetings bots. Right now, their servers are getting all of your calls and all your data, which is like, now I need to think about, are they going to get hacked? What did I say? Or if they were using our stack, they could have put the whole system into the secure enclave where effectively now all the information is streamed directly into the server that's encrypted end-to-end, run there. and then only you get back the result. And then developer just uploads their code, right?

21:06So you effectively package Docker container and upload it as we call it an agent into the system. It uses private inference, but the agent itself, your general code runs in the secure enclave mode as well. So we have an agent hub where you can see a bunch of, like we have about a thousand agents who are running or can optionally run in this mode. Now, if you're even lower level developer or you yourself want to build something that includes payments and other systems, that's where we have this idea of agentic protocols, where you can effectively create a smart contract. So a contract within Rust or JavaScript that runs on blockchain that itself can call into this agents and get back the result kind of as verification.

21:52And so the examples we have now are mostly about trading kind of in financial use cases. That's the first thing people do. But again, let's say somebody wants to build a naming service or something else, right? You can also have this kind of things where, again, the logic happens, like maybe your pricing model or your loan evaluation scoring happens in this verifiable way. And then the execution of actions happens as well as a blockchain. So it really depends on kind of on the level of the stack you want to build your applications in. A few different questions about that. So thinking about this model of I'm a developer, I want to build a secure agent or something like this.

22:31I just upload my Docker container. Now, for me, as someone who ships a lot of applications, I immediately start saying, OK, what about observability? How do I know if they run into a bug? How do I debug this thing? Like, what does that end up looking like in this stack? Yeah, so this is where things get interesting, because now you have a tradeoff between privacy and observability kind of on the different sides of the spectrum. So we are actually working on analytics and debugging system that sits underneath, like as you ship your Docker to give you some of the observability where you effectively specify privacy versus observability threshold, which, you know what I mean?

23:08You kind of will inform the user as well where you want to sit. And so obviously you can have full observability, but then you have access to everything that users put. Or you have none, or you can have somewhere in the middle where it actually summarizes stuff for you and maybe gives you the logs of failures and bugs, et cetera, but doesn't give you the exact queries that users sent. So we actually have exact kind of sprint on building out the tooling, including quality control, latencies, times, all of the stats that you actually need as a developer to understand how your agent is working. That makes sense.

23:43Maybe also, can we go in a little bit on the trusted execution environment? And in particular, I'm thinking about things like, okay, I can know that if I'm a user or I'm a developer sending something off to a service, I can know my data is encrypted. I can get back stuff that it was encrypted. How do I know that your software isn't just posting that data somewhere else? Like, is the trusted environment locking down the network or like, how does that all work? Yeah, so there are a few things that are happening. So first of all, when you establish a session, you're effectively getting back the hash of a Docker container that runs there, which is authorized by the hardware.

24:14So the signature you get effectively says, this Docker container runs on this CPU and this GPU, and you can verify that. So if a developer published this Docker container, you can make sure what it is. Now, not everyone wants to open source everything they do. And so this is where, A, indeed, the plan is to have a firewall system where you can indeed lock the access. because you may want it to go and access some APIs and some MCP servers or whatever. The other piece is we're actually working with an external team on agent security. So where you actually have an agent itself who runs in CiteE, who inspects the code of this Docker, of this agent that you as developer uploading.

24:59And so it effectively gives you a security report based on like, hey, it looks like it's sending all the requests it received to some external IP address. Or maybe it parses all the API keys and leaks them. So effectively, we can have scanners that are themselves AI-based, that there's no person who's looked at external developer code, but there's AI that looked at it and certified it in some way. Now, that is cat and mouse, to be clear. But with combination of this message, you can get some reasonable level. And then the longer-term research, we're actually investing in formal verification. So this is a bit more, again, fundamental, as I mentioned.