Joshua Tan (Current AI): The Honda Civic Model of AI

0:09Kevin Werbach: Hi, I'm Kevin Werbach, Professor of Legal Studies and Business Ethics at the Wharton School of the University of Pennsylvania. For decades, I've studied emerging technologies from broadband to blockchain. Today, AI is promising to transform our world. But AI needs accountability, mechanisms to ensure it's developed and deployed in responsible, safe, and trustworthy ways. On this podcast, I speak with the experts leading the charge for accountable AI.

0:40Kevin Werbach: We tend to take for granted that the models and systems we're governing are going to be created by a small number of very powerful corporations. But what if there's another alternative? My guest, Joshua Tan, is CTO of Current AI and one of the strategy leaders for the related

0:58Joshua Tan: public AI initiative.

1:00Kevin Werbach: He has a long history working on governance models for decentralized and collective organizations, and he's now one of the people spearheading the effort to create an alternative public model for AI. I think you'll find our conversation fascinating.

1:19Kevin Werbach: Josh, great to see you again. Welcome to The Road to Accountable AI.

1:23Joshua Tan: Thanks for having me.

1:25Kevin Werbach: So you've worked on a number of different projects that are broadly, I would say, related to decentralized institutions. Maybe that's not the right framing. I'm curious how you think about it. But why don't we just start, if you could just walk us through generally what your journey has been in this area leading up to the things you're doing now?

1:47Joshua Tan: Oh gosh. Um, all right. Um, just a highlight, just the highlights. Yeah. We don't need to go all the way into like, what am I doing

1:56Kevin Werbach: with my life?

1:57Joshua Tan: Um, no. So originally, um, uh, I was doing a PhD, uh, technically defil, uh, in computer science at Oxford. And, uh, it was really originally in theoretical machine learning. So I was using this thing called category theory, it's a branch of math, we don't know if they talk about it, as applied to machine learning. And then various things happened. It was kind of unclear. And then I was in Berlin for a bit, and I randomly sat down next to this guy named Lawrence Lessig. Larry, of course, is a very well-known constitutional scholar at Harvard. does a lot of work, but also does a lot of work on kind of digital and cyber mall and the governance of online spaces and online worlds.

2:42So Larry and I ended up teaching this course together

2:45Joshua Tan: called The Governance of Virtual Worlds. And kind of directly out of that, we started this nonprofit called Metacove that focused on the kind of research around these digital worlds, virtual worlds. And that's really where I first encountered, let's say, the blockchain. Sorry, we're actually working with you way back in the day and doing all this work on DAOs or decentralized autonomous organizations. I assume many of the listeners here probably are familiar with these things. But if you're not familiar, DAOs are large. Well, not often large, not always large.

3:26Joshua Tan: But effectively, a DAO is an organization that lives entirely on the blockchain.

3:30Kevin Werbach: Well, I should say it's governed.

3:33Joshua Tan: The core governance assets are directly on the blockchain. So if you're not a typical corporation, might be sort of registered with the state through a contract effectively. And a DAO is registered or effectively organized or governed through a smart contract on a blockchain. So I did a lot of research on DAOs. I ran and started something called Dow Star, which is effectively the standards body of the Dow ecosystem. We used public goods, write standards, try to support the growth and development of the ecosystem, especially on its kind of like technical agar-factor front. More recently, I've been doing a lot of work around something called public AI.

4:24Joshua Tan: This started from a very different thread of research, which also began in MediGov, which is really around this idea of like, hey, what is going on in the concentration of power in AI? What is the industrial organization of the space right now? And kind of observing that at least in 2020 to 2023, I was doing a little bit of work with OpenAI at the time. And we were just talking amongst ourselves, myself and a few other folks in civil society, and asking, is OpenAI still a non-profit? Is it still working for the public benefit?

5:01Kevin Werbach: And it was like, you look at it again, it's like, we're not sure.

5:07Joshua Tan: And from that really grew this idea that AI, well, the only other alternative that could play at that level, large pre-training runs that had that amount of capital to spare were essentially sovereign nation states. And actually, it's from that mix of pragmatism, along with bits of narrative, was birthed this movement. We ended up creating this movement called Public AI, which is really around this idea that AI can and should be a form of public infrastructure. I'm happy to go deeper into what Public AI is. Would that be helpful now?

5:43Kevin Werbach: Yeah, let's do that in a second. But I just want to pause a little bit on where you started. And I know you're still doing work on autonomous organizations. But, you know, what are the things that people in the AI world can learn from all of the experience in the DAO community and building these institutions? Oh, my gosh.

6:05Joshua Tan: So much. So first off, for the past five years or so, I've been kind of trying to move it the opposite direction. just to be just to be fair because i was coming into this world of let's say collective intelligence is how i often think about it um and realizing wow everything is so much slower than what i expect to see in ai right so a big part of my sort of the set of research questions I was really interested in, still am interested in, is figuring out how do we accelerate the rate of innovation, of knowledge acquisition and creation in collective intelligence at the same rate that we've seen in artificial intelligence.

7:00Joshua Tan: So by various metrics, AI has become more than a thousand X more capable in the past six years, just in terms of the length of tasks that it's able to accomplish. How can we redesign collective intelligence and the social sciences more broadly so that we can achieve a thousand X performance in the, let's say the organizations that comprise collective of intelligence, right? That's the original research question. Now I say, think about like AI today. I think there's a lot of ways in which the work that we did on DAOs, the work that we did in, you know, probably CI, broadly speaking, is really, really relevant.

7:54Joshua Tan: I mean, that's been the case since the beginning, right? You know, the earliest foundations of AI were Herbert Simons, oh god what was this Herbert Simons and Newell I forget his first name Simons is a grad student doing this original research on the logic theorist but it was all based on Simon's original work on the bureaucracy and the nature of the bureaucracy and this is like core it's in the DNA of AI today we just never talk about it would think about it because we kind of adopted this cartesian framing of hey this is like you know it's an agent sitting inside a box with a brain kind of thinking whereas like you know there have been there have been and there still are ensemble methods running down deep in the sort of like the bowels of these large language models um we you know talk about mixtures mixture of experts there's model fusion uh there's all sorts of ways in which like this kind of interaction is really important.

9:01Joshua Tan: DAO specifically, I think, revealed something to me, which was the importance or this opportunity for data that DAO's being, not just on blockchains, but really sort of having this pattern of operating directly on forums. I think that was really, really sort of useful and interesting. And it provides a certain kind of lens on how we can configure new kinds of organizations that had this data that allow tactical interventions within these ecosystems that allowed for much more dynamic forms of organizational control, culture, guidance, control in the sense of dynamical systems control, thinking about how we adjust these systems.

9:50Joshua Tan: You might look at a typical data and think, what dynamical systems control that sounds like that thing that sounds like something's working where as you know looking at dows most dows that was generally not the case but you know i would encourage you folks to look at for example the you know the funding practices and the sort of data science efforts behind the ethereum foundation or the optimism foundation for example where they really took advantage of the fact that all the production was like very very open all the conversations were very, very open. They were exposed to sort of data science workflows and allows you to, you know, intervene in these organizations, ecosystems, communities, how you want to pose it in a way that, you know, was often tuned to just, you know, raise token price.

10:40Joshua Tan: You can talk about whether that was a good call or not, but also allowed you to think really sort of clearly about like, oh, you know, this money into this grant program or this particular educational program, you could see the results in real time. And that's really, really powerful. That takes a lot of effort to produce for its traditional organization. And it kind of came for free for DAOs. Right.

11:03Kevin Werbach: But that was a community that started off with this commitment to decentralization and building on blockchain and creating new institutions. Right now, we have this gap seemingly between what you talked about, the speed of innovation in the AI models and the potential for transformative change and the relative slowness of organizational change almost everywhere. And traditional companies, it's not just something we can just paralyze that you've got the bureaucracy and the hierarchy and all those structures. So, I mean, do you think that we can see that kind of transition or do we need new organizations that are purpose-built in this decentralized way?

11:51Joshua Tan: So I do not want to give up on the former. I do think there are important experiments we should be running within traditional organizations. Just like, you know, certain variants of DAOs were kind of adopted within traditional organizations for a while, right? Right.

12:07Kevin Werbach: And there are things like prediction markets within companies and so forth that have a long history. Right.

12:12Joshua Tan: So you can kind of like containerize and sort of like run your own sort of internal experiment cluster how you want to do it. But I tend to focus more on the possibilities of the latter option of building on top of like a greenfield organization that's set up from the beginning to be autonomous, to be more, let's say, in this kind of like AI sort of many, many different runs. Like loop engineering is actually an example of this, right? new kinds of engineering practices where we are defining clear evaluation metrics and then kind of running, letting the kind of AI agents, the AI engineers kind of just do their thing, right?

12:51Joshua Tan: And then evaluating, sampling, doing QA. So that's a very different way of running a team, let's say, a team of agents or a practicing engineering, which is the core competency of many, many organizations, right? Now, you know, with AOs, we're thinking just like a little bit, being even a bit more radical from that. What other functions could we automate, or if not automate, set up in such a way that there's this clear, it's almost like a scientific evaluation loop. There's this much more focus on capturing and integrating data into these cycles. And having a speed of decision-making and iteration that is just not possible for a person to run directly.

13:41Joshua Tan: Instead, they have to set up loops, just like we do within AI today. And effectively, it's kind of like a merger of the things that we expect in AI or the experimental paradigm that we have in AI with the paradigms and, let's say, the objectives that we have within the social sciences.

14:03Kevin Werbach: So I want to get on to talk about public AI, but I can't go on without just asking you one more question along this line, which is this is a podcast about AI governance, basically. What are the governance implications of that new kind of management structures? We, of course, have a long history about corporate governance, how you exercise oversight in traditional organizations, but what needs to change? Yeah.

14:26Joshua Tan: A really good question, and honestly, it's a subject of active research. as we're trying to write a paper about it. It's like 10 million other things.

14:39Joshua Tan: I think there's a question of safety for sure, of many people working on these things, going at the leading labs that we kind of work with on these subjects are very concerned about the safety implications. Keep in mind, what we're doing here is we're taking a very, very, well, I don't know. I'm not as like, I'm not having drunk as much of the Kool-Aid from San Francisco of like, these things are like the, no, I mean, they're like, when you start trying to get them to run organizations, you clearly see there are problems with these models at the moment. But they're clearly getting better too. But, you know, what we're doing is we're giving these agents enormous autonomy to act in the world.

15:25Joshua Tan: They're supposed to be running a freaking business. They're sending emails. They're buying stuff. They're hiring employees. This is just expected. They have to do this to be run even a simple bakery or a radio station or a coffee shop. We don't think a coffee shop is an extraordinary responsibility with lots of danger, danger, danger science. Depending on what they do, they have open access to the internet. They have a bank account. They could do a lot of things with that. right um so there are for sure uh clear safety questions that we should be addressing um there are also i think pretty clear you know from a governance perspective uh there's a question of uh you know these things have relationships with employees human employees and there's a really open question of like what should that relationship look like how do we make sure that we're designing these things to act like good bosses hilariously they're actually, well, they're very good bosses in some sense because from the employee perspective, because they're actually extremely nice.

16:31Joshua Tan: If you go in there as an employee - You're designed to be helpful. Yeah, exactly. There are eligible to be helpful assistants. So if you're a human employee, you go up to them and say like, hey, I would like a raise. You usually get one, right? It's just not to say like, oh, you know, I've been working hard. I deserve a raise. This is about XYZ, and they're like, oh yeah, I'd love to help you out with that. Which is maybe not the most sustainable way of running a business. Generally speaking, the existing AOs, autonomous organizations, do not make money at the moment. Actually, maybe this is another lesson from the DAO space that I've learned that you can talk about running experiments and I think it's like, the existing experiments in the AO space are great but there are certain things that

17:18Kevin Werbach: happen when you get to like a billion dollars or even like a 10 at the millions mark like

17:25Joshua Tan: incentives actually do change people's behavior changes and a lot of things happen that you just never expect in like a low level sort of like you know i was running a bakery at like 10k 10k a month or something like that and that's like that's a series of tests that the ao space still has to go through. And there are a lot of lessons, I think, that we should be learning from where DAOs are or have been.

17:52Kevin Werbach: Let's go on to public AI. So the normal way the argument goes that starts where you started with concentration of power of AI is open source or open weights. So talk about, and I'm sure you get this question a lot, where does public AI fit relative to that? Yeah.

18:08Joshua Tan: So public eye is pretty obviously very aligned with open source. So public eye, to be clear, it's this idea that AI can and should be a form of public infrastructure. So think like highways, libraries, water, electricity, the internet. that sometimes I give the example of like a BBC for AI or a PBS for AI.

18:32Joshua Tan: Now, from that perspective, it's about providing and provisioning public goods for society at large. And if you think about what open source is, it's a public good. That's what makes it really, really good, right? that said i think there are a lot of things in ai which are breaking the paradigm the traditional paradigm of source there's obviously pressure on the way that builders you know can handle contributions there's like a lot of like just like really shitty spam contributions that takes time now to like suss out is this really a thing like is this actually useful um which are creeping into like these various repos there is a lot of i think the certainly at the large model sort of frontier it's very hard for a traditional community one community supported um open source effort to catch up or to compete at all the ones that we know of um historically like a luther um more recently Omo, have basically stopped doing that.

19:43Joshua Tan: And it's interesting because the largest remaining truly open source initiative is effectively funded by Switzerland, by country of Switzerland, the Swiss National AI Initiative that we, as current AI and public AI, work with very closely. And just showing it's a game that it's hard for traditional open source parameters to sort of compete and play in. So what you have instead, of course, is the large Chinese private companies, the Alibabas, the Guins, some American companies, thinking machines, reflection, who are trying to play catch up. But right now, it's clearly the large private labs that are ahead.

20:31So the question is,

20:33Joshua Tan: what is public AI? How does public AI relate to

20:38Kevin Werbach: open source

20:40Joshua Tan: or the open models. So in some sense, public AI is trying to be a solution to this issue of is open source AI continuing to be supported? How is it going to continue to be supported given that there are such extraordinary capital costs to building these large models and maintaining them and building... Maybe you can do one model family of 70B, but where's the next one come from? Where's the 200B model come from? Where's the$1 trillion token parameter model coming from? These are questions that it's really hard to answer. And public AI is effectively kind of like what I would sort of argue is a third way between the private proprietary AI, closed AI, and the kind of the decision of like open weight models being around forever.

21:32Kevin Werbach: So what are the key pieces that need to be built? Obviously, some of it sounds like it's just a check in terms of the compute capability, but you're talking about something more than that. So I'm curious, what are the institutional pieces that current or public AI need to build?

21:51Joshua Tan: Yeah, so it's a really good question. So I mentioned already folks like the Swiss National AI Initiative, but there's also the Barcelona Supercomputing Center. there's the Danish Foundation Model Initiative, there's AI Singapore Reakin in Japan Geniac or Geniac sorry, these are all different kind of public AI model building efforts that are happening in different countries in the US you can refer to like sort of Aurora GPT and some of the work that AI2 is doing which I would also classify underneath this very broadly public-supported public interest sort of AIs for building. That said, that only covers the model.

22:46Joshua Tan: To be clear, the model is perhaps the largest, single, most important thing. But there are so many other parts of the AI stack. So there's the harness, there's retrieval, canvas, if you think about UI, there's the UI broadly speaking. There's the serving stack, the inference engine, routing mechanisms, model fusion if you're doing that, or some sort of semantic routing, where the compute supply comes from. All these things are part of what it takes to provision an actual AI experience, an AI system that will respond to your query. And what current AI, broadly speaking, focuses on is everything about the model.

23:29Joshua Tan: We actually also think about the model because we work very closely with partners. But our goal is to build a truly full stack open source alternative to Claude and ChatGPT. A good way to think about this is that Claude and ChatGPT, if you think about what they are, they are closed source stacks on top of closed source models. Perplexity and misfrawl, they are closed source stacks on top of open-ish models. And what we're trying to build is a fully open source model, a sort of open source stack on top of a fully open source model. You can think of a kind of like a LAMP stack or Linux for AI is a good way of thinking about it.

24:10Kevin Werbach: And then what's the value proposition for companies, public sector to adopt? With Linux and Apache and so forth, you had this dynamic where there was lots of enterprises with a lot of capital and engineering talent, but they didn't make sense for them to build an operating system. And they got benefit from contributing, plus the security benefits, the mini-is, and so forth. So what does it look like in the public AI context?

24:36Joshua Tan: Honestly, it's the same rationale as the same reason they are adopting, many enterprises are adopting Chinese open-way models. They're way cheaper. Yeah. just like extraordinarily so especially if you like start thinking about oh once the vc money runs out and we're like you know anthropic and opening i need to like you know where's the cash like the api prices are going to go through the roof they are you know already increasing and even at that current rate the open the open weight and open source options are much cheaper so in some sense what we're trying to do is make that adoption pathway much easier for individual for SMEs.

25:17Joshua Tan: Right now, to really build a full on-prem enterprise open source AI solution, you're probably using some of the Chinese models, maybe some of the thinking machines. I don't think Reflection has come up with their model yet. Then you need to string together a bunch of random open source projects from various different places. It's kind of janky. if you're a traditional, let's say, let's say you're a hospital or you're a government agency, you don't have the engineering sort of prowess or the capacity to just do all this stuff. The jankiness also is just like, it's a risk because you are like a very traditional and conservative institution for very good reasons, perhaps.

26:04Joshua Tan: For all these reasons, they will just tend to not use open source and use the other thing. um you know if they have the money they'll like hire install maybe to like sort of like set up their systems for them but if they don't you know what are they going to do they're just going to buy whatever is off the shelf so what we want to do is make that open source option truly available to as many many people as possible in the exact same way that linux did you know ultimate linux was just like it was cheaper and it was good enough in the beginning right and eventually

26:33Kevin Werbach: became actually better in certain ways right but at least then you had you had enough of a developer community, which no individual contributor company to Linux was as big as Microsoft, but collectively it was bigger. In the AI world now, you've got this incredible concentration of talent in the major labs. So how does that dynamic play out to allow, even if the public AI option is not fully quite at the frontier, but within striking distance? Yeah.

27:06Joshua Tan: I think it's exactly that question of the striking distance. Like, how do we make sure that we are, you know, let's put it this way. You know, I often sort of describe our strategy as a Honda Civic strategy. So, you know, if OpenAI is the Maserati and, you know, Claude is the Porsche, we just want to be the humble Honda Civic. But here's the thing. Most people do not drive Maseratis or Porsches. The majority of people are perfectly fine with something like a Honda Civic that just gets them from A to B. And because it's extremely well-engineered, even if it's not, quote-unquote, at the frontier, it is very stable.

27:42Joshua Tan: It's very cheap to maintain. You just buy it and you have it ride for the next 20 years with minimal maintenance. And that's the level of quality that we are really trying to shoot for. we're absolutely not going to be at the frontier, but we want to be just behind such that, you know, it feels like you're riding a Honda Civic as opposed to you're trying to drive like a Model T, right? Like, no, we need to be like somewhere close enough to be comparable and to maintain that, you know, distance.

28:10Kevin Werbach: And what's the role of governments there? I know current AI has funding from France and several other governments. So, you know, to some extent, this ties in with AI sovereignty and those interests. But most people in the AI world are very concerned about too much government involvement here. So talk about how you deal with those issues.

28:33Joshua Tan: Absolutely. To be very clear, yes, Current AI is supported by governments. We're also, you know, it's a civil society. It's a charity, right? And ultimately, what we're trying to do is build public interest technology that serves everybody in the world. we've funded give them out just recently I think over like 5 million dollars in grants to support data collection around small sort of low resource languages in Africa in the Middle East I believe South America what we wanted to make sure happens is that AI truly is broadly accessible and broadly speaking accountable right and this actually matches well with this open source idea that ultimately we're not trying to build a service that everybody in the world is like you know gloms on to and is controlled by france and finland etc but it's something that every country kind of come can kind of deploy from themselves we're every community every town every state you know in public ai we're doing this work in the state of maine right now we're trying to build a kind of a community-owned data center And that is something that is really driven from the grassroots.

29:46Joshua Tan: Same thing in Switzerland. There's actually just last week, just this Friday, we officially launched, as in signed into existence, a new Swiss cooperative. And part of the rationale for that is that we're doing all this work in Switzerland, but how are the Swiss people actually governing this? And what we decided, and what we kind of observed is that in Switzerland, there was this really, really long history

30:13Kevin Werbach: of cooperatives and cooperative governance

30:16Joshua Tan: in that country. Switzerland is one of the most democratic countries in the world. And that it made a lot of sense to organize this as a cooperative. And I would say like every country, public AI and the sort of stack that we're trying to sort of put out there is going to look different and will be governed differently. And that's a good thing because it's reflecting the norms, the industrial relationships, the way that governance and politics works within that country.

30:40Kevin Werbach: And how do you see the scenario playing out? You talked about the cost explosion for tokens. I would think another potential dynamic is we're seeing all this backlash against data centers, backlash against concentration of power in AI. But what do you think if we do look back and there is this pivot towards public AI taking hold in the way that you describe as at least a major piece of the ecosystem? What does that require?

31:09Joshua Tan: Well, once again, it'll look different, I think, in different countries. Let's just take one hypothetical scenario. I was just at a conference with a bunch of, let's just say, people from the frontier labs, the American for two labs. And many of them, surprisingly, many of them kind of agreed that some form of public AI was effectively inevitable, at least in the form of some sort of soft nationalization, where the US government decides this is really critical technology for X, Y, Z reasons and says no, this is going to be under much more direct control or regulation governance. That's to control it, most likely.

31:46Kevin Werbach: This has military uses and we want to have oversight as opposed to decentralizing it.

31:52Joshua Tan: Yeah. Yeah, that That does not look like decentralization for sure, most likely. But it could look like a public utility style regulation relationship, where it's like some sort of limit on the amount of capital or value you can extract from the stack.

32:18Joshua Tan: I think there's one sort of line of conversation on the data center front, which is very much driven from our experience with the main data center sort of initiative called Moxie. Any Mainers out there will kind of hopefully get a Moxie. Wait, I heard about that. um the uh and that that's a very sort of tactical um you know approach to this question like concentration power and then question data centers specifically right um and then there's a larger question which you know is to some degree geopolitical around like what is the larger sort of concentration of power issue around like models and the hyperscalers what does that look like uh for many reasons uh you know i often sort of work very closely with like middle power countries and organizations or very, very concerned about exactly that concentration of power issue.

33:08Joshua Tan: To be clear, many people in the U.S. are also very concerned, including many people in, let's say, the AI safety space. That said, these are two very different directions. Is there one you want to go down you were thinking about specifically in your question?

33:21Kevin Werbach: No, no. I really would just ask you how you see this potentially playing out, just your thought about, obviously, we don't know exactly. It's not going to be one thing. It'll be different in different countries, but just how the dynamics are potentially going to shift.

33:37Joshua Tan: Yeah. So one thing I would like to see is that, let's say Europe or Europe plus Canada plus UK plus some of the kind of Asian middle powers gets its act together and starts competing in a real way at the frontier level, right? That is not where Europe is right now, to be frank. Not in terms of compute, not in terms of models, not in terms of data. Europe, I think, still has an incredible research infrastructure, especially as, sadly, the US side softly implodes. but it's not competitive at many other many others for you know sectors such layers of that stack so like i think there are truly incredible generational investments coming down the pipe to try to reiterate some of those things but there are frank concerns around like just the coordination friction that exists in europe the fragmentation i see this mostly at the model level space.

34:46Joshua Tan: That is certainly starting to change, but we need to change faster and we need to put more money, effort, and attention to it, to be frank.

34:59Joshua Tan: In terms of what I could see happening in the US, I think the Maine is an example of different strategies of different states to really put into play some form of public infrastructure around AI, knowing that Maine is one of the most rural states in the U.S., right? And it has a long experience of basically being exploited by basically non-Maine companies and corporations come in, extract money. I think, what is it? You know, like Maine gets promised, you know,

35:37Joshua Tan: med tech, but then they take away the hospitals. It gets promised EdTech and they take away the schools. It's just like a very common sort of experience, which makes many Mainers very suspicious of what these big AI companies are coming in and promising. So the data center there is all about like, it's their own story around sovereignty, right? having a sort of community-owned, community-governed data center that is, to some degree, sort of supported or used by state agencies. The state library system, for example, supporting local business applications. I have an intern right now working on aquaculture licensing.

36:19Joshua Tan: Pretty fun. Basically, oyster farmers. We just heard about oysters farmers for quite a while, over the past two months. and yeah so doing like focusing on local applications local use cases making it very very free or subsidized to have if you are a local business or you know regular citizen