The quest to keep OpenAI honest

0:01Hey everyone, welcome to Decoder. I'm Alex Heath, Deputy Editor at The Verge and your Thursday episode guest host. We've covered OpenAI in several ways this summer, from how it's navigating the AI talent wars to how people are starting to feel attached to ChatGPT. Next week, we're airing a recent conversation I had with Brett Taylor, the chairman of OpenAI's board. But today, I want to zoom in on the battle that's brewing about the future of OpenAI itself. Despite being one of the most valuable companies in the world, OpenAI is still technically a non-profit. That structure was designed back in 2015 to keep investors from steering artificial intelligence in ways that could harm humanity.

0:44It's also what set the stage for the dramatic board coup in 2023 that briefly ousted Sam Altman as CEO. And now OpenAI is trying to shake this nonprofit structure so that it can raise even more money and eventually go public. There's a lot at stake here and not just for OpenAI. To unpack why it matters, today I'm joined by Catherine Bracey, CEO of TechEquity, and Orson Aylad, CEO of Latino Prosperity. They help run Eyes on OpenAI, a coalition of advocacy groups that's challenging OpenAI's attempted restructuring. As you'll hear, Catherine and Orson argue that OpenAI has enjoyed the advantages of being a nonprofit while drifting away from its mission.

1:29It's a complicated, fascinating story that cuts to the heart of how the most important AI company in the world is structured and the impacts of that structure. Here, I should note that OpenAI declined to comment on this episode specifically. Instead, I was referred to a May blog post from Sam Altman, in which he wrote that the company's proposed restructuring will still create the, quote, largest and most effective nonprofit in history. Okay, here's my conversation with Catherine Bracey and Orson Aylad from Eyes on OpenAI.

2:15Catherine and Orson, I'm so excited to have you both on because I think this is an important topic that more people should actually understand. And they may in the coming months as this all comes to a head. That said, no one from OpenAI is here in this conversation. So I also kind of want to play devil's advocate here a little bit. And so my first question for both of you is simply, why should we care about OpenAI's nonprofit status? Why does it matter? I think it might be helpful for your listeners to just be reminded of OpenAI's history and why it was founded, which is important to understand the governance structure.

2:58It was founded in order to ensure that this technology that its founders believed was imminent was going to benefit humanity and not undermine it. They had a strong understanding of the potential risks of AGI and they wanted to build something that could harness that technology for good. And that was a lofty mission. And I think a lot of us at the time were very supportive of it. They structured themselves as a nonprofit explicitly to protect that technology and that mission from the imposition of investors onto the technology. And I thought this was very telling. I just wrote a book about venture capital and interviewed Sam Altman for the book and wanted to understand his thinking behind setting OpenAI up as a nonprofit.

3:51I didn't know that he would be so explicit about doing so because he wanted to protect it from investors, but that's exactly what he told me that he knew. And he knew very well because he was running Y Combinator at the time. He knew what investors would ask of this technology. And he knew that those asks would create risks for AGI to not achieve the mission of benefiting all of humanity, but instead be focused on profit and in the focus on profits create a lot of risks for humanity in the process. So the nonprofit structure was explicitly meant to protect society. And I think the fact that since I've had that conversation with Sam, they've taken a turn away from that mission and that structure to protect the technology and protect the mission is worrying.

4:41And so Orson and I and a coalition of other organizations are trying to make sure that at least the mission itself is protected, if not the governance structure. Talk about this coalition and who makes it up. What is its mission? How did it come together? This coalition came about, you know, we were all reading the same articles. We're hoping I wanted to basically abandon its nonprofit roots and go fully private. it. Many of us were concerned. One opening eye clearly is developing technology that will have severe consequences for society, will impact a lot of things. But we were looking at it mostly from a charitable asset law, where we knew that there was precedent saying that if you do convert to a for-profit and abandon your nonprofit roots, then you have to leave behind your public assets, your charitable assets, which at the time that we came together were valued roughly around $100 billion.

5:45Today, it's about, could be$500 billion. So the coalition came together, various nonprofit organizations, economic organizations, social justice organizations, educational organizations, and also some labor organizations. And so today, it's a coalition of over 60 organizations that continues to press the Attorney General in California, who can oversee charitable mission and charitable law to ensure that OpenAI truly acts as a nonprofit and that they truly stick to their mission statement. For those who don't understand how this all works, can you give us a sense of the legal and monetary benefits that OpenAI has enjoyed so far as a nonprofit?

6:34And also, are there restrictions legally that it operates under being a nonprofit? Neither one of us is a lawyer, so should make that caveat here. But both of us are founders of nonprofits and have boards and lawyers ourselves that tell us what we can and can't do. So, I mean, I think we can speak to it from that perspective. Obviously, any nonprofit can take donations tax-free to the donor, and nonprofits also don't have to pay taxes. And then they operate under a set of rules that in exchange for that tax-exempt status, they must do things that are in service of a public mission and not to benefit a private profit motive.

7:18There's a variety of different ways that can look conflicts of interest in private environment and other things. But essentially, OpenAI, starting as a research lab, created this mission. And I think as part of this mission, which made sense at the time, they were going to build this technology that could be in service to humanity. Over the course of time, crept away from that mission. And I think at a certain point became, for all intents and purposes, a for-profit startup that was competing with other for-profit companies that were building similar technology. And they were able to do so under a nonprofit banner with all of those tax benefits in place.

7:59This is wild to me. So are you saying that, and I think just people need to understand this if this is true, that OpenAI, the hottest tech company in the world, that's about to be the most valuable private company in the world at$500 billion potentially in the coming months, is taxed exempt? The nonprofit entity is tax exempt now. The nonprofit entity, okay. Yeah, how much money they're running through the nonprofit. I don't think anybody's out now donating money to the nonprofit and, you know, getting a tax benefit on that donation. But again, not a lawyer. It's difficult because if you look at their structure today, they're interconnected with a variety of LLCs, strong connection to Microsoft.

8:40what we do know is that the nonprofit status implies a strong legal and an ethical obligation to put the public interest ahead of shareholder returns. That's what we want. And if that mission gets diluted, if they override that mission by private motives and they shift towards maximizing revenue instead of maximizing public good, then you're in violation of your nonprofit charter. We both run nonprofits. We can't get rich of our nonprofits. We are highly regulated as nonprofits. We have to put together reports, audits that are available to the public. The same thing applies for OpenAI. And we just feel that they are operating with an enormous amount of secrecy and that they're putting together the profit motives ahead of mission.

9:28It's a little bit of a charade to say that they are operating as a nonprofit in the same way that like a traditional charity. This idea that they should receive the same benefits that like a soup kitchen receives is kind of ludicrous at this point. And, you know, if I was competitor to OpenAI and I was having to follow a set of governance laws that applied to for-profit companies and OpenAI didn't, then I don't know, that would make me kind of upset. So I just think it's time to throw off that charade and stop allowing them to use this nonprofit banner. In their interest, they want all the benefits of a nonprofit status without any of the responsibility.

10:06And I think that's wrong. We have to take a short break. We'll be back in just a minute.

10:20Support for this show comes from LinkedIn. Imagine if any of the movies that included the line, I need the right person for the job, settled for, I'll just take about anyone. How many heists would have failed? How many deals would have fallen through? How many secret spy missions would have ended in disaster? So why would you accept just anyone when hiring for your business? When you need the right person for the job, you can turn to LinkedIn Jobs. And now LinkedIn Jobs is stepping things up with their new AI assistant. So you can feel confident you're finding top talent that you can't find anywhere else.

10:54With LinkedIn Jobs AI assistant, you can skip the confusing steps and recruiting jargon. It filters through applicants based on criteria you've set for your role and surfaces only the best matches. So you're not stuck sorting through a mountain of resumes. Hire right the first time. Post your job for free at linkedin.com slash partner, then promote it to use LinkedIn Jobs' new AI system, making it easier and faster to find top candidates. That's linkedin.com slash partner to post your job for free. Terms and conditions apply.

11:36Welcome back. I'm glad you brought up competitors. Competitors have actually weighed in on this issue. Meta, for example, sent a letter to the California AG's office about this. And they made the specific argument that OpenAI, quote, should not be allowed to flout the law by taking and reappropriating assets billed as a charity and using them for potentially enormous private gains. And it also made the point that this would be a, quote, seismic thing for Silicon Valley and would essentially allow a path for other companies to do the same thing, for them to start as a nonprofit with tax-exempt status and then switch later on when they decide they want to start making money.

12:18What do you guys think the second-order effects are if OpenAI gets its way here ultimately and gets to pull this off? Maybe they pay a fine, back taxes, or they have to give more of their assets to the charity than they want. But essentially, they get through all this because the train has left the station. What are the effects on other companies? You both run nonprofits. What are the effects on other nonprofits? The reason the law exists in the way that it exists is to prevent companies from doing exactly this. I mean, it is legal. If they want to transition to a for-profit, they can. But they cannot take the assets that they built as a nonprofit with them as they do so.

13:03So I suspect that they had a conversation with the attorney general about this transition and were told, I mean, I don't know, but, you know, they were told that they would have to leave a certain amount of those assets behind if they decided to make this transition. And they have since decided that, in fact, no, they're not. They're going to maintain their current status as sort of a Potemkin nonprofit that oversees the for-profit and at the same time remove the profit caps on investors and let investors do whatever they want and take the company public. And that's the kind of thing that I think is not acceptable.

13:37If they were just transitioning to a for-profit and leaving the assets behind in the charitable sector, then that would be fine. I would totally support them leaving. But by trying to take the money with them, they are making a mockery of the rule of law, which I think is one of the impacts it would have on California and the entrepreneurial environment, and mocking its competitors, creating a permission structure by which any startup could try and do the same thing. If they are allowed to do this with no consequence, no real consequence, then it will provide the blueprint for other companies to do the same thing.

14:13And the AG won't really have as much, you know, moral authority to stop that from happening. If I can add, I highly doubt that the AG would let them go on just because they might feel as though the train has left the station. And just because of the issues that Catherine raised, the precedents, right, the appearance of public interest governments to attract goodwill and resources and then just abandoning those. If they get the green light, I can't imagine how many other nonprofits would try to do the same thing. So because of that, I do think that the AEG would sue them. And we would, of course, urge the AEG, the people of California, to sue because of the precedent that this sets.

14:58At the same time, I think you both can appreciate that for them to theoretically leave charitable assets behind, I mean, that's essentially the whole company. I mean, what else is there? They've become the ChatGPT developer platform company. The charity as it exists is really nothing. I mean, the charity entity. I'm not sure what it does. The last time I saw the IRS report on it, it had negligible assets. It was like maybe tens of millions, whereas the rest of OpenAI is worth hundreds of billions. So for them to, like you were saying, Catherine, properly leave behind the charitable assets, they have to basically start over from scratch.

15:39Is that what you guys are asking them to do? Legally, that is what the law says. I think the AG has some latitude on what kind of settlement he could negotiate with them. I think he does have to balance the interests of maintaining California's position as a place where innovation can happen and also the public interest. But it is true that these assets were created with the support of the people of California. And the law says that if you make this transition, you have to leave behind assets. And so those assets, some significant number of those assets should remain in the public interest so that they can be used to advance the mission of AI benefiting all of humanity.

16:25And practically, this looks like the assets are shares of OpenAI. Like it's just straight money. I'm being kind of silly here, but it's not like the people of California will all of a sudden get governance over like chat GPT usage in California, right? We're talking about financial assets. Can I clarify one of the things that we are pushing for is, you know, essentially OpenAI has two functions as a nonprofit. One is to adhere it to its mission. And that means safeguards to ensure that AI is used for the benefit of humanity. So there's the governance piece there that we think still should continue.

17:06And there's a second piece, which is the distribution of assets, which they mention also in their mission statement. What we're arguing is that they continue to govern with that mission in mind because they have to. They've decided that they want to stay a nonprofit. but that the distribution of assets be free of conflict and that those assets be spun off to an independent entity. Nobody here thinks that that independent entity should receive the full 500 billion. I think right now the AG of Delaware and the AG of California are trying to value what the assets might be. But our goal is that there continue to be strong governance so that OpenAI adheres to, safeguards some of the things we really need in the AI sector.

17:53But then that we have an independent entity that is fairly valued and perhaps over time funded, right? It could remain connected to the OpenAI so that there are a lot of talks of OpenAI going public one day. This independent nonprofit could get shares of that. So we are asking for two things, the ongoing governance and an independent entity that will oversee the distribution of assets. Let's talk about the governance piece and the mission statement a little bit more, because I think the first time a lot of people probably realized that OpenAI was still controlled by a nonprofit was when its CEO, Sam Altman, was briefly fired by the board of directors in 2023.

18:36This ruined my week. I reported on this extensively. Very dramatic. RIP your Thanksgiving. Exactly. And that board, that same board, it's not the same. It's almost completely changed over. But legally, that same board still controls the company to this day because OpenAI, despite the headlines, and a lot of people probably think OpenAI has not been a nonprofit for the last while. It still is legally a nonprofit because it hasn't done this transition. So it's the same governance structure as what it had fundamentally when Sam was fired in 2023. There's a scene from that saga, that board coop that I remember vividly, where a bunch of OpenAI execs got in a room at headquarters and they got on the phone with a group of board members who fired Sam.

19:22And I'm paraphrasing, but at one point, one of the execs said something like, what you're doing is killing the company. You know, OpenAI will go to zero if you follow through with this and you don't bring Sam back. And one of the board members on the other end of the call said that would be perfectly in line with the mission that we have as the board. And everyone that worked at OpenAI, the quote unquote for-profit company, was just like aghast at this. Because it is a crazy thing. Because like OpenAI at that time was already a super commercial entity. It was resetting the tech industry fundamentally in the AI race.

19:55And so to hear that, wait, like we're controlled by this board that the interest of the board could mean that the company shuts down. It's a very strange concept. And I'm wanting to double click on that with you guys. Like what exactly is the governance role of the board as it exists today? And what does that mean? It means they can just shut the company down. You say they are still legally a nonprofit. I would say they're actually a nonprofit on paper. But we don't think that they are operating legally, which is sort of the point you were making. And in order for them to be operating legally as a nonprofit, yes, the board, its only commitment, its only fiduciary responsibility is to the mission of the organization.

20:38My sense from OpenAI is that they don't understand their responsibility to the mission. They do treat this like a for-profit company, and they have sort of diluted themselves into thinking that by building the most powerful, valuable for-profit entity that is ultimately in service to the nonprofit because it will deliver more financial benefits to the nonprofit that it could then do something with to further the mission. And that's actually not the point at all. I mean, as a nonprofit leader myself, my responsibility is to turn down money if it is going to affect my ability to execute against the mission.

21:19And it is the board's legal responsibility to provide oversight to ensure that I do that as the CEO. So there is no world in which maximizing the profitability of this company for shareholders, for employees, for whoever is always aligned with the mission of the nonprofit. And when the mission diverges from the profit motivation, the mission has to always override, even if that means taking the company to zero. And if they can't live with that, that's totally fine. They should spin off the for-profit and leave a significant set of assets in the charitable sector. That's all we're asking them to do.

21:57It's not really clear why they refuse to do so. And we should just be clear that the nonprofit mission is to develop AGI that benefits all of humanity. I have a lot of problems with that because it's inherently very squishy and can be defined however you want. What does benefiting mean in that context? What is all of humanity? But yeah, maybe we can talk about that. Like, does the mission need to change for the nonprofit? Is it too vague? because at the time in 2023 when Sam was fired, that board decided that firing Sam was in the benefit of AGI for all of humanity. But it's hard to get there perhaps now.

22:35It's hard to see exactly how you execute that day-to-day, I guess. We haven't weighed in much on the specifics of the mission. I think what's important is the governance and the practice align with their current mission. I think their current mission is fine. What's not fine is the conflicts of interest that we see. We asked earlier, what is the board doing now? I think that's the biggest question. We're not sure who's really in charge here, right? Is it Microsoft? Is it the investors? Is it the employees? It appears employees were given shares, right? Current and former employees were given shares of this company.

23:13And so of course they don't want to see their nonprofit go down because their shares would go down, right? But that's, again, a big reason why there's a huge conflict here is because when you're giving shares to your employees, we know that they're not going to be in service to the mission. They're going to be in service to wanting to get paid, right? It's the tech mentality, the startup mentality. And it seems like they built it that way. So I think that the mission we can live with, what we just need more answers is on the governance and the practices that align with the mission. One set of people we haven't talked about yet very much are the investors.

23:50And I just cannot reconcile this current proposed structure that they want to live under. maintaining the nonprofit ownership of the for-profit, but converting the for-profit into a basically no different from a normal startup where shareholders can have equity stakes, Sam Altman presumably can have equity stakes, the employees can have equity stakes, that they can eventually go public. And that SoftBank and Thrive are going to be okay with the nonprofit board being able to drive the company to zero. I'm sure there was a conversation at some point before the$40 billion funding round where they said, we're not going to do this unless you can guarantee that the nonprofit does not have the control to do that.

24:39And I suspect they were told that, yeah, we would make sure that the nonprofit didn't have control. And so they tried to spin out into their own entity. I think the major question is why they decided to stop going that route and to maintain their current structure with the nonprofit ownership and what assurances they've given to those investors that make them okay with maintaining the nonprofit ownership of the for-profit company. It doesn't make any sense that they would be if the nonprofit is still in a position to put the priority of the mission above the profit priority. So do they have some agreement between the shareholders and the company that the nonprofit only has control on paper and will not ever challenge the for-profit?

25:29These are questions that have not been answered. And I think that their decision not to actually spin out the for-profit into its own standalone entity raises more questions than it answers. And so that's why we are still asking the AG to investigate. We met with OpenAI's team shortly after the decision where they retreated for profit. Not only would they not answer questions, but they said they didn't have the answers to a lot of the questions. So to Catherine's point, it seems like something happened and they retreated quickly without truly knowing how this was going to play out. We have to take another quick break.

26:10We'll be right back.

26:19Support for the show comes from Rippling. If you're a business owner, here's the truth. SaaS promised to make work easier, but now the average company is buried by hundreds of apps that silo your teams, slow you down, and simply don't work together. That's not SaaS. That's SAD. Software as a disservice. That's why you need Rippling. Rippling is the unified platform for global HR, payroll, IT, and finance. They've helped millions replace their mess of cobbled-together tools with one system designed to give leaders clarity, speed, and control. By uniting your employees, teams, and departments in one system, Rippling removes the bottlenecks, busywork, and silos your software created.

27:02Automated, perfectly in sync, and seriously simple to use, Rippling gives your company one source of truth for your people, their data, and everything they touch. With Rippling, you can run your entire HR, IT, and finance operations as one. Or pick and choose the products that best fill the gaps in your software stack. And right now, you can get six months free when you go to rippling.com slash decoder. Learn more at r-i-p-p-l-i-n-g dot com slash decoder. That's rippling.com slash decoder for six months free. Terms and conditions apply.