Can We Have AI Progress and AI Safety at the Same Time?

0:00Today on the AI Breakdown, we're looking at some actual policy proposals on how to have AI progress with safety and democratic participation. Before that, on the brief, new reports that Apple is spending a billion dollars a year to finally catch up on generative AI. The AI Breakdown is a daily podcast and video about the most important news and discussions in AI. Go to breakdown.network for more information about our YouTube channel, our Discord, and our newsletter. Welcome back to the AI Breakdown Brief, all the AI headline news you need in around five minutes. Today we begin yet again with more leaks or at least internal reporting from Apple around what's going on with their AI strategy.

0:41Apple's lack of presence in the generative AI space has of course been one of the big themes and one of the big questions for many industry observers, but recently we've been getting a few more hints about what may be coming down the pipeline. Over the summer, the information had a number of reports, including one that suggested that Apple was now spending millions of dollars per day training AI models. and we also got a little bit of information about what those AI models actually looked like internally. The TLDR on all that is that it seems like Apple had trained its own model and was experimenting with it, but that A, it didn't feel it was really differentiated from other products in the market, and B, they didn't know exactly how to integrate it with their devices.

1:16One of the big barriers for Apple is of course that they have a very privacy-first approach to data, and so tend to like running software on-premise on devices in other words, rather than in the cloud, which can be challenging with the size of these large language models. Well, at the end of last week, reports from people who watch Apple's supply chain suggested that we might start to see AI integrated into Apple products by the end of 2024. And over the weekend, we got an even more authoritative report from Bloomberg. The piece was called Inside Apple's Big Plan to Bring Generative AI to All Its Devices.

1:45So a couple interesting things from this piece. First of all, it directly counteracts the narrative that Apple has been trying to push, that they've been working on this stuff for years, and that they're just doing their normal Apple thing of being best rather than first. As author Mike Gurman writes, CEO Tim Cook says that Apple has been working on generative AI technology for years, but I can tell you in no uncertain terms that Apple executives were caught off guard by the industry's sudden AI fever and have been scrambling since late last year to make up for lost time. A person with knowledge of the matter told him, there's a lot of anxiety about this and it's considered a pretty big miss internally.

2:18Now, I don't think that anyone who's listening to this show would have really thought otherwise. I think widely speaking, our assumption has been that Apple just hasn't quite known exactly what to do with this space, even if many of us out there have confidence that when they do figure it out, it will still be a major mainstreaming moment given their track record of strong consumerization of new technologies. Now, the people who are leading Apple's AI efforts internally are two SVPs focused on AI and software engineering, John Gianandria and Craig Federighi, and they're also apparently joined by Eddie Q, who is the head of services.

2:47One splashy note from this report is that the group is spending around a billion dollars a year now on this effort. The report also gives some information about where AI might find its way into the Apple system. And again, it's not necessarily super surprising, but is confirmation of what many people assumed. Bloomberg writes, Gianandria is overseeing development of the underlying technology for a new AI system, and his team are revamping Siri in a way that will deeply implement it. What's more, it sounds like this development may be ready for implementation as soon as next year. This is felt to many like the lowest hanging fruit and most obvious place for Apple to make a bigger AI push, given that Siri as a personal assistant is already so central to the iOS operating system.

3:24Now on top of that, the software engineering group is also starting to work to bring AI into the next version of iOS, which is of course what MacRumors and others had reported at the end of last week. Writes Bloomberg, There's an edict to fill it with features running on the company's LLM, which uses a flood of data to hone AI capabilities. The new feature should improve how both Siri and the Messages app can field questions in autocomplete sentences. That same software engineering team is also trying to bring generative AI into developer tools in and around the Apple ecosystem. And it sounds like Q, who again is the head of services mandate, is to quote, add AI to as many apps as possible.

3:57That means things like auto-generated playlists in Apple Music, as well as integrations into productivity apps. Think auto-created slide decks in Keynote or helping people write in pages. Finally, they're also looking to bring generative AI into customer service. Now, what's pretty stunning about this, and I'm sure that many of you are thinking this right now, is just how table stakes all of this is. Microsoft's Office Suite already has AI up and down it. Google's Workspace just had BARD integrated as well. No one's going to be impressed when Apple comes out with Pages Writing Assistance or Slide Deck Help and Keynote.

4:29Those features are going to be more than a year old and totally standard by the time that Apple gets there. Same with things like auto-generated playlists in Apple Music. Spotify just feels light years ahead of that. When it comes to developer tools, these are some of the front lines for the very vanguard of AI. So I can't imagine that Apple's going to get a lot of traction there. In other words, at best, they keep parity with other players, which really leaves Siri as the big opportunity, given that no one has really nailed that personal assistant experience yet. Now, the piece also confirms what we've already known, in that this debate around how to deploy generative AI is a major barrier for them.

5:01Is it going to be completely on device, or is it going to be some hybrid approach? Bloomberg, for their part, is betting on the hybrid. Anyways, I think to sum this up, and obviously I took a little bit longer than I normally do on a brief, but I think it's worthwhile given just how big this question of Apple's role in the place looms large over the entire space. Frankly, it feels to me like they have their work cut out for them. I guess that maybe at the end of the day, Apple's bet is that device integration, more than state-of-the-art AI capacities themselves, are going to be the thing that wins.

5:28And so all that matters is deeply integrating these tools across the suite of applications that people are already using on their iOS devices. and who knows, that might work. But it is strange to see Apple so absolutely far behind, and at least as of now, seemingly bereft of novel innovation. Of course, they could surprise us, these are just insider-type reports, but from where I'm sitting, Apple's playbook looks tough from here. Now, speaking of insider reports and leaks, let's hop over to the world of chips. The latest Qualcomm Snapdragon chip, the Snapdragon 8 Gen 3, is going to be announced at the end of this week and is likely to end up in the Samsung Galaxy S24.

6:06And surprise, surprise, as The Verge puts it, it's full of AI features. So this is obviously an interesting parallel with the Apple iOS story, given that we're talking about Samsung's mobile devices and how AI will come to them. But according to these leaks, the new chip will create the opportunity for a variety of AI applications to happen right on the phone, including a number of different camera tools, like the ability to remove objects from videos, generate fake backgrounds, which is something that we've seen other companies like YouTube talk about a lot, or expand areas of a photo, which of course has been a big hit on social media as those features have come to Adobe, MidJourney, and other image generators.

6:40Now perhaps more importantly, and impressively if it's true, the chip claims to be able to run AI models on-premise. The models that it says it can handle include Stable Diffusion and Meta's Lama 2. This is powered by an upgraded hexagon neural processor, which the documentation says is 98 % faster than last year's model. There's also a bunch of updates that aren't exactly about AI, but very clearly that is the main focus. These chips will be in a number of different high-end Android phones. And so in addition to putting pressure on companies like Apple and their own AI phone plans, it also puts pressure on Google.

7:12Remember, Google's Pixel is not powered by Qualcomm's chips, but by their own Tensor processors. And so even within the world of Android, there is some seeming competition. Lastly today on the brief, a twin set of articles that were right next to each other when I was searching, and I think perfectly sum up how the world is handling and interacting with AI right now. Both of them have to do with AI applications for health, and the first one is incredibly positive and exciting. It's from NPR, and the article is called, With the help of AI, cardiologists can predict who will develop AFib. Now, for those who don't know, AFib isn't a regular heart rhythm.

7:43It's very treatable, but can also have serious health consequences. Well, now a group of cardiologists have developed an algorithm to detect it up to a month before it happens. As with so many of AI's advances in the health field, it has to do with the ability of AI to identify patterns that humans just aren't able to. So there we go, there's the one positive article. But then on the flip side, from Axios, study, some AI chatbots provide racist health info. So this is research from a group of Stanford doctors who ran a set of questions through four different AI chatbots, including ChatGPT and GoogleBard, and the information had some problems.

8:16According to Axios, all four models used debunked race-based information when asked about kidney function and lung capacity, based on old incorrect tropes about black patients having different muscle masses. Said one of the professors involved in the study, there are very real-world consequences to getting this wrong that can impact health disparities. We're trying to have those tropes removed from medicine so the regurgitation of that is deeply concerning. Now of course, this is a great example of the garbage-in-garbage-out principle. When chatbots are trained on information that is out-of-date and incorrect, there can be serious problems when it comes to applying that information to new and real scenarios.

8:50Now, of course, on the balance of being able to have massive medical innovations, like the ability to diagnose problems before they've begun, versus having these challenges with chatbot usage, obviously I think the former outweighs the latter. However, that doesn't mean that the latter isn't something that has to be addressed, especially as these systems get put into practice. Anyways, I just thought it was fascinating that these two pieces showed up right next to each other, and sort of perfectly summed up how diverse the coverage of AI is today. It's challenge and opportunity, friends. Challenge and opportunity.

9:16But we will wrap it there for that slightly longer than normal brief. Next up, the main AI breakdown. Welcome back to the AI breakdown. Today, we are talking about AI safety. And obviously, if you've been following along with the show, this is a huge topic of conversation. It has moved firmly from the ranks of people debating on Twitter to something that is a major, major policy issue. Indeed, part of the reason that the conversation is increasing at the moment is that the UK's upcoming AI Safety Summit is putting a real magnifying glass on the issue. Today we're going to look at a couple of really interesting pieces around AI safety, along with a very intriguing hint from OpenAI.

9:58But we're going to start with that AI Summit, where as we discussed last week, there is some indication that the attendance may not be what Rishi Sunak and his government had hoped. Wired published a fairly scathing piece today called Britain's Big AI Summit is a Doom-obsessed Mess. UK Prime Minister Rishi Sunak's Global Summit on AI Governance will focus on extreme scenarios of algorithms causing harm. Many British AI experts would rather he focus on near-term problems. The piece writes, Doomerism is supposed to drum up support for the UK government's Global Summit on AI Governance, scheduled for November 1st and 2nd.

10:32The event is being billed as the moment that the tide turns on the specter of killer AI, a chance to start building international consensus towards mitigating that risk. But just over a week before it begins, the summit looks to be simultaneously doom-laden and underwhelming. Two sources with direct knowledge of the proposed content say that its flagship initiative will be a voluntary global register of large AI models, an essentially toothless initiative. Its ability to capture the full range of leading global AI projects would depend on the goodwill of large US and Chinese tech companies, which generally don't see eye to eye.

11:03And indeed, the piece then gets into all of the scuttlebutt around the back room, negotiations about who's actually going to be there. The piece continues, sources close to negotiations say that the US government is annoyed that the UK has invited Chinese officials, and so are some members of the UK's ruling Conservative Party. The attendee list hasn't been released, but leading companies and investors in the UK's domestic AI sector are angry they've not been invited. And other AI experts say that the government's focus on the fringe concern of AI-driven cataclysm means the event will ignore the more immediate real-world risks of the technology and all of its potential upsides.

11:36Said one Oxford University lecturer, Keegan McBride, I don't know what the UK is bringing to the table in all this. They're so narrow in their focus. Now, of course, this is something that we've talked about a lot on this show. How on the one hand of this conversation, you have the accelerationist style tech optimists who with various levels of intensity say that this technology must be allowed to proceed. And then on the other end of the spectrum, you have the people who are completely convinced that perhaps we have already sowed our own doom. Then you have a third group, however, who are also in the concern camp, but are focused on a set of issues that are being drowned out in many ways, at least in the media conversation, by the doom side of the AI safety conversation.

12:13And of course, in the true middle of all of these, you have what I would consider the average opinion of regular people, who have spent a little bit of time thinking about this, but not too much, who are pretty open to the idea that there is good and bad here, but who are not yet sure exactly what to make of all that. Now, on top of just catering to the extreme ends of this conversation, it appears that the event has also ruffled feathers by only focusing on the efforts of companies that are basically just from America or China. Said that same Oxford professor, given the focus on frontier AI, quote, only a handful of companies are doing this.

12:44They're almost all American or Chinese, and the infrastructure that you need to train these sort of models are basically all owned by American companies like Amazon or Google or Microsoft. Indeed, the piece writes, Wired spoke to more than a dozen British AI experts and executives. None had been invited to the summit. The only representative of the UK AI industry known to be attending is Google DeepMind, which was founded in London but acquired by the search giant in 2014. This, as you might imagine, has not rubbed the UK AI community particularly well. Now, the piece goes on and on with all sorts of other things that the UK might be embarrassed about here, including their apparent unwillingness or at least disinterest in investing in UK-based AI companies, as well as the fact that they haven't pushed forward with any regulations that could be models for the rest of the world.

13:23But really what it comes down to for me reading this is that any hope that the UK had of using this event as some global moment of coming togetherness to align around the challenges of AI just seems not in the cards. There's too much fracture and they haven't assembled any sort of consensus. And so while the event might be interesting or useful on its own terms, it's certainly unlikely to get some sort of meaningful political agreement that can be built upon in future sessions. So, this is the setup to this conversation around AI safety. But, of course, the UK's AI Safety Summit is not the only game in town when it comes to people trying to advance some sort of consensus thinking.

14:00In Time Magazine at the end of last week, Turing Award winner Yoshua Bengio and Daniel Privetera wrote a piece called How We Can Have AI Progress Without Sacrificing Safety or Democracy. Now, of course, these folks are no strangers to this conversation. Yoshua Bengio, along with Jeffrey Hinton has this year been one of the biggest voices saying, hey, we need to focus on these issues. And indeed, they frame the reason to have this conversation now as that upcoming AI safety summit. And so I think that one thing that the folks who are involved in that event can hang their hats on is that if nothing else, it is certainly increasing the amount of conversation and focus on the set of issues.

14:33The authors argue that there are three core values that underpin most policy proposals in AI regulation. The values are progress, safety, and democracy. Progress, of course, refers to all of the opportunity of AI. Increased scientific discovery. Help identifying or curing diseases. An unbelievable increase in human productivity. These are things that people who are excited about AI are, of course, very focused on. The second value, they say, is safety, but this is where things start to get very complicated. They discuss how even within the framework of safety, it's not exactly clear where the costs and trade-offs lie.

15:06As they write, it can make sense from a safety perspective to limit the open sourcing of AI models if they can be used for potentially dangerous purposes, like engineering a highly contagious virus. On the flip side, open source code helps to reduce concentration of power. In a world with increasingly capable AI, leaving this rapidly growing power in the hands of a few profit-driven companies could seriously endanger democratic sovereignty. Now, and of course, that brings up the third value, which is democratic participation. In other words, an answer to the question of who gets to decide where those costs and trade-offs really land.

15:34And once again, they point out that there is no real clarity here. On the one hand, AI could entrench existing power imbalances, which might lead us to want to democratize AI, but at the same time, only focusing on democratization could lead to those safety outcomes that people are very concerned about. They call this a tragic trilemma. But, and this is really where the piece pivots, this picture, they say, is not accurate. And they go on, it can lead to a dangerous, well-studied phenomenon called false polarization, in which people perceive disagreements as more profound than they actually are.

16:03So, they say, we actually don't have to choose between progress, safety, and democratic participation. We can have all three. And then, of course, they suggest four policy goals to get us there. This, they say, is their beneficial AI roadmap. So what are these four policy goals? The first, and somewhat obvious, is to invest in the innovative and beneficial uses of existing AI. Now, one point that this section makes that I think is actually strangely lost in the debate sometimes is that there's broad agreement, at least among the vast majority of participants, that for as powerful as the current models that we have are, call it Up2GPT4.

16:37These are not the shoggoth paperclip threatening models that lurk in the fears of some of the AI safety folks. So they say we should be democratizing access to this technology. That could mean making sure that little startups don't have to deal with big regulatory burdens that come from frontier models. It means things like governments existing in AI use cases that don't have as much market value, but could have important social value. Basically, this says lean into what we have here and make it as useful as possible for society. Their second policy goal is what they call boost research on trustworthy AI.

17:08The challenge they point out is that, quote, while billions of dollars are spent each year to make AI more powerful, funding for research to make AI understandable, free from bias, and safe is tiny in comparison. They echo here the call that we heard over the weekend, for some equivalent to the Intergovernmental Panel on Climate Change for AI Safety, and they basically just call for much broader and larger public and governmental participation in and around important AI safety research. Policy goal three is democratizing AI oversight. They write, Many of the world's leading AI experts now think that human-level or even more capable AI could arrive before 2030.

17:41Regardless of the exact timelines, it's clear that unelected tech leaders should not decide whether, when, how, and for what purpose such transformative AI is built. This means that while we might not want to democratize direct access to potentially destructive technology for safety reasons, we urgently need to democratize AI oversight, i.e. participation. Now they point out that some of this is happening with individual countries and groups like the EU bringing their own AI regulations online, but they also note that we need global level efforts as well, as they put it a minimal set of binding rules governing AI R &D worldwide.

18:11And they point out, for those thinking that Western antagonists like China will never sign on for that, that leading Chinese and Russian researchers have participated in statements recently about warnings around AI's extinction risk to humanity. Their fourth policy suggestion is set up procedures for monitoring and evaluating AI progress. Now in this, they're making a strong argument for something that looks to be of interest to the U.S. administration, which was considering putting this in an executive order, but is also highly controversial, what they call compute monitoring. In other words, quote, tracking globally who is using the chips needed for building AI models.

18:44As they put it, with increasingly capable AI models, we will want to know whether somebody in North Korea is currently using 20 ,000 AI chips for a huge training run to build their GPT-6. Basically, their suggestion here is a combination of surveillance of who's using compute as well as a licensing regime that restricts who is actually allowed to train next generation and even more advanced models. They also think that governments need to mandate approaches to red teaming and external evaluation before new models are released. So what to think of all of these? In many ways, I think that three of these four are sort of just stand in for the most obvious pieces of this and ultimately just getting to the real meat of the argument, which is in and around what to be done around more advanced models.

19:23I think that we should be investing in the best use cases of existing AI. I don't think there's any really good argument around that. I do like that they're at least nominally pointing out that there's broad agreement that what we have now is not dangerous, at least not in the way that people talk about AI being dangerous in the future, and that there is a lot to lean into just from where we are right now. Boosting research on trustworthy AI, I also think that this is a very good place to put government dollars. I think deploying government-level capital to redress the imbalance between the money spent on developing more capable models and the money spent on safety and alignment is a good thing to do, if not guaranteed any particular outcome or another.

19:59Democratizing AI oversight is another one where I think that many would agree with this principle, that it shouldn't just be the leaders of a handful of AI labs that get to decide when and how this technology comes to humanity. But what that means in practice is hugely messy. I'd love to see a little bit more focus on what, for example, those quote minimal set of binding rules governing AI R &D worldwide would actually look like. I think the less we talk in generalities and the more we talk in specifics at this point, the better we're going to be. Which of course gets us to this last part, setting up procedures for monitoring and evaluating AI progress.

20:30I am generally of the opinion that the vast majority of time that we spend talking about AI safety in the abstract is, at this stage, kind of wasted breath. It is absolutely inarguable, based on every survey that is done, that this has now entered American and global political consciousness as an issue that is important to people. Now, we could have a whole different debate around whether the reason that it's so important to people has to do with the propensity for media to want to latch onto the scariest headlines, but that's not really what we're talking about here. And I also think it doesn't really matter.

21:01For whatever set of reasons, people are taking this threat seriously. And yet, I find that most of the debate and conversation still stays in the realm of the abstract. Where I think it would be more profitable for us is to actually dig into proposals like this, or frankly even more detail, around what type of new surveillance, if any, are we comfortable allowing for? What are the thresholds which trigger our concern around how many chips are being used to train any given AI? What are the actual rules and licensing processes for labs that want to create or release more advanced models? What are the actual steps needed and mandated to be taken before those models are released?

21:35How many of these things can be agreed to across governmental borders? I do think it's notable, and I think we've turned a corner, that some of the leading voices in this space are now starting to get into actual policy, if not proposals, broad outlines that others can build on. But I think we need to go a lot farther into the specifics. Optimistically, maybe that is the next phase of the AI policy conversation in the US at least, but I think we'll have to wait and see. Now, one last note, a little teaser because we actually don't know exactly what it means, but Zvi Maushawicz pointed out in a recent conversation on Twitter slash X that the head of alignment at OpenAI had answered a question around how do we know reinforcement learning from human feedback won't scale, to which they said, quote, we'll have some evidence to share soon.