The AI Impact Overstatement, ByteDance’s Profit Drop, Meta Muse Agent Impresses

0:13Welcome, everyone, to The Information's TI TV. My name is Akash Pasricha. It is Tuesday, September 15th. Today on the show, we are continuing to hear some industry perspectives on whether or not it will be practical to pace the development of Frontier AI models. And if it is even a good idea at all, I'm bringing on the CEO of Handshake momentarily for a discussion about that. We'll then break down exclusive reporting from the information on ByteDance's latest financials as the company pours more into AI. We'll also get the latest on Meta's Muse Agent, which has gotten strong early reviews. And we're going to close out the show with new reporting we have on Claude Code, where developers are finding ways to use Anthropics tool with rival AI models.

1:00It's going to be a great show, so let's get right on into it. The conversation around pacing the frontier continues to reverberate through the AI sector. I want to bring in one founder whose business has increasingly become central to the AI research conversation. Garrett Lord is the co-founder of Handshake, the company behind expert data labeling platform Handshake AI. Garrett, welcome back to the show. It's great to have you here. Thanks so much for having me. So let's talk about this pacing the frontier conversation. What is your view on it? Should we be pacing? Should we be tempering? Should we full seam ahead?

1:34Where do you stand on it? I think that we need to be super thoughtful around making sure this is the right environment for researchers to make a big impact in the world, but also be thoughtful around the unintended consequences of models. And I kind of am a firm believer that the progress of AI right now is just incredible. I mean, if you look at what's happening just quarter on quarter and the impact to our organizations, the organizations we work with, it's really exciting. But it's hard to understand that exponential. And so I think that the industry needs to be thoughtful. And I'm excited.

2:06There's many loud voices talking about that today. So it's thoughtful. So does that mean full steam ahead? Does it mean pacing? What does thoughtful mean? Well, I mean, something I always struggle with is like the Twitter narrative versus like what's actually happening in white collar jobs. Like today we're launching a benchmark called Atlas, Atlas Finance, which actually is kind of a measurement of how agents do in real world finance scenarios. So we actually put an agent in a white collar finance professional's job. And although on Twitter we talk about this like takeoff event and, you know, what's going to happen with humanity and what's going to happen with white collar workers, the reality when you look at this benchmark that's released later today is like agent gets a 12 % score, like the best model, right?

2:52So these models, I think there's a huge exaggeration on the impact these models are actually having in white collar job disciplines. obviously in software engineering, we've seen this incredible productivity increase. The average software engineer is three to four times more accelerated than we were a year ago. But when you start looking at oil and gas or manufacturing or retail or finance, yes, these are helpful co-pilots, but they're not actually able to do the jobs of humans. And so I think my perspective would be that the impact of the overall economy is being exaggerated from at least today's perspective as to what this looks like two years from now and RSI.

3:29I mean, people can speculate, and I am a believer in the exponential, but the current state of models today and where they'll be at in the end of the year is, I think, being dramatically overstated. And so I hear you. So you're basically saying that the level of AI that we need for enterprises and everyday people to see the value, I mean, it's not anywhere near the strength of model that would approach RSI or something like that. So we're sort of talking about two different buckets of AI here, I guess, in some sense. What's your view, though, on the security threats of, you know, having a model that could go broke?

4:08You know, the idea that it might not be an enterprise using it, but I don't know, some version of Astra, you know, who knows what could happen? How do you think about the security threats here? Yeah, I mean, we have, I think, one of the industry-leading safety team. We're super proud to work with many of the Frontier Labs to make sure their models stay aligned to human preferences. And yeah, I mean, we've had to do a ton of work around agent sandboxing and making sure that agents are actually, you know, interacting with the world in a way that's aligned to professionals and, you know, corporations' interests.

4:39I think that as agents start to touch enterprise workflows, governance, and policy models that align those agents to actually how large companies want their data to be interacted with and their consumers to be interacted with, it's going to be a huge issue. And I also believe, like, if you look at the hugging face incident, you know, it's quite scary what these models are able to do. I think that there is the reward hacking nature of agents and like, you know, how they're able to do such complex search and discovery and what's happening and discovering like zero day attacks. And I think the labs are, you know, I love the narrative around that and the focus on that.

5:14I think, but you know, there's, there's a big, I think, difference between what's happening and reward hacking and the actual economic productivity gains that agents are having today. Like when you, when you put them in white collar jobs and you give them access to the emails and the tools that real people are doing, you give them like a 15 hour task. Agents are unable to do a lot of white collar knowledge work. Most of white collar knowledge work beyond software engineering. And so I think - Why is that? Why is that? Help us out. Why is that? Why is it that we can have these incidents and then not able to do the simple tasks?

5:48Help me understand the disconnect here. I think there's a term people should understand is kind of verifiable or unverifiable, right? If a domain is verifiable, a la math, a la chess, software engineering, there's like a discrete, quantifiable, well-known outcome. Reinforced learning is extremely good at finding that answer. And when you start looking at most of knowledge work, it's like semi-verifiable or unverifiable. Like most of what you do in your job does not have like a correct right answer or the correct right answer is not written on the Internet. And if there's no correct right answer, models really struggle today and actually, you know, in reinforcement learning, which is where most of the gains are being made today in, you know, learning from what human beings are doing and learning white collar job scenarios and settings.

6:36And so a lot of what we do in our industry does is helping take the kind of unverifiable nature of white collar work and codify that into the shape of data that models can learn from. Right, right. Tell me, the conversation around open weight models, how much does that play into what you just described, which is getting, you know, we call them knowledge workers, white collar workers, everyday folks in the corporate sector or otherwise getting them to adopt it. Do open weight models, do they offer any kind of advantage here insofar as how you can tune them? Are you still seeing that traction in your own business?

7:18Yeah. I mean, we're big believers in frontier model companies, like all the major labs. I'm also a huge believer in open source. And I think large enterprises use a mixture of both, right? They're going to be constantly picking the model that's best for the job at the right cost and quality and accuracy curve that they want internally. And, you know, we're big advocates for actually starting off your agentic transformation in context on a frontier model. But we are seeing and we are working with many enterprises to actually build on top of open-wide models. And the reason that is, is because a lot of what companies want to do falls outside of distribution of what the frontier models are trained on.

7:56Like if you take a large oil and gas scenario working with a, you know, multi-hundred billion dollar public company, you know, they have a ton of legacy data. They have a ton of operating data. They have employees that have been engaging in the supply chain and logistics process for decades. And they're unable, they don't wanna give that data to the labs themselves, right? And they're not seeing frontier models actually achieve the outcomes they want, the ROI they want internally. And so that's where many enterprises, they're leveraging. I think the future is like an ensemble of models. You're leveraging, you're riding the frontier and all the capex that's being spent, picking the right model for the right job.

8:30And you're also most likely leveraging many medium-sized or small models train on your own data with your own experts internally. Right. The conversation around pacing the frontier, in all the Twitter discourse that you've been following very closely to, I gather, I mean, there's sort of the one perspective that pacing the frontier is just a defense against competition from open weight models. Do you see that to be the case? Yeah.

9:08I think pacing the frontier is an extremely important topic. And I think that the industry needs to be laser focused on making sure that these models are aligned to humans and are aligned to the way we want them to interact in the world. I also think, and I'm like all for focusing on that in all the ways that people are talking about today. I also would say that it is a highly convenient narrative to always be focusing on kind of the fear of what models can do and distracting from the actual progress these models are making in the real world. Like, I'm talking to CIOs and CEOs of Fortune 500 companies all the time, like, trying to get ROI out of these frontier models.

9:55And outside of software engineering, we are struggling to find companies that are truly, like, agentifying end-end processes and seeing the ROI today. Their token costs are skyrocketing. Their employees are using these tools. And yes, there are productivity gains, but we're not seeing this like takeoff two to three to four X increase in employee productivity in end to end processes. And CEOs, I think like this is the 2026 was the year of like getting people to use these tools. And 2027, I think is going to be the year of like reconciling the cost and money they're spending on these models and the actual ROI they're seeing.

10:30And so I think there's kind of both conversations that need to take place. It's like, how do we pace the frontier? How do we make sure this align human preferences? But at the same time, in order to see trillions of dollars spent or hundreds of billions of dollars spent on tokens, we're going to need to drive far more economic progress in all our job disciplines and science disciplines in order to see AI truly play out the way we want it to. And so, Garrett, what is the solution to that? Is this, I mean, the models, the strength of the model is one thing, fine. What are the other levers that need to be pulled to get that ROI that you're hoping to see for white-collar workers?

11:08The direction of travel for all modeling companies, including open source, is like you're scaling compute against more data. So we're obviously compute constrained today. Like we're power constrained. Like I think the world's going to need far more power and far more compute than, you know, on our wildest dreams. I'm a big believer in what these models will do in the world. and they also need, you know, way more data. Like the direction of travel is models are getting bigger, more compute's necessary, and they need more data. And the reality is like we've scraped and kind of asymptoted out the pre-training distribution.

11:38Pre-training is like the scrape of the internet, the common ground. Like we've kind of gotten it all. And now the next decade is going to be really focused on like collecting what's in private companies and what's in people's heads into rewards and the shape of data that models can learn from. Right, great. Well, Garrett, it was a great conversation. I want to thank you for coming on. That is Garrett Lord, the co-founder and CEO of Handshake here on TI-TV. The information has exclusive reporting that ByteDance's net profit is dropping as the company continues to spend more on AI. Asia Bureau Chief Jing Yang and reporter Juro Osawa wrote the story.

12:16Here is a clip from Juro breaking it all down. ByteDance's net profit in the first half of this year declined to$20 billion. This is mainly because of AI investments. The huge cost of AI is a universal challenge for all the tech giants, and ByteDance is no exception. ByteDance develops its own AI models and operates China's most popular chatbot app. It's also expanding its AI cloud business to sell its models to enterprise customers. And we reported in May that ByteDance is developing chips for running AI models. All of these efforts require a lot of investment. So that's why its profit declined by a single-digit percentage.

13:02Good news is that ByteDance's revenue grew 30 % to$120 billion, thanks to international revenue from TikTok. This makes ByteDance the biggest tech company in China by revenue, bigger than Alibaba or Tencent or Huawei. To put this in perspective, ByteDance's revenue is about the same as Meta, which is TikTok's biggest social media competitor. Meta's revenue in the first half was$117 billion. But there's a huge valuation gap between those two companies. Meta's market cap is about$1.7 trillion, but ByteDance's valuation is estimated at$630 billion. And what's driving ByteDance's revenue growth is TikTok.

13:50TikTok is delivering strong growth after a deal in January with the U.S. government. ByteDance sold 80 % of TikTok's U.S. data security unit in that deal, and it helped eliminate the concerns about a TikTok ban and brought some stability to the business. In the long run, ByteDance is hoping that AI businesses will drive its growth. Its model sales through APIs are growing, and customer demand for its AI video generation models called C-Dance is very strong. But competition is really intense. Chinese open-source models from Moonshot, DeepSeek, and others are reshaping the global market. But ByteDance is trying to carve its own path in China's AI race by focusing on closed-source models and choosing not to rely on distillation of U.S.

14:44models like other AI companies in China do. So there's a lot to unpack from ByteDance's earnings, and we'll be watching the company's performance very closely. Meta's Muse agent has gotten some great early reviews. Sasha Kolecki is a managing partner at Creator Ventures. He published some thoughts on the tool this week that got quite a bit of traction online. I want to bring him on to share his view on all of this. Sasha, welcome back to the show. It's great to have you here. Thanks for having me. It's great to see you. So you've been using Muse, I take it, last week or so? I have. I've moved, I would say, the majority of my AI search volume from ChatGPT and Claude personally to Muse, yeah.

15:27And I have instinct as well. I dabble in all in GrokBot. I'm pretty easy with them. But beyond search, I mean, what is it actually doing for you on the agentic side? yeah so the basically big breakthrough in these new tools kind of the first one to launch that really really worked was instinct that was after grog bob which is a kind of bit more expensive not quite as good for consumers uh is that it it can do tasks for you as you said so it can for example if you want it to book a ticket to the cinema for you it can instead of just showing you the link what chat gpt might do and telling you what you need to click or maybe doing browser use on your machine, Chattipity and Claude could do.

16:07What these guys can do is they can actually spin up a virtual browser, go to the site, you can talk through them the process as they do it, and it'll actually book you the ticket itself. Right. And it does the whole thing. So you've been using it to book tickets for you then? Well, actually, I thought about using Instinct to book Odyssey tickets. I ended up just doing it myself, the old school way. I didn't want to take any risks with the exact seat they chose. There are some things that are too important to hand off to an agent. Right, right, right. But yeah, I've been doing it for everything.

16:39I've been doing it for booking. My apartment building has an amenities area, for example, and you can use it to book them as soon as they become available so you never miss them. Right. Well, look, the reason I wanted to bring you on is because you made a post yesterday about the traction that Muse has been getting, and I I believe you called it, you know, one of the most successful consumer launches since ChatGPT. And you referenced some data in there as well. Tell me a little bit about the data that you were looking at and why you decided to make that claim. Yeah, so with, you know, with launches, you know, they're never going to get to the same DAU numbers as the big apps from day one.

17:17But what you can track is the number of downloads, which is the number of new users they're onboarding. And if you look at the number of downloads, that Muse has received in the first few days of launch, it's averaging the high tens of thousands between kind of 50 ,000, 100 ,000 downloads per day, just in the US. It's only available in the US at the moment. That's a lot of downloads. That puts it above all of the other meta apps, except Instagram. It's just below Instagram. It's about 3 ,000 below Instagram, but it's above obviously threads, WhatsApp, Facebook, et cetera. So it's a huge launch for them in terms of new downloads.

17:53and I think probably more importantly for them, the sentiment has been super, super positive. If you remember the first day that Threads launched, which also got a huge number of downloads, it was quite polarizing at the beginning. People were unsure whether it was required because Twitter exists and everything like that. This has been kind of universal love. Yeah, I'm trying to figure out why, what the difference is in the, I mean, Threads, that was launched at a time, I mean, look, I think Elon had just taken over Twitter. There was a lot of people quitting Twitter at that point. This is, I mean, this is sort of, it's a pretty open battlefield right now.

18:31It's not clear who the winner is. So is it the strength of the model itself that Muse is using? Is it the platform? Or is it really just the fact that people are eager to try out the latest thing? I think it's all of the above. You know, if you're Mark Zuckerberg, you obviously hope it's the first one versus the last one. I think what they've done, the big breakthrough they've made is they've combined a very close to the frontier, state-of-the-art LLM with incredible browser use, which is really truly state-of-the-art. And with a virtual browser, you can see, as well as this new persistence that LLMs are starting to develop, which Asia is starting to develop, which can kind of finish tasks for you and will not give up until it's done.

19:17and those three things have combined to make a really really really powerful tool um so i i this category is definitely here to stay there's no doubt that this is not a fad this is how people are going to be using their phones to a considerable extent in the future the question is whether meta will ultimately win it what about the limitations of the agent right now what can it not do for you so far yeah when there are still some websites that it can't do some people have made complaints on x that like for example they one person said that they're Actually, this was for Instinct, not Muse, but one person said their Resi account got banned because it sent so many requests to the Resi servers to try to book a restaurant that they got banned.

19:55Some other people have said that they've been blocked out of certain websites. So, you know, there are obviously limitations, but we're weeks into this kind of mega trend. Of course, it's not going to be perfect, but I don't think that changes the fact that this is going to be how people use the Internet in the future. How do people, I've seen these benchmarks now for agents that people put together. And I think Muse has been very much at the top of that list, at least over the last 24 hours or so. When you look at those benchmarks, what are the different criteria that people evaluate the agents on and rank these things?

20:35I mean, walk us through some of those assessments. Yeah, I saw a few benchmarks. I mean, it's all, everyone's just figuring out as they go. I saw somebody made a draft benchmark. I think it looked at, for example, booking flights was one thing it looked at. Booking tickets was another one. Oh, so it actually gives it specific tasks. It literally did it with specific tasks. And actually Muse and Instinct were obviously the two at the top, and they had different strengths. I think another thing that Muse did that was really smart was they actually allow you to see the browser that the agent is doing your task in, which is so satisfying because when you give a task to instinct i'm sure instinct will do this very soon um but when when you give a task to instinct and by the way i'm a big fan of instinct it's incredible product but you kind of just trust that it's doing the job you can't be sure you kind of hope back in 10 minutes with an answer the amazing thing about muse is you're actually watching it work and you can step out and step in again but it gives you such confidence that it's doing the right thing right well i mean you are obviously a power user of these tools if you think about the everyday person and the trust that they need to build for these agents, how do you ultimately think these agents are going to build trust with everyday people?

21:48Because I don't know, for a certain class of people, I'm sure it's a little freaky to watch the cursor move on its own and sort of trust that it's going to do the right thing. Like, I mean, what do these companies need to do to build trust in users? Is it just a matter of showing results? Because I don't know that it's as simple as that. Yeah, it's kind of like self-driving cars. People are nervous at first. They have their friends try it. There's no problems, no crashes, and they learn that way. I think my friend Sophie Bacalar responded that she's kind of seeing some people in her Equinox community start to use it to book classes.

22:27And I joked back that if they're starting now they're going to really need to because all the classes are going to get booked at 12.01 AM the next day. So at some point, there's almost like a bit of a tragedy in the commons or collective action problem where once everybody else is using agents to get ahead, you kind of need to, otherwise you're going to fall behind. So I think this is, again, I feel very confident this is the way the world's moving. Right. And last question for you, I mean, um walk me through the security precautions that you've seen meta use to perhaps ensure that these things don't go rogue is there anything visible on the surface that you've experienced that you sort of thought that you know that this was a good decision given what we've seen with rogue agents yeah there's two there's two main ones the first one is they have a um a credentials vault um which is basically you store your credentials somewhere which they say that they don't see um which i think in the case of meadow is probably going to be true and i'm sure the other the other ones are taking that pretty seriously too and finally um when it goes to payments instinct originally partnered with link which is basically a stripe product that allows them to you know provably not see your payment details and then you can just prove every payment um individually um so yeah there will be some screw-ups of course this is going to be a hundred there's gonna be a hundred dau product and then hopefully you know eventually this will be a billion dau category so there's going to be screw 100 million 100 million dau oh yeah 100 million 100 million dau first then it'll be 1 billion dau eventually um got it you know there's going to be screw-ups there's going to be problems but there's there's um there's no doubt they're taking this super seriously and uh you know as a percentage of the users um that will end up being a lot of people but it's still going to be a very, very small chance of happening to you individually.

24:17Great. Well, Sasha, I want to thank you for coming on. That is Sasha Kolesky, co-founder and managing partner of Creator Ventures here on TI TV. Claude Code has gotten a lot of traction, largely because of how great anthropic coding models are. But now developers have found ways to use the tool with other companies' models, which has stirred some controversy. My colleague Alice Couture wrote about that in this week's AI agenda. I want to bring her on to share more. Alix, welcome to the show. It's great to have you back. Hi, Akash. It's great to be here. So I didn't know this, but you can actually use cloud code with models other than Anthropics' own models?

24:57Yeah, and that's actually been an ongoing problem for a very long time. Cloud codes is basically the coding agent, and then the model is like the brain, what does the reasoning. So you can use CLAT code, which is called the harness, and it's basically the structure that has the memory, the editing, and adds context to the request. And then the harness reads your request, translates it, and sends it to the model. And then the model is doing the reasoning. And you can definitely use CLAT code or codex, which is OpenAI's coding agent, with a lot of different models. And actually, ClackCode has been used with more than 360 different models over the past month.

25:46Oh, wow. Yes. And why is Anthropic allowing this? I mean, if I was... Look, they haven't really taken the brand position of being the, you know, open AI. That was a bad way to put it. They haven't taken the position of being sort of an open ecosystem player because they have closed source models. So why are they allowing other models to be used in the platform? Well, it's more complicated than that. They don't really allow it. It is not supported by Anthropic. Anthropic's model has been built with Cloud Code. So they work together and the configuration is for Cloud Code and the model to work together.

26:35So they're not really allowing it. It is definitely not supported by Anthropic, but they can't prohibit it actively. So they have already banned it for subscribers. So if you're an Anthropic subscriber, you can't do it through your subscription. However, if you want to use a router or if you want to do it in a different way, you can do it. And Anthropic doesn't ban it actively because it would be a huge commercial blow for Anthropic. because the whole community is all about being open-minded and being able to try different combinations. So Anthropic can't really prohibit it actively and publicly, but it has already done it for subscribers.

27:21So it's a little complicated. It's not prohibiting. And so are developers increasingly using Cloud Code with other models? Is there a trend here? Yes, they're definitely using it more. A router is basically a software that allows you to use different models and to route your requests to the cheapest models. And developers are increasingly using routers to do that because they're just realizing that now you can achieve so much better results by combining different models with different harnesses. And it is now widespread across the community. And they're just trying a lot of different new things and different combinations to achieve better results.

28:10And so you pointed out in your column that there have been episodes where certain users have done this. They've used other models with Claude Code. And I think there have been episodes where their accounts have actually been shut down briefly by Anthropic. What has been the reaction from the AI sector to those incidents and what ended up happening for those users? Yeah, well, it gets a little technical here because there's two different ways of combining cloud code with different models. One is through the routers and one is through proxies. proxies. So proxies is basically a piece of code that allows you to connect different harnesses with models on your local server.