DOGE’s AI for SEC Rule Cuts, AI Recruiting, Cursor’s Data, and Databricks Disruptor | Sep 2, 2025

3 Sep 2025 · 36 min

Ask about this episode

Ask anything about it. ChatGPT or Claude reads this page and answers with the times it was said.

Connect VO and ask about every podcast you hear, including the moments you saved. Add to ChatGPT · Add to Claude

In short

Podcast Summary: The Information's TITV - Episode on AI in SEC Regulations, Recruiting, and Disruptors

Episode Details

  • Title: DOGE’s AI for SEC Rule Cuts, AI Recruiting, Cursor’s Data, and Databricks Disruptor
  • Air Date: September 2, 2025
  • Hosts: Akash Pasricha
  • Guests:
  • Sylvia Varnham O'Regan (Washington Correspondent)
  • Brendan Foody (CEO of Mercor)
  • Marc Freed-Finnegan (CEO of Chalk)
  • Natasha Mascarenhas (Reporter)

---

Key Takeaways

  1. SEC's AI Utilization
  2. Introduction to DOGE:
  3. DOGE (the efficiency arm of the U.S. government) has been quieter since Elon Musk's departure.
  4. Focus remains on integrating AI to improve government processes.
  • SEC Collaboration with DOGE:
  • New AI tool developed by DOGE is being used to analyze SEC regulations for potential cuts.
  • The tool mimics a chatbot, scanning rules and generating a list for review.
  • Support and Criticism:
  • Support from leadership at the SEC; however, some critics argue it may lead to inefficiencies and unnecessary labor.
  1. AI in Recruiting
  2. Interview with Brendan Foody (Mercor CEO):
  3. Mercor valued at $2 billion; focused on utilizing AI for recruitment.
  4. Major trend: Transition from traditional academic datasets to professional domains for AI training.
  5. Emphasis on high-caliber talent to enrich AI development rather than low-skill contributions.
  1. Data Licensing and Cursor's Strategy
  2. Discussion with Natasha Mascarenhas:
  3. Cursor, a leading AI coding company, is in talks with major AI firms (OpenAI, Anthropic) about data licensing.
  4. Two main types of data sought: preference data and demonstration data.
  5. Cursor is cautious about sharing data that constitutes their competitive advantage.
  1. Disruption in Data Services
  2. Interview with Marc Freed-Finnegan (Chalk CEO):
  3. Chalk aims to disrupt established companies like Databricks and Snowflake by providing real-time data processing solutions.
  4. Current data services often operate on outdated batch processing, which is less efficient for real-time applications.
  5. Chalk's unique approach focuses on delivering fresh data to models at inference time, enhancing user experience and revenue.
  1. Perspectives on AI's Future
  2. Brendan Foody's Insights:
  3. Discussion on how AI's evolution will create new job opportunities, especially in physical roles, despite fears of displacement.
  4. Emphasis on the need for educational adaptations to prepare new graduates for an AI-driven job market.

---

Detailed Discussions

SEC and AI's Role

  • The conversation led by Sylvia Varnham O'Regan outlined the SEC's proactive use of AI to streamline regulatory processes, despite some skepticism regarding its efficacy.

AI Recruitment Dynamics

  • Brendan Foody discussed how AI is changing recruitment, focusing on sourcing top talent to build better AI systems.

Cursor's Data Licensing Dilemma

  • Natasha Mascarenhas highlighted the complexities Cursor faces in deciding whether to license its data, balancing potential revenue against the risk of losing competitive advantage.

Chalk's Disruption Strategy

  • Marc Freed-Finnegan detailed how Chalk is positioning itself as a game-changer in the data services space, emphasizing real-time data infrastructure over traditional batch processing.

---

Conclusion This episode of TITV provided valuable insights into the evolving landscape of AI in government, recruitment, data licensing, and competitive strategies against established firms. The discussions reflect both the potential benefits and challenges posed by AI's rapid integration across various sectors.

Additional Resources

  • [The Information's TITV](https://www.theinformation.com/titv)
  • [Subscribe to The Information](https://www.theinformation.com/subscribe_h)
  • [AI Agenda Newsletter](https://www.theinformation.com/features/ai-agenda)

---

Note: The episode is part of a daily tech news series, highlighting critical developments within the AI and tech ecosystems. Tune in for live broadcasts Monday through Friday at 10 a.m. PT / 1 p.m. ET.

Written by AI. May contain mistakes. Listen to the episode to check what was said.

Hear the part that matters, and keep it.Open this episode in VO. Double tap your headphones to save a moment as you listen.
Get VO free

Transcript

Automatic transcript. May contain errors.

0:13Welcome, everyone, to the Informations TI TV. My name is Akash Pasricha. It is Tuesday, September 2nd. Hope you had a fantastic long weekend. We are going to get right back into it today. We've got a story about how the SEC is using AI. We're going to bring on our Washington, D.C. correspondent to talk about that one. We've also got an inside look at how AI coding companies are getting offers from big model companies to license their data. And we've got two founders coming on the show. The CEO of Mercore and the CEO of Chuck are coming on today. We're going to ask Mercore about the future of AI hiring, and we're going to talk to Chalk about how it is trying to disrupt Databricks.

0:52A lot going on. Let's get right on into it. News about Doge, the efficiency arm of the U.S. government, has been much quieter since Elon Musk left a few months ago. But this week, the information published a story about how that group is using AI to target SEC regulations that could potentially be cut. and I want to bring on our Washington, D.C. correspondent, Sylvia Varnham O 'Regan, to tell us more about that. Sylvia, welcome back to the show. It's great to have you. Hey, Akash. Thanks very much. Always good to be here. So, look, I think Doge has gotten a little quieter. I mean, Musk has left now.

1:28I mean, I haven't paid too much attention to it. Where is Doge at right now? What do we need to know before we get into the details of your story? Yeah, it's a good question. I mean, I think when a lot of people think about Doge, they think about the early days when Elon Musk was there and they were doing mass layoffs across the federal government. Their tactics were very aggressive. And after Elon Musk left, the group got a little bit less exposure in the media. At the same time that Elon Musk left as well, his sort of right-hand man, Steve Davis, also left. And now Steve Davis sort of ran the day-to-day operations of Doge.

2:06So the leadership structure became a little bit more squishy and hard to pin down. So Doge is still around, though. That's the thing. And I think what people need to understand about the structure as best we can, because as I said, the leadership is quite... It's murky. I mean, it's very... It's more decentralized now, right? But a lot of these people who came into Doge actually have full-time jobs inside government agencies. So they're quite enmeshed in the government. And advocates for Doge would say that that sort of shows that Doge's mission, so to speak, is become more integral in the federal government.

2:46But the group is less defined, but it's by no means gone. And so, sorry, are these mostly people, like their side hustle is working for Doge? Like it's like a side project alongside their government jobs? Is that the idea? no so they came in to be a part of doge and then ultimately in some cases were employed as full-time agency employees and this is getting slightly into the weeds of employment structure but some of them were special government employees which has a which is defined by a period of time whereas a federal government employee is someone who can stay on for a longer period and got a different job status yeah okay all right so musk is gone there's still a doge doge is still doing some work.

3:29And part of that work, as you wrote about this week, is helping the Securities and Exchange Commission use AI to find rules that could be cut. I mean, tell me more about this. Yeah. So the Trump administration has made AI a focus and that goes beyond Doge. And remember that Doge teams are often embedded in specific agencies. And you also have people in the group who go between the different agencies, not to confuse matters further. But the SEC does work with Doge employees, and there are people who have been in that agency for some time. And I have been trying to stay across what Doge has been doing in the SEC as it's such a consequential agency.

4:17And what my reporting showed was that some of the people associated with Doge had developed an AI tool that essentially scans the rules and regulations of the SEC. And the idea is that they can use this tool to sort of identify, using different criteria, certain rules that could be cut because, of course, the Trump administration has a big deregulation focus. They want to strip regulation generally. So this is to sort of advance that idea. The tool, from my understanding and my reporting, works sort of like a chatbot. So you can ask it to look up rules based on certain criteria, for example. And from there, it generates a whole lot of rows of different rules.

5:04And then these would be reviewed. And I mean, the process for actually cutting a rule is more involved, but it's a starting point. So the SEC now has its own chatbot internally that it can use to sort of scour through all the internal regulations. That's kind of what it looks like on the ground, I'm gathering? Well, to be clear, it's not something that's widely available to all employees. This is a project, this is a small project within the agency. But yes, and it was actually modeled on a similar tool that was developed in another agency and used in a similar way. And how much support is this idea of using AI to find rules and regulations that could potentially cut.

5:47I'm sure there are people for this idea, people against this idea. Help me sort of understand the arguments for and against using a tool like this in the government. Yeah, well, from my understanding, the project has support of leadership, but it is still in the early stages. So there are several steps to go before the rules that are identified by this chatbot are altered or cut. But the SEC in general has, like I said, made AI a bit of a focus. They assembled an AI task force in early August, a chief AI officer. So I think there is support for this initiative. But from my reporting, there are people outside the agency.

6:30I interviewed a former advisor within the SEC under the Biden administration, for example, who said that it was not a good idea, it wouldn't be effective, and in fact would just create unnecessary labor because all of the rules identified would need to undergo further review. And he argued it wasn't a good use of resources. So there are certainly different schools of thought. And that's one of the fun things about covering Washington is that everything is political, right? And so you get a lot of different opinions on these things. Right. Hear me out though. So I'm just imagining, and look, I don't know anything about what it's like to be in government.

7:09There's a lot of different layers of power that you have to go through. But I feel like at its simplest form, using AI or a chatbot to, for example, just go through all of the rules and regulations. I mean, again, we don't know about the impact. These rules and regulations are in place for a reason. But, you know, it's just the same way lawyers are using it, for example, to go through these Word documents that are hundreds of pages long. Doesn't that seem like an okay use case? I mean, I understand not getting AI to make the decision for you, but it kind of seems like a bit of an obvious use case for AI.

7:48No, am I thinking about it wrong? Well, I think it depends on the quality of the tool. And to be clear, we weren't able to independently verify how effective this tool is and whether it's sufficient to navigate what are very complex rules and regulations. We don't know how advanced the technology is. We know it was built on a number of different models and it was trained on governmental data, including legislation, you know, conservative policy proposals, executive orders, things like that. But I think it really, I think that question would really come down to the details because sure, you could argue that it's a good starting point and why not deploy it.

8:32But if it isn't able to sort of identify things in the way that you want it to, if it's clumsy in whatever way, and again, I didn't verify either way it's efficacy, but I'm not sure I have a clean answer for that. No, no, no. I think it kind of just comes down to, again, the idea that the SEC and that Doge are using this technology at all is remarkable because we know how slow a lot of these government processes move. And so what I can't wait for is for you to find more details about the effects that these chatbots are sort of having on government operations because right now they're just using it as an introductory tool.

9:18and tomorrow it could be making more decisions. We don't know. So it's a fascinating story. Thank you, Sylvia, for coming on the show and talking to us about it. That is Sylvia Barnum-Oregan, who covers everything Washington, D.C., and tech for the information. Well, we all know how important data is to making generative AI useful and effective. And this morning, my colleague, Natasha Mascarenas, published a story about how big AI model companies have talked about the possibility of licensing data from Cursor, the all-star AI coding company that has taken Silicon Valley by storm. And I want to bring on Natasha to tell us more about that.

9:55Natasha, welcome back to the show. It's great to have you. Great to be here. So who is interested in Cursor's data? Tell us what you found. Yeah, so many of the large model makers. So I reported this morning that OpenAI, XAI, Ananthropic, as well as other labs have had conversations with Cursor to license or purchase its data to train their own models, making it even more interesting because each of the companies I just mentioned are also building their own coding development software as well. So we're really seeing them not be able to acquire these businesses, but maybe try and take a step closer and get access to that valuable data.

10:33Okay. So we've got all the big fish, OpenAI, Anthropic, XAI. So I do want to get into the bigger picture here, but just so I'm clear, so what kind of data are we talking about here? Yeah, so it's two kinds of data. It's preference data, which shows which answer a user's like better, and demonstration data, which just shows if the answer is correct when someone asks a coding prompt. I would say the other angle to look at it really is the scale of data. These labs that I just mentioned are already having software engineers develop code for them, track those answers. but Cursor has millions of engineers developing, you know, that much more code on top of it.

11:13So they're also looking for scale of data when they're pursuing these kinds of deals. Got it. And so by scale there, you just mean a lot of data, essentially. A lot of it. Yeah. Got it. And is there any indication if Cursor is going for this deal or not? Do they have any interest in selling their data? You know, they're entertaining the conversations from what I understand, but I'm not getting a sense that there's an imminent deal. To me right now, it's sort of, The playing field looks like, you know, if you scratch my back, will I scratch yours? And so we know Cursor is very much dependent on these models to power its app.

11:46And so I think right now the conversations are out of place where they're seeing if there's a version of this that works, which is interesting in and of itself, as we've been talking a ton, you and me, about the unique deals that AI startups in this space need to notch in this moment. And so right now what I'm seeing it is more exploration. Got it. So, okay, now let's take the step back because I got to be honest, I read the story and I was flabbergasted because it seems to me like you have a company like Cursor, which is building its competitive advantage on the data that it has. It's trying to build its product.

12:18And here, I mean, you have OpenAI and Anthropic and actually, I mean, they're building competitive products. Wouldn't Cursor just be giving the marbles away here if it gave OpenAI access to the data that makes it different at all? that's honestly the question that made me pursue this story in the first place so you're exactly right it is a big question mark on why cursor would consider this but there's a few reasons for one they could decide to sell a portion of their data let's say the free users net new users that they're tracking anyways and they're maybe not getting that much value from or they have answers from so maybe there is like a small version that makes a customer that you know they're happy with and then they also can sort of monetize the flip side of it is maybe cursor doesn't see its biggest differentiator as data and we're entering a new place in the ai race where the differentiator is the research that cursor applies on top of those insights or you know the user interface when i talk to developers about why they're choosing a certain coding product over another it's not coming down necessarily to the brand name as much as it is it's the easiest to use i have to think the last and cursor you know has become really popular for for pretty um simple features such as autocomplete where it's auto completing code so to me this could really you know depending on which way this goes we could see cursor kind of giving us an answer on where the ai application debate stands um you know is the value mostly in the data or can you sell some of that and still really own a valuable business um that's that's to me really like like where this goes next.

13:57So do you think they do it? Right now, there's more questions than answers on why this makes sense for Cursor. Obviously, I'm sharing a way that this makes sense, because it's interesting and it would tell us a lot. But right now, if you had to ask me, I don't think we'll see anything in the next few months. I mean, Cursor has been seen as one of the biggest rocket ships in AI, but it's still a two-year-old business. I don't think you make a bet on something that you are, you know... I don't think you make a bet on something that is so powerful and give it away just yet. I think a lot of things would have to, you know, change.

14:30Maybe it's pressure on margins. Maybe it's a really fascinating deal from one of these model makers that gets it to change. Yeah, I mean, I'm going to be honest. I think it's like throwing in the white towel. I mean, I don't know why you would do it this early, at least. I mean, I understand, like you said, if margins are really that tight. We talked about margins. Margins are hard in this landscape. But it's a young business, you know? Maybe there's something we're not seeing. I think it's interesting. I think it's obvious, actually, that the larger model companies would come with offers because it's them saying, we couldn't buy you or maybe buying is too expensive.

15:07Give us some access to the goods. So I think it's an interesting question. Natasha, I look forward to hearing more about where these deals go, if they go at all. And so we will have you back on the show when that happens. That is Natasha Mascarenas, who covers venture capital and startups for the information. Well, we all know how competitive the war for AI talent has been, but that has also meant a lot of opportunity for companies that can use AI to help companies find that talent at all. One of those companies is Mercor. The company was valued at$2 billion earlier this year. It is led by CEO Brendan Foody, who is a Teal Fellow.

15:44And I want to bring on Brendan to talk more about where he sees the war for AI talent going. Brendan, it's great to have you. Welcome to TITV. Absolutely. Excited to be on, and thank you so much for having me. So, look, I do want to talk about Mercore, but, you know, the story we put out this morning about the phenomenon here where larger AI model companies are approaching companies like Cursor for their data, whether it's a licensing arrangement. I mean, you talked to Natasha for this story, actually. Tell me a little bit about what your reaction was to this happening at all. Yeah, it's not surprising because of a lot of what you're saying about how valuable the data is.

16:27And the largest trend that we're seeing in the market is the move away from academic data sets to get PhD level reasoning or Olympiad math abilities, you know, from generally students towards these professional domains of how do you get data that teaches models how to use all the tools that a software engineer would use and reason in the same way. same across all these other professional domains. And so I think it's definitely consistent with that broader transition in the AI landscape. Okay, so it's consistent with it, but this seems like if a cursor would ever go with this approach, I mean, I see it again as throwing in the white towel.

17:10I mean, do you not see it that way? I do. I mean, I think that the evals or data sets for the model are almost like the PRD for what the model does. If the model is the product, then the eval is the product requirements document. And I think you're making a good point. Just simplify it for me. What do you mean by that? So right when we're building software products, we would have a product requirement document. We'd give them to software engineers and then they would translate that into a software platform. When building models, the way to create them is that we build an evaluation set or benchmarks and then we give them to the AI researchers to translate that into a set of capabilities in a very similar way insofar as small changes in the eval set making very meaningful changes to the end outcomes and capabilities of the model.

18:05And so a company like Cursor or whatever enterprise giving away their eval sets or even broadly their training data sets is giving away the core IP and secret sauce that will allow them to drive a long-term competitive advantage. And so I agree with a lot of your points about why we think it's unlikely. So now your business is kind of interesting because you kind of sit at the opposite side of this, which is that one approach is that Cursor, and I'm not going to say, Cursor's not going to sell their data so quickly. Okay, but one approach is that you have application layer companies offering up their data, let's say, to the model companies.

18:46The other approach is something that your company is betting on, which is that actually getting humans involved in creating this data for the model companies. You've actually got people on your platform that are getting hired by the AI companies to literally code to then produce that data for an open AI. So, I mean, this is kind of an interesting platform that you've built. Tell us a little bit about what got you interested in making this hiring platform and how it's different from something like LinkedIn. Yeah, absolutely. So we started out automating all the processes of hiring people for ourselves and our friends, where similar to how we would review resumes, we'd conduct interviews, and we decided to hire, we automated all of those with LLMs.

19:30And then we met one of the major labs, and we saw that there was this enormous transition in the human data market, where it used to be this crowdsourcing problem of how you get low and medium skilled people writing barely grammatically correct sentences for early versions of LLMs. But it was very quickly moving towards this sourcing and vetting problem of how do you find some of the most exceptional people in the world, the high caliber software engineers, the bankers, consultants, lawyers, and doctors that can work directly with researchers to build these data sets. And the business now, 18 months later, works with all of the top five AI labs.

20:08We work with almost all of the top application layer companies in Silicon Valley, as well as six out of the mig seven. And so it's been this exciting ride, and it's really driven by the core point you were making earlier, which is that these labs are not able to get training data that they're allowed to use for foundational capabilities from the application layer or from enterprise customers. And so they come to us to build those data sets. So your company helps people find jobs in the era of AI at its most fundamental level. I kind of thought AI was going to displace jobs. So how do you think about that?

20:51Well, I find it to be a very fascinating dichotomy, right? Which is that with every technical revolution, right, of the calculator, the industrial revolution, all of these things, everyone talks about job displacement. But all of them also came with all of these new industries, all of these new opportunities that drove abundance and far more prosperity. It was a way better life to drive the tractor than it was to plow the field oneself, right? Or to do arithmetic problems relative to using the calculator. And we similarly see that there's this trend in knowledge work towards teaching models, how to do whatever capabilities people want them to be able to do.

21:35and that's going to be this enormous new category that very few people are paying attention to. Right, right. And I should say, you know, it's, you must have studied up on the history because I should say you are, you're a young found. I mean, you're a teal fellow, right? Are you 21, 22 years old? Yeah, I just turned 22 in April. Okay, so you've done a lot of history reading on the calculator and the effects that it had. Look, you were on this podcast, No Priors, as Sarah Guo's and Elad Gil's podcast. One of the things that you mentioned on the podcast that kind of interested me is they asked you, they said, what do you think happens when we see white-collar work get displaced if we see that happen in the era of AI?

22:18And one of the things that you mentioned is that you think we're going to see a lot more in the physical world. What did you mean by that? Yeah, well, I think that models will certainly take much longer to automate processes in the physical world of how do they run a chemistry experiment? How do they move a construction item from point A to point B, et cetera? And so as we start to automate more of the monotonous day-to-day tasks of how do we send a recruiter message or schedule a meeting, I think that there's going to be a lot more opportunity in the areas where models aren't yet able to do specific things.

22:56And so the landscape of what people do every day will certainly change dramatically over the coming five to 10 years. So you think we're going to have a lot more people doing physical jobs in the era of AI? Is that the idea? Absolutely. Yeah. Okay. And okay. All right. Fair enough. I agree with you. That has to happen. The other thing I wanted to ask you about, and again, this goes back to you presumably having a lot of friends who are new grads or looking for their first jobs out of graduation. You know, I always wonder how new grads are changing the way they look at engineering jobs or how schools are changing the way they're teaching engineering.

23:40And so I wonder, the friends that you talk to who are in these engineering programs or people who wanted to be product managers in sort of the older tech world, are they changing the way that they are teaching in these schools or the way that they are looking for jobs? I think there's a lot of inertia, so it doesn't feel like there's been meaningful change, but there certainly are a handful of professors and teachers that are enacting change, right? Where tying back to the analogy about the calculator, instead of asking people to do arithmetic, they're seeing what they can do with the tools. They're saying, how can you use ChatGPT to build an entire application in an hour or whatever the equivalent is in the respective domain for writing documents, for creating products, et cetera.

24:24And so I think that that is undoubtedly the direction that the future of education is headed. Great. Well, Brendan, thank you for coming on the show. Really appreciate it. That is Brendan Foody, the CEO at Mercor. Well, the enterprise software boom of the past few decades has given rise to big data services companies like Snowflake and Databricks. And you better believe there are data services companies now popping up alongside the AI boom, trying to disrupt those very companies, Databricks and Snowflake. It is a big bet, but one such company is Chalk. And I want to bring on its CEO to talk about the company's strategy.

25:02Mark, welcome to TITB. It is great to have you. Thank you so much for having me. It's great to meet you. So we're going to get to the big questions, but perhaps the biggest question I have for you is, you are a new father of how many days? Oh, wow. It's been about 10 days now. 10 days. Okay. So I've got to ask, how does being a dad change your perspective on the AI boom? I want an honest answer. Yeah, thanks. Well, I think we're still not going to do screens for a few years, but I'm optimistic about the future. I'm an optimist. That's why I started a company. That's why I had a kid. And I'm excited about the world he's going to grow up in.

25:40Well, I think no screens is something probably all of us can agree on. So mind you, 10 days is, I mean, gosh, I'll come back to you in a year and we'll see how things look. Please, thank you. All right. So look, I want to talk to you about a couple of things. Chalk is an interesting company. It's admittedly a little bit difficult to understand. And so I want to get into that. But one of the things I wanted to ask you about Big Picture Topic is you are familiar with a lot of the different model companies and a lot of different models that application layer companies have to use and lean on for their products and services.

26:17And at the information, one of the things we've written about is how the cost of using these models, it started to come down and now it's started to plateau a bit. And again, this is an arena that you have spent a lot of time in, I'm sure, given the nature of your business. Why do you think the costs of these models haven't continued to come down? Yeah, for sure. And this is so important to really the entire ecosystem. I think there are probably a few things to look at. Number one is hardware. Obviously, if you look at GPUs, there are a lot of supply chain constraints. And at the same time, there is really just insatiable demand.

26:53And so I think what you're seeing is that even as chips get more powerful, workloads are scaling even faster. So that means more parameters, larger context windows, those larger models. And so they're kind of outfacing. So we got the chips. Yeah, for sure. I'd say number two on the software side, most AI is still running on general purpose infrastructure. And if you look at what's really required to do inference really effectively, you need special purpose chips. And that's really just not happening for most use cases today. Right. And I'd say, kind of number three is you just look at the market.

27:26You've got a handful of hyperscalers that really control the compute. And that really means they control the pricing now. But at the same time, energy costs and data center costs are not exactly going down. And so, I guess I'd say costs don't fall just because models improve. Costs fall when you get more efficient at what you're doing and when you learn how to use infra in the right way. And that's really why Chalk exists, not to build bigger models, but to help people have better infrastructure on the infra side. Got it. And so you say that based on what you've seen, it's all the input costs for the model companies that haven't started to go down.

Read the full transcript

28:01One of the arguments that we heard on this show late last week is we had the CEO of Replit on, and he made the bet that, look, I don't think we should bet on the costs of the models coming down. It's kind of a question mark, and he actually didn't really think that they were going to come down. He thought we've kind of reached a bit of the minimum cost, I guess that it could be. He's saying we got to price our products to sort of widen that margin. Do you agree with that approach? Do you think that's the right approach to take for these application companies? Yeah, you know, I think that when you really look at your inference costs, and really costs are shifting from training, which is kind of this one-time event, to inference, which is serving that model and running it again and again.

28:46When you really look closely there, it's a little bit of a reality check. Those margins can be pretty scary. I think startups often underestimate those costs, and they're not scaling in the way that you want. So I think the tactics are relatively straightforward. I'm not going to say anything revolutionary here, but obviously you want to right-size the model. Not everything requires GBT for class reasoning. We might be comfortable with a smaller model, a domain-specific model, a distilled model. I think also you'll hear as Chaw, you'll hear us talk a lot about computation on demand, and getting the freshest possible data to compute an answer in context.

29:23But sort of by contrast, I would say caching has its place. If you can tolerate a value that was calculated in the past, you can save money by not reconcuting. And sort of the other thing I would add with that is batching also has its place. Choff can run one row at a time, which is something Databricks doesn't do. Databricks does batch, but batch is lower cost. And if you can do a batch job across a billion users, that is more efficient than one at a time. The results may just not be quite as exciting. Got it. Okay. So, and you're starting to get into it. So let's go there. I want to understand how Chalk is planning to disrupt Databricks and Snowflake.

30:03And you started to get into it, but I just want you to simplify it a bit. Explain it to us as if we were sort of an eighth grader. How is it that you're planning to disrupt Snowflake and Databricks? Yeah, thank you. Well, they're incredible companies, first of all. I guess I'd say that really there's a gap in the market. For companies that care about getting fresh data to their models at inference time, there is not a solution for inference data pipelines. And companies wind up building their own brittle, bespoke infrastructure. So if you ask, who do we compete against? It's in-house solutions.

30:38Chalk is a data platform for inference. That means that if you want to get data from the source at inference time so you can do real-time computation, we provide that. A great example is one of our customers, Whatnot. Before Chalk, they were doing a nightly batch job to generate a set of product recommendations to show in their app. And if not everyone knows, Whatnot is the largest live streaming marketplace in the US. It's like an auction platform. I've seen it. Yeah, it's pretty fun. You get to auction off all your - It's like a mashup of eBay and QVC. Right. And so the idea is, so every night, if I'm using the platform or any tool, my personal experience on this app is determined by sort of this nightly refresh of the data that happens, you know, call it every 12 to 24 hours type of thing.

31:31And it says, okay, here's the new data, whereas your platform is feeding the platform data on demand. Is that the idea? Yeah, so it's not our data. It's not our models. But before Chalk, Whatnot did an overnight batch job to generate a set of recommendations you would see in their for you feed when you loaded the app. But that's from last night. So historically, you shop for sneakers, but this morning you're shopping for baby toys. Right. They're still going to show you seekers. Right. With Chalk, we update those recommendations on demand, and that's really powerful for generating new revenue.

32:07And also, imagine the case of a brand new user where you've never run that batch shop. I'm a new user. I'm in the app for the very first time. With Chalk, we're able to update the recommendations based on what you just did, what you just clicked on, and that creates a much better experience and also, of course, drives more revenue for one. And so from what you know, Databricks are so like, they're not doing anything like this? They're not. They're focused on large batch trading jobs. And the way you can know that, you look at the newest query engine that Databricks just came out with. It's called Photon.

32:42They spent years and probably billions of dollars working on that. And if you go to just the docs page, it says, this cannot be run for anything intended to take under two seconds. We don't run photon for that. And that's fine. It's optimized for batch. But the reality is in today's world, faster computation, on-demand computation, fresher data creates less broad, more revenue, better user experiences. It's not the market they're going after. Obviously, they've done a lot of acquisitions. They're building a full stack for AI. But we are laser focused on real-time inference, which we really think is where the market is moving.

33:20Last question for you before Before I let you go, we've written a lot at the information about the corporate data wars that are coming up in the era of AI. And what I mean by that is, you know, you've got these sales forces of the world that are sitting on a lot of the data, for example, that's in Slack, right? And then you've got the AI agent companies that are trying to pull all the data from Slack and other applications and stuff like that. And Salesforce is kind of putting up their hands and saying, or putting up a wall of some kind of saying, look, no, no, no, no, no, no. We're not going to let you, you know, access this data.

33:49and there's a big debate, right? It's like, you know, whose data is it? Where does the data belong? My question for you is not so much where you stand on that, because I think I know the answer to that question, but my question is, where do you think this eventually goes? How do you think it ends? Yeah, I think that it's very clear that from a consumer perspective, consumers own their data, and certainly at an enterprise level, enterprises need to own their own data. I think that, you know, one thing, for example, that's very differentiated about how Chalk deploys, we deploy our software into the cloud environments of our customers, period.

34:23Their data never leaves their environment. We're not trying to own data, control data. We deploy software into their environment to preserve their privacy, their data control, and everything else. And we think that's the right model. And so hosted solutions have their place also. But certainly when it comes to data access, companies need to have access to their data and really need to own their data. It's often one of their most valuable assets. Right. Great. Well, Mark, congratulations on everything you built with the company, but mostly congratulations on being a dad. That's great news. Like I said, I think no screen time is something we can all agree on.

35:01And so best of luck with everything. And I look forward to having you on the show again as we see what happens with your company as it grows. Thank you for coming on. Amazing. Thanks for having me. Okay. Well, that does it for today's show. A reminder that we are live on this stream Monday through Friday at 10 a.m. Pacific, 1 p.m. Eastern. I want to thank Amazon Web Services, who is our presenting sponsor for this production. And I want to thank you for tuning in. We really do appreciate your viewership. I am already excited for our next show tomorrow. And so until then, bye-bye for now.

From the publisher

The Information's Sylvia Varnham O'Regan talks with TITV Host Akash Pasricha about the SEC's use of AI. We also talk with Mercor CEO Brendan Foody about how AI could overhaul recruiting and Chalk CEO Marc Freed-Finnegan about his company's strategy to take on Databricks. Lastly, we get into AI data licensing with Reporter Natasha Mascarenhas.

Articles discussed on this episode: 

https://www.theinformation.com/articles/replits-margins-illustrate-high-costs-coding-agents 

TITV airs on YouTube, X and LinkedIn at 10AM PT / 1PM ET. Or check us out wherever you get your podcasts.

Subscribe to The Information: https://www.theinformation.com/subscribe_h

Sign up for the AI Agenda newsletter: https://www.theinformation.com/features/ai-agenda


More from The Information's TITV

All 304 episodes
DOGE’s AI for SEC Rule Cuts, AI Recruiting, Cursor’s Data, and Databricks DisruptorThe Information's TITV · 36 min
Listen in VO