1006: In Case You Missed It in June 2026

0:00Jon Krohn:This is episode number 1006, our In Case You Missed It in June episode.

0:09Jon Krohn:Welcome back to the Super Data Science Podcast. I'm your host, Jon Krohn. This is an In Case You Missed It episode that highlights the best parts of conversations we had on the show over the past month. My first clip is from episode number 999, where I sit down in San Francisco with the revered Chip Huyen, a two-time mega bestselling O 'Reilly author. great guest for episode 999. In this episode, we confront head on the question that so many people in our industry are quietly asking themselves. If the cost of building software is going to zero, where does the moat go? You were kind of talking about this earlier, this kind of existential crisis, which I feel as well.

0:47Jon Krohn:You know, it ranges from across everything I do. You know, I had, I did a PhD in AI and that gave me a real moat around my career. You know, other people couldn't create a machine learning classifier or, you know, understand problems with labeling data or these kinds of things. But now all of those kinds of things, a machine can do, you know, no problem. For the podcasting too. I mean, it lowers the barrier to entry. You know, you don't have to be a great writer to come up with great topics, you know, to script episodes. So, you know, there's a lot of people in a lot of industries who would feel like the moat is going away from them.

1:22Jon Krohn:I'm wondering, so obviously focusing on physical systems is one way to create a bit of a moat for yourself because hardware R &D cycles are going to be longer than software. And there's lots of different ways that hardware can specialize and Anthropic or OpenAI aren't going to next month just all of a sudden have a robot that does that too. So there's a moat in physical AI. So maybe that is just the answer. But are there any other ways? Yeah, what other ways do you recommend? And I guess you also had the systems design idea right at the beginning of this episode. What other tips do you have for our listeners on how they can try to future-proof themselves a little bit in this time?

2:02Well, nothing. This is really funny because I have a friend who is an economist. He's one of the smartest people I know. and he does like consult a lot of governments and coming up with like technical policy tech policies on like yeah how how to get their nations like stay up to date in the AI era and he was straight up telling some of them it's like yeah there's nothing you can do it's like you you don't have enough budget for it like just don't do anything so it was like some people do have a very very pessimistic views of the world but I think like I'm more on the optimistic side I do think that's like AI I can solve a lot of problems, but I can solve things that will never stop being problems for me, for us to solve.

2:54So for one thing, it doesn't matter how many AI models are there or how good AI models are. I will never stop being angry at people on the internet. There will always be people that piss me off. There will always be customer services I'm unhappy with. There will always be things that's like collaboration. is just not quite straightforward. So recently, there's a founder, and I realized the founder, he's very smart. So he came to me and he pitched this idea of another Asian orchestration framework. And then he told me that all the problems that he has seen with a lot of companies is that there's not enough communications between product and engineering, which is very classic.

3:37And he was like, okay, and my Asian orchestration platform is going to solve that. and I was like I don't think that's a technical problem you know like usually when product and engineering people don't talk with each other that requires people solutions. Usually you don't solve that by like okay here's another tool we magically make product people and engineering people get along so there are a lot of people problems that it's not quite like so. So I think there are several categories of tasks I think that's like are not are not entirely completely clear following how to solve. One is human AI collaborations, right?

4:19So a lot of AI tools nowadays kind of built upon legacy systems and things about how we interact with old software systems. So just an example of coding tools. So originally, we have a lot of coding tools that are just part of VS Code because VS Code existed, right? And then we have a lot of coding tools as part of the terminal, because terminal has always existed. But then this made me think, wait a second, why are not taking apps like VS Code and terminals, why do they need both of them? I mean, just going back, why is it different stuff? And another thing is, why is terminal so hard to use?

5:07So a lot of engineering, for me, I have used terminals in school and stuff and for work. I use it, but I'm not crazily happy with it.

5:16Jon Krohn:Right, you've never been a big Vim person. No, okay, so I have friends who are crazy Vim person. I have friends who just don't use VS Code or PyCharm or anything. They're just straight up coding their terminal. So it's a lot faster with all the key, right? So in theory, you could use terminals as an IDE. Yeah, as an IDE. yeah but so but like yeah okay so so so we know that terminals exist but like because of like coding tools like cloud code a lot of product people or like people who never used terminal before are sadly exposed to terminals and they go like okay why is this so hard to use you know it's like it's very painful and i think it's like okay terminals maybe that terminals are hard to use on purpose because terminals are actually very powerful like terminal basically give you access your control plane for you to like control the computer you could easily like remove like rf like everything right like yeah like you can just just do that so so maybe like you make it hard to use so that only people who are willing to get used to it use it so they are less likely to make mistakes so so so yeah so so i think maybe but it's also like a legacy thing and i think like why don't we have something like in between like yeah we have something that can be both can give you access to the file system, access to the computer, like a control plane, a terminal, but also easy to use, like an IDE.

6:39And I think that is, I'm going to talk about it, and actually OpenAI and Anthropocene introduced a bunch of services like desktop app, right? Like the Codex desktop app, which is basically the same idea of like, okay, very easy to use interface, but give you access to a lot of things, the ways that a terminal can. So I think that is evolving. and also there's a bunch of like how to access your agent when the computer is not working. You probably have seen people complaining about like, okay, like why I have to keep my computer open all the time because my coding sessions, like my cloud coding codecs are doing their things.

7:15Yeah, so I usually just get into an Uber and it's just like, can my computer open? I look like a freaking nerd. But I was just like, it's just making me think like, there's no reason my computer should be open, right? Yeah, because it's totally run as a cloud. And then we need something that can access through the phone. I think a bunch of people are building it. Okay, now you've wanted things on the phone. Now you need some kind of sandbox, right? Because how do you share context between the phone and the computer? So anyway, I'm going to very much use genres. Shut the s*** up. She's talking with you all.

7:48Jon Krohn:Well, I'm just wondering what this has all been very interesting. But what is this all? Are you saying that there's lots and lots of problems that we can still solve? So I think it's like, I use it, not a great example, obviously. But I was saying it's like, one thing is like we don't quite have a good understanding like what is the optimal way for humans to use AI. So human AI interface is on big things, right? Another category of problem is it's like, she's not done. It's just like just the first, it's one of my first in the 20 points. No, this is really important. This is good. Yeah, another thing, if you will, let me say it, is how AI interacts with the world.

8:30So we have a lot of techniques to make AI good at using tools, right? But I think for AI to interact well with the world, we do not just want to improve AI. We can still make the world more AI-ready, right? Like, how do you, like, for websites, for apps, we can make it, like, better documentations, better APIs that agents can call, and better, like, security, like, making, okay, this kind of, like, actions are dangerous. So maybe you should, like, have less permission to AI, you know, and stuff like that. But how about physical work? So I'm not sure you've seen this very cute video of, like, a food delivering robot.

9:09A food what robot? Food delivering robot.

9:12Jon Krohn:Food delivering. Right, right, right. Yes, it's a very tiny robot. Not a foot-delivering robot. Foot. That would be weird. Do you have the extra feet to, like, deliver? I'll take six feet, please. So, yeah, so the robots are very cute. And then the robot just couldn't cross the street. So the robot had to ask the pedestrian, like, hey, can you press a button for me so that it turned green so that I can't cross? And the pedestrian was like, what the heck is going on? So I actually talked to someone who worked at one of those robot food delivery companies. And he told me the hardest part is just how to get a robot in track of the world.

9:53So actually some cities have this streetlight API. So that's a robot could connect to a streetlight API. So it can turn, it can press the button via the API.

10:05Jon Krohn:Oh, because it can't press the button to say that I want to walk. Yeah. So I'm just saying that part of how you could provide tooling to make the world easier for AI to operate in. Yeah. From the broad question of where Motes live, we zoom into one company that has been building inside this changing landscape for years and to great effect. In episode 997, I chatted with Andre Kerenkov, co-host of my favorite podcast Last Week in AI, and the founding AI lead at Astrocade, a wildly successful platform that allows millions of people to create video games without writing any code. Andre walks me through how his team was effectively doing vibe coding before Andre Karpathy had even coined the term, how his whole company adopted Claude Code a year ago, and the most important lesson he's learned from this.

10:53Jon Krohn:Yes. Your title has shifted over the past couple of years. And so there's kind of, there's two journeys actually that maybe we can kind of cover in one chronological sweep. So I think that you starting, you know, your company rather starting to work on Astrocade and doing this kind of like vibe coding for video games, it predates certainly lovable being popular. Yes. Or these other kinds of vibe coding platforms being popular. Yep. So you guys were working on this Vibe Coding platform before it was popular. And then you, within that ecosystem, your title has shifted from ML Scientist, when we recorded the last episode, to founding AI lead.

11:33Jon Krohn:So yeah, how has the company transformed? How has your role transformed as the company has grown and gone from just developing behind the scenes to now having tens of millions of active users? Yeah, it's quite the story. I joined in April of 2023, just after finishing my PhD at Stanford, where I was actually doing machine learning and robotics. And initially, that ML scientist label made sense. You know, my background was in machine learning, and I was going to be working on the AI side of things. But we sort of shifted that label a little bit because ultimately, you know, it's not machine learning to build agents or to do prompting machine learning.

12:17You need to understand machine learning and what is involved, but a lot of it is understanding bigger systems of prompting context engineering and just generally the more hands-on practical problems of building something that works rather than being scientific of doing research. So I think my title shifted just to reflect the nature of the work itself. And yeah, in that three years, I joined April of 2023. The company as a whole started on this direction of building basically what we have now, which is a user generated content platform for games back in February of 2023. This is like two months after ShareGPD came out, I think two, two and a half.

13:06So at the time we had LLMs, we had LLM APIs, but they were still stupid relative to today, right? They're not anywhere where we used to be. So the idea of straight up code generation, one of my very early things at the company was actually experimenting with code generation. And they could already write little small functions and so on. And I remember even back in 2023 when chat GPT was coming out, people are demoing, oh, like it, it brought pong and it was mind breaking or some website. Um, but, um, as anyone who's followed AI over years can probably tell relative to today, going back to 2023, LMs were much more limited in many ways of hallucination, reliability, general intelligence, and certainly being able to write code that actually works and has no bugs.

13:59Sure.

14:00Jon Krohn:I mean, it's really, I would say, and I've talked about this on this show a lot and with guests, and I'm sure it's the kind of thing you've been talking about with Jeremy a lot on the Last Week in AI podcast, but it's really since February that we have reliable text-to-code generation with the Opus 4.6 release embedded in the Cloud Code environment. Yeah. I mean, I think I'll be a little more generous. I think people have come to realize and wake up to it more since February. And it's one of the interesting things where I remember our company, we adopted CloudCode, basically everyone starting in about June of 2025.

14:37And I still remember CloudCode came out around, I want to say February of 2025. So the realization that these alums were now able to do coding has been sort of brewing. I think Andre Acopoffi coined the term Vibe coding in early-ish 2025. So it actually just hits the one year mark since the term itself has been coined. And what really happened was that this entire type of product and experience matured, right? So Cloud Code came out. Very quickly, the people kind of down in their minds doing the work with Cursor realized, whoa, this is actually on another level relative to just Smart Autocomplete.

15:26And I remember kind of seeing the hype and like not being sure if it's actually that different. Then I tried it and like a few days after trying it, I was telling everyone in the company to start using Cloud Code. But as you say, I think this year, since February, January, as the new models came out, it's just gotten better and better and more and more reliable. And I'm sure at some point you've talked about the matter time horizon eval, whatever, where.

15:59Jon Krohn:Oh, yeah. Yeah. The meter thing. Yeah. I mean, I talk about it. Every single talk that I give, I have a meter chart in the first few slides. Yep. and actually I was just recording the episode that'll actually come out next week with Chip Hu Yen, episode 999. And I talk about the meter charts in that with her. So yeah, it's definitely, I mean, it's, yeah, I'll have a link to meter in the show notes, but basically it's just, yeah, it's just showing this crazy exponential increase where like with Mythos, I mean, with Mythos, it broke the meter. Yeah. You can no longer evaluate how long the task that AI can reliably do because the tasks are too long and it's hard to evaluate.

16:42So yeah, it's been a progression where there was an inflection point about a year ago where it got to a point where they could do tasks if you were babysitting them and they could write code. And now it's getting to a point where they can do it without you babysitting them. And that is another sort of shift that is on top of Vibe coding as a thing.

17:06Jon Krohn:Yeah, it's wild. What can you tell us? I mean, you kind of just disclosed one thing there that's happening behind the scenes at Astrocade, the cloud code usage, obviously without divulging anything that would be an issue publicly. What else can you tell us about what it's like behind the scenes in terms of the tech stack building a platform that is generating games being used by millions of people? Yeah, I'm not entirely sure how much my CEO and CTO want me to say, but... You don't need to say much, though. I will say, I think the truth is there's no secret sauce fundamentally, right? There's an agent, it has some tools, we use an OLM, it's all the standard ingredients.

17:53So the ingredients aren't special, but the way you mix them, the way you put this whole thing together is kind of a tricky part. So you need to, like, we have a lot of effort to benchmark and to evaluate the way we do our harness, the way we do our prompts. And it's actually a very challenging problem to benchmark because it's one thing to benchmark, like, a multiple question, answer, test where you know the answers. When your task is like implement this Tetris crossed with a merge game that also has like a puzzle component, there's like a million possible answers. You can't even do LLM as judge.

18:33Jon Krohn:At least one million possible ways of doing that. At least, yeah. So the key to what we do behind the scenes is the finer points of how you put together the LLM, the prompt, the tools, and make it all function. I can't even imagine how tricky that would be. This is the tricky thing, I think, that still provides a moat for product designers across the board, which is that when you get into any particular niche like this, there's tons of opinionated bets that you make as a development team, as a product management team. and some of those are going to be wrong. And you kind of, you get that through, you know, having good user feedback metrics.

19:20Jon Krohn:You can kind of learn, okay, going in that direction was the wrong way. Let's try this other way. And then over time, over many years, you accumulate, okay, we've kind of gone in the right direction overall. And that gives you a moan. Yeah, I think you learn a lot and we have learned a lot over years. One thing you've learned over time and I'm still, oh, a blog post, just detailing all the many things we've learned in doing this over a few years. But one of the things you learn is you have to be very careful around scaffolding around AI in the sense of, you know, as we were starting out in 2023 and even in 2024, you couldn't do vibe coding yet.

19:58So we had to come up with a way to let people make games where the AI was helped out by some sort of structure, which we can call scaffolding. And so it took these pre-existing pieces and it put them together and made things you could use in the game. But then you hit 2025, you get to a point where you could do vibe coding. All that scaffolding now is limiting you as opposed to becoming more powerful. So one thing that we are very mindful of is building in a way that it is very future compatible. You want to build your system in such a way that when the LLMs get better two months from now, three months from now, whatever you built isn't outdated.

20:39And it's one of these tricky things where I think probably in startups and research and everything, you learn over time that you have to know what to keep things simple and really deeply understand how to leverage technology and build something in a way that's compatible. so I don't know if that is actually interesting but it's something I've had to learn the hard way

21:10Jon Krohn:Agents are getting smarter every day but even the smartest agents get stuck without the right context and the right tools that's where Notion comes in With the recent launch of custom agents Notion became the collaborative AI workspace where teams and agents work side by side and now their new developer platform is turning that workspace into infrastructure developers can build on What sets Notion apart is that the collaborative workspace and the platform you build on are the same thing with permissions, context, and governance baked in from day one. Workers, for instance, are Notion-hosted sandboxes where I can run database syncs without standing up my own infrastructure.

21:43Jon Krohn:That means I can pull guest research, episode analytics, and my consulting client data from all the scattered systems where they live, then keep them synced in Notion databases automatically. My agents and my human team work from one single source of truth. Learn more about Notion's developer platform today at notion.com slash superdata. That's all lowercase letters, notion.com slash superdata to try Notion's developer platform today. And when you use our link, you're supporting our show, notion.com slash superdata. For all this software abstraction enabling our AI systems, there is a very physical reality behind it, the data centers themselves.

22:19Jon Krohn:In episode number 1003, Lightning AI's Vice President of Infrastructure, Frank Basso, takes us inside AI data centers and explains the industrial reality of how the AI sausage actually gets made. A lot of us can probably picture a photo that we've seen online of these long hallways of server racks. But is there anything else that's kind of interesting, maybe particularly interesting about an AI data center when you're physically standing there? When you're physically in there, one of the differentiators from a traditional data center is the noise level. These systems are very noisy. We call them screaming banshees.

23:01Jon Krohn:Oh, my God. I had no idea. Yeah. So especially within air-cooled and inside the data hall, the levels range from way beyond what you'd hear at a rock concert if you're in the front row. And so hearing protection for our staff, we require two types of hearing protection at all times. You have both like the buds that go in your ear, you can use molded ones or not, you know, listen to your music or whatever. So something occlusional in your inner ear, and then cans, right? And cans being all passive, you cannot wear or use active noise canceling systems within a data hall. This is a big thing that people are like, yeah, I put my noise cancelling on, it's great.

23:47Well, if the data hall is 105 to 110 decibels, to cancel the noise, noise cancelling generates 105 to 110 decibels. So that does just as much damage.

24:00Jon Krohn:You're not hearing it, but it's damaging your drum. Wow, I had no idea about that. It makes so much sense now you say it, but I had no idea that with my noise cancelling headphones, I'm here thinking, I'm wearing noise-canceling headphones right now. Obviously, it's not canceling 100 decibels of noise. I don't have screaming banshees in my recording studio, believe it or not. But that is, that's actually, that's an interesting take-home tip. For anybody going to a rock concert or whatever, you need to have passive noise, not canceling, but just suppression. Yeah, occlusional or suppressive. And so really, it blocks the two different kinds of hearing protection, the inner ear hearing protection, and then the outer ear blocks different frequencies of noise as well.