If you can imagine it, you can probably build it | OpenAI's Satya Tammareddy and Rowena Westphalen

19:22Satya Tammareddy:And so humans are used to expecting the computer to give you the same answer and for it to be a very black and white answer. And these systems are completely the opposite. That's why we get hallucinations. And it's a really good example of a strength overplayed becomes a weakness or, your weakness is also your strength, right? So these AI systems would not have learned or evolved as quickly as they had if we didn't have the, that old type of technology was very brittle. This flexibility is what gives us our strength. Then when you put that into workflows, we currently have businesses that are reliant on very deterministic workflows.

20:00Satya Tammareddy:And so there's like a whole range of things to rethink when you, if you actually want to take full advantage of the probabilistic side of things. And so that's where, as Satya was saying, evals become so important because the key thing to getting this right is like having a consistent way of ensuring that the range of answers that you get is acceptable, but you're still catering for the range.

20:21Rowena Westphalen:How many customers are capable of designing an eval? Like, cause it is a new skill, right? Like there's so many new skills developing to get the most out of AI. Like, you know, I remember it wasn't that long ago when this concept of having a prompt engineer emerged and I was like, what is a prompt engineer? And now we've got all of these additional new role types. Like how many customers are in a position to be able to sit down and say, gosh, now that you ask me, what is the range of acceptable like responses here? Are we finding, is that an area where you're also having to do most of the educating or is the market ready for that?

20:56Satya Tammareddy:It depends on the customer. There are lots of organisations that I think are ready for that. You just have to, I think they need the permission from their leadership and they need to be open-minded. But when they go back to basics, I think they can do that. It's worth saying prompt engineering is not going to be a thing for much longer, I don't think. The reason you needed good prompts and prompt engineering is so that you got the right answer and it was unambiguous. And these latest models are incredibly good at inferring context. And so maybe evals will just be a short-lived thing. What do you think?

21:28Satya Tammareddy:The interesting prediction.

21:30Rowena Westphalen:I think it'll take time. To your point around context, even with the move from organizations using AI purely to respond to things. So think back to chat right we've back to chat like or we knew it to be like a year ago yeah and now we're we've moved to agentic right and so to your point the agent goes and finds that context and comes back with an answer for us or has completed a task for us i still think evals will be important particularly for really high stakes work um and humans will also be very important for that kind of work but I think day to day to your point we may not need them for that that much longer or at least it'll be easier to template it'll probably prompt you with like you might want to think about this as this yeah oh yeah you're right that is that doesn't seem like right to me yes so actually I want to come back quickly Satya to the aha moment I think you were sort of describing actually Roe the aha moment for a lot of customers is like when they finally start going from a sort of interior design version of a renovation to an engineering and, you know, demolition style of renovation.

22:48And I'm really interested in what you're learning from particularly enterprise

22:52Rowena Westphalen:customers as they deploy these sort of first and second, you know, use cases into their business. Is there a measure of success that you find is just radically faster at building adoption and ease? Is it like time saved or like you took a bunch of really painful and repetitive tasks off my team and they hated them and so they're happier? Or is it money saved? Is there for you a sense in what is likely to drive that aha moment first? Or is it as simple as like a storytelling and it touched the CEO's work, helped that CEO and that's when you start to get that buy-in? Yeah, it's a great question.

23:34Rowena Westphalen:On the question around measuring and what is driving that, there is no one answer there. I do think someone getting to that magic moment, and if that is the CEO or a leader in the business, that I think is the pivotal moment where they get it and then they're encouraging their team or teams or company to experiment and get to where they are. But I do think it's a journey. We see a lot of leaders that have had that and, you know, you can tell they're just living and breathing the tools even though their day job is completely different. And so I do think that has a lot to do with inspiring and encouraging organisations to get their hands on the tools.

24:26Rowena Westphalen:Now to your point on what are the metrics we're seeing, it really varies. And there is the element around saving time, saving maybe dollars of investment in building a product, getting to market faster, like leaner teams driving output. But also what we're seeing is 75 % of enterprises have said AI has allowed them to do things that they weren't able to before. And so often that may not be easy to measure. There's a really interesting customer that I'm working with where they've actually asked ChatGPT work to help them quantify various tasks around what is the ROI on this based on what my employees are telling me.

25:12And so I think on the measurement piece from what I'm seeing, it is a journey and a lot

25:19Rowena Westphalen:of folks are struggling around how do I actually prove ROI on this? Yeah. Yeah. I'd love to then kind of get into the other part of ROI is cost. And, you know, the big topic at the moment is tokenomics. Satya, you have a degree in economics, started your career as an economist. You'd be the best place. Can you define what tokenomics is for part of the audience that may not know? So, tokenomics is how valuable a token is in terms of delivering a task. And so I would think of it in terms of cost per task. And so we're seeing a lot of attention around this topic, given the cost of using AI has for a lot of businesses has become exponential.

26:06Rowena Westphalen:And so that is both a combination of the tokens they're using, but also the tasks and then the complexity with which those tasks are delivering outcomes.

26:16Satya Tammareddy:So it's super interesting. If you think about the cost of the token has actually decreased significantly in the last year, right? It's like there's two stories going on because such is absolutely right. Lots of organizations are like, oh, how am I spending all this money? And at the same time, if you look at the individual token cost, it's gone down quite a lot. Actually, efficiency in tokens and ensuring that our customers have the most cost-effective access to AI and ensuring that all of humanity can access and benefit from AI is like a huge focus of ours. I imagine that there would have been a time when they made a light bulb affordable to run, to like run one light bulb and it'd be like, hey, that's so cheap.

26:55Satya Tammareddy:And then they decided to switch on an entire city and they were like, oh wait, that's lots of money, right? So it feels a bit like that. We've actually been working on efficiency and effectiveness of our models for like the last 18 months, two years. I'm really impressed at the vision that OpenAI had around this because even if you look at, so our current model is 5.6, we'll have a new model coming out imminently. So maybe by the time this podcast is out, we'll have a newer model, but two models before that was 5.4, not that hard to work out the numbers. But if you look at the performance between 5.4 and 5.5, it looks almost exactly the same, like in terms of achieving an outcome, but literally the token usage is a third from 5.4 to 5.5.

27:38Satya Tammareddy:And it's gotten even more efficient with 5.6. And then on top of that, we're delivering different size models. So in 5.6, we talk about Sol, Terra, and Luna. And if you think about like the moon, the earth, and the sun, and you think about the sizes there, that kind of gives you an idea. So Sol is our flagship model, premium. Terra is the medium-sized model. And then Luda is so affordable compared to what it was. It's like Luda has now reduced costs by about 80%. It's like, it's so affordable. But even our most premium models are much more cost effective than all the other frontier labs. Like that's where we're really leading the market.

28:21Satya Tammareddy:I'm sure they're going to catch up. Like that's the way the AI market goes. But right now that's kind of where we're seeing a big differentiation.

28:30Rowena Westphalen:And we look at it on a cost per task basis. So Ro mentioned the cost of tokens has come down significantly, but also it's more efficient. And so to actually get a task done.

28:44Satya Tammareddy:Yeah. I mean, the whole token thing's not that useful because it's like each AI lab has a slightly different definition of a token.

28:50Rowena Westphalen:It's not exactly like a commonly understood unit anyway. Yeah.

28:53Satya Tammareddy:And so it is much more useful to look at, you know, what's the business outcome that you're achieving? And that's where it's incredibly exciting to see the sorts of things that we're doing with the likes of ChatGPT work. So, you know, we've had ChatGPT for years. ChatGPT work is the agentic equivalent where you can actually delegate tasks. And like the typical use for that is coding, right? Helping a software engineer deliver and develop code. And so our coding, just in Australia, the coding output has increased six times since January. And in fact, for women, three times for women, which I'm really happy about, lots of women out there coding.

29:30Satya Tammareddy:But we then realized that about 40 % of what we thought was coding work was actually being done by non-coders. So it's like, oh, people are producing PowerPoint slides and creating spreadsheets and chat CPTs creating websites to describe things. And that's really exciting kind of output and delegation. So yeah, it's been really exciting to kind of see how these use cases kind of evolve into new opportunity areas.

29:57Rowena Westphalen:I do want to come back a little bit to tokenomics because, I mean, it's a thing that's very top of mind for a lot of our portfolio companies. It's been in the news, obviously, a lot recently where, as you've described, like there's sort of so many forces happening at once. On the one hand, you've got firms like yours who are constantly pushing out new models with new capabilities. And as you alluded to, I imagine one of the most interesting strategic questions for Frontier Labs is like, what am I optimizing this next release for? Is it efficiency or efficacy, or am I looking for moving further into the frontier of answering the hardest questions at any cost?

30:36Rowena Westphalen:I mean, all of those things have trade-offs, right? And you're kind of always thinking about what should be done there. But I think one of the interesting things is if you are a customer of OpenAI or any of the others, or indeed you're building technology that then enables your customers to be users effectively of the open AI models, you're having to think about like, are they calibrating their use of those models to match the task and the complexity required? So, you know, most of them we can choose, right? You mentioned, Ro, the sort of Sol, Luna and Terra models. And there's an instinct for people to say, give me the best always, even though it's like I'm asking for a recommendation for which cafe to go to around the corner.

31:19Rowena Westphalen:I'm really interested in where you think this is going to go from a product standpoint. It seems obvious that AI will get better and better at defaulting to the model that it needs, saying like, this is not that hard, like you're not asking me for an mRNA new vaccine. You probably don't need the frontier model to do that. But also recognizing that you want to be demonstrating the full capabilities of the model, knowing that the cheaper models by definition have to cut some corners. How do you think this is going to play out in products? And maybe I'll start with you, Satya, and then come to you, Ro.

31:51Rowena Westphalen:Yeah, this is very top of mind at the moment. In terms of product, I think two ways. Firstly, currently you have to select both model, but also how fast you want an output. And so, there's a world where we abstract all of that away. And so, it's really easy for you when you ask your tool to do something, the choices are made behind the hood. So that's one. Second is empowering businesses to, so there's an education piece, but also empowering them to set defaults for certain, it could be certain, so coding obviously is the most advanced use at the moment in terms of what we're seeing. And so enabling those that are using coding, more powerful models versus there might be tasks that don't need, you know, the soul.

32:44Satya Tammareddy:I mean, I think there are some coding use cases that you'd use a medium-sized model for. It just depends on the complexity of what you're doing. I feel quite strongly about using the right model for the right task. And there's kind of two analogies I've got for that. So the reason I care about it is because I'm always thinking about the resources that we're using for our planet. And so like, if I'm angsting with ChatGPT about like a recipe replacement, I don't really want to be thinking about using premium resources to do that. But then on the more high stakes stuff, then obviously I want to leverage that correctly.

33:17Satya Tammareddy:But then it's also this element of like, if you thought of these models as individuals in an organization and it's like, would I really go to like that senior consultant and ask them, can you staple these printouts for me, please? You know, meanwhile, I've probably got an intern who's excited to be in the office that is prepared to just do anything that probably doesn't mind stapling these things for me. You know, and so it's that kind of side of things as well, to kind of like use them for what they're best for. The nice thing about the AI is it doesn't mind either way, it'll do it. But I think there's a responsible element there.

33:51Satya Tammareddy:As long as you say thanks.

33:52Rowena Westphalen:I'm interested. Do you say thanks? Do you say please and thanks?

33:54Satya Tammareddy:Yeah.

33:55Rowena Westphalen:Didn't Sam say like it's going to cost the planet X amount if you keep saying please and thanks? But it's very natural to want to do it.

34:00Satya Tammareddy:I don't necessarily say please or thanks, but what I do do is everything we're doing in AI is actually replicating the knowledge we've had in the world. I think to get the most out of the model is probably everything you would do to get the most out of a human. So a human that feels appreciated and nurtured is going to be better. And I've got this weird theory. I don't know if it's true at all, but I've got this weird theory that if I'm more constructive with the model, it'll do better things for me.

34:26Rowena Westphalen:Are there any other funny things you do with your use of ChatGPT or the other models? that you think is like a quirk? I've been using voice a lot more in the last few weeks.

34:38Satya Tammareddy:Yeah, should we show you? Yeah, go for it.

34:41Rowena Westphalen:The biggest benefit here is, I mean, I'll give you an example. We had a business review a month ago. I was walking home from the bus stop and using voice, I prompted chat and work to look at a deck I had from last time and then populate it for the update over the last three months. And I got home, looked at it later that night, and it was 90 % there. I didn't have to type anything. I just spoke to it. Yeah, it's great. So I'm kind of then interested to kind of go into a bit more around how you are responding as leaders in this AI wave to the way that AI is expressing itself. I mean, you mentioned earlier, Satya, you've been in market since December, which is a very, very long time in the era of AI, but no time at all.

35:29Rowena Westphalen:Like we are all still learning what this is going to look like. Maybe starting with a kind of customer perspective, what's something that you think customers are focusing on that is probably not a thing that they need to worry about? And maybe what's a thing that you would love to be getting more attention in this next wave of AI to build the best version of the kind of AI version of life that we want to seat. Ro, can I start with you?

35:55Satya Tammareddy:There's a lot of theoretical debate around AI. So there's a lot of discussion about the potential risks and not to minimize them in any way, but there's almost too much theoretical discussion rather than actual doing. The sooner you start to prototype and use these things in a safe way with guardrails, the sooner you'll actually understand what it should actually do and how it should actually work. I think there's a little bit too much discussion about the potential risks of what if, and maybe not enough discussion about the potential risks of if we don't adopt it and what will it mean to Australia if we just ignore all AI.

36:34Satya Tammareddy:And I'm by no means saying, let's just take all the holes off and just do anything.

36:39Rowena Westphalen:But what do you perceive as the biggest risk if we don't embrace it?

36:43Satya Tammareddy:I think there's a real concern that I have about relevance and productivity in Australia. The the value chains are being changed. Ways of working, new service offerings, new ways to engage customers. Traditional enterprises that aren't really thinking how to fundamentally change their organisation are potentially at danger of having a startup come in and completely replace them, completely change the way the customer service is being delivered. And then at a global level from a community perspective, Australia needs to, I would love Australia to stay at pace with where the world is at, you know.

37:24Satya Tammareddy:I think we're doing that, actually. I'm not, I don't, I don't want to have it interpreted that I don't think we're doing that. In fact, I'm really proud at like the level of adoption that we have in consumer here, like one or two adults in Australia use ChatGPT every week.

37:37Rowena Westphalen:Yeah. I mean, this is one of the most interesting things is that actually adoption rates in Australia and New Zealand are amongst the highest in the world. Yeah. I think across both OpenAI and Anthropics platforms, actually higher per capita use than the US the last time I had a look at it. And yet the tenor of the debate does feel different here.

37:55Satya Tammareddy:Well, I just wonder how much it is people using it as a consumer and then actually involving it in work. One of the things we did notice was 40 % of the consumer conversations was actually work-related. So then you have the situation where you've got maybe a regulated industry who's being super conservative and has gone like, okay, I'm just going to switch off everything because, you know, we can't risk anything happening. And like the essence of what they're trying to protect isn't wrong, but like just switching everything off isn't useful. And so as a result, you wonder how many people are just like doing their work stuff on their personal machine because they know it's going to help them be more productive.

38:34Rowena Westphalen:Yeah.

38:34Satya Tammareddy:So getting that balance right is important. What do you think, Sats?

38:37Rowena Westphalen:I completely agree. One thing that's also surprised me in all of this is, actually, I'll ask you a question. Can you guess outside of coding, what function or industry has been, we've seen the most adoption? You mean in a business setting? Yeah. Customer service? No. Legal. Legal. Interesting. Which is really interesting. With legal, we've seen a hundred times increase in how they're using agentic AI across the globe. Anecdotally seeing similar trends here in Australia. Gilbert and Tobin, for example, is one of our customers. And you would think legal being regulated. Usually a more conservative voice.

39:22Rowena Westphalen:A very conservative. Privacy, confidentiality, all of these matter a lot. It's interesting seeing how Gilbert and Tobin has evolved and their leadership has been very, very innovative, but also keeping in mind that they are in such a regulated industry. And they've taken quite a measured approach to how they're thinking about rolling out AI. And I've been really impressed and to the point where they told us that their clients have actually asked them to be using AI more because it reduces the number of billable hours. Well, it's interesting you should say that. I actually heard a very senior partner in a law firm, I won't name that law firm, describe, this is probably close to a year ago.

40:07Rowena Westphalen:So again, you know, the dark ages relative to now, but describe one of the challenges that they were having managing clients that they actually had to keep two very distinct lists was almost bifurcated for them at that time. They had the customers that were saying, I want you to promise you're using AI wherever you believe AI can deliver the service for me faster and more efficiently and with fewer billable hours. And I won't pay to have a junior person do that if there's realistically something else they could be working on, the higher order and an AI could do it. And then they had, excuse me, another list, which was under no circumstances can you use AI at any point in managing our work.

40:48Rowena Westphalen:And so internally, this law firm was trying to manage two quite distinct customer expectations. And I just thought it wasn't an absolutely fascinating likely temporary move as everyone gets comfortable but this really really stark difference in what their own customers level of comfort was and it we I mean it would make for a really interesting experiment on like in 50 years time is the first set of customers still around and the second set not because their mindset on adoption of new technology is just very different Yeah, that's, I mean, matches what we're seeing as well. And it's great to see law firms be really mindful of that and be progressive, but also cater to their customers.

41:34Rowena Westphalen:Yeah. So I'm then interested, actually, you've both spoken about the opportunity set and you work with large enterprise customers across Australia. Is there an industry or a sector where, from the perspective that you have about knowledge of what the technology can actually do, you'd think this is obvious. This whole industry has a huge opportunity with AI, but it's surprisingly laggard and what's it about that? Like are there areas where you think why hasn't it taken off faster there? From my side, marketing. And when you look at industries that are embracing, like I wouldn't say marketing is a laggard, but given the potential, There's content, there's multimodal, so images, text, there is analytics around campaigns and what they're generating.

42:26Rowena Westphalen:There's so much that I think marketers can be doing. And so I've been personally surprised that marketing hasn't been, you know, the leader, like I was saying with legal. You must have a hypothesis. Why? It is interesting. Speaking to marketers, there is definitely a strong, I mean, obviously, human judgment and creativity are very important. That is coming into play where maybe there is an over emphasis on the fact that we need to be really human. And when you look at marketing these days, yes, there is an overstimulation for all of us around content. And so that human touch is really important.

43:07Rowena Westphalen:But I do think the best marketers that we've seen, they are leveraging AI, but also remembering that human touch, the judgment, the taste, all of those things that make marketing really human and make that emotional connection for us.

43:22Satya Tammareddy:You also see cases, I think, where marketing is now being done by non-marketers because they can, similarly to design.

43:30Rowena Westphalen:But that is sort of resulting in us all being drenched in AI slop, right?

43:33Satya Tammareddy:Well, I mean, I suppose there's an element of that, but I think there's also an element of like, it's not just content, right? It's like programmatic. That's right. Pipe gen, stuff that you would normally have to have relied on someone else to do. You can just be like, actually, no, this is important for my bit of the business. And so I'm going to run with that. The other emerging bit that's super interesting is the ads in ChatGPT. I mean, people are using ChatGPT as the equivalent of search anyway. And so we've launched ads in Australia and the first few companies are lining up for that. So that's going to be a really interesting area to see how that evolves and changes.

44:11Rowena Westphalen:I'm really interested in that. I mean, Google, for the longest time, built one of the world's most successful businesses over figuring out the answer to a very similar question. it's right in the line of sight of the big frontier models you have a prediction for where

44:24Satya Tammareddy:we'll go i mean i think it's going to change the way we think about things and i think it's so it can become so much more useful than the way we think of an ad today because the ability to for it to be hyper personalized to your context and so it is much it can become much less of an ad much more of a real a realistic recommendation i mean we all use chadgpd for recommendations of

44:48Rowena Westphalen:what to buy. I mean, I'll give you an example. I've got my friend's first birthday coming up for her son. And so I asked chat, hey, one year old, my friend is very into these things. And then I got a list of toys and books on Amazon that I could purchase. And so it was just, it just made my life so much easier because I don't have to go and research all of this.

45:15Satya Tammareddy:And it's just so nice to have a discussion around that, right? Because then it's like, Like they suggested, but, and it's like, oh, I didn't tell you that actually they're vegetarian or like they don't want anything with leather or, you know what I mean? And then suddenly it's like, instantly it's like, oh, okay, I understand. Here you go. It's going to take a lot of friction out of some of the consumer processes, which is going to be fun and interesting.

45:37Rowena Westphalen:I mean, I think one of the things that's curious is like context is king at the moment, right? Like, and actually whoever can really own the maximum context and then of course have the memory to hold onto it. Those are some of the biggest questions we experience at the moment is going to be able to win that. And of course, in the past, all of the other big players have sort of purchased context from others to be able to make the best recommendation. But, you know, you'll be in an interesting position to draw on thousands and thousands of conversations and tone of voice and a whole bunch of other things that allow you to do things differently than the kind of, you know, bygone era has allowed us to do.

46:15Rowena Westphalen:I'd like to move to the sort of founder perspective. And you've just done a founder event last week, as I understand. So I want to get your sense on who are the founders that are really getting the most out of your technologies? Like, what do you think distinguishes them? Is it the type of questions they ask, the type of team that they have, the type of problem they're trying to solve? Because there's so many people in our audience that are early days founders trying to think about how to make the most of this set of tooling? Satya, can I start with you and then we'll move to the road? We're seeing probably two types of founders.

46:51Rowena Westphalen:One, what we would consider AI native. So what they're building is entirely built on the technology at hand, right? So the thing Steve from Lorikeet, Jackie from Relevance, Heidi Health is another one. We're also seeing founders where maybe they've got businesses that can leverage AI. Let's say a more digital native company that are building with AI now. The best founders we've seen, I mean, we had Founders Day last week. We had 150 amazing founders from around Australia join us. I was blown away both by the types of founders. We had a few year 12 students that were building companies, which is incredible.

47:39Rowena Westphalen:But also, you know, some of the founders I just mentioned, and they're all, I think the best founders are really thinking about what customer problem can they solve rather than how do I embed AI into what I'm doing? They're starting with the customer and the problem. Really going from there, how do we also help our teams build faster? How do we have impact faster? And so, I mean, Jackie from Relevance mentioned to us, he's spun up a new team at Relevance that's focused on helping customers with their internal tools. And there's five people on that team. And he was saying two or three years ago, that would have taken 70 people for him to dedicate to that.

48:22Rowena Westphalen:And so, they're able to go faster, right? Because hiring 70 people, like just the time it takes, but also they're able to just go and have that impact. And so I think it's really amazing just to see everything starting from that customer problem and working backwards. Right.

48:40Satya Tammareddy:I mean, exactly what Satya said. I can't go past a founder with a vision. Like nothing beats vision. And I don't mean vision for AI. I mean vision for a customer outcome. I think of Ben from Employment Hero. You know, they've got like this really strong perspective on like what employment could be and what it could mean. to individuals. And you see it across any sort of leader who has just got like this clarity on like, hey, we can get here and then is able to then slot in and be flexible enough to try the opportunities. A lot of this is a mindset shift. The person I follow a lot, and we're very lucky to have access to him in OpenAI is a guy called Pete Steinberger.

49:23Satya Tammareddy:So he's the inventor of OpenClaw, which is that viral agent that went viral end of last year. And Pete's amazing. So like, you know, he's got agents that you see these pictures of him and he's got like 12 screens that he's working off coding and that sort of thing. And so then he's like, well, I've got an agent that does this and an agent that does that. And then he was updating his agents and then he's like, wait, why am I updating my agents? Like I should have an agent that updates my agents. And then like it's this whole kind of – so that was just a mindset thing, right? Like it's like actually I could have something else predicting this and doing that.

49:58Satya Tammareddy:And so it's that same sort of ability to really think in a big and aspirational way and then translating that into the technology. But increasingly the technology can do it, right? It's there. It's about having the imagination. I love that. I think vision. Yeah.

50:15Rowena Westphalen:So one of the big topic areas, of course, with AI deployment and adoption at the moment is hacks and the sort of cybersecurity and trust element of it, it's probably top of mind for a lot of your customers. How's that playing out in customer conversations? Satya, I'll start with you. Yeah, there's been a lot of interest around it, particularly how can businesses defend themselves proactively from these threats. We're working with a lot of companies across Australia and New Zealand to help them with that. and that can be as it can start from let's say a developer that's building their products or is behind their code we can actually help them check the code that they've built and and so there's those micro moments where through our cyber models they're able to check and and strengthen the code base that they're building but also it goes much you know much more macro from there as well, you know, into defending.

51:19Satya Tammareddy:Ro, are you seeing, how are you seeing that play out? Yeah, we've got a huge amount of interest and it's, I think it's a really important area for us to support Australia. And like, we're getting a wonderful response and partnership from the, they call them CISOs, the chief security information officers that we deal with. As such, you mentioned, so the package around it, we actually call Daybreak because it's a combination of not just the cyber capable models, which are very intelligent and very clever, but it's also the harness that we put around a harness is this fancy word we use for the the coding platform and then we also have these kind of reusable models and so as such as said the simplest bit which we call day back blue which is much more kind of a proactive defensive perspective and is the much more common thing that we're doing in australia which is you know as we scale coding and as we reduce backlogs and produce more stuff we're ensuring that we're not increasing vulnerabilities along that way.

52:16Satya Tammareddy:So the engineers who are being made incredibly productive in producing new solutions aren't actually creating vulnerabilities along the way. They're empowered to check the work that they're doing and ensuring that it's safe. And then for certain organizations under the right circumstances, then we can help them with what they call the red teaming and the more proactive stuff. And that's incredibly important for critical infrastructure and that sort of thing. And it's actually great that we've got the ability to leverage models like that.

52:44Rowena Westphalen:And what's the part of the debate that you can expect coming next? I mean, you see things coming down the line well before your customers. What are the sorts of things you're advising them to start thinking about now, even though it hasn't become topical yet?

52:57Satya Tammareddy:I don't know if this is necessarily a new thing, but cyber is not a technical problem. Cyber is a business problem. So I guess the organizations that are still kind of putting it to the side in some like small IT team here and leaving it to the problem of the tech leaders, that's the thing you'd advise them around. Because as we've seen in businesses that have had issues, the whole business suffers from it. And so treating it as a holistic business approach is probably the most significant thing. What might that look like?

53:27Rowena Westphalen:I mean, why involve other teams in that problem set?

53:30Satya Tammareddy:Well, because of awareness and responsiveness and the general build that you can do to be, you know, it's an everyone problem. And then it's also integral to the way you go to market. It's not just an afterthought or something like that. So it's a responsibility of everyone and it should be led by the business, in my opinion.

53:55Rowena Westphalen:Yep. Satya, anything you think the forefront of the customers should start thinking about? So I would agree with Ro and making that everyone's responsibility. And so I mentioned earlier around when teams are actually building the products having that kind of vulnerabilities checked at that point so it doesn't just sit with the sisso it sits with all the teams building the other thing i would say is with the proliferation of agentic coding we're seeing that previously the build phase was the bottleneck and now we're having a lot of product leaders say hey, actually it's like the product managers that are now just bottlenecked in terms of - Yeah, they're drowning under review of, is this the right product?

54:44Rowena Westphalen:And so how do you think about building your teams so that they're flatter and they take into account the shift that we're seeing? So in some ways it's organizational structure. And this is a question we're getting a lot of companies that maybe are facing these bottlenecks. How do we adjust, flatten, maybe align PMs directly with the business, right, so that they're no longer that bottleneck? Yeah, it's interesting, actually, so much attention over the last couple of years has been placed on like who's not going to have a job from AI without thinking of the next step of like, well, if AI makes certain tasks and capabilities ubiquitous, who then becomes decision makers that then, you know, we may need more of certain roles.

55:36Rowena Westphalen:and actually product is an interesting example that, you know, a few years ago people would have said you don't need product because you can just go straight to build and you can do customer discovery so much faster through prototyping. But we're increasingly hearing from our founders that actually those roles, the decision-making, the taste around what to build for whom and why is actually becoming, it's rising again as the skill set that they need more and more of. 100%. I love that.

56:03Satya Tammareddy:Better things for humanity.

56:06Rowena Westphalen:my final question for both of you what's something you've changed your mind about we'll start with you bro uh change my mind about in anything personal professional something you used to believe and now you believe something different what have I changed my mind about I do change my mind a lot actually

56:25Satya Tammareddy:yeah I think this I think I've changed my mind quite a bit about hiring recently as I'm like looking for people so like there's there's stuff around hiring that's signals around hiring that you always need. Like in my case, you know, you need to check the technical capability and they need to be coachable and they need to be self-aware and that sort of thing. But I think it's going back, going back to that vision bit, I'm now like really testing for like, you know, what do you imagine could come from that? Like the, yes, there's a bit of hunger and yes, there's a bit of, obviously there's competence and capability, but like having a, having a really wonderful aspiration about what you want to do is something I probably not so much changed my mind, but have added along the way.

57:08Satya Tammareddy:What do you look for there? I just ask them like, hey, tell me some stuff that you're building. Like, what are you curious about? What do you think this might be? And then just see if it's just kind of, you know, the same, same stuff, or are they really stretching the thinking and kind of increasing the curiosity themselves? Satya, something you've changed your mind about?

57:27Rowena Westphalen:Six to 12 months ago, I mean, we spoke about hallucinations and how much the models have improved in the last six to 12 months. For me, I still, I value writing and communication a lot. And so, I used to be skeptical about the outputs and, you know, I always edit and review both what AI has given me, but also what my team's giving me. And I remember six months ago saying to someone, hey, that reads like it's very clear. You used AI to write that email. But what's changed for me, I think, is the fact that you can actually – there's someone on my team. He's very good with the tools, but he trained Codex, which is now chat work, to mirror my tone.

58:22Rowena Westphalen:And he sent me a draft the other day, and I did not have to change a word. And so what for me has changed. And that was uplifting, not creepy for you? No, because it saved him time. But I think we all know that, hey, if you're sending something, it's still coming from you. And so you should make sure it's authentic and like you own it, right? And so for me what's changed is the models are there and I was sceptical earlier about the more creative and expressive parts of, you know, leveraging AI to represent you. And I still think there's a strong need for that human judgment, taste. We talk about these words a lot.

59:07Rowena Westphalen:Taste is really important, but I've changed my mind around leveraging it for writing, which I'm really passionate about. Actually, on a practical note, we were talking about this in the office just last week about everyone's working hard to build prompts to get the models to really replicate the way that they write. And there was an extensive discussion amongst our team about like, oh, I've written this thing and I've embedded it into various parts of the stack so that it really knows it's like every piece should come out like this. And it's not just about the ellipsis. It's more than that. Have you got any tips on like, maybe this is your colleague who did this for you on like the ideal way of prompting to get an outcome which feels authentic and is consistent over time.

59:53Yeah, I think the context memory is important, but also my agent that I use, I've actually

1:00:01Rowena Westphalen:trained it on my voice. When I get an output, it's almost there, but I still like to add my own touch. And so, I think you can train it to get almost there and in some cases there, but all of those things I think are important context the memory but also you providing feedback

1:00:20Satya Tammareddy:and just like you would to a human yeah yeah the feedback I think is the that's how you train it right you're like hey here's the stuff so going back and say these are the things I changed yeah after what you did yes thanks for what you did but here's what I did and this is what I'd love you to learn from and there's two kind of chatty tips that are good around that so you can easily create a skill around that. So that's just basically a text file that you associate with it. But if you just say, hey, I want to create a skill around that. And so feeding in all your historical stuff, all the stuff that you think really represents you.

1:00:54Satya Tammareddy:I mean.

1:00:55Rowena Westphalen:And actually on that, it was interesting because it created the skill for me without me asking. Yeah. Even compared to three months ago, I feel like we've come a long way there.

1:01:04Satya Tammareddy:Yeah, which is nice. And then that makes it much more easily shareable. Well, thank you both for saying yes.

1:01:11Rowena Westphalen:I know you've got loads and loads on your own. That was fun.

1:01:13Satya Tammareddy:It was great. Yeah, thank you.

1:01:17Rowena Westphalen:Thank you all so much for joining us for another episode of Wild Hearts. If you want to learn more from other ambitious people building, designing and creating the world that we all want to live in, then please hit the subscribe and follow button. It would mean the world to us, the founders, the operators and the investors who join us on Wild Hearts. this podcast is a labor of love from blackbird and our production team the show is produced by joel connolly of blackbird our marketing genius is laura coford and our editor is andy jones thank you all so much for listening and we'll see you all next week