SED News: Perplexity’s Chrome Play, Meta’s AI Freeze, and Intel Becomes Too Big to Fail

0:10Hi, and welcome to SED News. I'm Gregor Vand. And I'm Sean Faulkner. And this is the version of Software Engineering Daily, where we just take a spin through the last month's headlines. We go into a main topic in the middle, where we go into a bit more depth about something very specific. And then we take a fun look at some hacker news highlights, and then just give some predictions for the week ahead. As you may be able to hear, I'm a little bit sick this week. The news must go on, though. But I might not be as chatty as usual. So Sean is going to be doing more of the chatting today. So I thank Sean for that.

0:47Yeah, absolutely. I like to chat. I like to talk. So I'm happy to step in, step up for you. But as you said, podcasting waits for no one. There's no sick days in podcasting. Absolutely. What have you been up to over the last month, Sean? Yeah, so it's been a busy month. I had some vacation time. And then the company I worked for, we launched, my team launched three products recently. and now we're sort of in a sprint from now to the end of October where we have our big annual conference current New Orleans this year so there's going to be a lot of work between now and getting ready for that with a lot of different travel in between it's exciting times but what's going on how about yourself yeah it's well it sounds like summer is very much over for you and yeah it feels the same this way I've managed to actually take a bit of a break of the last three four weeks which is very unusual to get up in Scotland that's probably when I come to Scotland I always seem to get the flu or something.

1:40So that's that. But yeah, it's been good. But yeah, likewise, summer is definitely ending. I'm going on to some different projects in the next few weeks. So that's exciting, but definitely ready for a lot of work ahead. So with that, maybe we'll just run into the main news headlines. So these are things that might have popped up on the main news outlets. So like Wall Street Journal or Financial Times or like TechCrunch, whereas Hacker News is kind of more the niche stuff. So the first one is Perplexity have been making a lot of waves. We've got a couple of things on them. Yeah, I mean, this is kind of a crazy one where Perplexity offered allegedly$34.5 billion to buy Google's Chrome browser, which is almost double Perplexity's own valuation and way above what probably most people think Chrome is worth.

2:29And then while this is going on, there's a federal judge is weighing in antitrust remedies after ruling Google illegally monopolized search, perplexity seems to be signaling like, hey, if you force a sale, we're ready to buy it. And of course, I doubt Google's not interested in this. And they've said that selling Chrome would obviously hurt their business and even raise security risks from their perspective. Analysts don't think the sale is likely, but the bid itself shows how I think aggressive some of these AI companies have been. And they're interesting getting a full hold in search. But to me, it feels like not a real acquisition attempt and more like a PR move to stay in sort of this antitrust conversation.

3:10There's a lot going on. Let's kind of attach ourselves to that. We can make some headlines. Yeah. Because I mean, the bid that they made,$34.5 billion, is almost double their own valuation. So unless they're going to go get a bunch of debt on that or something. Yeah. Can you get like a mortgage loan from a bank to buy Google Chrome? Yeah, exactly. It's a lot to watch that meeting. So yeah, and as you say, Google said it's not interested, but that's the point is to get it kind of into the news around antitrust. I mean, there's been a lot of noise around Google needing to break up. I think it's funny, we still talk about Google as Google, even though the parent company is Alphabet.

3:46Everybody just seems to have forgotten that and it's still Google and all the Google products. Yeah, I mean, it's kind of like Facebook with Meta and stuff like that. I mean, a lot of these conversations remind me of the early 2000s with all the antitrust stuff around Microsoft as well. And there's been sort of rumors of this coming for Alphabet for quite some time. Yeah. But also perplexity, I guess, maybe another PR move, perhaps, but they're talking about a slightly different ref share model with publishers. Yeah. So there they want to essentially introduce this new version of their revenue share.

4:21And they plan to distribute the money when an AI assistant or search engine uses new articles to fulfill a task or answer a search question. So, I mean, it makes a lot of sense. Like if you are switching from offering blue links as search results to something where you're giving essentially a fully baked answer, can you monetize the answer in some way where you give credit back to where the source material came from? And I think this is something we touched on last time of how might that impact the types of answers that you get? Like if you can essentially pay for the answer or sometimes be part of the answer because you're paying for it, are you necessarily surfacing like the best answer and doing right by the users?

5:00Now you can make the same argument for conventional search, but at least with conventional search, it's clear, well, somewhat clear, which are the ads and which are the actual like organic results. So I think it's something interesting to like pay attention to. I am a little bit wary of ultimately how that could impact the value of what some of these models are able to provide users. Yeah. And I mean, I think, yeah, to your point around sort of paying to play, it's going to be interesting, you know, this is cured at publishers. So like, which publishers are they picking? Because if they kind of let everyone in the door, this is just going to be clickbait article heaven, you know, where, oh, we're designing this article to be featured in an answer.

5:42And, you know, interestingly, the model is moving to comet plus which is their browser offering they're offering 80 of that revenue to the publishers which is very interesting i mean that's a polite way of putting it where they're going to like take literally just the revenue off a product and say if you're featured we're going to distribute i mean there must be some kind of ratios depending on how you come up in results but yeah it's a very interesting way to where to peg it and 80 sounds like a lot to me but i mean that will come down probably, but yeah. Yeah, definitely. It does seem like a lot.

6:17And I think it's one of those things where whenever there's kind of like a new form factor with which to get your message out, like a new sort of marketing channel, what inevitably happens is the early adopters of that channel will be a lot of times like very successful because they're kind of like early to it. It's like if you're early to Facebook or something like that or social media, then it's like, oh, wow, it's just like untapped channel where everybody else is stuck sending emails. Emails is really noisy. Let's exploit this new channel. The problem is once people show success there, it's like everyone just like drives in there and eventually the channel becomes this noisy channel that gets devalued and people pay less attention to.

6:56So I kind of think that we might be leaving the heyday of the chat LLM experience if this becomes too popular. Yeah, definitely. So moving on, next one is Intel, which has just sold 10 % of itself to the US government. Yeah, I mean, I think this is pretty big news in the world of semiconductors. Essentially, that 10 % is worth$9 billion. It's one of the biggest government interventions in a private company since, I think, the 2008 auto bailout. The deal came directly out of negotiations between Trump and Intel CEO, Lit Buhtan, and is framed as this way of strengthening America's chip-making leadership.

7:34And recently, I read the government AI action plan for the U.S. government. And part of the strategy outlined in that document was making sure that the US stays ahead of the rest of the world when it comes to like CPU, GPU innovation. So this seems to be somewhat in line with that. And we saw shares of Intel jump based on this news. But I think it raises some big questions of like, can the government ownership really turn Intel around when it comes to struggling to keep up with rivals like NVIDIA? or does this make Intel too big to fail in some fashion like with taxpayers kind of now on the hook?

8:08Yeah, it is very interesting. As you say, it's like the biggest government intervention in the US company. I mean, this is a public company. So there's public shareholders as well who are like looking at this and going, well, now one of our shareholders is the US government. That's quite different to most companies. It does kind of have echoes of China, for example, where virtually all of their companies have state ownership in some respect. This just feels, yeah, could this be like tip of the iceberg where the government is starting to step in on certain, I mean, Intel was struggling. So like, it won't just sort of start jumping in on things.

8:43But yeah, I think too big to fail is a good way to think about it. Just given that these people that can fab chips effectively, it's very hard to do that. Only a couple of companies, two or maybe three can do it properly obviously tsmc is the big one taiwanese so they're kind of in a weird gray zone between china and the us so yeah this just feels i mean i i can't say i support this exactly but i can totally understand where it's coming from and at the end of the day intel's had a really rocky ride recently they've really struggling on the ceo front and i've still got that book high output management by andy grove and he was he was the one who took intel to its highs.

9:22And then he died quite a little while ago. And they just don't seem to have hit their stride at all. After that, I had many MacBook Pros with Intel processors that were absolutely shocking. So I was very happy when Apple moved on to its own silicon. So it's one to watch, certainly. I mean, it's a good example. I mean, who knows what's going to ultimately happen to Intel. They're one of those companies where if you went back 20 years ago, it's like hard to imagine Intel being in the state that it is now. It's like, there's all these companies that I think we talk about today, where it's impossible to think about the world without those companies.

9:58But these things happen. Not that Intel is necessarily going away. But like, there is always the fear of a company becoming sort of like the next like Xerox or something like that. And I think that some of the things that we're seeing now around this like battle around search, even going back to the stuff we talked about with perplexity, like, if all the high value searches, essentially the equivalent of high value searches go to chat gpt gemini perplexity these types of interfaces then what does that ultimately do to like a behemoth in the industry like a google yeah so moving on to our kind of final big headline it's meta unfortunately again but we did try not to put meta at the top of the show again i think they're just trying to make news so this is around the fact that they are now freezing ai hiring is their strategy is to make news and press is a very expensive strategy.

10:47So we talked a bunch about meta AI hiring and AI talent battles that are out there in the last show. And apparently meta just hit the pause on AI hiring after months long spending spree, where they post like more than 50 researchers from open AI, Google Anthropic and others. And sometimes there were offers of like hundreds of millions of dollars. Zuckerberg's been personally involved in the recruiting, even paying out 14 billion for scale AI to land as co-founders as Meta's chief AI officer. So a lot of money, a lot of expense. I'm not shocked that there could be some investor pushback from that.

11:23So Meta's frozen hiring and they're reorganizing their AI division in the four groups with one focused on super intelligence. I think the other challenge, it's kind of like if you do a lot of acquisitions at once. It's one thing to be, hey, we acquired all these great teams and companies and software. It's a whole other challenge. So like, how do you actually integrate that and make sense of it? And I think they could be in a similar state where they just because you acquired all this talent doesn't mean that talent can work together behind one singular vision and actually deliver on something.

11:50So I think the jury's still out on terms of is Meta's lavish bets going to end up leading to a new generation of innovation for the company and they'll be able to catch up in some of the AI races and offer something compelling. Yeah, it sort of feels like they made such a splash with the amount of money they were putting on the table or the package. amounts they're putting on the table. And you can only imagine sort of internally, there was probably just a lot of like, all right, how am I going to get on this AI team and triple my salary and all this kind of stuff. So there was probably quite a lot going on there.

12:23But excuse the pun, I feel I did make a bit of a meta prediction last time, which was that this could be a vanity project. I can't see how just assembling kind of the super band of AI and hoping that the album is great like i just don't know how that works i've never seen things work where people just throw money at a bunch of disparate people put them in a room and say go i don't know i can't think of many examples where that's worked or like go but go really fast and like we need you to go as fast as possible that never tends to actually create great things so yeah let's see and they spend a lot of money developing like the metaverse which i don't you wildly successful.

13:08So I think if they end up in a situation where they are spending wildly on AI initiatives that don't necessarily go anywhere, it could be problematic for the company in the long run. Yeah, for sure. All right. So let's move on to our main topic. So something that Sean is quite the expert in. So we're just talking everything agentic. We're just kind of looking at where are we with agentic and agentic workflows and so on is kind of what everyone is trying to anchor themselves to at the moment is kind of my feeling you know whether you're anthropic or whether you're like a company that's actually been going for say five years and now you're you know you need to tag ai on your name but it's not just ai you're then saying oh but we're the agentic blah blah blah but the reality is kind of where we're trying to look at today so maybe if we kind of look at this in three stages just like where is the disconnect between reality and what's going on what is actually happening in agentic and like this real and then just looking at any of the main innovations in agentic workflows but i'm gonna handle it just on you're the expert yeah so i mean just i think looking broadly there's a lot of i think recent news around what is truly the roi for this reports on hey most of these projects are going to be cancelled in the next couple years it's unclear what the business value is we don't really understand the risk controls.

14:30There's all these both technical and also, I think, sort of business operational challenges around bringing these AI systems into production. And I get that. I think one of the challenges in the industry, and I often talk about this with different businesses I work with, is that there's the reality of what you can do with agentic workflows or even models in general today and where you can get value as a company. And then there's sort of the hype cycle around this, where people are vibe coding demos or like you see these keynotes that look really impressive and things like that. And I think there's clearly a separation between what's happening in the zeitgeist of hype and what you can do in reality.

15:13And I think unfortunately, a lot of the hype is distracting for where there actually is value. Because I do think that there's generally value, but most of the hype is kind of focused on aspirational things that I don't think are realistic right now. Yeah, like hype being, oh, we can have like a fully agentic employee is like a extreme example. I think that all that does is create fear in the market. And I also think it's unrealistic. There's so many challenges right now of actually being able to build something that is fully autonomous that you could trust and run reliably. If you think about like a multi-agent system, where you have a bunch of, let's say you basically have a bunch of nodes in this, maybe in a graph or maybe it's a dynamic graph structure.

15:55Well, if each of those have a probability of success of, say, 90 % success and 10 % failure, but each of that compounds, then you end up with essentially the probability calculation, you multiply the error rate. So by the end of that, if this thing's like 10 nodes deep, you might have only like a 30 % chance of success. So that's not a very good success rate. So the idea that that could replace a person right now in most tasks is, I think, unrealistic. That being said, I think that there are a lot of sort of meat and potatoes, unsexy use cases that are tremendously useful. And a lot of it has more to do with these kind of human on the loop experiences where you use a model, maybe component of it is sort of agentic in some fashion where there's some dynamic decision making, and it's able to create some sort of first pass type of document.

16:47Let's say you take support tickets and then you want to be able to give like a support engineer, arm them with a potential solution, or at least the materials that help them solve the problem faster. Well, I think that is a realistic scenario. We run tests on those types of things where, and had humans, you know, evaluate and score those, where we get like quite good success rates. And if you can make those people significantly more efficient and kind of take away some of the not fun mundane tasks of just collecting all this data together, going inspecting certain logs, and you can have an AI system basically go and do some of that grunt work and kind of bring it together.

17:23That's very, very valuable. And I think from an ROI perspective, if you have to pay somebody to kind of run around and stitch these things together today, that's a very expensive use of human talent, where I think the token cost is certainly justified. You know, if you even think about like the legal domain, there's a whole pocket of legal where you pay people to basically go into these rooms and like find an obscure document, needle in a haystack. Well, LLMs are very, very good at that kind of activity. Yeah. I mean, I guess maybe two of the core examples that keep coming up as quote success cases is the almost like it's either like IT support or employee onboarding.

18:02And where does Agentic become helpful in that regard? well with IT support, you might in Slack say, hey, my VPN isn't working. And it can go off and actually look up not only just the documentation for that VPN, but it can actually, for example, maybe in a browser, look at the configuration of the VPN and sort of reason about why that might be a problem. Capital One's tech team isn't just talking about multi-agentic AI. They already deployed one. It's called Chat Concierge in a Simplifier in Car Shopping. Using self-reflection, and layered reasoning with live API checks. It doesn't just help buyers find a car they love.

18:40It helps schedule a test drive, get pre-approved for financing and estimate trade-in value. Advanced, intuitive and deployed. That's how they stack. That's technology at Capital One. The other example that again seems to be just getting reality check is good. It's coding again. And it's a lot of it now is in the terminal, cloud code, other such things. So it's interesting. we're already slightly moving away from in the IDE to just in the terminal. What's your take on those? Yeah, so I think one of the reasons why some of the stuff in engineering is kind of the tip of the spear when it comes to being able to leverage some of these, I guess like the edge of what these models are capable of right now is because you have ways of kind of validating the output.

19:28Because one of the biggest challenges with a lot of these problems is it's very hard to validate whether what the model produced is actually correct or not. Like it might read totally fine, but it's hard to know for sure, unless you're an expert, of how to actually like validate this. In a large scale system, how do I validate that or have some confidence at scale without having to like have a human always in the loop? But with engineering type tasks, I have some checks and balances. Like if it's code generation, well, I can compile that code. I can run the code in a sandbox environment. I can run unit tests.

20:02I can run integration tests. So I have a bunch of stuff that I can essentially have some level of confidence that it's executing correctly and doing the right thing. I think we're actually starting to see some similar things in sort of the DevOps world where if it's something like automatically diagnosing an issue in, I don't know, let's say cloud resources, and there are ways of essentially trying things and then undoing it. So you need to be able to have something that like is automated or at least you have like a lot of confidence around that doesn't need as much handholding. You need a way to kind of be able to roll back changes if you're allowing like an agent to take certain actions.

20:40Like it has to be action and you have to be able to almost think of it like a commit log in the database where I can run some sort of transaction, but I can undo that transaction. I have this kind of log that can go back and replay and so forth. And we've built a lot of technology around that over the years to help us solve some of these problems. And I think not all those things have been baked into some of the things that we're building and trying to productionize today. I think there's a big disconnect between being able to put together like a compelling demo internally at a company and then getting to a place where you can productionize that.

21:10And a lot of those challenges have to do with how do I eval and test these? We don't have great solutions for that today. It takes a lot of hand curation to essentially do that. And how do I have some level of confidence of validating the output is correct? And then there's also this big challenge around sort of the data that describes the data. So when you're building any kind of Gen AI experience for a business, the big hurdle is that these models are really powerful and they're very, very smart kind of about like public information. But they're kind of dumb when it comes to like your specific business, your specific task, your customer information.

21:45So you have to essentially go and collect that data and provide it to the model so that you can steer it in the right direction. It has the right context, essentially, during prompt time assembly to be able to come up with an intelligent response. And there's a number of challenges there. One is simply like, how do I go and retrieve the data from all these different locations? But on top of that, there's kind of this larger issue of even if I have the data, is the model necessarily going to understand how the data relates? Because when you're talking about human language, the way these models worked and are able to get this understanding and air quotes of human language is because they've been trained on essentially billions of documents where the relationships between words and sentences and so forth, the structure, they're able to form essentially a statistical model that helps them understand that by just seeing this multiple times.

22:39But those patterns don't exist in sort of the land of spreadsheets and databases and where customer data is today. Like the model is not going to understand that some obscurely named column in your database represents, I don't know, like some sort of customer ID that has to be factored in as a foreign key to another table without explicit instruction. So this metadata, the data about the data is really, really important. And most companies don't have a good way of essentially conveying the metadata because most of it exists inside their heads. I mean, have you seen any real cases, because this feels like from a fiction point of view, obvious, but any real cases of where companies are actually training their own foundational models on their data, which is different to say, let's just not putting out an advert here, but Glean is, hey, we'll pull in all your data from the organization across lots of different places.

23:32and you can query that and get more contextually better results but what we're talking about here is if you're going to set up an agentic workflow well which large language model are you using and as you've just said if you take one off the shelf it's not going to have like this super context or understanding of your business from all sorts of angles and like well if it could have seen that spreadsheet before it answered the question it would have probably given a much more accurate flow and output so yeah what have you seen anything like that i mean some people do some level fine tuning there's not too many companies i've come across that are like from the ground up building an lm maybe a handful that you could count on like one hand but the most people aren't doing that some level fine tuning but even there there's challenges because with fine tuning since you're adjusting not all models you can fine tune there's not necessarily ways of doing that if there are these, you know, API based models that are hosted somewhere else.

24:30But then on top of that, there's a challenge of if you are making sort of these model weight adjustments and then new models come out, or I want to try out a different model because another company comes out with the next big innovation models. I don't have an easy way to kind of transfer that knowledge. So it can become actually a sort of barrier to innovation. And most of the people that I see doing some type of fine tuning have very specific reasons for doing it. Like They're quantizing a model to shrink it down and run it on an edge device, and then they're fine-tuning it to get the performance that they need for specific tasks.

25:03Or it might be really domain-specific, but I would be cautious about doing that from the get-go. What most people are doing really is trying to figure out how do they contextualize the prompt. There's this whole area around context engineering now and also gathering the data. And then how do you also encapsulate the metadata? So there's a lot of things going on in the industry right now where I think the new battleground for data is really about the metadata. Because the most valuable for like an AI perspective for being on these models and make them task specific are the most valuable data you can feed them.

25:39Besides the raw data is the data describes the data. So that's like semantic layers, knowledge graphs, ontologies, like all these technologies I think are having a huge resurgence. But it takes a lot of work to actually get these things out of people's head and encode them so that you can make the models reliable. So that's a big barrier. I think a lot of the challenges around building these systems successfully today has fundamentally to do with companies just have poor data posture to begin with. And it's all about the data. Like all these AI problems are fundamentally data problems. So we have these powerful models, but most companies aren't in a place from a data perspective where they can even take advantage of them.

26:17Yeah. And that slightly runs against at the moment, if you look at any companies selling agentic workflows, they're all saying enterprise, that's where they're trying to get to. And yet, here's you saying, but sure, but they don't have the data in a place that can make that a reality. And then, you know, okay, well, could the enterprise do as an internal project, for example, but, you know, it's that then classic thing of like, well, how much time are we going to sink into this prototype? is it just to show everyone it's possible and then walk away from it or are you as an enterprise really invested in trying to automate away sort of drudgery of not necessarily coding because we're kind of doing that already but like drudgery of other roles because i think that hey we'll automate your drudgery is kind of the sell for most agentic to enterprise companies but i just don't have a feeling that there's just a lot of uptake other than probably very low paid prototype type projects basically i mean i think the things that i tend to see that actually hit production are you know a little bit more work flowy in nature than these kind of like you know truly egetic systems where you just have like a bunch of kind of nodes floating around the ether and you're going to figure out the graph dynamically i think they're a lot more hey like we want to take this particular input and be able to process it to some portion of automated processing to hand off to a person to then sort of justify.

27:45So if you think about, I don't know, something like, you know, loan underwriting, well, loan could come in, that could be the input signal there, you know, like, okay, all inputs are going to be a loan application. So you already have constraints around what you need to interpret. It's not like anybody can just come and ask anything. So it's a much more like sort of closed world problem where I understand what the inputs are. And I can break that down into kind of like a workflow that a person might execute themselves. And there'll probably be some form of maybe dynamic decision making as part of that, like which data resources and tools it might need to interact with.