Claude Code for Finance + The Global Memory Shortage: Doug O'Laughlin, SemiAnalysis

26:06Doug O'Laughlin:It makes issue, it makes errors on the margin, but you can kind of force it into like, for me, I love using rubrics, right? Hey, I care about X, Y, Z out of 10, score this, and then you can really do multiple things. It helps with the stochastic thing. Do you put it all in one prompt, like the task and the rubric for the task? Or do you put the rubric after all the tests is done i actually have two versions of this i'm like hey you can pull all this stuff together just run the put for the rubric or whatever yeah or you can do the task and the rubric it just depends on how you want to do it yeah because obviously if you put it task and a rubric then it can iterate itself but if you put it after then it's probably more like you pay attention to the rubric yeah exactly and well in the other part part of it too yeah it will iterate but like the context rot doesn't matter i kind of like it to be separate because the thing is it's like okay it needs to be this like fresh look at it you have to think of it kind of like it would perceive anything anywhere right it just each context window is just opening it up and i think sometimes if you have done if you do it together it commingles the information to the point where it becomes biased or susceptible opus 4.6 as you know is like super sycophantic like it loves to like say yes okay yeah i'll do this for you yeah i think having it separate keeps it like keeps some of that drifts kind of away and that's like one of the things that i've really personally i like the results better but it's it's just complicated like part of this is really weird because i am i'm weirdly now opinionated on taste in terms of how you should design things because you can like for example the context rot thing until someone explained it i was like oh my god thank god someone said it this is a huge deal there's this like meme where it's like yeah Well, do you see the meme?

27:45Doug O'Laughlin:It's like of mice and men. And at the end of the book, I can't remember which character shoots. I've never read it. Yeah. So one character shoots the other guy. And it's like some guy made a meme about it being like, oh, this is after your cloud code is garbled, you know, five million tokens. You're like, OK, it's time to put you down because the context raw is huge. So, yeah, this is an example where. So what are your compact practices? Do you sort of aggressively compact manually or? So I personally, with the new OneMill, I feel like I try to do it all in one complex window. I'm not doing ginormous.

28:17Doug O'Laughlin:The OneMill is very new, right? Yeah, OneMill is very new. But it's a big deal too. Because your skills and whatever your Cloud MD is a percentage of OneMill is so much smaller. So you just get so much more oomph, right? Because the 200Ks are just wiping over and over and over. That's a big deal. I think it's a huge deal. And also with how the agents are working, the sub-agents will have their own contacts window. And then the pasting kind of like really saves that big, you know, the one million. You just want a really high quality project within that. That's the best, in my opinion. Compacts just kind of start the compression of the noise.

28:53So, yeah. Mentioning sub-agents and multi-tribes. So, first of all, I wanted to give a shout out to this thing from Enthopic Research where they were like, here's our production traffic. and they did a report that was kind of like, they're equivalent on the meter chart. And there's a lot of people saying that, oh, you should, you know, software engineering has PMF, but here's the next list of everything else. But what if they're also just software engineering, right? Like software engineering is like 50 % right now, but there's nothing stopping me from continuing to go to 80.

29:23Doug O'Laughlin:I think maybe what's going to happen, this is like maybe a giant dream take. It has like data analysis in here, which that's what you were doing. Yeah, that's, in my opinion, that is downstream. So I think how we should think about it is software engineering might all be downstream of chips, which is downstream. Like chips is upstream, and then it's AI, and then it's software engineering. It is all the extension of that same compute hierarchy. And I think the teaching where machine and code kind of inter or in the world intermingle right now is code. and so that's just going to be the bleeding language that's used to to figure out everything else that's that's my belief like yeah it doesn't make sense to build like for example this is a perfect example this is like excel cloud for excel is much worse than cloud code using python to use the excel skills to then deposit into it's all much worse it's much even even all the work they're doing yes a hundred percent because if you think about it it's it's a legacy why make a car engine fit into a horse carriage.

30:24Doug O'Laughlin:It should just be in a car. It's like a backwards compatibility thing where it does work because LLMs are relatively generalizable like this. But why bother? Because that same abstraction of information on Excel, it's just in that because it's human formatted for us to understand. And I think that that's the important distinction. All of this information stuff, all this software stuff is just to be consumed by humans. It doesn't matter. Yeah. If they're just as good at putting the data together, we should be much more concerned about machine focused of like software consumption. And so they can like, you know, the LLMs and the agents can put and synthesize all the information and deposit God knows however you want it to be.

31:04Doug O'Laughlin:I don't need to make a chart in PowerPoint or Excel. It will just deposit the Matplotlib in a chart to me, in an image. Fine. Are you trading Matplotlib? Yeah. Wow. why why you know it's better it's better i know understanding that code yeah so why ever make a chart again yeah if it if it's better it's just like it could be inconsistent with like the other charts that you do yeah i don't care that much about i i don't think we would care that much but i think one our new charts are better than our old charts yeah and number two i think if it increases the speed of information that matters a lot yeah and so i think we're much more so pretty much the new charts will outweigh the old charts because they'll just grow so yeah i think it it is a little inconsistent we have the same watermarking honestly i think it's better than our old formatting anyways well the first thing this looks reminds me of is bloomberg i was like you guys are just like you know becoming bloomberg uh which is a nice yeah couple things i wanted to sort of double click on because this is just a cloud code like brain dump yeah and one of the the biggest set of cloud code shows in the world which is sub-agents and agents swarms i don't know if you've tried i have tried them pick pick either one at whatever you want i have a controversial opinion that claude does not do rl on the agents forms or agent team yeah it's just an experiment it's just an experiment uh thank you thank you because no one not we exactly because the pro it's just via prompt and it's actually very bad i think sub-agents are okay because they usually We have a quad MD to go do whatever.

32:34Doug O'Laughlin:But the agent team is actually really good. Okay. Well, we can't knock it because it's experimental. Yeah. No, no. What do you try it on? Well, it was like some big data analysis of like many, many different companies with different KPIs into a dashboard all in one. I was like, hey, can you just make this all whatever, split up the teams? You know, speaking of that, though, you say that, but can we, can we to one agent swarm is actually good? I have also tried that. It is. That is actually really good. So I did some like, oh, example of things that I was never available to me, like internal benchmarking of these models and be like, hey, here's a set of problems I'd like you to do 20 times.

33:10Doug O'Laughlin:Can you do them? And then I can measure the performance between them and then like do qualitative. But like, what's the difference between X and Y? Yeah, that that was completely out of the hands of me, a normal guy like three months ago. OK, now it is completely available to me. That's awesome. Like I am very I care about this stuff. And now I have the tools that's able to automate and do a lot of this stuff because, hey, all the software engineering is like partially automated. And so, I mean, my experience is the 2.5 Swarm actually improves the model's performance meaningfully. The agent team makes it meaningfully worse because there's clearly not RL done.

33:44Doug O'Laughlin:So it isn't context aware of what's the best thing to be done. And yeah, so I think it's interesting. I like subagents because it's usually a little bit cleaner on a task to go do it and then come back. but the agent team is just very they had some post about how they did stuff yeah where it was yeah there's a bunch of rl for for this and i tried it myself i thought it was like pretty it's cute how they do all these like little games and stuff yeah yeah also it's crazy how like the setup you have to it's a lot of compute to just run the swarm i think it's like a 16 node of h100s okay and you're just like dang so you and i are not gonna be running and this is just to run and i'm sure there's concurrency available but yeah i think it's really cool and that's like i think then that's the sign of what's next.

34:25Doug O'Laughlin:Because, you know, these agents are going to get better to a certain extent. They're, you know, it's another benchmark to hill climb, right? But then it's going to be how many of these together in a bigger chain can you get to work? That you could argue it's kind of like a scale out of the reasoning problem too. Hey, how do you get these, like this one agent to essentially get a verified whatever, put it into a bigger process and do more information work? That's the next thing. and it's important to have context windows that don't garble up into random stuff and is able to do just like good enough with token efficiency, I think is a huge part of that.

34:59Doug O'Laughlin:Yeah. So yeah, that's kind of what our experiments have shown, at least in terms of like the agent swarm versus like not. I think it's very clear the agent team out of Claude is an experiment. But Kimmy, show better. No, they'll definitely do better. But the Kimmy 2.5 tells you that this is already, boom, perfectly great new place to do more work on, completely available to us right now. I think it's huge because if these agents get any better, like, I don't know, I'm never going to sleep again. Honestly, like, it's very interesting, this sort of moonshot AI, and this is a tangent. We're not really going to focus on this very much, but you know how, like, the sort of AI tigers out of China were DeepSeek and Quan, and then you were like, well, who are these, like, Kimmy guys and these sort of newer names, like, I guess, Minimax as well would be there.

35:46Doug O'Laughlin:ZAI has been around longer, but only recently, much more active yes so like i i noticed that kimmy is much more in the productization phase like as seen as opposed to like the quen's of the world the deep seeks of the world who don't really care that much i mean quen because of how it's attached to alibaba right like yeah yeah they they have a way to productize it but it's like it's like kind of like the gemini version they have so much stuff to do elsewhere right yeah but yeah kimmy kimmy's pretty interesting they're pushing so hard they got everything i know they got kimmy mammoth kimmy claw kimmy claw yeah i know kimmy claw i haven't yeah dude i was gonna say have you messed around with open claw because i did i oh yeah i remember what was it first called cloud bot cloud bot yeah dude i was gonna say it was really really euphoric i was like having it read all my emails and my calendar and do all this stuff and i was like wait wait wait this is really really really prompt injectable and i was like this is pretty secure and important stuff so i like i was like you know cloud code psychosis is good enough for me at this point i mean so what i do is i just have multiple emails right and there's there's a safer email to give to bots and i can let it use that and if it impresses me then i can upgrade it but cloudbot didn't impress me at the head i'm gonna honest with you i wasn't impressed either that was the reason why people were freaking out about this notebook i was like bro have you actually used this shit because it's not even right now on code code in a relatively focused terminal it will be like oh blah blah blah i'm like dude in the dot emv there was an like in the dot emv there was an api i told you to use for this sub case of problems and it's in your cloud md like please focus up like it still is like making mistakes yeah this is not like truly agi and there is harness you still have to wrangle this thing but it's not like a perfect skill follower and the the context in each attention window is going to like change and sometimes they'll be lazy sometimes they won't be but it's definitely good enough to do a lot of information with yeah i was gonna so i i use i use our i use our discord as basically like a a way to just bring information in and out i i just thought i saw this too where basically a lot of people are just setting up things that they could have done in Zapier with CloudBot because they're like, well, now I'm like AI pilled.

37:47But actually, they just done it more securely with Zapier. I think it's kind of interesting. I guess.

37:52Doug O'Laughlin:I do think it's kind of interesting. But I think there's... But the difference, though, is Zapier... I mean, I remember I've tried to use Zapier before. Yeah, it's also not very good. It's also not very good. The difference, though, is like... And that's okay. Like, it's okay to be early to something and just wrong because you weren't the one that made it happen, right? Right. CloudBot, the Cloud Code, CloudBot, whatever, all this stuff. The reason why it's so powerful is it gets to completion. Right. And like, OK, Zapier, maybe you can get to completion all the time. But like, man, it probably took you like eight hours of clicking through things and like copy pasting crap to make sure it all works and it's all secure.

38:28Doug O'Laughlin:And it's like, well, CloudBot did it or Cloud Code did it in like, you know, four and a half minutes. And that's good enough for me. You know, that that's a faster achievement. And so it's totally okay that they were right, but they were just not the right mechanism. You see this happen in the history of compute. I think there's also an innovator's dilemma thing where Zapier, as a pre-existing business, had this view of the world of automations as very strict on-rails workflow type things that their giant user base already uses. They couldn't really pivot that much. So that's why I think one of the co-founders left because they were like, well i can't exist within this like yeah like like yeah you you end up becoming with you know the the box will control you you you you are it's your it's your golden handcuffs yeah it's just like your cage you know you're gonna act like how you are in the cage and so yeah that that sucks for honestly that's i feel like that sucks for the framing i have is like your priors become your prison oh that's pretty good that's pretty good that's pretty good your priors yeah i haven't bogged that yet but i should you should you should your priors become your prison i like that a lot Coming back to Cloud Code, I also want to make this like the sort of Cloud Code.

39:35Wait, I'm so sure. No, no, no. I want to indulge because that's how natural conversation goes. And I think people enjoy that. And probably that's the only time we'll talk about Kibbe.

39:44Doug O'Laughlin:Yeah. So do you use hooks? Give me the Dougal Woffland Cloud Code setup. I had just essentially a few base skills. And then I have a lot of APIs. And then we've also made sure to work, and this is all work in progress as well, to have APIs for some of the semi-analysis information out as well. And so that way we have an internal server. An internal server that is accessed by people with an API so that all the semi-analysis researchers are able to hit some basic level of context because I think the context is really what matters. I'm too dumb to be really smart in order to have... Well, I guess I do have some hooks, if that makes sense, in terms of like...

40:23Doug O'Laughlin:I think hooks are very underrated, right? Yeah, I do think... Because you can do like a Ralph loop just with a hook. Yeah. yeah i feel like i underutilize hooks i i think that is true but i do i do run some version of them on like skill calls effectively like hey on this then you have to start pulling all this stuff but i think in the beginning i tried to do all this like hook stuff and like compounded stuff like that and i found that like you know the gas town ralph loop era it's like it it is a sign of what will come but i just don't think there's enough fidelity to like make crazy multi-turn something happens.

40:55Doug O'Laughlin:So like, okay, actually less is more, try to have like a strong set of smaller skills with a good amount of context information to be pulled in. And then at the beginning of every session, ask and focus on what you want to do. So that like it prompts the like, not like, you know, a cloud within a cloud, whatever. So here's the goal to finish within this single context window, and then get it done. And this is like my generalized research thing. Hey, I want to look at the price of NAND since 1984 or something like that. This is what I want to do. I want to, so like the, actually, no, let me just give you the best example.

41:29Doug O'Laughlin:That is probably not going to work. I would like to fine tune a time series foundation model to predict NAND and DRAM prices. Okay. I'm going to first start by gathering as much information as possible for all this stuff, blah, blah, blah. And then we're going to fine tune it, evaluate which ones we're going to do. I chose Kronos 2 because of covariates, blah, blah, blah, blah. Try to set this all project up we'll make it a versatile dashboard internally for for semi-analysis maybe we'll external if we want if it's a good enough product okay so then it like does all this stuff and then i just like start plowing away hey can you go research series of search api serper or exa or whatever you want to use to go look for all these different information sources and then bring it together right so this agent goes and gathers all this information this agent goes and like works on like considering the fact that the price isn't perfect to do all this fine tuning on and then we like throw it in i also had it of like well what do i use it showed me which gpu whatever we're renting on an hourly basis and so yeah we just pull all this stuff together then we fine tune it and i'm like okay cool how did this work and then we just have this constant iterative loop until i try to finish something i got to the point where i was like okay this this time series ln is probably not gonna work but unfortunately unfortunately the you said it was because of regimes or something else i think so it's regimes yeah there's no way this regime is so messed up For a lot of people who are new to finance, this is why I have an issue with all these kids doing stock trading games with LLMs.

42:54They have no idea. They've never studied finance. Past does predict the future a lot until something fundamental changes and the macro shifts and risk on versus risk off. They've never heard those terms. I had to explain it to people at Cognition. And yeah, the rules invert, completely invert. Like what used to work is exactly the opposite of what you need to do when you have a regime change. Exactly.

43:19Doug O'Laughlin:And it's very, very, very hard because, and the other thing too is you really like, okay, each of these are almost like a one-off onto their own. Right, which reduces your sample size. Yeah, which reduces your sample size. And so then at the end of the day, you end up being like, well, it kind of just like, I guess it's here's some heuristics, good luck, have fun, right? Here's your checklist to see it might be over, but you really don't know anything until then. So, but like, okay, an example of where this project was helpful. And it's like, okay, I'm not going to have the magic LLM tell me what the price of memory is going to be.

43:51Doug O'Laughlin:Hey, it was a good weekend project. I did burn quite a few tokens. But I do happen to have, after all this like information, synthesis and analysis, all of the memory prices of everything I could possibly find, plus the things behind API that we paid for, plus, you know, enhanced data sources. And I have all the covariates. So like, hey, WFE, what was the consumer sentiment? Every macro thing of all time. And, you know, what's really interesting is I am going to just be like, okay, well, now can you go make a summary of each and every memory regime and what it looked like and what created the beginning, middle, end, and put that in a dashboard so it's relatable and, like, easy, shareable, consumable within my firm and company?

44:30Doug O'Laughlin:Yes, I'll probably be done with that today. And that, okay, so that you're like, well, that's just gathering, doing information and stuff like you don't understand. No one's ever done that in the history of time. I know for a fact as the guy who like is like the cycle semiconductor guy I've written and done more work on the cycles that I think anyone else has at this point. Especially for like the older ones like the 80s and 90s and 2000s and 2010s. And like when I did it first time the human grok way brain is I went and I read these old annual reports and I put it together. I try to string a narrative through it.

45:01Doug O'Laughlin:And I brought through all I'm like what was GDP this year. What was all this stuff. and you have to like make all this giant sheet to whatever and then make the narratives. No, not on that shit, dude. I mean, this is like too much information to gather. It's like a lifetime of work. It's like a PhD project. I did it in a day, two days. Yeah, I mean, I think the pushback would be that then you don't have enough expert information to criticize the reasoning that went into the report that you're slopping out. Yeah, there is some slop. I definitely agree with the slop. So I think of it once again.

45:36So right now. By the way, that's also existential for you guys if you get caught putting out some slop to your clients. You have to at one point be extremely AI-pilled and number one in the world at applying AI to your productivity. Great. But also you got to. You have to.

45:54Doug O'Laughlin:So I think the thing that's really interesting is that this whole thing is a game of hygiene now. Yeah. Because I think it's like this is really hard. and I think about it all the time, I feel very comfortable with doing all this work because the thing is my, at the end of the day, it's done the work. And I've done the work. I have like a lot of like embeddings in my brain, a lot of information. The vibes that have got me in here is actually like tons and tons and tons of information, set up scenarios and like pattern recognition, right? But yeah, you're right. This crap makes mistakes all the time, all the time.

46:24Doug O'Laughlin:It is still just like a, like I think of it once again, as like a junior analyst, right? The analyst goes and does all this like really pain in the ass information. You bring it all together to make a good decision at the top. But the problem is, historically, what happens is that junior analyst, who I once was, went and gathered all that information. And after doing this enough times, there's a meta level thinking that's happening where it's like, OK, here is what I really understand and how this type of analysis I'm an expert in. Actually, I'm very good at. I consistently have a hit rate. Now I'm the expert.

46:52Doug O'Laughlin:Right. I don't think that meta level learning is there yet. We'll see if L1s do it right. But everyone who's spending one quadrillion dollars in the world thinks it will. It better happen if you're spending a trillion dollars and there's not meta-level learning. But for me in our firm, that massively amplifies everyone who is an expert. And we are a firm filled with experts. And so it's this hard part where I wonder if new people, we will be less lenient in terms of how much AI tools you're doing. Are you like junior or new to the firm? Junior to the firm. yeah a junior and like like because like you have to still do something that you can just like slop it up it's very obvious to me what it's slopped right when it's slopped and there's no cognition then it's like like whatever the artisanal last five percent is like that really matters but for me i know inherently what the five percent is i can like write it away with some really easy heuristics and time and like be like okay well this is the last five percent you fix this is what i believe just fucking make up these assumptions instead press enter okay cool we're good to go you know and so that's kind of the hard part that's a real hard part there is still a human in the loop right now one day someday it'll be superhuman but i definitely believe the where we're at today where we're there it's not there like you just compound all this noise and it becomes just like garbled just like all contexts rot but in terms of like the capability that is overhit like you know the human cpu in these this agentic swarm is very very powerful now yeah you know a huge huge huge multiplier of what you're able to do and for me that was enough to be like i feel agi pilled honestly because if i define agi as many common jobs not like i'm not i'm not doing asi that's like religion can it automate or change or or take or you know completely shift a lot of the information work yes 100 yeah like data analysis is a perfect example hey every quarter i want you to just find me some examples of some information that might be interesting.

48:47Doug O'Laughlin:I just can't imagine if I was an entry level worker doing data analysis that a 22 year old, an average 22 year old would, would murder the hell out of a relatively well thought out agentic system. And so you're like, yeah, that job actually does seem at risk. And so that, yeah, that the 4.5 capability enough like that, that we hit some level with gentically where it seems to work and do bigger information work. That's when I'm like, okay, yeah, this, this does change everything. And so, yeah, there's, there's all kinds of mistakes. I, It's a new level of hygiene that we have to do. You're going to have to understand what the absolutely of Gentic work is back to you, right?

49:22Doug O'Laughlin:I catch it making errors all the time. It doesn't always pull skills. You can definitely tell context windows. It gets dumber over time. It's not AGI today, but it can do these crazy long tasks. And as long as you finish it at the end and deposit it as information work, that's very valuable. Yeah, amazing. So you do a lot of client visits, obviously. By the way, transistor radio, amazing. for like understanding like what your world is like yeah are you also cloud code pilling your analysts and your eye you know on the other side i've definitely cloud code pilled the analysts everyone in the new york office i'm like must try it i like really tried to like not i mean not your like not the semi-analysis or your customers and all that and like my perception is they don't adopt any of this stuff okay so yes and no some people are interested but you have to remember it's relatively more conservative but i think but if you ask any analysts if they're using ai every single one of them will tell you yes i use it every single day of course how could i not this like a vital skill and so the the the basic the basic inference that i'm doing is i am a bleeding edge adopter i'm a relatively smart dude who knows what he's doing and if a tool is useful or not and i've evaluated the tool and i'm like wow this is an amazing tool that i literally like pry it out of my dead fucking cold hands okay i'm like this even if it's like makes mistakes i will be using this for all kinds of work forever then i look around to everyone else and be like most of these guys are enough like me that if they have an opportunity and an edge they will obviously apply it and they look at this tool and they start to use it if they start to use it and they're thinking like me they're gonna obviously adopt it i'm like well i don't understand why everyone doesn't adopt it i would argue what we'll see in the 24 month view it will be a base level i think I think cloud code, co-work, whatever is going to be a base level of all information work very soon.

51:12Doug O'Laughlin:And, you know, you see one, my friend was telling me how his portfolio manager found co-work and he's like getting it to read his emails. And he's like, oh, my God, I love this. Right. Everyone's moment is going to be a little different. But I think my moment, it feels like GPT 3.5 or 4 for me, where there's that first time where you're like, OK, I know I made some shit up. But like, this is better than like, if I went for hours searching, putting information together, it can, and then also it's like the analogy power, you know, where you can say, Hey, this is the setup. Can you describe it in this?

51:43Doug O'Laughlin:These like really strong pattern matching skills that are really powerful. I just think it hits some level of capability. I can't tell you what it is. It is like my tape, my personal taste where I'm like, Oh wow, this is completely over the, the, the, the chasm of what needs to happen for it to be a very, very powerful tool. And so, yeah, that's my cloud code moment. I think. There's some kind of automation chart. You know, XACD has this automation chart. And I think we need a version of this that is the cloud code. It needs to be much... But what's crazy is this, the cloud code thing like murders the axis.

52:14Doug O'Laughlin:Exactly. It just strips everything right or something. But also, what I was trying to figure out is, well, okay, it is maybe dumber, less human attention, but because you can spin it up so quickly and it can spin parallel so quickly and it gets done, you get more turns at the wheel. Yes. whereas in as a human you get one turn you get one turn but with with clock code maybe you get three turns and they the sort of review process is the thinking yeah and you just need to get very good at review or yeah or hygiene yeah i think of it as hygiene the thing that's like really going to be painful though is like a lot of my expert opinion has been built by like you know it's like pre-phones and not right like your attention span like you know the children are cooked okay like you know the attention spans are really bad all this stuff like i don't i read this like really sad thing we're like oh we're getting dumber or something first generation i don't know i'm not gonna maybe that's like you see the coinbase earnings yeah i saw the car so so like you have this thing where it's like okay and it's cute and all but like it's such an addictive technology that like i feel very grateful that i'm like well i understand what i'm doing have this history of doing stuff and able to apply a tool but like people who are riding this curve it's gonna be very dangerous it's like giving everyone right now i'm strong yeah that's so funny i i think you should just do that yeah well we we we do we do with some of the semi-analysis memes you know and and the thing is you say some of this brain rot is like so bad which is it is terrible but some of it is also like you know it is hitting some attention mechanism in my in my my deep primordial monkey brain i mean you yeah it's stimming me and you're like you know i can't look away from the the subway surfers so you couldn't look away i was i had to pause it yeah that was i literally i won't well hey there's like have you ever been at like a bar when they play like these like weird like they'll be like like tiktok videos for lack of whatever and you just watch there's tiktok bars in new york not tiktok bars not tiktok okay it's like there there's like essentially a b-roll channel that they'll like sometimes play in public spaces and you will just find yourself like being engaged with it like there are certain things that just it works so yeah sorry that's completely off but but i wonder this quad code pill is very powerful for me i believe it will change how it all works it'll shift all of that over massively the the chart but it's just really weird because if you didn't pay any like human cognition to get there i don't think you're going to be a great reviewer one of the reasons why you know what what makes that that human feet that loop well is because once upon a time you did that and you could make the three like yeah yeah idiot you're not thinking about this problem in this way you're missing this this like you know whatever you're not considering this 90 you know like the 10 tail something like that yeah and so it's like yeah i know you said this but like you know i i know you i know that i told you the valuation is the only thing that matters but like it's also a fraud you can't do both right like if you think about like the analysis stuff you have to know when your own and personal embedded model is like, yeah, actually this one overwrites this one.

55:16Doug O'Laughlin:That's through learned experience. And I wonder if we're just reviewing, we won't be building and embedding those assumptions to understand judgment. Right, right. Because you're just checking for mistakes rather than trying to do original thought by just doing the work. Yeah. Yeah, I think that is a danger. Yeah. And that's what hygiene sounds like to me. Like, hey, it's really addicting to be like, you know, whatever, press the button over and over and over. but sometimes you do actually have to like think, you know? So I think that that's, it's going to be really interesting. I mean, have you tried?

55:47Like, so, I mean, the way to model the sort of meta learning as element is like once a night you do a batch job of like, look over everything I've done, like extract some learnings, you know, and OpenClaw, I think one of the interesting things I really liked about it was this heartbeat. Yeah, heartbeat. And I'm like, people aren't like excited enough about this because like, well, this is the first instance where like the agents are just always on like always living always reflecting yes and like what is it sold on md2 so i think it's much more for character and like yeah whatever

56:17Doug O'Laughlin:but like yeah harpy yeah it's harpy harpy's the con yeah i mean i think yeah that's a good way to put it yeah and so like that's the powerful thing about all this stuff is that like okay yes we know that the contact like it gets garbled we know that open open claw doesn't always do everything you asked to ask it to do initially but you can see the design patterns like the heartbeat md is a perfect example can see the the design patterns where it's like well you know is all of our tasks every single day actually us having this like genius thing or do we like sit down in a single session finish a single project and get up and get some coffee then come back if it's that and you can just fuck you can make the heartbeat.md consider the like the session to session and like hey meta learnings all this stuff and it's only specialized and focused on one form of doing something so it actually does have a context of all the like let me i'm thinking like a customer service agent or something like that it does have the context in fact it can look at every single time it's ever happened that's actually information and context no human could ever hold you're like wait that that feels like like like that's effectively good enough to do a huge information yeah and have enough context and be able to fetch it and maybe like there would be some verification to make sure it doesn't just totally mess it up.

57:30Doug O'Laughlin:But that to me feels like a design pattern that you can build something on. And so that's the vibe is that we've hit some capability that you can build these much bigger blocks now. And those bigger blocks are not just like the single line of code. It might actually be a business. It's kind of crazy. I wouldn't have put myself as AGI pilled. I think 4.5 is like actually. I think my own timelines have moved up a lot yeah are you guys watching gdp val i to the best i can but i feel like i'm mostly just trying to no no no to me when gdp val came out so i i'll just yeah gdp val is like uh basically a like a broader street bench let's call it where it's like applied on every discipline every profession that is white collar that you can model and it is above like something like two to five percent of gdp something like that that's why it's called gdp val and they they had human experts do the tasks and as well as UPTs.

58:24And here's the results, right? Like where 50 % is parity with industry experts. Yeah.

58:29Doug O'Laughlin:Yeah. Coin flip, exactly. So like you can see the nice increase from 4.0 to Opus 4.1. And since then, obviously 5.2 and Opus 4.5 have already exceeded. We're at 70 something now. Yeah. Which means models are consistently better than industry experts at these things. Yeah. So to me, like this is the AGI definition, isn't it? Yeah. Yeah, this is. And so like, I think the problem though, Yeah, I would say that that is the definition. So the thing that's crazy is because there's like this ASI element that people are like really, really focused on. We're moving the goalposts. Yeah, we're moving the goalposts.

59:00Doug O'Laughlin:But I'm like, bro, I the the goalposts like I mean, we'll see if this is actually the machine god and Shoghith will come and talk to us and vibrate on our same. What if I do think so? OK, I don't I'm honest with you. I'm very open. I will change my mind often. I'm not this is not something I feel intuitively I got today. Maybe it's the next next next thing. But when it comes to like the GDP valve version of this, yes. Yeah. This is white collar work. White collar work. Which is most of the time. Very boring. Knowledge. Like actually it's almost all, not almost all, but it's a huge portion of all of work in the world.

59:35Doug O'Laughlin:It's like now we just made, like my favorite stat is like once upon a time, 90 % of people were farming. Right? Now today less than 1 % of people are farmers. it's kind of like this crazy shift where technology is going to massively change the relationship with all of that and it's going to be like this 99 one thing i don't know if it'll be quite that drastic or whatever maybe you know everyone's just doing leisure so far my experience is everyone just works harder and it's been my experience but it just it just feels like a massive moments happen like the the steam engines invented and you know the the trains are here and and everything is going to change in knowledge work.

1:00:11Doug O'Laughlin:And it's kind of crazy. There's a sort of economic cycle from my macro days that I can't remember the name. I can't look it up. But it's basically like there's this stages of economic development where like your economy starts up majority agriculture. Then it discovers like manufacturing. Then it discovers white collar work. Then it discovers, it builds like a very mature financial sector. And like these are like a layer cake that's all declining over time. And then the new things increasing so my theory is like there's this like fifth layer that's like has to open up that starts to happen because i do fundamentally believe we'll just invent new work i do believe that you know i percent like humans are very adaptable that's like my favorite thing i've learned you're able to adapt to god like coldest coldest place in the entire world the warmest place humans are in every latitude that's in a physical sense but i think we're gonna find a way to make utilization go up but we'll we'll invent more work for sure but i think the thing that's crazy is just like things change so quickly and that five to ten year period like 10 year gap can be drastic and crazy and that's just societally It's wild.

1:01:19Doug O'Laughlin:And it's happening in our lifetimes. It's happening in our life. It's happening right now. It's really crazy. It's very... So this is a complete side task. I'm really curious of when we start to see it in a much bigger way in the real economy. That's my pet. Yeah, why is it not showing up in GDP yet, right? So there's going to be... Some people are going to be like, oh, the facts. The facts, the internet, same thing, information transfer, whatever. I think I'm actually scared for a third worst thing, which is like now now this is a complete crackpot theory please don't hold me to this internet but what if ai is massive massively deflationary and and also i think one of the more interesting conversations i've had in a bit is like what was gdp was invented once upon a time as a way to figure out how much we could divert you know normal economy away just to war during world war like one or two or something like that okay my spiciest take is i feel like gdp itself is going to be very, very challenged by AI because information work.

1:02:18Doug O'Laughlin:Yeah. So how we, how we capture it effectively is all of an economic good. And then the service hours divided by hours. Okay. So there isn't like a widget to widget difference, but in theory, if we could break all of information work down into units, we're going to have a lot more information work for sure. Like more work will be done. I don't know what the value that's going to be. Is it going to be so much increase in supply it's deflationary that seems to be like a real concern it's possible yeah and then we'll figure out how to use it but like there may be a great depression of ai yeah where like we figure it out yeah well i i wrote this whole thing about railroad stuff because it's my favorite okay my favorite capital side it's on fab yeah okay i can't remember it's like railroad fab it's about all the railroad stuff over time okay pretty much because we're like everyone was first looking for the internet we've well massively passed the internet in terms of the absolute size of the build out it's not even close like what numbers are you thinking of like i think i think a trillion was a trillion all in was essentially the real dollars for a version and i think we are well past like we like whatever this year and it's cumulative right we will well pass that i think railroads the reason why it's so interesting is because honestly it's way crazier but but part of the problem and craziness of it too is like railroad was literally like one of the first added layers of the layer cake, if you think about it.

1:03:43Doug O'Laughlin:Before it was agriculture and railroad was like, okay, well, how do we move this agriculture around faster? And then banking got, I kid you not, like one of my big takeaways is banking effectively got invented by railroads. Oh. Because there's no need to finance it. Yeah. So much money was needed that like effectively 85 % of all paper or whatever was essentially just railroad debt. Yeah. One of my favorite anecdotes was before there was a federal bank, a federal reserve, Andrew Carnegie was the federal reserve. Yes. Yeah. There were individuals. Yes. Yeah. And so all this stuff. So it's like the whole thing.

1:04:13Doug O'Laughlin:I kind of did some work on the Gilded Age, all this stuff. But like my takeaway is like that was a really interesting cycle because it was so big and took so long to deploy. It actually was 45 years of like there's three cycles. Actually, there's three boom busts. I don't know if it'll be quite that long. All the cycles kind of collapse. Yeah. Because, you know, information moves around faster. Exactly, yeah. And so you have all this stuff where I think it's going to happen faster, but, like, I would be really shocked if it was all in one go. That's my vibe. Yeah. Where it's, like, it's all in one instantaneous up-down, I think it's going to look like some multiple cycles.

1:04:48Doug O'Laughlin:So, yeah. Kind of just wrote about railroads. There was, like, a baby railroad cycle. Then there was a huge railroad cycle. The modern railroads invented out of it. that's like my favorite analogy for this because like i think it was like gdp percentage of cap x each year were like high single digits for sustained for like 10 years yeah but what's crazy is like that amount of spend is like we're like well on track for that did you do percent of gdp because i think that's i think the way you make it convertible stargate itself two percent of us gdp and i mean it's gonna go up like yeah yeah that's a yeah and it's not all gonna be in one year right but it's okay so yeah so total capex for it was 4.8 percent of gnp and 25 percent of total gross fixed capital investment okay so 25 percent of investment every year and four five percent of gnp i think we're there you know stargate plus and tropic plus whatever yeah we're right there xai yeah so we're at a railroad build-up which was like at one point like but the thing is crazy we should exceed it like probably yeah yeah no no probably like we should yeah okay like Like, this is bigger.

1:05:54Yeah. Okay.

1:05:56Doug O'Laughlin:I'm like, I would like to say, yeah, sure. Yeah. We will do it. I'm worried. I'm like, dude, we're going to get all the money. That's like such a, like, the pedestrian concern. Yeah. It's not a pedestrian. I mean, that's what happens every capital. I worry, like, we must answer the Middle East. We'll flip the thing. We must. We must. Well, this happens every single time. That's the reason why the, like, bubbles happen, right? It's like, we essentially get so big, we're just like, this must be built. It doesn't matter the price. And then all of a sudden we look at it, it was like, ooh, that was a steep-ass price.

1:06:23Doug O'Laughlin:but i think i mean the thing i think about this is like how i think about the big picture is there is a demand curve and a supply curve and we have no idea when they cross they will cross one day and every single year the demand then we're finding that demand curve and then the supply curve we're just like we're doing our best to deploy it and i think for me like i don't know when that number is i'm not i don't want to say a number go up forever because i feel like that's like intellectually dishonest but quad code for me is the first time where i'm like and we're bringing it all back together where you're like demand go up so much i am now guzzling as an individual like for example i'm we're we're off i'm off max it's not enough it's not even anywhere near enough like i mean some people buy like five maxes yeah well yeah so so i i so i'm on fast i'm on fast with one million on api which is that is like an addiction level if any sense but yeah i i really think it's the first time we're like okay well actually how much is this worth to me on a yearly basis, I think it's like$20 ,000 to$30 ,000 easily, if not more.

1:07:23Doug O'Laughlin:I don't understand what's the... I can't price it. I have no idea of the elasticity. Yeah. You pay for a perfectly compliant junior analyst. Yeah. Right? And so what's that cost? Yeah. $90 ,000. That's able to work in parallel. You can have 100 of them. It's kind of crazy. Yeah. So it's... It's a skill issue if you cannot manage a junior analyst that is$20 ,000 a year. Yeah. 100%. Which, like, I mean, okay, like, you know, skill issues, like, it's your fault. But no, like, we have to learn how to do this. Yeah, it's like, it's three months old. Exactly. That's the correct way to put it. It's like, it was definitely a skill issue that you didn't know how to get, like, your settings on your iPhone to work, you know, at one point in time.

1:08:02Doug O'Laughlin:But, like, in the very first month of us having it, no one's going to be like, yeah, you idiot, you rube. You don't know how to use your completely new technology that got birthed last month. I think it's just about a time. It's a bit of time. And it's like kind of interesting because you're like watching. I mean, it's cool is that if you're like on this absolute bleeding edge, you get to see the design patterns like blossom in real time. And like with this like really old older guy who's like been through the history of technology since like forever back then. He's like one of the most interesting intelligent people of semi-analysis.

1:08:32Doug O'Laughlin:And he talks about how, who is it? Tanch. Okay. So like you said this, like we, we, we had the conversation one time. He was talking about like early internet, how like it wasn't actually sure if the browser was going to win. It was like a remote web file service. Some people thought like, well, it just, I'm just going to reach and play with someone else's web files remotely. Right. Who knows? Right. And that kind of, you know, it kind of is a remote web file. Who the hell knows? They were design pattern searching back then. And I think we're at that again, where all the design patterns are open.

1:09:01Doug O'Laughlin:and it's like really interesting because there's many different ways this could go and we're going to have to kind of collectively agree what's the best set of hygiene set of design patterns what's the level of abstraction and then like all the rest of how much sass it will disrupt all everything else who the hell knows but you get to watch it like front row seat right now yeah i mean my biggest one and i i do want to bring it to samiz in in a little bit but is the ide two months ago they we had stevie eggy from gastown talk about how 2026 would be the idea died and i like two weeks ago three weeks ago i recently was like shit he's absolutely fucking right it's over no i'm really wondering too because like ide so so i think that same my like my the reason why i'm so excited about this is i get to like look i never my daily driver was never an IDE, right?

1:09:53Doug O'Laughlin:My daily driver was like Bloomberg or Excel or something like that. But I have a personal belief. It's not happening yet because we're not quite there in the maturity curve. Like software is just gonna be first. But the year like of, you know, Excel is dead for finance. Yeah, like it's - Excel is the IDE for analysts. Excel is the IDE for analysts. Bloomberg is the IDE for analysts. I believe every one of these IDEs are done. It's dead over and going and dead. I just think it's, why, why? It doesn't like, just imagine the concept of you. Like I remember when I learned Bloomberg, I had to like watch videos to learn about all the random subfolders and keys, how to use this, how to use this, you know, the tactic knowledge of using this function versus that function.

1:10:35Doug O'Laughlin:That's like crazy to think about. That is like horse and buggy, okay? The agent with the information that can perfectly retrieve and analyze stuff is gonna have the ability to pull that all together in a better UI than it was with no legacy, whatever. I think all of that is dead. And this is why my spiciest take of all is Microsoft is a lot to lose. I think they have the most to lose of everyone. Because Excel is a human IDE for information work that's generalizable. So is PowerPoint. So is email. Those are the base core level of abstraction that I decided to be broadly generable. But I just don't think that matters anymore.

1:11:13Doug O'Laughlin:I think Claude Code or Co-Work or whatever is going to be the year that like it will destroy all of that. All that information work that that where you sat every single year, it's over. I think that's the one that's like more shocking and scary that like people don't believe. Like I believe in my stomach with conviction because I have already had that moment for me. Yeah. I will never make a chart in Excel again. I actually believe. Yeah. It's hard to let go because I have so much like ingrained knowledge of like manipulating things directly in Excel. Bloomberg, I have. So there's no way that you would know this, but like my very first startup was an attempted Bloomberg killer.

1:11:46Sentio decay office. I remember. I remember. Are you in an office? Yeah. Yeah. No, no. Oh, yeah. Yeah. You were one of the few. You had an OD. Sentio. I was a Sentio customer.

1:11:53Doug O'Laughlin:You were Sentio. I was a Sentio customer. Like I rolled in, dude. I remember how dare they acquire Sentio. I had a patent. We filed for a patent for similar tables. Anyway, one of my conclusions was like Bloomberg is just like three things. It's Slack and it's the journalism, which is amazing. And then it's the data feeds. It's actually not really the UI. Yeah. Yeah, but I think for the first time in my life where I just think that like, I just wonder if that like, okay, if you can get obviously, so you're telling me that the future, the undisputable future is just like, it's IB and nothing else.

1:12:25Doug O'Laughlin:And then like a terminal that types in some stuff. I think that if you are marginal and only curious and not hyper interconnected, which I would argue that I am at some analysis. Like, for example, I'm trying my absolute best to just rip Bloomberg out. We're going to fax it API. Like, all an API with a cloud code is my belief in the future. Hey, verifiable data source that you trust. Yeah, yeah. Hey, scale. For you guys, you can do it. Yeah. For traders, we're so... For traders, no way. I understand. Like, there's an information network that's, like, outside of this. Then you do deals in IV. Yeah.

1:12:54Right? And we start track by the regulators. Yep. 100%. Yeah. 100%.

1:12:57Doug O'Laughlin:It's totally... I completely... But as an analyst, yes. As an analyst, yeah. And so, like, but I just think that, like, okay, that doesn't really... So, you're right. The core cash flow Cal thing will continue onward. but like each iteration of this ai thing i was like yeah i'm still gonna be using bloomberg right this first time it's like actually no i don't care anymore the ib is my utils of like marginal value from from ib is like now outweighed by how like clunky this is and i want to just make some charts right and so that's immediately you save 10 to 20k yeah for switching down yeah there you go it's amazing yeah but by the way what was your cloud code end of your prediction 25 yeah i want know i sandbagged the ever-loving show oh okay i i just believe 25 is very like i like it's like a the rate it's on is like whatever 50 or something like that but i think i feel i wanted to give a 95 confidence interval i think 25 is within the 95 confidence interval sure so it's between 25 and 50 something like that yeah yeah it's just absurd but you know and it could also be codex are you watching yeah yeah so it's gonna be clear i'm actually even willing to comment on that because like i know we've done a lot of shit of being codex haters yeah i i think i i'm by the way when i put cod code codex agent whatever all in percentage that we can publicly see i would argue the for the ratio outside of that's probably higher too but whatever yeah i think together yeah we're watching codex i actually think codex is codex is pretty good 5.3 i think so we had the whole thing i because like i wrote most of the articles like oh token efficiency to context rod all this is the same one or yeah it's in the bottom it's in the it's in the paid section okay okay so but like tldr i was like well you know the reason why quad code is so good anthropic is so good is because all of this token efficiency the token efficiency is better than chat gpt all this stuff blah blah and then like 5.3 codex came out and it's like yeah that that completely doesn't matter anymore they're like they're they're so back i really think 5.3 codex is awesome in encoding though but you can watch it like the reason why i like opus 4.6 so much is because when i'm using it i'm using it for like coding in is the way i interact with it but I'm using it for broad generalized information work, right?

1:15:01Doug O'Laughlin:But I think the difference is Codex wants to code because it's RL to be so good at coding to win on Sweetbench that like you're trying to use it for general information. Like, hey, I'm trying to, can you go research and search all these websites? And like, I don't even think they have web search in it or whatever. Maybe, maybe you can give it an API or whatever, but it's like, great. I'm creating a piece of scraping software to go look at these websites. I was like, no, no, no, no, no. Just like, just ingest tokens of what's on the website. It's like, okay, great. I'm still, like, it's so coding-pilled on the URL that I think it isn't generalizable in the way that 4.6 is, where it's like, oh, I could have it.

1:15:36Doug O'Laughlin:I could have it make some rubric or do some research or do something like that versus Codex. It's very, it's very, coding Codex is coding-pilled. And so that's what, but I am very optimistic actually on Codex and we do track quite a bit. You can, they have a meaningful amount of thing. Share, you could see the Bloomberry. They have the chart, bloomberry.com. So the Cloud Code definitely is in the lead, but I think the part of it too is like the like-to-like comparison. There's a ratio of Codex that's not available because it doesn't sign off every commit. It does sign off on pull requests. That ratio is much closer.

1:16:10Doug O'Laughlin:So all the OpenAI people like Rune will tell like, well, we're not accounting for it. Yes, we didn't account for it. But like, I think Codex is better. I think there is some real problems and issues, but I bet you the second that they have a new pre-train with the RL, because the RL stack on Codex 5.3 is amazing. Like it's very coding code. That's when it flips over. And yeah, look at the other players here. It's just like, I mean, my favorite thing is that how GitHub Copilot is like number one. And like, I've never heard of like, do you know anyone who uses GitHub Copilot? Yeah, look, okay, that's a bubble talking, right?

1:16:45Okay, that's a bubble?

1:16:46Doug O'Laughlin:Yeah. Yeah, that's the CSF bubble. Yeah, yeah. Like, you know, there's like all these Windows users and you don't talk to them, right? Like you do, but we don't in Samsung. and that's fine. That's definitely bubble talking. But yes, Copilot has a billion in ARR, I think, at least. Yeah. What's crazy is Cloud Code has a ratio. Their attribution of Cloud Code and ARR is 2.5. Yes. So that on this, the daily install counts, right, is an order. Which is, by the way, just the VS Code extension. Yeah, I know, I know. That's not even a default way to use Cloud Code. Yeah, you're right, you're right. CLI, NPM, Damos, another way to track it.

1:17:23Doug O'Laughlin:But I think they have their own costs their own installer now anyways all in all definitely heard understand it's very hard for us to like actually track it but like i'm not i'm not criticizing i'm just like i think codex a big thing i'm watching is well it's codex back because they reported like jan to feb they double users yeah okay so so i have some not skepticism just because they have such a big chat gpt portal that It could be like TriAtlas. Like the modal that pops up can really move big users. Like they're not quite a Google.com in terms of having so much ability to like siphon off users off.

1:18:00Doug O'Laughlin:But I wonder like the like to like. But that's like my skepticism. I have an answer for that. Alexander Embericos was just on the LennyPod saying that they actually haven't invested enough in the web experience. So like I think the attribution for that is zero. Okay. Yeah. I guess I just saw a modal be like, oh, TriCo. try code i mean but the but a modal isn't and then to be clear codex in the in mac is great i'm actually i mean yeah yeah it's they actually read the app the app launch the the app launch is actually pretty good so yeah and i think i'm pretty bullish that honestly especially for coding because it's like very coding code i just can't get it to work as well for non-coding stuff then my you know you use conductor no i've not used conductor oh okay i thought i heard you say on a podcast no I've not used Conductor.

1:18:45So basically, like, the argument for any first-party app is that they're only going to prefer their own first-party plans. Yes, 100%. Which, like...

1:18:54Doug O'Laughlin:They're already doing it. Like, I feel like this is how they're going to differentiate, right? Like, they're going to... Well, then you have a conductor where you can use codecs and cloud code for different tasks as you see fit. And so this is the clean superset. No? In theory, yeah. But, I mean, this is like... Okay, so then you can argue this is the clean superset it feels kind of like i guess my design pattern on that is really skeptical of building on top of something that is growing very quickly and has money and whatever like i just think my favorite one is like platform as a service if you remember that one was like infrastructure as a service platform as a service sas software as a service and like oh this platform is a service and it's like it always just ends up being in the middle so it just gets even by one or the other i i think of that like middleware layer unless if it's a really really really compelling case often dies but that being said in this moment i agree i actually you i like to have them like review each other like having them yell at each other is really great i might actually try this soon i haven't used i haven't used conductor personally yeah i'm mostly just been you know going deeper into the psychosis yeah and this is as a former cloud analyst very typical of like do you want a multi-cloud or do you want to go all in one cloud and the the classic argument for multi-cloud is well then you can use the best of everything exactly but if you go all in one cloud you can exploit yeah uh the sort of minor features of everything and you know it makes a market and there's no right answer for everyone exactly yeah yeah i mean it's yeah that's what that's one of those things where like even the really small percentages in ai still really matter because they're the huge and like yes we're very happy very productive okay it's good to be an analyst in the space because it's fun to keep up with it right like i agree like i i think everything everything we like the horse race yeah i like the horse number one number two oh yeah yeah but yeah no no i know but then you have to your brain also have to be like number two is really big too and then i i just think like for me someone who likes the history of all this like like likes history of innovation and competition and disruption and stuff likes new technology it's like a very fun time to be following this stuff all together tech during the like 2017 and 2020 years so boring yeah at least for me anyway i thought it was pretty boring to it yeah sorry i was i interrupted you in mid no no i i i remember what i was talking about that's okay it's just fun time it's fun time to be it's okay things are happening okay i wanted to transition to a little bit of a spicy thing where you were on tbpn and their title that they chose for you was dougal often thinks microsoft is out of ai and oh did you did you not see those okay so so i wouldn't say out of ai no i did okay so i didn't watch it i never re-watched his thing Okay, so how I think about it is...

1:21:36Doug O'Laughlin:But you said things like Microsoft is scaling back investment. So it was the previous conversation I was talking about. Yes. How Microsoft has the most to lose. They have the most to lose of everyone in the entire world. Because they're the software company. The horizontal software company. If the craft is dead, they're bucked. Yeah, exactly. They're the horizontal software company that humans use their software to do information work. Okay? No, like I cannot paint a bigger target, okay? I cannot paint a bigger target. Salesforce. Yeah. Okay. That's another two numbers. Microsoft is automatically bigger.

1:22:08Doug O'Laughlin:Yeah. That's worse. Yeah. But the other thing too is they have this Azure business. I don't think I'm completely out of the race. I'm like, you know, it's a really great clickbait title. But the problem is the Azure business with OpenAI, right? You're essentially renting barbarians at the gate. You're like, you know, this is ancient Rome and you're like, hey, we need some extra guys. So we're going to pay money for these barbarians to burn. The Golden Army. Exactly. They're in Game of Thrones. Yeah, yeah. The Golden Army. And the problem is like each year they become more powerful. And then at some point they're just like, you know, we could just like scale these shitty walls.

1:22:38Doug O'Laughlin:So that's the problem is the wall and the moats every year are getting more dilapidated as they continue to rent GPUs to the barbarians. So it's just like Google Yahoo again? Yeah, it is exactly like that. And so it's just like this weird process where that's a terrible setup too. Because what happens in the history of that is you have to choose one or another. okay if you do either poorly you're like you're like somehow in a third worst place you either all in become azure like maybe in the telecom era right because you're team t guy you become dumb pipes okay that is the the azure becomes what is it charter right uh-huh yeah but then oh the other version of this is you say no no screw these guys i have to like reinvest back in and like essentially steal copy their their features and build up my moat that means i need to stop investing in Azure for the stock.

1:23:28Doug O'Laughlin:That really sucks because the stock is very much weighed on out your Azure revenue. And meanwhile, if we actually had to value Microsoft X Azure, the multiple would be really low right now. I think about that all the time. What would this trade X Azure? This is just Microsoft. Eight times earnings, 10 times earnings. It was trading like that before actually. Oh, geez. And like remember the 2010s era when it went all the way down to like 10 times earnings in the Steve Ballmer era. And then it inflected outward as it did Oath with Azure. Yeah, yeah. Azure and Oath 365. Yeah, there we go. That's right.

1:24:01Doug O'Laughlin:Yeah. Yeah. Okay. I don't think you have the answer, but I just like, this is one of the most bizarre, I want to call it f***, but I don't know if it's a f*** or not even. Yeah. Because it's a clear decision where they were the lead investors in OpenAI. They had the deal and they consciously obviously stepped back. They're still good partners, but like, what happened? Like, so. I think the biggest blunder of all time, that the part that like is kind of crazy to me about that one is like, yeah, I definitely think there was a financial decision. Because when you look at it, it looks like a conversation of shareholders, ROIC, and how much are you willing to burn cash?

1:24:41Doug O'Laughlin:Because like, you know, effectively, you look at all the other peers and Google, I would argue, is going to free cash with zero. I think Meta will go to free cash with zero. Microsoft is still like, you know, Satya did not make the company. he is a professional manager and there is a board and there's a conversation responsible yeah yeah he's being responsible right but the problem is that responsible this like an innovator's dilemma right like do i maintain maximize shareholder value and cash flow today or do i have a deep belief that ai will kill the hell out of my core business and i need to all in invest you know am i ready to bet the entire company on on a trend and it seems like satia is not a believer you know we've been talking about AGI, he is not ASI pill, okay?

1:25:20Doug O'Laughlin:He doesn't have any fear of the Shoguth. He thinks it's just like a new, it's a new loaded, it's a, you know, Lotus and Excel came around, right? Like, it's just a new tool. But I think at the same time, this, this conflict between renting GPUs to the barbarians who will disrupt your business, or, you know, your actual core business, it's clear how they're feeling. In the call of earnings, they talked about they could grow a lot faster if they wanted to, but they're trying to reinvest back into the internal capabilities. That to me sounds like we are not going to hire as many barbarians. We're going to pull, you know, we're reinvesting these walls, pull in together and try to defend the core mode.

1:25:54Doug O'Laughlin:Right. Because the dream of this, in theory, you're like, oh, remember in 23, when they did the first big deal, you're like, wow, Microsoft's going to win it all because they already have all the distribution and they're going to have the perfect product. And boom, they're going to have this giant business that makes them, you know, whatever, a hundred billion, a hundred trillion dollar okay whatever number you want to say but reality is Claude for Excel Claude for PowerPoint is literally exactly what it's supposed to be Microsoft should have built it Microsoft should have built it yeah and so now you see the barbarians and this isn't even your primary barbarian issue the guy who you know this is like this is the this is like the tribe over the hill yeah yeah barbarian this is the tribe over the hill you know and though the tribe over the hill is like like you know on a nightly raid easily sack the hell out of your castle and you're like dang, this is an issue.

1:26:40Doug O'Laughlin:So Microsoft now is super stuck in the middle. And so how they're going to have to do this is totally different. I think they're going to keep pulling back in. We're starting to see that. They're going to do internal training. They're going to try to do more foundational models. They're going to try to use the weights that they have access to. Is MEI? Yeah. But I'm very skeptical because their execution has been kind of dismal. Well, you know what it may be seen? They are one of the big companies in the world with all these resources yeah i i just want to push back on the the sort of responsibility part like you know so oracle picked up the slack yeah is oracle being irresponsible you know i i'm actually if we're going to talk about oracle i think so let's talk specifically about oracle because this is where we're going to go i think oracle was irresponsible because the magnitude of what they did okay like the thing is like i think the slack they should have done it but like the whole setup in my opinion on oracle is own goal they messed up the messaging they messed up the fundraising and in my opinion if they were not like like one of the things that happened is they went so aggressive out the gate did the quarter where they said like 400 billion dollars right they they said our prize the wolf they promised the world then they proceeded to raise as much money as possible and like this is the first time they've ever done these giant build outs and so now there's delays everyone's like whoa you did this much right capitalism is kind of like hey hey pump the brakes and and seriously i think that if they just teared it out better meaning that they didn't do it all in one period played a little bit of expectations management this year's revenue from the deployed gpus should partially help start to keep self-funding and that's how you make this work in a glide path without going up down up down a big bang and so they i think what really happened is the big bang that really screwed them up was the debt side they they just offered so much debt it's kind of funny because in high yield tmt it's such a big part of the entire index like the issuance is so big it's like a debt index yes i have zero familiarity with that okay i'm pulling some numbers up i did the numbers forever ago i'm like i i collusated him whatever forget all the precision let's just say all of the investment grade tmt is like 500 billion okay i think oracle is like 135 of it so that's like that's so big and so each time you have to you put up a huge new issuance you have to give someone an incentive to go buy your debt instead of someone else's.

1:28:57Doug O'Laughlin:And so you just kind of like, they're screwing up the liquidity because these issuance are so big diluting the whole pie. It makes all the terms a little better or more favorable for investors. So literally the entire index is selling off because it's like, yeah, it's a supply thing. Right. And that's the thing that's like crazy to me is like, so they, they massively overshot. And I think that we're like a weird bottleneck. I never, ever, ever, ever, ever thought us about would ever, ever hit. And I think you could appreciate this uniquely is like one of the bottlenecks is like supply of debt into the market.

1:29:31Doug O'Laughlin:Like capitalism cannot like absorb that much capital demand because the order of magnitude, it's totally different. These hyperscaler businesses have been completely self-funded since the history of time, had never gone out and issued anything. First time they want it, they turn around and they're like, hey, instead of like, can you give me a$10 billion loan limit? And like, we've never done that before, right? So the absolute size is kind of screwing it up. And I think that Oracle specifically was way, way, way too aggressive into a relatively illiquid market. And so like, you have to do this, like you have to kind of lay yourself into it if it's gonna be like that.

1:30:05Doug O'Laughlin:But they like super jolty did these big, huge incremental ads and kind of flipped the whole thing. Oracle, CDS, people all freaking out. I think a lot of it's mechanical, specifically on how badly it was done from a supply demand perspective. And I think they can pay for it. You want to hear it all right? Microsoft could have just internally funded this. Microsoft could have internally funded this. It would have been totally fine. I 100 % agree. And this example where it's like, yeah, I think that that's a blunder. That's a perfect example of a blunder because Microsoft's cost of debt is the same as the United States government.

1:30:35Doug O'Laughlin:It's like the cheapest you'll get anywhere else. This is correct. And just from the math perspective, no one else, they're better than Oracle. They just, by their credit rating, they have a 2 % more profitability at a capital basis. That's like, you can't beat that. I don't know why they decided not to, but now they're in this weird thing where they're like, they're kind of wavering. Like to win, you have to be like really bold, right? And they're kind of like doing this one thing over here, being really defensive with Copilot. Satya is now, you know, the product manager of Copilot. And then they're also pulling back from Azure.

1:31:11Doug O'Laughlin:Meanwhile, the competitors are pushing in for the supply. it's a really weird game. I think Microsoft has to choose a direction. We'll see. We'll see. That's what's going to make it fun. I'm more than happy to change all of my opinions when new information comes around. Yeah, and I'm sure we'll have more information that emerges. I wanted to touch on TPUs and then go into memory. TPUs will hopefully, I don't know, maybe a short one, but like, you know, for a long time you could not buy TPUs, at least current gen TPUs externally, and now you can. And Google's open as a as a as a supplier i guess i think sergey doesn't want to lose and i think the thing that happened was up until like you know he wasn't no one was there a part of the whole deep mind story was we will hoard all the tpus because we were first you know and so like why sell you know why what gives anything to the topic i think it's because at least last year it became pre-gemini three it was like dude we have all these tpus we're gonna hoard them all but like people aren't using our products anyways and and like like hey what's all what what good is all these tpus if we're getting our asses kicked in consumer i think it's an interesting thing too because the other thing i think about is there's a lot of different ways to to like break this down one we wrote about it in tpv8 like whatever we think ruben will be much more competitive i think ironwood v7 is the peak gap between on between tco between nvidia and tpu right so if you are at your absolute strongest point.

1:32:39Doug O'Laughlin:What do you do? There's two ways you could do it. You could try to maximize and squeeze the juice and make margins, or you can gain market share. I think the perspective of doing this externally with Anthropic is to gain market share. Because the biggest gap you have, and one of the reasons why there hasn't been a second merchant chip, and also you can argue NVIDIA's most valuable company in the world, what's the value of TPU in Google? It's huge. You must have done the math. I've done the math. It could be like, it's like - A trillion? It's like a trillion. Yeah, it's like a trillion or something like that.

1:33:09Doug O'Laughlin:Assuming it gets like 30 % market share or something like that. Everyone has been trying to crack the merchant silicon mode, right? And now they have the biggest absolute outperformance. A lot of the people who did the original TPU program are like now at OpenAI. Some of them are medics. Some of them are, yeah, you're exact. Some of them are medics. They're all over, right? Like the core team that did most engineering have like since really dispersed. And so I think the gap might close over time. And so at this absolute period of time, they're going to win the market share. And then what happens is if you have an install base, you have an incentive to upgrade your install base.

1:33:42Doug O'Laughlin:That's like the hugest problem with AMD, for example. No one wants to buy new AMD chips because it's not like they have old AMD chips. They're not upgrading from anything. And so when you have that number two place, you have to win definitively. And then also you have an opportunity to win again next year. I think the install base issue has been a kind of huge one. And so TPU is at the point where the software ecosystem is mature enough The hardware is definitely mature. The networking is really mature. You have a really good external customer who actually knows how to use your product. If you want market share, now's the time.

1:34:13Yeah, that would be insane if they actually sort of pump the gas on that stuff. Are you also hearing, I don't know if this is something that affects your analysis at all because I don't have any appreciation for the sizes that we're talking about here, that JAX is helping TPUs win or JAX is winning relatively to PyTorch, at least in like the academic arena. which is a leading indicator of what it's going to be using.

1:34:36Doug O'Laughlin:I do not have as, I don't have a special purview on that. The thing I'm most excited about and like very much TBD we'll see is inference X will have TPUs eventually. That's something we want to do longer term. I think that that will really show in numbers. As a benchmark? Yeah, as a benchmark. How do you expect them to come in? Pretty good on a price basis. I mean, our expectation is like they're the best TCO by a meaningful amount right now. Anthropic's very clear how they feel. Like everyone is very clear. i think even open ai would take i think everyone would eat as much tpu v7 as possible if you had it in a perfectly unconstrained world it would probably be at this exact moment like you know the hottest kid on the block until reuben comes out but the reality is supply chain really matters and that's just that's not available and so that tco that tco advantage is at this absolute biggest aperture then like jensen essentially gets it gets their stuff together it's competitive and boom it closes so this door only open right now probably tsmc is the biggest blocker so there's no yeah yeah what what can you do it's this cascade right they wish i think you've talked about yeah like all the way it goes all the way back to the fabs yeah yeah well it's interesting because it's like even more than the fabs like like on the optical link i should be pulling up yeah that that's it that's it just need to have it or google that goes swing everything yeah so so yeah it all goes back to the fabs it all goes to who's making the chips and like i think one of the big differences too is just like the per it's just like a really good cleverly designed system architecture and it's relatively stable and it's clear that they you can pre-train big models on it which is like a huge huge swipe at open air right now that being said like i think open air will get their their act together very quickly and so yeah that's kind of like the narrative i think it's going to be a good story for probably like a year or two but then the real question is the V8 we just don't think will be as competitive to Ruben and that's when your special window starts to close what's the technical reason why HBM HBM 4 versus 3 and that's a secure is the strategic decision by NVIDIA yeah I think one so NVIDIA is always if you think about NVIDIA they're always trying to gas it as hard as they can like they like it is a high performance ship it is a it is an F1 like it is as maxed out as possible TPU is kind of like this like replicatable pod in a very large, with like very high stability, right?

1:36:54Doug O'Laughlin:Which if you know the history of Google, that's what they do. That's what they do. Yeah. That's what they do with Infra. Yeah. But I think GB200 would have completely mogged, you know, V7 if it came out on time and stable. It came out a little delayed and it wasn't stable. And so I think there's a lot of different ways to kind of course correct that. And the one thing that's important is like, I think on the infrastructure on the supply chain side, bar none, NVIDIA is the best. They own the entire supply chain. They really do. Like, you think all those HBM price increases, they're going to come for TPU just like NVIDIA, but NVIDIA was literally in Asia.

1:37:30Doug O'Laughlin:You saw him drinking with everyone, with the SK, with everyone, with all the Korean guys, all the TSMC. He's doing the shots with everyone. Why do you think he's doing love shots with everyone, okay? It's because he needs to get the chips, okay? So, yeah, this is Samsung's chairman. Yeah, that's the Samsung's chairman. Yeah. And it was the other guy. But let's put it this way. That's a huge deal. That's a huge, huge, huge deal. Do you think Google was out? Hyundai. Yeah. Do you think Google was what, you know, do you think Sergey was out in Taiwan drinking to get supply? No, 100%. There's an opportunity here, but there's only so many TPUs that can be made because of all the bottlenecks, right?

1:38:09Doug O'Laughlin:And so NVIDIA has all the supply chain locked up. And so they're going to have like so much of that kind of constrained there. And so it's going to be really interesting. They're going to get the best, most performing HPM. They're going to be first on the roadmaps for even more rack density. They're going to have like the best connectors, the best, you know, the whole system will be once again, turbo jammed again for as hard as it can be. And the people who made V7, like they made, the chip was done like three or four years ago. Like the talent dispersion aspect where people who worked really hard on this team to make this great chip has really kind of gone all over.

1:38:44Doug O'Laughlin:That starts to get worse. And so if that gets better, which takes some time, I think our current read is that like the HBM specifically and the memory scale up is going to really go in Ruben's favor. And so that's the big difference. And I think, as you know, that's what makes the context windows. That's what I'm able to do bigger, bigger, everything, everything. Yeah. And so they're going to really jam it. And that's going to be a huge advantage in performance. One thing I love about your analysis is it's not actually just the context windows. It's not just the KV cache. We also have to offload it to non-HBM.

1:39:13Yeah. Every other part of the memory. It's like such an interesting cascade waterfall of like just like a short squeeze and everything. It's not a short squeeze. It's like a surprise squeeze.

1:39:24Doug O'Laughlin:Yeah. I mean, I just want to. I mean, if I read one ratio. I'm like, yeah. Okay. So it's a three to one. Quantified to three to one to four to one ratio. I think next year, so it's in the Memory Mania post that we just put out of like the 4 to 1 or the tradeoff ratio. Yeah. Scroll down somewhere and you'll see. Yeah. So basically for listeners, it's the idea that like when you convert to HBM, because there's a huge demand for HBM, it takes three times. One HBM sort of unit is like three times of the other sort of DDR or whatever, right? Yeah. So some amount will always be lost in production because yield isn't perfect.

1:39:57Doug O'Laughlin:And so effectively you're trading some, like you're trading. i i mean i actually wrote a really funny piece like i called it like super oil but let's say this is a better one but pretty much like in order for this higher grade of jet fuel has been invented and the only way to make it is to like actually get rid of all all your other fuel and you have to like massively condense it and refine it okay so now what happens is if there's any demand here that it's an instant shortage and so we we hilariously enough came out of like the biggest shortage ever in nan and d-ram like terrible like catastrophic the worst one ever like the the last analysis i could put to is like 96 or something like that seriously it's like a history one and then meanwhile we have all this new demand hbm specifically the highest end you need the most memory the trade ratio is crazy so each you know each bit of of hbm is essentially a 4x multiplier onto d ram and then now so we completely constrained took all the d ram capacity we just came out of this shortage so no one invested in any clean rooms or capital equipment or anything like that.

1:40:57Doug O'Laughlin:People got like massively free cash from negative. No one's spending a cent, okay? People could go bankrupt, you know? So they haven't invested in these three-year-long lead time items. And then now there's like more demand than God. And it also evaporates the middle layer because the KB cash offload. And then boom, you're just looking at the supply demand. You're like, yeah, this is not going to catch up for like two years. I think the thing that's like so interesting is the supply chain squeeze because these clean rooms take two years to make, man. And effectively, everyone paused. And how bad the last cycle was really forced everyone to completely pause altogether in terms of adding any new capacity.

1:41:33And so now we're a few years later

1:41:36Doug O'Laughlin:and all the supply is gone. So, I mean, people are, I mean, it's just, it's crazy. We, our post, our conclusion is like, we could see DRAM prices like go up 100 % again. Like it's going to be the point where, and this is like also example, like really interesting in the whole thing. another 100 i think is demand destruction i think you will start to have demand destruction from what does that look like where hyperscalers maybe purchase less or something like that on the margin on the margin right because they're like okay well what if i just really focus on this energy aspect instead and also ironically all the energy so like every not every data center in america but like many many many most of the data centers in america are delayed so you had this thing it's supposed to come on in 12 months it's coming on in 18 maybe what you can do is you can play chicken with memory prices and you can kind of push out of course everything you have in the pipeline you you pull forward as hard as you can okay you pull forward you double triple order and then the DRAM and HBM guys are like oh my god how look at all this demand and then at some point in time what happens is you say well we pulled this all forward you know we have the power is going to constrain us anyways we're going to like kind of chill out the orders and historically that's in the memory market that that's what causes the crisis the the prices to drop realistically just looking at at the aggregate demand of how much we've purchased in terms of power it just seems like the gap is just huge it's completely off to the point where the most obvious logical leg of the ai the ai trade is effectively investing in memory is a capacity yeah well not to say okay it's you could say sk hynix and and say micron micron and all the all the semi like semi cap has been ripping which like by the way when i was in baliastny we a majority of a lot of money we made which is being long micron yeah in the last like yeah it's a good example yeah so like you have all the semi-cap stuff right like all everything that is even remotely related to investing in capacity for memory that is like the ultimate bottleneck right now and also for listeners it's gonna affect like your phones yeah like yeah i apple i think apple's moving i have to buy an sd card for this thing yeah it was a bucks yeah that's that's nothing too that's and that because like that's just the nand side dude have you looked up like like i want to say like 64 gigabytes of of d-ram like like i'm moving up like i i need to refresh my iphone i'm moving it up because i'm doing this research oh yeah you need to do as soon as buy your iphone now yeah yeah you buy your iphone now because what's going to happen is when iphone go into the spot market it prices are going to go up 100 of them that's insane and so they have the past we're going to be buying like old iphones and then taking them out for them there's no more that's actually there's a there's a whole super duper deep in the weeds there's this whole like technology that was very focused on cloud era called cxl which is memory expanders for cpus in order to have like whatever just like elastic pools of computer cpu and d-ram and whatever memory attach and whatever it never really took off because essentially hbm was like the way that really crushed it all high performance best of breed wins but this cxl technology that kind of never really took off is going to take off just because what they're going to do is they're going to take ddr4 they're going to take the oldest every bit of spare memory they can find and they're going to put them into racks and then they're going to attach them via cxl so like this oh yeah exactly that yeah it's exactly that but the thing that's so crazy is like this dead technology is like having a shot on goal because of how bad the storage or how bad the memory constraint is yeah like yeah that that you know i i was like a cxl bowl for once upon a time then became very clear it was going to die and i was like it's back but only because the entire express intent is to have these d like old chips pull the old chips attach it to something new that's what it's gonna be like the the memory shortage is just like it's crazy so yeah it's incredible yeah so obviously this is lower level than i usually go to which is which is why i'm having so much fun one thing i i do tell people about is like well you know everyone including sam by the way is like predicting longer context windows we've been kind of effectively stuck at a million for two years now i've actually been thinking about that a lot and like this is not going to go to 100 million context windows it's not going to go to trillion like we're this is it yeah this is it for like five years 10 years pretty much okay so the question is will yeah i mean yeah probably actually will capitalism work will we will there be a way for supply to show up probably but on top of that i wonder if there's gonna be like like hey his history of compute what happens is you have to like you have to like make a curve of the of the context windows like does free context windows go to like 1000 hey you can use chat gpt free now but you your context window is like a thousand tokens or something like that and then you just like somehow do a tiny like a tiny parcel for that just so that you can then charge like you know 100x more for 1 million the 1 million context window is like a mansion you know that's the real you live in a mansion i live in a mansion right now yeah oh my god the word just context rationing just came to me i'm like fuck like we're gonna have like vouchers for like okay you can have this amount of context today like it's like yeah you have to you have to learn how to use it well because of the d-ram yeah okay so i actually have a question i know because like okay long context to me makes a lot of sense right hey that's like like that's like the memory scale up version like if you're thinking about chips but in the AI world.

1:46:51Doug O'Laughlin:I just am like always been curious because it does feel like at least in my stated experience really long context like you see in the papers they kind of like drop off they like actually don't use all the context so that's like kind of the thing I've been most interested in is like does the hundred million context actually matter if if it's not possible to use it all? They you know versions of 100 million do exist today they just suck in various ways they're not actually applying full attention right yeah you can you can use a state-based models or even like LSTM to like, you know, to process 100 million tokens, but you're not paying full attention to those 100 million tokens.

1:47:25And so I think like the way that we have context about today and those curves, they will improve over time and they have been improving a lot, but we're just not, we're never going to use all of them, but we'll improve like on the algorithm side. I think for me, what matters is you represent the physical constraints that us, the software side can never surmount because there's a physical constraint. And, well, I mean, it just, like, physically we cannot double. We can't even double in how much it's 10x. Yeah. Yeah.

1:47:52Doug O'Laughlin:What's the point of talking about it? Yeah. What's the point? Yeah. I was going to say, like, we can invent a lot of things. Context rationing is pretty good. I really like that one. Context frugality or, like, budget or something. I feel like everyone's going to be like, whoa, you're running out of context window today. You know? Like, maybe that's what happens next year. We're charged on context window. And then one of the more recent obsessions is recursive language models, which, again, is just reusing the same context window on an office. I've been pretty interested in that. But, like, to be clear, I'm a total idiot.

1:48:21Doug O'Laughlin:I have no idea if Claude tells me what's wrong. You're the semi-sky, man. Like, you're really good on your stuff. One thing I wanted to spot check on was Talos. I have not messed around with it. Okay. You don't have to mess around with it. It's just this general theory of custom ASICs burning the weights into the chip. Yep. So you don't need memory. Yep. That's pretty good, actually. Right? I think that that makes sense to me. This comes at the perfect time. It does. But I guess, okay, so historically, the question is how big does it scale, right? But, like, I mean, you know a lot of the models are kind of actually smaller than you think, right?

1:48:54Doug O'Laughlin:So, like, sorry, what do you mean? A lot of the production. Yeah, they get distilled to shit. Yeah, they get distilled to shit. So, it's like the push and pull there is going to be like, okay, can you just burn in enough efficient Pareto Frontier in terms of performance to be burned in straight onto the silicon. That doesn't need memory. And then boom, you can scale this forever. Versus like, you know, the performance edge of the long thing. It's pretty clear to me that like Talos has a place because you're kind of seeing this market bifurcate a little bit. You can argue the pre-fill, decode, disaggregation, stuff like that.

1:49:27Doug O'Laughlin:The focus on performance inference serving is going to be a subset of the market and then the training and the whatever and the big production. like you're we need to kind of break it in into smaller parts in order we've been using the same yeah that makes no sense just in order for the compute to be even remotely okay it makes sense to you and i mean tbd on the sort of practical implementation but otherwise burning the weights into the chip why didn't etched or some of the other guys get there first um that's just pretty interesting i don't know okay i mean i'm not gonna speculate i mean i'm not kind of like super speculate i mean the thing the thing is like their thing is like okay how do we have a big systolic array, right?

1:50:04Doug O'Laughlin:But like, they didn't burn weights into the chip. That's a little different, right? I mean, like, look, like, I mean, yeah, I just think the way to speed things up is to never transfer anything. Yeah. Yeah, that's the fastest way possible. And so, but the thing is, the bet on this really large systolic array is effectively everything is compute bound, right? Like, I don't think that that's really the case in terms of like where we're actually seeing issues in production markets today. It's like, you're actually seeing all the issues in the memory right and so like i just don't know if that's like going to be the perfect solution there is definitely a world and space and like a design space where they're going to be very valuable and cool but like also the reason why my hit rate for every ai accelerator trip is like very like i just don't believe in them it's because like where are they until cerebrus and grok honestly they were all considered failures and even then we're like what are they going to do with grok what are they going to do with cerebrus so is is sominova no i think sominova is like a much more interesting one but i think there's like there's like all kinds of deal issues with that.

1:51:01Doug O'Laughlin:I haven't been keeping up with that one as well. I always try to mention them as part of that cohort. Yeah. Yeah, because I kind of forget about them too. But yeah, honestly, I was going to say they were... You know, once a year they show up. Yeah, they do. And they're not so bad. Yeah. You mentioned actually some CPU shortage stuff or CPU... What's going on there? I think it's okay. So, okay, I have one. We'll start with the conspiracy theory that I think is really funny. Have you been noticing just like, I feel like web services have become really unstable. It has been down a lot. I like and like me this is pure like you know a schizophrenic hat brain because I have a schizophrenic trend hat brain I'm wondering if it's two things shipping vibe code slop to prod that's number one that's definitely something that's possible but it's happening to all the clouds at once I feel like it's not just the AWS thing it's not just like a github azure thing we are kind of right at the exact five to six year period of the refresh cycle of COVID.

1:51:55Doug O'Laughlin:So COVID, we had this big 2020, 2021, you bought like$100 billion of CPUs and stuff like that. And so we're right at the natural end of life for these chips. And so usually what you do is you have this big refresh of all these chips. But what's been happening instead is everyone has essentially scrounged all of their budget as hard as they can. But then like, I feel like I've seen it in like Azure, like, hey, last night, my Amazon Prime thing doesn't work. And I was like, it'll probably work in the morning, babe, don't worry about it. I think Azure is just, like AWS is pissed tonight you know something like that but i think so we have this five-year thing everyone's every single dollar they could to essentially invest in as much as ai is impossible and just do maintenance capex on cpu ironically at the same time for all this cloud code stuff is actually if you have this coding agent just generate you god knows how much compute how much like software where is the software going to run on cpus so i think we're going to see some increasing utilization as well as the fact that rl is like actually heavily used for like rl gems you have to you have to simulate software and it uses a lot of cpus so the or not quite like the orders of magnitude and gpu stuff but it's just such a big trend even when it steps slightly in a place mass amounts demand you feel like we might actually be seeing a cpu shortage partially because this refresh cycle and partially also because like i legitimately believe the cloud code cloud code is increasing software creation and then on top of that there is real yeah and from rl yeah and just general production agents as well you know we just yeah every like rlms take compute and you know open claw takes more compute and it's just it's just a different slope but at the same sort of direction still an upslope and in a slope that to be clear has had massive underinvestment for the last two years because everyone like how did the same problem that happened massive underinvestment because they're like screw it we're doing maintenance only we're all we're going to do is maintain maintain the past we're not going to add anything else and then all of a sudden just a little tiny slope on top of it you're like boom shortage yeah amazing that means guys say that means numbers go up that's one way to put it the thing that's crazy is we talk about the demand it's like you're right it's for sure show me where I'm wrong definitely not but the thing that's crazy memory prices are going to go up so much that we're going to have to choose which that's the crazy part to me historically memory has never been a constraint like this where I said actually you're not going to get your low end phone you're not going to get a gpu this year for gaming none of that stuff you can't do these things because your price out of market that's what's crazy i didn't say that is the first thing that's happened a long time that's gonna be really interesting to see where that shortage and how how it's like digested and felt that is amazing thank you for that breakdown i feel like i really understood it by talking to you yeah let's go to transition to a couple personal things and yeah sure as the dn how do you write because you you write a fuck ton yeah i do i have been writing a little bit less these days now that I'm like in the semi-analysis mega mind I definitely write a lot and like you kept going with fab because for a while okay dude to clear that was really so so so look I'm still trying to do fab because I I do feel deeply connected to writing let's just specifically talk on this a little yeah just just just like explain yourself okay so the thing before LLMs came around the thing I felt the strongest about my my number one information skill is I was able to read and synthesize and process at like really high speed, really high throughput, decently high comprehension.

1:55:19Doug O'Laughlin:The adjustment is speed in terms of comprehension, almost anything like when my like when my friend gets a PhD, I go read their paper, I was like, I have a pretty good idea of what you're doing. I was like, like, hey, when I was interested in semiconductor book, I literally raw dog some textbooks, whatever the comprehension was not very high. But like, hey, whose comprehension is you know, but I was able just to like push through these books and learn. So I've always loved reading. That's like my, my number one original competitive skillset differentiator. And also something I like loved as a kid, crazy reader when I was a kid always have been.

1:55:49Doug O'Laughlin:And then starting the sub stack, which has been really fun actually, because I just really wanted to get my story out. Like the things I cared about, close the loop for writing for me. Cause I love, I love reading so much. It makes a lot of sense that I love writing. I think what really helped is I wrote every single week for like since October 21, like consecutive streak for a long time. The streak has been a little broken as of late. Semi-analysis plus fabricated knowledge is pretty hard to do. Like all of 24, I think like we're just talking just like every single day, every single week I will put something out, right?

1:56:20Doug O'Laughlin:Is it like a hard rule like one a week? It was a hard rule one a week. At least an attempt to. And so I think one of the best ways all the people who write about writing all say the same thing. You need to just be writing. Yeah. And so that's how I start. I'm writing every week, they know. Yeah, it really helps. Well, I was going to say what's crazy is like it's it's kind of hard these days and i and l lms kind of have really i don't know i don't like l lm writing i do like it for ideation like making outline yeah yeah here's my un unorganized thoughts make it into an outline and then like you know i'll even be like put bullet points in the outline and i'll literally read the outline and then like ideate and write in parallel but yeah that's that's how i feel about writing i guess write more i have a strong for non-fiction writing i really like this book called on writing well that's just a really good classic book it's actually summarized and synthesized into a into a skill for me oh yeah yeah yeah yeah so hey please edit this use these use this style guide use the like learning from this book so yeah just something like that yeah okay and then do you like have a topic idea list that you groom like i i put mine in apple notes now but bro it's no never never i'm one and i'm just i'm just a one shot whatever is on your head yeah usually i one shot the the idea all the way Yeah, yeah.

1:57:34Doug O'Laughlin:Usually I think about it for quite a bit. So it's been bouncing around in my brain. And then at some point in time, I've like condensed enough information to make a really crappy outline. And that's usually when I just one shot go. For me, like it's hard to one shot and bounce because you will forget. Right. And sometimes you have like really good stuff that you forget. And sometimes it's actually, so I call this mise en place writing, where you basically just have a store where you're just kind of writing, working your ideas in parallel. And then every now and then you cook. Yeah. And so this is async and this is sync, right?

1:58:06This is like passive, like, oh, here's a data point. Here's a quote. Here's a thing. I'm just going to slot it in the right thing. And then I bake it. Historically. Okay.

1:58:14Doug O'Laughlin:So how that pre-writing actually works today is probably in the semi-analysis Slack. It's just like all the little things. Then you just search it up when you need it. Yeah. Search it up when I need it or something like that. But like I do most of the pre-writing, I think, in my brain and I have places that I put it out that I reference it later. but my favorite thing too is like when it comes to the because like okay well once by a time much more on the beat oh hey here's earnings read every single one and put it all together but like my favorite skill or tip or whatever is like hey do the pre-writing think about it all that stuff and go to sleep and wake up the next the fresh context window in the morning is my number one advice on writing helps you decode better it helps so much better like literally if i'm like hey i need to write something right now i will do i'll write it all down i'll make outlines i'll do all kinds of crap except for writing it and then i'll be like and i'll go to sleep and then wake up and the first thing i do i'll open up a new tab and i will write it go and then so usually that will get me to 60 75 of something even if it's like an outline where i like have gotten all the ideas enough to know how to fill it out the rest of the way and then that's that's how i take it from there cool amazing last thing hike yeah so one bit of context for me is i i just i just i've never taken a break never and And I feel like, you know, if you take a break in this time, you're like, just going to be so behind.

1:59:28You're just going to so miss out. I just found out my friend from OpenAI took a break a year off to bike through Japan. How could you?

1:59:37Doug O'Laughlin:I'm like, you're going to miss it. You're going to miss everything. But he's like, I'm good. You know, like I'm, you know, having kids and whatever. You did a sabbatical as well. And like it was pre-AI. But it was interesting. You did the Appalachian Trail. So there's three big ones in the United States. It's the Appalachian, the Pacific Crest Trail, and there's a Continental Divide Trail. So I did the Continental Divide Trail, which is the longest and most remote of the three. Sometimes considered like the older, bad, whatever. But like, honestly, the PC, they're all different trails. Like I'm pretty steeped in hiking culture.

2:00:10Doug O'Laughlin:I think mile for mile AT is actually the hardest. But I did the CDT as my first trail, as my first through hike. You know, you learn a little bit about the three when I was choosing which one I wanted to do. And the CDT was the one that scared me the most. I was like, hey, this would be the hardest, biggest accomplishment I could possibly imagine. And I thought, if I never have an opportunity to ever do this ever again, which so far seems to be pretty correct, which one I'm going to do to feel the most like, hey, I did the thing that I really wanted to do because I've always wanted to do a long distance hike.

2:00:38Doug O'Laughlin:And so I chose the content of the Vite Trail. I did that in 2021, pre-AI. But after the GPC3 essay? After the GPC3 essay, yeah. I felt like I was missing out a lot. And there's like a huge, it was a huge year for Substack. I feel like I missed out like a very big year of like the big growth. You're doing okay. I'm doing fine. But I just think that for me is something I always deeply wanted to do from an intrinsic perspective. I think something is like life fulfillment. And I would definitely do it again, but I probably. To be clear, for people, it's like four months, five months, six months, six months.

2:01:09Doug O'Laughlin:Six months, 2800 miles, we'll call it on the route, 2850 or whatever the miles I went. And like you meet people on the way, but you're mostly alone. Mostly alone, did alone, you get the trail name. it's a whole audiobooks i listened to audiobooks until i hated them listened to music till i hated it got bored as hell like you just you just you go you you go through all of it actually yeah yeah it was awesome six months i think the thing i think about is so far in most in in my life up to that point you get kind of get kicked from situation to situation right you create a view a form of yourself you think you know yourself you have ideas of what motivates you how do you reacting to situations, blah, blah, blah, blah.

2:01:48Doug O'Laughlin:I think the one, the CT about like, it's just like, I like, I like the outdoors. I like hiking. I'm like good at it, whatever. It's just something I really appealed to me from an adventure perspective. Like when in modern life, you get to say, Hey, I'm going on an adventure. Never like, and that's what it was. It was, it was an adventure for me. And one that I got to like, really, you, you, you know, it's like, Oh, the journey is a destination or whatever. You learn a lot about yourself. In fact, I learned And it didn't grow me up per se, but I feel like I am more well-defined of my view of myself.

2:02:16Doug O'Laughlin:I understand how I react. I actually know where my exact line or it's like, you know, you're like, oh, I'll go do this. It's like, actually, no, I know my exact line where I'm like, I would not do that. I know exactly where I'm not. That's too scary, too hard to whatever. Yeah. I know my limits a little better. I feel like I know just more about myself. It is a very condensed version of a very intense life. and yeah I wouldn't give up that experience for anything in the entire world it was extremely personally meaningful to me I think it's very fun to go back to the lower part of the Maslov's hierarchy of needs like all this crap what we're talking about today is so abstract it's like totally fake and we were not born and built for it we were born to like you know our human evolution got us to like scrape a living in the mud okay hunt and gather hunt and gather and just not die it's kind of interesting to go backwards and to see what feels like dude i was so hungry so scared so alone so like but also like super low that the phrase is like lowest lows and highest highs these crazy lows where you're like what am i doing what does all mean highest highs and me like holy crap it's so good just to be alive all these things where it's like it's just so like the raw experience of life is so meaningful and you don't get to experience it well doing it that way And so, yeah, I wouldn't, I highly recommend it.

2:03:29Doug O'Laughlin:It's very, I would do it when you're younger. I wish I did it after college. Yeah. Like right after college and said, hey, like whatever, kick us out a year. I think it's good to learn about yourself. It's really important. Your self-mastery is your most important tool use of all. So, yeah. I love that. Yeah. Self-mastery is a most important tool use. Yeah. Amazing. Well, thank you for jumping on and like covering everything. Yeah. I feel like I got like, got to go through the sort of quad code psychosis all the way to the semi-usual, all the way to the hiking. Yeah, thank you. Thank you for having me.

2:03:58Doug O'Laughlin:Yeah, great to catch up.