242 - I Replaced My Research Team with This AI (Here's How) with Elizabeth Knopf

0:00This is not deep research. This is just Sonnet 4.5. And it created this really good comprehensive plan that before you would maybe get a fraction and you'd have to sort of iterate through. You can actually get that full workflow within the browser now, which is just like mind blowing. You think about a year ago, you could not do any of that. You're basically building out your second brain. It's your AI second brain.

0:26we are going to be reviewing some of the updates that anthropic has released in the last week the reason i'm the most excited for this is because at least for me claude anthropic has been the most intuitive with helping me sort of work through this interactive software building i don't even know what you want to call it artifact building for those who don't know artifacts are for lack of a better term anytime that you're creating something within any of these LLMs. Let's say it's like, hey, create me an LOI template or hey, create me a checklist for X, Y, and Z or hey, create me a mock website.

1:00That is an artifact. And what Claude does really well is it saves those artifacts. So you can actually go back and look at them and reference them. However, they haven't been dynamic. And what I mean by that is they're pulling on static data. They're not going out necessarily to the web and crawling things and then returning something. they're using static data because there was no API integration. Well, it looks like now there is some API integration that Anthropic has enabled so that it's turned these now static artifacts into dynamic artifacts, kind of like web applications or even websites themselves.

1:34Is that a fair characterization? Absolutely. And I would just say like, you know, it's not going to be totally obvious until you start actively using it. So a lot of the things they just rolled out, which we can run through in just overview of their blog, as well as this cool little visualization I built out using its new capabilities. But a lot of it, it's more nuanced in terms of like, you won't get it until you see the output of what you're prompting. Okay. So with that, let's just show one of the outputs. So Liz went into Claude and she said, show me what's new with this 4.5 update within Sonnet.

2:11Yep. So I asked it to demo this. And then I asked it to imagine it. Like I had to ask you what this is. So there's this new term imagine, which is kind of the vocabulary that Anthropic wants to use for create or generate. Okay. So I asked it basically to build out a way to showcase its capabilities where it would actually demo them on the fly. So here it sort of gives a couple of summaries as to the highlights of what it does. So number one, the benchmarks, it just keeps improving for software development. Again, unless you're a coder, you're probably not as aware as to how that's improving. Actually can have computer use, which is what we demoed the other day, which you should check out that episode, using the extension that it has.

3:02And then also this is sort of underappreciated, but they've shown as a model can think longer, it has better outputs and responses. And so it can now focus for up to 30 hours, if not more. I've actually never been able to push it that far, but because it can have extended thinking beyond what it previously did, that output should be much more high quality and can handle more complex prompts and tasks versus in a lot of the interfaces like with the browser or in cloud code, you have to sort of iterate through tasks to make sure they're all done. Now you can provide a more complex prompt to get more output because it can extend its thinking.

3:46Let me give an answer for that or an example for that. So let's say you have a multi-step problem. And the first step of the problem is go out and find this data, bring it in. Great. Now, after you've found that specific data, I want you to analyze and synthesize the data. Cool. That's step two. Now, after you've analyzed and synthesized the data, I want you to measure it against whatever these benchmarks that I've set and find me, you know, XYZ criteria that I'm looking for. That's that's step three. In the past, you would give that prompt and it would show you step one. And then there'd be mistakes in step one.

4:19And so you'd have to be like, well, I tried step one, but it started failing at this point. And I was like, oh, I'm sorry about that. Let me fix it. And then you'd finally get past step one and you're into step two and you do the same thing. You'd like iterate through it. And then you finally get past step two and you get into step three. So it was like this back and forth that you would have with the AI in order to finally get to a working product. With this, what I've seen is it's actually going through and it's like, okay, step one, I got to go retrieve the data. I should go to this website.

4:45Let me test it out. Okay. I went to that website. I tried to pull it. Oh, it didn't exactly work the right way. And now it's like, it's testing itself and thinking through to make sure that it's not giving you this half baked answer. It's actually giving you something that works. Absolutely. Absolutely. You can even do within that, like in step one, you can say, here's a success criteria. And if it meets these criteria, great on versus if not go back to the first. I love this. I love this thing that you created because it's like, it shows me exactly what the new updates are advanced reasoning and math.

5:18So it solves complex math problems and, and you can click run a demo, right? Right. So here we're going to do run the math demo. So we'll do this simultaneously. Oh my gosh. As it can do is parallel tool execution. So before when you would prompt it in the browser, if it's using tools for MCP or which is more on desktop versus like, again, these tools that you have here, it could only do one at a time and you would be able to see it do, okay, step one, open this thing and run through. And now it can actually do multiple. So we can actually even demo that. Okay. So go up to that first row. Go up to that first row.

5:56run all of these just so you see they're all okay all right so run a math demo find all the integer solutions to x squared i can't quite see it yeah x plus y squared equals these and y cubed so i guess this was a hard math problem i don't remember from you know my algebra days but that's hard or at least for the well and it found the solution yep so it found that solution so math whizzes you can go test it out parallel tool execution so again this is sort of demoing It just used a couple different processes here for fetching, grabbing a CSV file, testing, and doing some front-end analytics altogether.

6:34Whoa, hold on, hold on, hold on. You're just like blazing through it because I know this is like old hat for you. The parallel tool operations, anybody who's like tried to use AI to scrape something, create a CSV, and then generate like a calculation out of that CSV. is this saying I could I could prompt it to do like hey take this data organize it in this way find me three most correlated variables like it will go through in that step-by-step process yeah so I think rather than it going by step-by-step it's now saying like do these three things all at once versus before again it goes one step at a time and sorry let me pop over so it's better visibility okay so yeah so basically doing all of these tasks at the same time so it was what would you use this for like what would you what do you use executing parallel tasks yeah i mean this is more of a time saver it's just thinking about like when you're like if you're manufacturing something you would do you know once like the the whole um toyota way as you do from a quality standpoint step one step three get the full output basically this is is more of of a time efficiency thing.

7:44So rather than going step A, step B, step C, which is how, again, most of these models are working where they're going one step at a time, this is to say we can do these three at the same time so that if they need to work together, they're all done and ready versus having to go do this thing, wait for them this other thing to go and then come back together after a lot of time. This now can say we're doing these two tasks at the same time to then bring them. So let's build a UI showing data analytics, but you also need to go run and capture that data and do calculations or synthesize the data. It would do those tasks separately versus now it can do basically build the UI while in the dashboards, while it's doing some functionality on the data.

8:28And rather than again, having one and then the other, it's coming to do it simultaneously and more dynamically. So that's just one simple example. Does that mean in the chat, I might be able to say, Hey, create me this UI. And then while it's extended thinking, creating that UI, I can, I can go into the chat and give it another prompt to say, go pull this data. No, it's basically saying, okay, here's all the, the list of tasks I would need to do now. Let me go and execute on those tasks simultaneously versus like you, like you still, it still stops you and it has that little stop bar when you prompt it.

9:02So you can't do that unfortunately, but it's more working. A lot of these updates are more under the hood and less obvious to the unit until you sort of see that it is just faster, more comprehensive, and higher quality outputs. Do you want to show any of these or should we go to the wash? We mentioned the long task planning. So that's, again, extended thinking. We mentioned the computer use, great code, autonomous agents. So this again is a little bit more less obvious where you can say like build an agent to do things. This I think is more obvious in cloud code, but again, it has more agentic capability in terms of how it is running the LLM.

9:48So it's actually, again, going out and getting data, searching more websites and doing more of that functionality where it's just less obvious. And then again, if you're into specific like law, medicine and STEM, apparently it's better. And then it's a little bit less sycophantic and deceptive. So they're correcting some of its personality and then file creation, which we demoed earlier this week. So those are really four updates for it. Actually, let me just quickly show because I think the extent of this is going to be more obvious. So previously for deep research, you would have to use deep research to get a really comprehensive output.

10:27So I prompted it to take Nick and his good friend, Chris Corner, his channel Corner Office, since they have a fun friendship, to basically analyze the two and provide just a strategy, compare them, create a formatted output, and to go also check out other viral channels and incorporate those recommendations. So this is not deep research. This is just Sonnet 4.5. And it created this, you know, this is like pages and pages, content, pillar strategy, distribution, strengths, weaknesses, channel profile. This is just like super, super comprehensive. It just keeps going and going and going. So I'm just going to keep scrolling until we get to the end.

11:10But essentially, it's created this really good comprehensive plan that before you would maybe get a fraction of this as an output and you'd have to sort of iterate through. And this, again, went out and did the research and used its agensive capability. I feel like perplexity, people used to use perplexity for a lot of this, like the deep research. Yeah. I mean, you're still scrolling. Yeah. This looks insane. Yeah. Like I actually should see, let me download it and we'll see how. Is it something like where you have to select deep research or is it, is it kind of deciding based on the prompt and the ask how much time it should, it should give to extended thinking?

11:48So I gave it, I told it to work for a certain amount of time. It didn't necessarily work for that amount of time. So I think it just depends on like how complex the ask is for that. So that ended up being 22 pages. I mean, I haven't read through it, but I'm also assuming it's not the typical AI slop. Like it's not 22 pages of chat GPT 3.5. Exactly. Yeah. Real analysis. Definitely like bullet points, but it, you know, it pulls out example episodes before, you know, it could get these wrong. Or again, this whole hallucination problem, I feel like is not quite solved, but it has been made it dramatically because it's actually able to go do, do the work.

12:27Wow. Yeah. So like my first, this actually nailed it. Competitive landscape. Like you guys are catering to specific audio. This, this totally. You got to send this to me. I need this. I will. This is Liz is going to start her own research firm. I mean, this is insane. This is incredible. Yeah. Like a lot of that entry level research and analysis. Oh, look, you've got a head to head comparison so you can duke it out with your bud. I wonder, could I also upload data sets and have it look more thoroughly than it has in the past? Usually when I have data sets, I have to give it explicit instructions of, hey, look at it like this or normalize for this data or tell me where the statistical significance is, which variables are correlated, which ones are not.

13:19Will it do that on its own now? I would imagine so. I don't know if you've prompted it yet, but I would say high probability yes, because again, it is thinking things through before it does it to identify what's important and what it thinks you're going to want. So my guess is yes. And again, I was able to download that into a PDF. You can download it into other file formats. So you could do like a really deep analysis for like an investor pitch deck or product demo deck or whatever kind of PowerPoint you need. Like you can actually sort of get that full workflow within the browser now from research, getting the content framework, the look and feel, all of that within the browser, which in my mind is just mind-blowing.

14:09If you think about a year ago, you could not do any of that. How do you think about getting the most out of this technology? Because I think of it as a fighter jet. I could fly it. Huh? Yeah. Yeah. I agree with you. Yeah. So like there's so much technology in it, but a fighter pilot is going to be the one who actually knows how to use it. If you put me in the cockpit of a fighter jet, I don't, I don't know what I'm, I don't know what buttons I'm supposed to push. I don't, I don't know how to fly the thing and whatever. So like, I see stuff like this where I can spit out hundreds of pages of analysis, but the stuff that you just said was like, yeah, now you can do this framework and have this comparative analysis.

14:51It's almost like, how do I even get up to that speed? Yeah. Does that make sense? Hey, I don't know if you remember this, but when we started this podcast, we entered into a social contract. I would spend time, energy, and money producing this podcast, interviewing these individuals and giving you insights into how to build, buy, start, grow your business. And you would like, subscribe, and leave me a five-star review. Now, out of that, we both get to talk to really cool people and hear really cool insights. We both get a ton of value. But I just want to help you keep your word. So would you do me a favor?

15:24will you go leave a five-star review for me on Apple or Spotify? It would really help. And if you want, even share this with a friend. Number one, you've got the best mentor and teacher available, which is Claude itself. So if you don't know what to do, literally just ask it, say like, I want to achieve this outcome and always do the outcome rather than say like what you think the process is. You can say the process and get feedback on it too. But if you just say like, I want to achieve this goal, what do you recommend? Give me a couple different options. So that's number one. And as you're going, like, again, ask for feedback and say, like, I heard on it 4.5 just came out.

16:00How can I use this? Give me ideas. Here's what I'm doing. Give it the context. And then again, you can use projects to give the context of like your life of, you know, everything going on in your life, whether it's personal, your business stuff, your health, your finances, you can, again, create cloud projects to put stuff in each of those places. and then you have that context. And as the new functionality comes out, you can just reference those over and over again. Or better yet, you go to Google Drive, create a folder, and then you can always update things and reference them within the projects.

16:32How frequently do you update reference documents and context documents? Yeah, so I have certain workflows that are, I would say, somewhat consistent. So for instance, if I'm trying to write viral scripts or posts or contexts, I have certain frameworks that I probably should update more, but I have both like automation workflows. And then I have just like pod projects that always like, like the principles are always the same. Like how you do storytelling, like that's not going to change. So I have static information there. And then I was like new themes or trends. That's sort of what I'm updating.

17:10So it's a mix of like where there's dynamic things that are changing and where there's not. So yeah. Do you like create time for yourself in your schedule to say, all right, I'm going to go update? Yeah. You do. Yeah. I have sort of like one day a week where I have a couple hours and that's usually late evening because then I have like family time and kids and kick up and I'm a mom. So deal with that. So late evenings is usually when I do sort of my work on updating my Claude stuff. I'm building out Claude code stuff where I have framework. So I have like a Thursday night that I do like framework updates.

17:45Mondays, I sort of have like a check-in of like what's been going on. And I'm even sort of building out like the Elon agent of what did I get done this week where it literally goes into - Really? Yeah, it goes into Cloud Code and it just basically logs everything and gives me a summary of like, what did I do this week? And you could do that and have it like go even towards like other applications and sort of like check out everything you're doing. There's a software tool called Rescue Time, which you could use too, but you could build out your own agent to do that as well. So I have sort of like general frameworks and personal systems that I'm really trying to nail that I have like a couple hours every week I'm working on.

18:19And then my fun projects that I'm doing a couple hours every night. And then the rest of my day is, you know, just normal work stuff, but I'm always using. How do you, do you think about the AI management in different buckets? For example, when I think about managing a business, I'm like, okay, there's five pillars to managing my business, there's finance, there's operations, there's product or quality, there are people, and then there's marketing, sales and marketing. Those are like my five pillars in any business. Do you think about like managing your AI? It's not even a tech stack. It's more like a AI brain.

18:57Yeah. So this is like, I've actually been thinking a lot about this and I don't have a good answer and ask me in a month and maybe I'll have a better answer because I've been trying to of my like my operating system I've been putting a lot of thought into like what does that look like for my life where I can use AI across the board from you know not my business stuff so I have like the business pillars which are similar to what you would say for normal normal business pillars and then I have just other other slices of my life so managing my kids schedule and everything they're managing my finances and everything there but I would say my health but that's non-existent right now.

19:36So, but like that, you know, that should be a slice too. So, you know, I would say those are sort of like the big buckets, but in our like maybe pillars. And then under those, there would be sub pillars in their own sort of sub operating systems. Is that what you're going for? You're more thinking of like - Kind of, kind of. I'm more curious around like, okay, this is, I have a frameworks bucket. I I have a prompts bucket. I have context documents bucket. It's like, in order to have that brain, the AI brain, it's got to have specific pieces to it. And I'm just trying to think through like, what are the specific pieces?

20:13That's great. Okay. So yes, is the answer to that one. Okay. So I do, and this is going back to how I've been building the architecture for my personal operating system. Because for instance, I've collected all these hooks. I've collected all these content frameworks or just like business frameworks. And I'm like, I don't want to keep putting them into every file folder. So I sort of have prompts that I use and I have just like a document that I've been using. I would say I probably could get that better. I've used tools in the past. Like there's this really cool tool called, you can't see my extensions, but TextBlaze has been keeping a lot of my prompts for the time being.

20:48And at some point I'm probably going to export them. And then I also just have just a little Google sheet of prompts I like. And I have built out a little automation tool that I basically have prompts I use that then can be saved into that Google Sheet. So that's like one thing that you could also build out in Cloud Code as an agent for prompts that you like, where you can easily from your interface Cloud or ChatGPT or whichever tool, have it be inputted into just a master repository. So that's one. So having a library, but if you want to just keep it simple, just like Google spreadsheet, then number two, I am building out an agent system where I have like a master orchestration where it's basically the general rules of everything.

21:33And this is, it's sort of like project management plus some other stuff because they're not everything fits in the project management bucket. But right now when I'm working on a new project or I'm building up something new, So there's like very similar things it's going to reference. Or again, there's like templates and then there's like content repositories. So I have the prompt library. Then I have like content frameworks and templates. And within that, I have like a subfolder of hooks. What's a content like that you're putting out? Yeah, I know. So I don't have enough content or good stuff that I'm like, this is my stuff.

22:07I did train it on my voice though. So I do have like my voice document where I had, I uploaded a bunch of my content. and then said like, what's my voice? So that AI could replicate my voice for AI generated content. Yeah. I have like my own personal voice documents. And then I have frameworks or people that I like to follow who I want to replicate. So I have like their scripts. And sometimes it's like, I like the framework in which they laid out a video or the way they built this. And so I'll either capture their voice as well. I'll capture the framework they've built out or just some of the ideas from that video.

22:46So that's sort of similar to the content cloud code thing I built. So I'm basically trying to get that to be accessible and not interfacing itself. But right now it's just literally a file of like stuff I've collected and curated where it's the content and stuff I like. So again, it's ideas, frameworks, my stuff, my voice. I've been thinking a lot about this as well. like you're basically building out your second brain yeah it's your it's your ai second brain yep and in order for the second brain to be functional like it's got to understand how you think yep which is one component how does nick or how does liz think about certain things how do they make decisions how do they approach problems and then it's also got to have context because it's one thing to just how does nick think about things it's another thing to know what's currently happening.

23:35Yes. Because one without the others, knowing how to think without context just is not, is not great. I just think about like, how do you, how do you manage both of those staying current? Yeah. Right. You got to have data aggregation. You got to make sure, okay, my, my zooms are recorded and it's got to go into my emails to know what's going on there. And I don't know, I'm just trying to think through, has anybody figured that out? Like, have you figured out a good way to have it? Yeah. So, okay. So going back to sort of the system and the framework I was building. So we'll take content as an example, and I can explain sort of the automation piece of it too, which I think there's a component you're referencing.

24:11So in addition to, so I have like content stuff, I have the prompts library, and then those are just like resources, which I, things that the agents can reference. Now the agents is where the action is actually happening. So that's actually where the decision-making, like you were talking about comes in because every time you build something in cloud code, or again, in your project where you put instructions, Or even in the prompt, you're giving specific instructions, which again, is that how do I make decisions? And so you could easily provide inputs or examples or history that you've already done for the AI to extrapolate those decisions.

24:48Or you can be explicit and just say, here are the rules I have for this specific bucket of things. It could be so specific or it could be super broad. And then say, whenever you do these actions, reference this decision matrix or whatever rules you give it. And so that's the beauty of having agents where it can go and say, here's the rules I have. And you can basically, again, make these super narrow. And there's a number of different ways to do that in cloud code. And then go execute this task that I'm giving you by building out this agent. And so it's actually doing the work from the content and context that you've given it.

25:26Like those ones, the building out the decision making, how are you updating those often are you like going at the end of the day in each one of your chats and saying please write me a context document no uh i wouldn't say like my decision making or those rules are being updated not dynamically right now i'm in the process of building stuff out for like like a system like a trading plan first like that's sort of my latest thing i'm right now trying to systematize so that i'm more iterative but at some point i'll probably stop that until I get inputs that are changing within the market that make those rules or that system less relevant or it's not getting the returns I want.