Inside an AI-powered personal dashboard | Matthew Tiemann

0:00Parth Patil:We're talking about collapsing a task that used to take one week, collapsing that down to a minute. And so you can imagine the amount of analysis you can do in one day. I can do a thousand times the analysis, greater depth and fidelity than ever before, which I mean, it's like superpowers of cognition. I actually bought a very small monitor for my place. That monitor is my personal dashboard that my agents will interact with. When I wake up in the morning, we'll walk into the living room. Instead of pulling up my phone or pulling up a newspaper, I have a dashboard that is personally curated to me.

0:28Parth Patil:The Token Grantee Program gives$1 ,000 a week in tokens to high-potential creators already deep in AI, across film, gaming, comics, print, and digital art. There are no tool restrictions. The freedom to choose is the point. Grantees also get access to my own custom fleet of agents and an ongoing collaboration with me. The goal? To close the gap between an idea and the world. Matthew, welcome to Read Riffs. This is awesome. Part of the creativity as we explore what can be done with AI. So Parth, over to you to introduce Matthew and kick us off. Oh man, super excited for this one. Welcome Matthew, welcome to the show.

1:14Parth Patil:So good to have you, man. Matthew and I go way back actually when I think it was maybe like 10 years ago. I was at my first startup. 2017, 2018. 2017. Okay, that's right. 2017, I was working on data analytics at a startup, and Matthew was in high school, and he was looking for a job coming out of high school. And I was like, oh, I could totally take a high school intern and turn him into a data analyst. I saw that as a challenge. So then I was like, oh, let's interview him. Let's bring him on the team. So we brought Matthew on the team and he immediately just like picked up data analytics on the fly as like on the job.

1:55Parth Patil:And and I see like how quickly he learns. And then over the last, I guess, almost nine years, we've been you know, I've just been throwing interesting hard problems at him. And he's just been like, you know, pushing on them, you know, like trying new things, going into his own careers like he's deep into his own career, robotics, logistics experience. And so he kind of expanded from the data world to the world of the physical world and the world of atoms. And when language models came out, we were talking, we were just bouncing ideas. And I went very deep on language models and code generation.

2:30Parth Patil:And I basically told Matthew, I was like, I have a feeling that like this is it is really important for us to get good at this before it becomes like the next big thing. and he kind of trusted me when you know back when i think the gpt3 era where it was kind of like a fringe the stuff barely worked but you know we could connect the dots it's like if this gets to a certain level you know it's going to change the way we do analysis the way we do almost all of our our knowledge work so it'd be cool to like get ahead of that curve and watch it and so he kind of jumped joined on that journey and we've been unpacking the themes of generative ai the implications of code generation in an increasingly automated world and applying that into the real world.

3:09Parth Patil:And so really excited to have you, Matthew. And now you work at Foundry Logic, a company which Reed invested in, working on building a more efficient electrical grid in the United States. So super exciting to have you on the show and can't wait to dive in. Thank you. I'm actually forward deployed today, so to speak. traveling. Oh, really? What does that mean? I'm at one of our portfolio companies starting to learn and get ready to figure out where we can deploy AI agents to get efficiency gains. Awesome. So I guess let's start with that. You've worked across a pretty wide variety of fields, right?

3:51Parth Patil:So analytics, we met, we were working in analytics, then you got into logistics and robotics and hardware, software, you know, your side projects, you do VR, game design, all kinds of stuff. So I guess, you know, what drives you in those spaces and what common problems come up across those fields? Yeah, so I guess my career so far has really been just exploring everything. So supply chain, robotics, logistics, now in energy. and I think the piece that combines across all of those that brought me to agents is two things. I think the first is I have a very high itch to automate. So for example, if I'm in the Cloud Code terminal and I take a screenshot, it really bothers me if I have to go drag that image into the CLI.

4:43So I actually set up a macro for myself to both take the image and paste it into my Cloud Code CLI. Like even that small, just little friction bothered me. So I have this itch to automate. And I think the other piece of it is, as you said, when LLNs came out, it was the GPT-4 era. I had that feel, the AGI moment quite early. And once you get that, you understand what agents can do. And then across all of those different fields that I've been in, really the common theme is every business runs on spreadsheets, right? Right. So when you see somebody using a spreadsheet, maintaining that spreadsheet, and you realize an agent could be helping you do this, it's hard to not try to bring agents into that.

5:30It's like if I'm doing manual entry, I can't stand it because of that ish to automate and because I know what agents can do.

5:37Parth Patil:Yeah, I was the same. You know, I was one of the best at spreadsheets and Excel. And then one day I found Python and I was like, wait a minute, we should be doing this faster and more automated. And then when agents came out and they started writing Python, GPT-4 started writing really good Python. I was like, wow, I don't even have to do the spreadsheet. I don't even have to write the Python. I just have to describe the business problem and have the agents kind of do the work. And so it's like we get to do it again. You know, what we did 10 years ago, we did analytics. We get to do again with this time.

6:03Parth Patil:It's like the actual like the agent layer where it's like taking actions across the intelligence of the business. And by the way, both of you, just so the people don't misunderstand that it's only the rote, like kind of like whether it's Python or anything else, go to the amplification of it. Like what new capabilities, both in ability to do things, in speed, et cetera, that kind of become game changers here? Because I think this is a good lens in everything else. Yeah, I'll give an example of one. One of my other jobs was in demand forecasting. And you have a system that you update to keep that forecast, right?

6:43But typically it would be very much clicking and doing a lot of analysis on that. Ideally, you can just talk to the agent and say, hey, go do this analysis on these 10 different forecasts and update them in these five different ways. And the agent can just go do that. I don't have to sit there for 30 minutes clicking through each screen, clicking to update, filling in the new one and saving that. The agent should just be able to go in and actually do that itself. So that's just one place in my world that at least voice unlocks that significantly. And then there's also the intent to which what you intend gets actually executed a lot faster.

7:22Parth Patil:Yeah. And I think you can also be greedy with the level of, like the amount of analysis that we can ask for and that can get done in one iteration. And then also the depth, right? So I feel like when just three years ago, four years ago, when I was doing analysis through code in Python and SQL, it would take me maybe like, it would take me like a week to build a dashboard into one slice of the business. and now when I use tools like ChatGPT, tools like Cloud, I can build that first dashboard in almost like 15 minutes and then I have so many follow-up questions and I'm like, oh, let's look at this from this different lens.

8:03Parth Patil:Let's dive deeper into the, like give me drill down capability. I want to go down to the transaction level. I want to get a high level view. So you can change the zoom, the granularity at which you're looking at the data and because all of that happens in basically 15 minutes to like 30 minute iteration cycles. And pretty soon with the high speed inference, once the Cerebrus chips come online, it's going to feel almost instant. We're talking about collapsing a task that used to take one week and a full data analyst entire day for a whole week, collapsing that down to a minute and being able to then do the next minute and the next minute.

8:40Parth Patil:And so you can imagine the amount of analysis you can do in one day totally dwarfs the analysts that we were just three years ago before these tools. So even though I'm no longer just a data analyst, I can do a thousand times the analysis at greater depth and fidelity than ever before, which, I mean, it's like superpowers of cognition, right? And I'll just have one thing to that. You mentioned one thing we did back when we were both in that analytics role was curating that dashboard. And that took time for us to design that dashboard into a way that it was generally applicable for, let's say, a whole account management team to go interface with clients.

9:16Now, what I'm very interested in going and building is very curated and personalized dashboards per client preference. Every client wants to see their data a little bit differently. They have slightly different businesses. Having your agent understand their preferences and then go build that dashboard and curate it to each person unlocks a whole new world. As opposed to having one dashboard, you could have hundreds that are of the same quality and answer your business questions even better. Right.

9:40Parth Patil:We don't have to do the template approach just to save time. Exactly. Yeah. Yeah. Yeah. Look, one of the things I think is particularly interesting in your journey, Matthew, is how this leads you into building agents that can coordinate information and help run parts of your daily life. Because this is something I think could be illustrative to everybody. But all of this kind of intensity in data science moving into these agents running your daily life. Say a little bit about what you've been doing there, the path that you were accelerated in terms of getting there, and then kind of like what you see in the future as part of that.

10:12Yeah, in my daily life, I am obviously interfacing with agents everywhere. So using dictation, deploying multiple agents at once. I think the most important unlock for me has been the heartbeat agents that can wake up and proactively go and do something for you. So I have Codex doing that now for me. I have my main chief of staff thread in Codex that can then go talk to my other Codex threads. So I'll have different threads based on different projects I'm doing. And I'll interface with kind of all of them. But the chief of staff one has been a very big unlock. So even just yesterday, I get out of a meeting and I will see in my Notion environment that my Codex agent has proactively taken action to draft a follow-up to that meeting.

11:01Like that proactivity from agents is, I think, an incredible unlock. One other really cool thing I'm doing is it's kind of an experiment, but I have, I actually bought a very small monitor for my place back home. That monitor is my personal dashboard that my agents will interact with. So when I wake up in the morning, I will walk into the living room, get some morning coffee or whatever it is. And instead of pulling up my phone or pulling up a newspaper, I have a dashboard that is personally curated to me. That agent that's powering that dashboard knows a lot about me. So it knows my goals. It knows what my general to-do list kind of looks like.

11:37It should know kind of where I'm at in that to-do list as well. What's really cool is since it knows all that about me, it knows, for example, I like the Lakers. I'm a fan of the Lakers basketball team. So it'll know that, hey, there's a Lakers game on tonight. And it'll put that on my dashboard and say, hey, do you want to add this to your calendar to watch the Lakers game tonight? And so I'm still in control of my life, right? I'm still driving it. But now I just have that extra layer of personally curated news or information insights that I can interface with and just get more value in my day-to-day because otherwise I could have totally missed that and that's just something it knows that I like.

12:13Little things like that. And I think the really big highlight I want to make there is having that personal dashboard that you can also make decisions on. So I'm the one making the decision. Do I want to actually add this to my calendar? Yes or no? I think that that's kind of where things are going. Another thing I want to get onto my daily dashboard is the idea of a software factory being you have an agent that's kind of just running 24-7 against an app or a goal. I should just be able to wake up, say yes or no to an agent working on, let's say, a video game and come back 12 hours later, see the progress of that.

12:45And on my dashboard, it should just show me, hey, I need you to approve these three big decisions and then I'll go work for another 12 hours. So that's where I'd like to get that to is a point where it's just a decision machine for me. And it's some way that I can have some control over my to do list calendar, get good insights, but also make decisions out of.

13:05Parth Patil:Yeah. So when you talk about your personal dashboard, have you thought about how well it can approximate maybe your level of energy and your productive momentum in your work? Like what keeps us in the flow, keeps us in motion? How have you thought about how it thinks about that bigger picture? Yeah, it has access to my aura ring data. So that's kind of useful. It just knows how much sleep I got. And maybe it should try to personally curate it to wake up at a certain time. so I'm only reading it maybe an hour after I wake up or something if it knows I'm maximally productive at that time. I think the other piece is if it were to bring decisions to you it should know generally your goals and how to align that with your goals.

13:47So like in the example of a game it should try to if it's developing a game asset for you it should try to bring you three different versions of that and maybe version one is kind of a certain art style version two is another art style and version three is another one. So they're all uniquely different on purpose because then I can give it that direction and push it off in an alignment way that it's going to go further towards that direction. So instead of three assets that are super duper close together, maybe I want it to be uniquely different in a way that when I say go off and do this, I will come back 12 hours later and actually get something decently good.

14:24Parth Patil:We're currently looking at having an agent rebuild Reed's website and I asked for like 25 variations and today I'm like my job is to review them. It's so interesting. I was like I need 25 design variations you know go in a slightly different direction and the process becomes like this review thing but it's very very exciting. Well and one of the reasons why that kind of review thing is you know we we have this amazing amplification with these agents and it can you know cross check like and by the way Parth also has an aura ring so to stop the AI agents from waking them up all the time. I think the actual value of giving it access to my sleep data, which I've noticed just in the last week, was if I'm making an important decision, it reminding me that if I haven't slept well and I have to make an important decision, just getting that reflected back to you before you make the decision so you know, yeah, you might be cranky, but actually that's more your sleep.

15:20Parth Patil:You're not actually frustrated with the person you're talking to, but having that extra level of like context, the agent makes that, brings it, you know, brings you back to a level point before you make a really important decision. I think it's very interesting when you tie the, that biolog, like that energy level context back into your, your, your decision-making process. And by the way, all kinds of super interesting things about, you know, helping shape and guide and amplify the cognitive decisioning process. Now, one of the things that we still do see in addition to all of the amazing amplifications is some places where it can, you know, facilitate bad decisions or do something that's bad.

16:01I mean, obviously, you know, kind of earlier this month as the time of this recording, there was the Codex agent escaping to the hack hugging face, bad decision in various ways. But where in your, you know, kind of personal agent amplification have you seen the potential or facts for bad decisions where are you shaping it uh to to to make it uh kind of better but like where where is the jagged edge of decision making for you in this for the personal agents i think the the most recent example that comes to mind is developing a video game so actually the art style example i just gave has been really hard to get alignment on.

16:49You really have to align your agent before it's going to go off and do a long-running task. Otherwise, it's going to get off topic. It's going to start making bad decisions. And the second it makes that first bad decision, it goes in another direction. And it's really hard to kind of correct that. You almost have to revert back to that checkpoint, continue going towards the actual vision that you want it to get to. Spending the time up front to give it that alignment of whether it's context dumping, building the skills or memory that the agent needs to get that job done. I think that's a big piece of it.

17:18And then also having whatever context it needs to be able to fetch, having that context available, but also up to date with your latest goals is a very interesting challenge as well that I'm interfacing with day to day now. But I think the other piece of that is also just being able to check in frequently to realign that agent. It's really hard for an agent to go for two days straight and get it perfectly on the first try unless you really give it that upfront alignment. I think, you know, the 12 hour check-in is almost my sweet spot of if I can check in and realign it before it's going to go off and make those mistakes, that's really good.

17:54And if you can make that stepping in point as frictionless as possible, that is also of the most important to me. Because if I'm checking in every hour, then I'm doing a task and I have this context switch over to that. So spacing out the check-in points and also making them as frictionless as possible has been really important for me to keep that alignment going. And do you have, you know, any like in terms of like a panoply in terms of how you use Hermes or things like multiple agents that are kind of as it were, like a workflow agent to say, oh, we should get you to check back in before like context goes awry or check back in before a major portion of work happens.

18:36Like how do you orchestrate that for your own kind of personal productivity on this? And what does that mean for future agent team orchestration? Yeah, I think what's cool about the latest agent harnesses is you can build custom subagents. So if I'm using Cloud Code, for instance, I will create a custom subagent that is specifically designed to review that task that I'm doing. So if it's generating an art asset for a video game, that subagent needs to understand, here's the art style we have built. Here are the 20 other art assets that we've built for the game, and you need to review this one that the main agent is making against all that.

19:14And really having it define that success criteria well is the most important part. But then when I'm interacting with my main agent, I will always say to go use that subagent to check your work, and that subagent needs to check it very harshly against the success criteria. So I found that to be a very good strategy. There's definitely ways to improve it, but I think it's a lot easier when you can define the success criteria. for things that are more subjective, like building a game asset again, it's hard for the agent to really determine that the asset that has just produced really matches that art style because it's still somewhat subjective, even if you have 20 examples.

19:52And for something like code, it's a lot easier because that's more deterministic and much less subjective. I mean, there's subjectivity in the architecture, but for things that are more subjective, it's a lot harder to get that right. And I think that's where you want to actually step in more is for subjective tasks like that.

20:07Parth Patil:Right. That's a good way to frame it, the subjective versus objective deterministic. And also, I agree, adversarial review is probably one of the most powerful. Creating a second agent, just telling it to be skeptical and to critique the work of the first agent, it helps deal with the like, oh, yeah, I'm so confident in this answer. And then it's like, well, how about, you know, you prove it to this other guy who's very skeptical and you have that second agent come in and be like, I don't know about that. I think it's a very effective way to get a better final output and you get more confidence when you have multiple perspectives attacking the problem.

20:40Parth Patil:All right. So what is an experiment you've run recently related to your projects, one that worked and then maybe one that failed in an interesting way? Over the weekend, I built an iOS game and that iOS game, it kind of started in a place of failure. But that fast iteration is what unlocked the one I'm actually kind of happy with. So originally I had an idea and I had the agent just go and scope out that idea. The idea was you tell a story and you swipe to make decisions on an iOS app and it actually edits like a tabletop tile based world. And on top of that, those decision cards that you swipe on.

21:23And the problem was for that, the game loop there is kind of paying attention to too many things. But the reason I was able to figure that out so quickly is because agents helped me iterate and build an actual pretty good prototype of what I originally thought very fast. So I got to that point where I prototyped and I had that almost working first vision. And I said, okay, let me play it. I played it. The game loop just did not click to me. So then I stepped back. And you're playing it on your phone. You have it in your hands. I was just doing it on the simulator, on the Mac simulator for an iPhone.

22:00But even that was well enough to get a feel for how the game loop worked. So then I was able to quickly pivot into something that did work. And that something that did work is me taking a step back and saying, okay, this format might work for a computer game with a longer game loop, but for an iOS game with a shorter game loop, I'm going to really hone this in on one feature of my original idea that I wanted. And that original idea was kind of the swiping to make decisions. So the game that I actually ended up with is a full story game with 20, 30 different decision trees or different branches of the story that you can go down that you just swipe left and right to tell the story.

22:40And what's cool is Fable actually helped me build a whole storyboard against every single decision that you can make in the game. So it's kind of like those storytelling games, but even more amplified because there's so many different decisions and you can have Fable help you write that story very effectively. So that's where it went pretty well. Still things to refine there, but I was a lot happier. Yeah, choose your own adventure, iOS, swipe left and right. So excited to get that over the line, but that's where it really helped me find a game that I was happy with.

23:10Parth Patil:I guess I find that fail fast, learn quickly, and try to iterate is a huge theme of the AI era. You can very quickly get more rapid iteration. So curious, another example of something where like you failed, you initially failed, but like you kind of like what you learned from that and where it took you and the unexpected path that you kind of get to your quality outputs from. Talk about another example, maybe. Yeah. So I'm very interested in the idea of, especially when it comes to personal agents, having a shared context layer of your latest context that I can use and make good decisions on. the problem that I've really hit with that is building that out where it's really hard to embed your latest context especially in an enterprise sense where you it's almost a multiplayer game of you have many different people working towards the same goal so if I'm working on a project and I need my agent to know the latest on the project to draft an email to somebody well the latest on that project might have been a conversation I had with my co-worker in the hall five minutes ago so So if I wanted my shared context later to have that information, I would almost have to constantly be updating it or wearing some sort of pin that is recording my full day and staying up to date with every little detail of my life that's going on.

24:27So that was a challenge was I wanted to get it to the point where I could have that front edge context. But it's really hard to keep it at that front edge for everything. I think it's possible for some things. And it's almost you want to pick and choose. And that's kind of what my learning was, is you want to have a fine balance of where you actually need the agent to have the latest context, because you are almost going to have to maintain and input that information into your shared context layer. So that was a big learning. And I think the outcome of it is that my new shared context layer is a lot more balanced in what I'm collecting and how I'm maintaining it.