I gave Clawdbot (aka Moltbot) access to my computer, calendar, and emails: Here’s what happened

25:49Clawdbot:I already gave you an identity, but please always identify yourself as an assistant. And it should, I think, knock on wood, store this in its memory and do this in the future. But it was a really funny learning in terms of prompting is really quite important. I thought I was being fairly careful with permissions, which I was. It could only do a couple things. But I underestimated how much it seems like this tool is biased towards acting as you, as opposed to acting as an assistant. And I'll have to look through the repository and I'll have to kind of get myself familiar with how it's implemented.

26:27That's not the intention of this podcast to really understand why that is happening. But prompting really, really matters. And I think the product lesson here that's kind of interesting is, yes, I could

26:40Clawdbot:have been really, really precious about prompting. I could have said, create a draft of this email to these guests, send it to me for review before you send it. But at the point that I'm doing that and each turn takes at least a couple minutes, this is not a productivity tool. This is not making me more efficient than sending that email myself. And so I do think there's this balance between these autonomous agents being user controlled and being really cautious about how you prompt it and being autonomous and probably doing some things wrong. And I think this is a prompting problem on both sides.

27:14It's a prompting problem on the product provider side. It's a prompting problem on the user side. And I don't think enough people are probably sophisticated enough to decompose why one prompt versus the other would do well if you're just a consumer or a prosumer. And so I think this is where a lot of the weird behaviors that you'll see are coming out. So, so far, what have I done with Claudebot? I've installed it. I have given an identity. We have rescheduled one event or we have scheduled one event.

27:42Clawdbot:We have given an access to email. We have rescheduled two events now and emailed guests about these events. And then this is where it goes crazy. This is where it gets fun. So I decided to give it edit access to our family calendar. This is a calendar where we have pickups and drop-offs and basketball games and piano practice and my ballet practice and all that stuff. Now, I love this calendar. It's very important to me. And if I needed to nuke it, I definitely could. So I gave it access. And what I wanted to do was, one, email my husband and I about upcoming week and get us coordinated on where there were gaps in terms of pickups or conflicts where I was across the city at a Purcell event and he was needing to pick up the kids for basketball practice.

28:31And I wanted it to fill out the rest of my calendar. My kids have started a new basketball season. Our neighbor is picking up the kids on a certain day. All those things I wanted to get it done. And here is the

28:44Clawdbot:problem. I gave it a bunch of instructions and it could read that calendar pretty well. It could categorized the events pretty well. And it had no idea what day it was. And so as I was on Telegram, going back and forth giving it can you add this? Can you remove this? Can you change the schedule? I thought it was doing a great job on Telegram because I wasn't really paying super attention. and it was confirming that it did all these things. And then I opened up my calendar and everything was on the wrong day. I mean, everything was on the wrong day. And if you are a parent, you get this. You're like, wait, wait, wait, wait, wait.

29:32Clawdbot:Is so-and-so picking up kid number two on Tuesdays or Wednesdays? And I know I moved piano, but I don't think I moved it to that day. So it took me a second to understand the damage it had done, but it had really gotten things wrong. You can see me say, stop, you are setting all these one day late. And it was setting everything one day late. And not only was it setting everything one day late, the CLI tool that it was using to add these events to the calendar could only set one-off calendars. And so it could not set a recurring event. So if I wanted to delete these broken events, I had to go through one by one and delete them.

30:13Clawdbot:And then the other problem with our crustacean friend here, when you're collaborating with them, is I was on my computer, this one, with my calendar open. It was over here in the CLI with its CLI open. And we were conflicting with each other. So I would try to delete all these bad events, and then it would go put them back because it thought something got broken. We were just, I was trying to add them in. I said, you know, stop. It did not stop because of latency and because of these sub agents. And so I went through and set up everything correctly and it went through and deleted all my work. It was, it was terrible.

30:53It was really, really stressful. And I said, you know, I had to completely redo. It's like emailing my husband every five seconds. And so it was not great and it actually never got it right and I will show and share with you the discussion we had about time zones but this is another thing that you know non-software engineers using something like this really have to be aware of is as I said on X the only remaining software engineering problem is time zone conversion and LLMs just have no sense of space and time. It just does not know when now is. It doesn't have a sense of time passing. Now, I will say Claudebot, because it has these daily files and daily logs, has a little bit more

31:40Clawdbot:of a temporal sense, but not a great one. And so if you don't understand why a computer could get dates wrong using a tool like this, you're going to get really frustrated. I could at least understand why time zone conversion, maybe there was a UTC timestamp in the Google API. I can at least understand why this was happening and help guide it towards a solution. But it certainly was frustrating and something that I don't think your everyday user would be able to do. So I'm going to entertain you all. And I'm going to tell you, as I was doing this, I took a pause and I took my two youngest kids to Target because we were out of stuff.

32:24So I asked if it could discuss things with me via voice. And it said, sure, you can send me voice notes. I can send text back. I could send you voice notes back, or we could go through Twilio and I could set up a phone call. I just said, let's set up voice notes to your text reply. And so I could press voice on Telegram and have it reply to me as I was on the go.

32:47Clawdbot:And so while we were in this back and forth on time zones, I want to share with you my delightful voice messages to Claude Bot, because this was a real, real energy. Let's see if we can hear them. Okay, so this is me at Target pushing a cart, getting really mad at Claude Bot. You put it back, but that is a a Friday. Friday is a date, so do not change anything. But can you please explain to me why you are getting days mixed up? This league game is on the correct day. Again, please do not change it. But I do not understand why you have the days mixed up. Okay, so I am getting super annoyed by this experience of getting days wrong.

33:36And it replies, oh my gosh, you are absolutely right. I see the problem now. I was off by one day. Here's all the new dates and they were still definitely off by one day. So once I sent my mean mom message, it came back with me and said, you are absolutely right. I apologize. Here are the dates, right?

33:57Clawdbot:The issue is I've been, this is very funny. I've been trying to quote unquote, mentally calculate which day of the week each date falls on. Even though the API is telling me what the date of week is, I should probably trust it. But I was using my LLM brain to decide. And what did I say back to it? Well, I said this. You are a computer. You are not doing anything, quote unquote, mentally. You are making calculations. Can you look in your logs at all and understand where the calculations come from or no? And if you did not enjoy this, that is my very, very new baby crying in the background as I'm lifting him from the car seat into the stroller.

34:39Clawdbot:It was quite an energy. And again, this is one of those things that as a software engineer, I get it. I have done time zone conversions for my whole life. I understand that APIs return things in all sorts of formats. I understand LLMs can't do basic math when it comes to dates. It's just too hard. We do not have the technology. And yet the fact that this model told me it was doing it in his head was so hilarious. So once we had the back and forth about this, it gave itself a rule to follow in terms of getting these dates right. And then I asked it to add it to its rules. Now, the final thing that we did is I asked if it could send me voice notes back.

35:25And this is where some of the magic of Claudebot really does come out. One of the things that people have been saying

35:29Clawdbot:about CloudBot that's so cool is you can just get it can give it self skills. It can learn things. It can just do things very magically. And if you're trying to get back and forth voice notes in Telegram, it would have been pretty hard to like figure out what API you want to use and what skill and hook it up and use CloudCode, all this stuff. And it just did it. So when I said, can you please send me voice notes back? It just sent me a voice note back. So let's see. Yes. I can send voice messages back to you. Let me know if you'd like me to use voice for replies. I can do that anytime you want.

36:04Clawdbot:That was a pretty magical moment. And, you know, I've been giving Claudebot a really hard time in this episode, not because I don't think it's an awesome product. The reality is going back and forth via text with something that has helpful access to your calendar, has helpful access to your email, can learn skills like voice that you can just chit chat to. I actually really liked the form factor of the experience. And I liked the concept of what it could deliver. It was just that the implementation of it had a couple of things. One, too technical for the everyday user. Two, too scary to the security aware user.

36:49And three, latency that took some of the magic away from the experience. And so again, I don't think this is a bad product from a capital P product perspective. I'm just not in love with the implementation. And we'll just summarize my what I did with Claudebot with my last use case, which is I have Claudebot use its history to create a Next.js app that showed the history of our conversation. And I asked it to redact names, numbers, URLs, email addresses, all that stuff so I could share it with all of you. So again, kind of a classic AI engineering, AI coding, vibe coding use case. Now, the one thing that I will say is a lot of people are really excited or say they're excited, I don't know if they've used it, to use CloudBot to spin off CloudCode to do coding for them.

37:47And where this wasn't the magic use case for me and why it didn't start it is I've been spinning off remote agents with computer access to do coding for me for a while. I use Devon, which has a virtual machine in a local environment and can spin up stuff, access to the web all the time. I use it from Slack so I can at mention Devon. You know, I have a Slack bot for chat PRD. So I'm at mentioning my product manager all the time. Cursor has background agents, you know, everything has codecs you can kick off online. So I don't know if people are just not using those tools, I guess, Cloud Code doesn't have one like that quite yet.

38:25I don't know if people aren't using those tools, but I've been coding by kicking off an asynchronous teammate, quote unquote, for, you know, two years now. And so that piece was never what I wanted to use Claudebot for, but I thought you got to vibe code something when you're trying a new agent. And I did that. So what I did is I sent Polly the Claudebot a voice note, and this is the requirements I gave it.

38:47Clawdbot:Okay, let's use voice from here on out. I want you to document our conversation in a Next.js web app that shows the back and forth of our full conversation from the very beginning today till the end in a UI. I want you to redact anything that is a secret key, a person's name or a specific place. And I want to toggle between two UI versions of this display. I want you to be able to show me a terminal style conversation back and forth, similar to a Claude code or you Claude bot C-L-A-W-D-B-O-T. or I want you to show me a telegram style text back and forth. The content should be in JSON. The same again redact names, emails, dates, etc.

39:42Clawdbot:Replace them with placeholders or redacted blocks and then generate the next JS app. I'm going to use this so I can share this conversation with others without sharing my information or having to do a screen recording. We are eventually going to deploy this to Vercel. Can you let me know when it's deployed to Vercel so I can look at it? So I sent it this message and it kicked off local development building a Next.js app. Now, when I got back to my laptop that Claude was running on, one of the things that I noticed is deploying it actually wasn't that simple. Claudebot didn't have a GitHub account.

40:23Claudebot, I didn't really want to add to my Vercel account. I didn't want to log into those things. It seemed like a big rigmarole. And so getting it to deploy without having to set up a bunch of accounts seemed not fun. So what I did instead, don't tell anybody, is I airdropped the repo to my own laptop here. I actually logged into CloudCode and made some edits. And to be honest, in terms of coding quality and just the back and forth, with the latency of CloudBot and the inability to sort to see what decisions it's making from a coding perspective. I didn't love Claude bot Telegram vibe coding is just too slow.

41:04The cycles aren't good enough. There are incremental enough. Clearly not like perfectly tuned for the coding use case. It's not like sending me a PR link, all those sorts of things. And so I just preferred working with it on my desktop and Claude code and deploying it through my normal system. So that's a little bit feedback there. One thing that I did think was really cool is when I was on the go and it said the app was ready, you know, I was in Target or whatever, and I wasn't at a place where I could run a local machine. It was pretty cool to say, hey, like, shoot me a screenshot of what it looks like.

41:41And it did. It shot me screenshots of what the app looked like directly in Telegram. So I do think there's some underappreciated aspects, really simple things. Email me that file. Share me a screenshot. That are really useful. to interface with a laptop or a desktop or a device at home. So I do think this is an underappreciated aspect of being able to chat with your computer. It can do things like send you files, take screenshots, open up browsers. That is pretty cool, especially since we don't store everything in the cloud. All my desktop screenshots are not in the cloud. Some of the PDFs that I download are not in the cloud.

42:18And so this was a really kind of fun use case for chatting with a remote developer. Now that being said, Devin sends me screenshots all the time. I don't think it's perfect for coding, but it's something to think about. So I want to end this workflow section with one workflow that I thought it did a particularly good job at. And it was good

42:37Clawdbot:for two reasons. One, the product interface was what I wanted. I got the full Claudebot bot experience. The second thing was the output was really good. So what did I ask it to do? Well, I asked Claudebot to go on Reddit and research what people would want from ChatPIRD. So I said, go on Reddit. I did this during VoiceNote. I said, go on Reddit, find what people want from ChatPIRD, find what they want from a product AI platform, and email me a report. And what did I love about the product experience? One of the killer features of Claudebot is the ability to message anywhere, anything, anyhow. I sent it to VoiceNote.

43:16Clawdbot:I could shoot it an email. I could text if that was faster, and it would reply in kind as text as voice, whatever. And it would also email me. So it felt very much like an employee that I was working with, hey, like send them a slack and they're like, yep, it's in your inbox. That sort of always on anywhere, anyhow, communication flow for the agent was really, really nice. The second thing I like from a product perspective is I've talked about this from a negative point of view, which is the latency is not great. It's just not super responsive and super fast and it's kind of broken sometimes. But if this is a research task that I don't really think should come back quickly, I don't mind waiting for ClaudeBot to do a good job.

43:59And it did. And it's very similar to an experience with an employee. If I give them sort of a research task or roadmap task, I don't expect it to be returned in 30 seconds, except they go out, do a bunch of research and come back to me. And so I wasn't as bothered by the latency here. And then the third thing is I thought the output was actually quite good. So I'll show you what it sent me, which is it sent me this chat parody Reddit research markdown document, emailed it to my inbox, and it listed out key insights from research Reddit, from researching Reddit. And what I thought was awesome is this is right, but it's presented in a really simple, punchy way that I can go action.

44:43This is exactly how I would want a PM or a research assistant on my team to come back with insights. And these are the things that we hear. So it was really accurate, you know, integration limitations, both on our side and customer sides hard. No one reads long PRDs. Let's make our PRDs shorter. You know, PRDs need to be living documents, all these things, a couple bullet points, a couple reference links to Reddit threads. And I have a full document that I can go build a roadmap off of. And in fact, that's exactly what I asked it to do. I said, go build a roadmap based on this, look at our current functionality and tell me what I should build next.

45:22This felt pretty magic. I'm probably going to steal some of these ideas. I'm going to circle around to that in a little bit in terms of what I think is next for ClaudeBot. But I do think there is going to be demand both from a consumer perspective and from an enterprise business perspective on a agent employee that feels like an agent employee. It has a computer. It has account access. It can do things. It does those things well. But I think there are some things we're going to have to figure out first before we let it loose. And again, you can see this year I asked it to do a roadmap. It totally just didn't do it.

45:57It forgot it, said let me check on the background agent that never replied. So again, we're hitting some, you know, sharp edges on the product experience. It's not perfect, but it is pretty interesting. So what have I showed you so far? One, I have told you a little bit at a high level what Claudebot is, although I haven't gone into all the detail about how it works. Not really a point of this episode. I've showed how to onboard with Claudebot, including how to connect Telegram to chat back and forth with it on text or voice. I've showed you how I give access to its own Gmail workspace account, as well as in one password.

46:34So it can interact with limited scope to my data. I've given you some warnings about what you should think about in terms of scope and access there. I've gone through a couple workflows, a couple admin workflows, which are simple calendar all the way to advanced calendar management. Did not do well because it doesn't have a good sense of time and space, but hopefully we'll figure that out. As software engineers overall, I asked it to contact partners and guests by email. It did not a great job there because it lost a sense of its own identity. I had it do vibe coding, which it did a fine job at, but is definitely not my favorite tool for AI engineering remotely and asynchronously.

47:13And then finally, I showed you my favorite use case was for it to do some complex research and analysis with tools with web and it did a really good job and came back with came back to me with something that I really like now one thing that I didn't go check that I'm going to go check next is did it teach itself all these skills was it really telling me the truth when it said it had rules a peek under the hood which I will probably do as a follow-up either on x or here on the podcast like how does this thing work behind the scenes. That was not the point of this episode. The point of this episode was to show how somebody would come with a blank idea, maybe a fresh Mac mini, install this thing from the command line and actually get it to do things.

47:54And I think I showed you it's good at some things, bad at others and scary across the board. So let's get to my final thoughts here, which are basically that. And I shared this on X, but the whole time I was doing Claudebot, the whole time I was using this, I thought two things one this is so scary this is a terrible idea nobody should be doing this it should not have access to all this stuff on my computer I should not be sharing these keys locally I should not let LLMs have access to Gmail OAuth even if it's a sandbox app I was like no no no no no no no SOS don't love it as I said this is the final boss of security training you should be very careful about what you give it access to.

48:39And one of the things that I'm most concerned about is probably you get the most power from CloudBot if you give it access to your actual inbox, to your actual calendar, to your actual documents, to your actual repositories, your actual GitHub. And I can imagine so many things going wrong with that. Just knowing how it's built, which is built in an awesome way. I think Pete's done an incredible job. I don't think there's any ill will or mal intent in how it's built. It's powerful. It self learns in installs skills, it asks for permission, it's pretty independent. All that stuff is great. Until you have full read, write access to your most personal information.

49:20And one of the things that I was thinking as I was, you know, preparing for the show is great, I just gave an autonomous AI agent access to where my kids basketball practices are like, is that something like, do we want to self docs to a AI crustacean? Probably not. So I think that's going to be one of the challenges of this product because the second feeling I had was, boy, oh boy, I want this thing. I want AI that I can text. I want AI that does not make it complicated to talk back and forth with voice. I want AI that when I say, hey, can you look at my CRM, doesn't say go to this web page and press this button and to enter this API key and do this and that.

49:59I just want it to happen automatically. I want all that. I just don't think this is it yet. Like this does not feel yet like the interface to get me there. And so I have this real tension between I think the product from product experience isn't quite there yet. It's not really for the non-technical. So it really is for tinkers and hackers. There's a lot of security stuff here that's super scary. And can I have it please? And so maybe this is an example of something that from a market category perspective definitely has product market fit there are gajillions of dollars to make here i just don't know if this open source yolo mode terminal tool is is the thing and in fact i'm gonna take you know this laptop soon and office space it um for those of you that are very young that means i'm gonna go like hit it with sledgehammer but like i'm gonna uninstall it i'm gonna remove those keys i'm gonna delete the telegram but like I don't like this this makes me nervous and I'm also going to go build one for myself and so I think there is why there has been such a zeitgeist around this product is it is actually really cool to be able to chat voice whatever a very smart self-sufficient agent and hackers see it and they're also you know more risk tolerance than the everyday person But that being said, like, husband, please don't connect your Gmail to this.

51:23Like, mom, absolutely not. Like, kids, stay away. Not safe for kids. Like, this is just, this is something that unless you have been through a security tabletop exercise and know what to know, I would just be really cautious about how permissive you are in terms of access. and that leads me to my final final question of this episode which is you know I think Claude will live in our hearts forever and in fact it's probably got a great feature in front of it I love how fast the team is going Claude bought but who's going to build this thing for real like who is actually going to build this thing for real who is going to build the consumer version of it who is going to build the enterprise version of it who is going to get it right and I think this is a complicated question I'm just going to pose some thoughts as we close out this episode So, you know, this should be Google or Microsoft's game to lose, maybe even Meta's on the consumer side.

52:14But this should be Google or Microsoft's game to lose. Like they have the data. They have your Gmail. They have your calendar. They have documents. They have the models. The models are exceptional. They just got to build, you know, they have devices, Android. destroyed they just have to build the product experience and have the sort of institutional fortitude and close your eyes legal team to allow some of this to happen because I think it's a really cool product built on top of the Google ecosystem I mean I think the same on the enterprise from a Microsoft perspective if Copilot did this this is pretty incredible that being said I don't know if those companies are going to have the velocity or the bravery to go as YOLO as Claude Bot did.

53:03So I don't know if we're going to get there with the big companies. On the flip side, you see Claude Bot open source, great for hackers, but super scary, giving like API key and OAuth access. Smaller companies, startups are going to see this and want to build this. And I think one of the things that I would warn startups, it's really hard to build on top of these data sources for real because Google doesn't want to give you read, write, go, do everything access to their data. Microsoft does not. You have to go through these compliance hoops and approvals and reviews. And so while I love the idea of a do everything, do anything bot, it's going to be complicated from a product builder perspective.

53:44It's going to be complicated from a large company perspective. Who gets the data? And then again, like, is Apple going to get in this game? This is just what everybody wants from Siri to do. Siri has all your apps, all your access. But again, it's a combination of product building skills and risk tolerance, I think, and willingness to experiment. Maybe Anthropic and OpenAI come in. Maybe we get our OpenAI OS and workspace tools, and maybe we get Claude Inbox, and we get some new versions of this. It'll just be really interesting to see how this shakes out. So in conclusion, what are my thoughts about ClaudeBot?

54:19It is scary. It is fun. It does some things really, really well. It is really interesting from a interface perspective, and it doesn't always work. And I'm not sure it's for, I was going to say everyone, but I'm not sure it's for anyone right now, except for people who are really willing to roll the dice with their AI bot. That being said, if you're willing to do that, the way this has been built, the way it self-discovers skills, the way it stores its memory, the way it gives itself access is really interesting and should inspire a lot of product builders thinking about AI products on what the interface of the future is.

54:59I think we are going to be seeing and hearing a lot more about agents like this. I am going to be giving you my honest takes about where they are now and where they're going to be in the future. and in the meantime, I'm going to go execute Polly the Claudebot. Thanks and I'll see you next time on How I AI.

55:19Clawdbot:Thanks so much for watching. If you enjoyed this show, please like and subscribe here on YouTube or even better, leave us a comment with your thoughts. You can also find this podcast on Apple Podcasts, Spotify, or your favorite podcast app. Please consider leaving us a rating and review, which will help others find the show. You can see all our episodes and learn more about the show at howiaipod.com. See you next time.