How I built an Apple Watch workout app using Cursor and Xcode (with zero mobile-app experience)

0:00Terry Lin:You used AI to make getting swole at the gym easier. I am so excited about this. Let's see how you built this. I started using the GPT mobile app as like speech to text. And I was like, well, if the model can understand when I'm talking to it like it is right now, why can't a workout app do this and then tag the data for me? Make it like a structured data set with analytics. Let's talk about how you built this thing. Because I wouldn't even know where to start in terms of the mobile and watch side of things. I was like, what if I have a Python script that takes these files and it takes a GPT-4-0 and it just suits it to me in an Excel.

0:31Terry Lin:Two months later, I now have an Apple Watch and an iPhone app. Now I did jumping jacks, 35 reps. He sends it to the phone. So whatever device you have, log your workouts pretty much with no work. It does simplify this experience of going to the gym, like the gym bros with their notebooks, where they're writing down their reps or in their notes app. This is awesome.

0:52Terry Lin:Welcome back to How I AI. I'm Claire Vaux, product leader and AI obsessive, here on a mission to help you build better with these new tools. Today we have Terry Lynn, product manager, vibe coder, and AI-powered Jimbro. He's going to show us how he made a mobile and watch app to track his workouts using cursor, Xcode, and index cards. Let's get to it. This episode is brought to you by Paragon, the fastest way to ship product integrations. Are integrations on your product roadmap? Whether you want to ingest your users' files from apps like Google Drive or OneDrive, sync data with their CRMs or automate tasks like searching Salesforce records, integrations are mission critical for products and SaaS.

1:36Terry Lin:But integrations take months to build, and existing solutions aren't good enough. Embedded iPaaS are great for automation but break under high volume, and unified APIs are limited by their endpoints and data access. With Paragon, developers can ship new integrations in days with purpose-built products to support any use case, from high-volume ingestion to real-time actions and automations. That's why engineering teams at u.com, Pipedrive, Cinch, and hundreds of other B2B SaaS and AI companies use Paragon, so they can focus their efforts on core product features, not integrations. Visit useparagon.com slash howiai to see how Paragon 2.0 can help you accelerate your products integration roadmap today and get$1 ,000 off.

2:31Terry Lin:Terry, I'm super excited to have you on the podcast. Welcome. Thank you. Thank you. Long time listener, first time caller. Well, speaking of first time, we have had a lot of web developers, including myself, on this podcast, but we actually haven't spoken to very many people building for mobile. And I know mobile apps have special challenges technically, you know, just building them. And then I'm super curious to see how folks like you are approaching using AI to build mobile apps. So before we get into the how, let's see the what. What did you build with AI? So I built a mobile fitness tracker called Copper's Corner.

3:11And essentially the problem I was trying to solve, maybe I put my PM head on for a sec, was I would go to the gym. I would try to be consistent. And then, you know, life gets busy. You start to slip one or two days and then you go back a week later.

3:22Terry Lin:You're like, oh, what was I doing? And so I tried these fitness apps where you kind of have to like create an account, set up your exercises, tell them what your gym has equipment wise. And I felt like doing a lot of homework. And even when you're working out, you have to like log what you did, you know, your rest timers, things like that. And then last year I started using the GPT mobile app as like speech to text. And I was like, well, if the model can understand when I'm talking to it, like it is right now, why can't like a workout app do this? And then kind of like tag the data for me and make it like a structured data set with analytics.

3:56And so that's kind of how I started down that path. And then I basically started with a little script. And then eventually, two months later, I now have like an Apple Watch and an iPhone app. So I'll run you through how it works, and then we can talk through how I built this.

4:09Terry Lin:Okay, so you used AI to make getting swole at the gym easier. I am so excited about this. As someone who definitely never, never skips a gym day. Okay, let's see how you built this. Okay, cool. I'll run you through how it works real quick just to get a – everyone has like a visual context here. So this is the login screen. Now, if you can see it, he's the hero of the image. Was this the good? Oh, Cooper. Cooper the dog. Yeah. And I don't know if you can see him, but he's sleeping right there, literally. Like on the side of my screen. Oh, hey, Cooper. Yeah. Okay. So the first thing is authentication, right?

4:46I'm using sign-in with Apple. There's kind of a UX decision there because I don't want you to create an email and have to, like, you know, enter a code. and so one of the first things you'll notice is when you log into the phone here it logs into the apple watch so there's some stuff i can talk about there but essentially you could basically just use voice and record your workout here and so i'll give this a shot i did dumbbell shrugs 35 pound dumbbells 10 reps and so what i built is you could record it from your phone or you could do it from the apple watcher because sometimes when you're at the gym you don't always have your phone too, right?

5:23So let's see if it got it right. There we go. See? And it has a transcript here. If I try it from the phone, right now I did jumping jacks, 35 reps. See, it sends it to the phone. So whatever device you have, it's kind of cool. It can log your work out pretty much with no work.

5:43Terry Lin:So for people that are not watching on screen, I says an app that goes across your mobile device and your smartwatch and you basically speak to it at the gym, say what exercise you did, what weight, what reps, and it just automatically populates a structured record of that exercise, including what time you did it. It shows a picture of the equipment you might have used or a picture of somebody doing jumping jacks or in this case, push up jumping jacks. And it does simplify this experience of going to the gym, you know, like the gym bros with their notebooks where they're writing down their reps or in their notes app.

6:24Terry Lin:And Apple, so you built this. It's multi-device. It's multimodal. It goes voice to text and text to structured output. But let's talk about how you built this thing, because I wouldn't even know where to start in terms of the mobile and watch side of things. Yeah. And there's one more thing to show you real quick. I'll flash it on is we were talking about how to be consistent. So there's actually like a history view here. So you can look at your seven day, 30 day, 90 day analytics to see how consistent you are. And you can see in like the last 30 days, I've been going to the gym 21 days. Right.

6:58So now I know when I've been slacking, when I haven't been. And also I can see like my top exercises. You can see I do a lot of jumping jacks here. I do leg presses and then you click into the leg press. You have like a whole history of what you've done. So now you have like a scatter plot of where my weight's going, like a history of if I'm actually progressing or not too. So that's kind of it. I could walk through how we built that, but any questions there?

7:21Terry Lin:No, let's go through how you built it. Okay. So one thing with iOS apps is that you usually have to use Xcode, which is Apple's IDE to actually build the code. So I do something called dual wielding here. So this is what Xcode looks like. And what I do is usually I do a side-by-side with Cursor here. So the way I link this is I make both of them point to the same folder on your computer. And then Cursor will do the coding. And then I will do the building and the debugging in Xcode. The reason is because sometimes on the phone, when you get errors, like compile errors, build errors. Xcode is really where you have to go to get that because Cursor, it's not like a web browser where you could just kind of look at the tools and then start looking at the console.

8:08And so there's a little bit of a kind of workaround you have to do. And I've seen some people online have these hot reload apps, but there's a kind of solution I've found that works best for me. And also if you have to build to the Apple Watch, you kind of have to do it separately. So that's kind of why I have this split workflow for now.

8:24Terry Lin:Got it. So we want cursor to somehow build a native integration here. So you have a little bit more cross-pollination if you could. Yeah. Yeah. And it's not like web apps where you just run a local host and then you kind of can see it in your browser. You have to kind of always build it on your phone. And right now you're seeing this in the iOS simulator. I would say even when you're testing mobile apps, like swiping around with my mouse here feels very different than when I'm on my phone testing in here. And it's even different when I'm in the gym actually like running around and like trying to record stuff.

8:54and sometimes audio is not great too. So I think when it comes to like mobile apps and you're testing this, you just have to get as close to the actual experience. Otherwise, you're kind of just testing in a box and not really getting the true kind of user experience there.

9:06Terry Lin:So how did you get this started from zero? Kind of what was your setup in terms of defining the product, getting cursor set up? How did you work through that workflow and then doing the build and testing and Xcode it on your phone? Yeah, so like we say in product, You want to like think big, start small, right? So the V1 of this was actually just using Apple voicemail, like on the Apple watch. I would record that and then you would copy it to my computer. And then around February this year, I started getting into Vibe coding and I was like, well, what if I have a Python script that takes these files and it takes a GPT-4L and it just suits it to me in an Excel.

9:46And so what that looks like here is kind of this spreadsheet here. This was like the very V1 of it, where you see the transcription. It attempts to tag it with different muscle groups and like rep sets. But the problem here is that this is not structured data, right? It's just whatever the model thinks is the exercise. And so then I was like, all right, well, then what is the solution then? Is then you got to put it into a database where it's actually structured data. You have like the foreign keys. You can actually manipulate it better. It becomes more consistent. So that's kind of where it went from this to a backend API that I built in cursor.

10:19Terry Lin:I want to pause here because I actually think this for folks that are listening is a pretty cool hack, which is using voice notes on your on your watch to just narrate your workout at the gym and then download that text, you know, put it in even chat GPT or some sort of like no code little flow and populate it in a spreadsheet. That's just such a simple and easy way to start. And then what you're saying is you're a PM and an engineer and you want foreign keys and a database. So you decided let's make this a full app. Yeah. And it's also if you look at the spreadsheet, how do you make sense of this, right?

10:56Over time, you're not really, you don't have these analytics that you see here on the right side of the phone. And so I was like, well, how do you then do that? You can ideate with GPT. I think when you work with models, it could either work with you or work for you. This is very much in that working with you side or just ideating on features and how you can potentially make like an MVP into like a low hacky tool that gives them higher iteration.

11:19Terry Lin:So, OK, so you decided to build this app. How are you actually building? What is your step by step workflow in Cursor? Sure. So I have kind of a three or four step process in Cursor. So first step is creating a PRD. Second is having a model review it. And then third step is executing. And so the way I have to set that up is kind of three rules here. There you see PRD create, PRD review, PRD execute. And so if we take a file here, as an example here, I have an existing ticket. What I'll do is I'll take an issue, I'll add it to cursor here, and then I'll have it run through PRD create. And what this rule does is essentially breaks it out into what you need to implementation.

12:09So any reference diagrams to actually do this task, what are the goals we're trying to do. One thing I do is I use Gherkin user stories to actually describe the scenario. So the format is like given something happens when the user does this, then do this action. And so there's also some investigation here that happens. So if I don't know how to actually do something where the model doesn't know what files to use, what databases to touch, that's kind of a checkpoint here. And then this is kind of like the V1 of the PRD where it's not fully fleshed out, but at least it's the structure of it. And then one thing I realized is sometimes early on when I was live coding, this document would not be enough.

12:50And that's kind of why I created a PRD review rule where it basically does like a check on that PRD. So one question I ask is, if another model were to take this plan, how would you rate this out of 10 if it had no context and it had to execute on this? And so you're basically sanity checking your PRD to see if it actually has any gaps that a model could trip up on down the line.

13:13Terry Lin:You've seen the doom and gloom headlines. AI is coming for your job. But the reality is a little bit brighter. In Miro's latest survey, 76 % of people say AI can boost their work. It's just that 54 % still don't know when to use it. As a product leader and a solo founder, I live or die by how fast I can turn fuzzy ideas into crisp value propositions, roadmaps, and launch plans. That's why I love Miro's innovation workspace. It drops an AI co-pilot inside the canvas, so stickies, screenshots, and brainstorm bullets can become usable diagrams, product briefs, and even prototypes in minutes. Your team can dive in, riff, and iterate.

13:58Terry Lin:And because the board feels like a digital playground, everyone has fun while you cut cycle time by a third. Miro lets humans and AI play to their strengths so that great ideas ship faster and happier. Help your teams get great done with Miro. Check out miro.com to find out how. That's M-I-R-O dot com. So how did you build these cursor rules? And how did you know what to build and how to put stuff in this template? Honestly, a lot of blood, sweat, and tears. I think when I first started Vibe coding in February or March, no one was really talking about memory banks or rules yet. And so the problem I was running into was every time I modeled, I got to do more and more complex tasks.

14:42It would at some point trip up over itself. It would make up stuff. It would make up file directories. You would get endpoints wrong. And so I realized after repeating it, telling it what to do time and time again, I started noticing these patterns and like how to give it context. And I was like, well, why am I repeating this over and over again? I should just make these into the rules. And so that's kind of how I ended up with these processes. And you notice I also broke them onto like three PRD rules. The reason is because originally my rules were super verbose, maybe 800 lines long. And I realized I was just running into a lot of context windows with the same rules.

15:16So you'll see like they're now no more than 200 lines long. And so I actually like I'm much more cognizant of how many tokens I'm using with these roles. And so over time, I've kind of gotten more efficient with that as I get more experience with Cursor.

15:31Terry Lin:And the second part of this, which I think is so interesting and mirrors how things work inside product organizations, is you write a PRD and then somebody looks at it. Somebody reviews it and says, is this good? And that person could be, you know, a PM lead. It could be a peer PM or it could be an engineer. And it seems like you're taking this from an engineering point of view and saying, like, as a model reviewing this zero context, what do you think? And I like this idea of rating zero to 10. Do you feel like the ratings are generally pretty accurate? Yeah. And it's more of a barometer for me and how well I prepared it.

16:04If it's like a 7 out of 10, I'll then ask, well, what are the three points? Why did you dock it? And it'll give me some reasons on why those gaps are. up that is this an edge case no we could ignore that and it'll kind of you want to get it to at least like a nine out of ten before you do it and the reason is then when you have it run through the plan it could probably one shot it pretty quickly and so i'm doing that here on the side you see there's a seven out of ten here and so it gives me like a straw man and steel man a bit and so you see the gaps identified i can then be like all right you know inconsistent api like are these actually things we need and then i'll kind of then kind of iterate this until it's a 10 out 10.

16:41And then that's when I then run the execute rule, which I think you've seen is like the checkbox rule. You have it in chunks. Something I do extra is I have a git commit before and after each phase and ask it to pause. So I don't just have it one shot everything, but this is when I know I can let it run, go get a coffee and then come back and just do some QA.

17:01Terry Lin:Got it. So your step is write it as a PRD. Gosh, you're poor AI PM. You write the PRD, then someone else tells it if it's good or bad. Then you iterate. Then the poor PM has to make epics and tasks. The AI PM has to make the checklist. And then you're giving very explicit directions on check-in at every single point. And then you say, here you go. Here's your PRD. Here's your checklist. Let it run. And then are you coming back and checking that code, doing the build in Xcode, and then verifying it all works and shipping it? Yeah. So here's an example of one where on the Apple watch, when I showed you, I hit record and I stopped it.

17:46It was saying send to iPhone, but it wasn't staying on long enough. So if you're recording this, you may miss it. And then you may wonder, hey, did it actually even go to the phone? And so this was adding like a little timer. So you can see that if you're kind of walking around the gym and there's like a feedback that, hey, this is, we got this. You don't have to worry about it. You don't have to worry about it failing. And so this is what the PRD looks like. I logged it originally in a tool called linear. And so I pulled that in here and then I ran it through PRD create. And so we created this doc.

18:13And so I'm very explicit where I even tell it what files you want to touch just because I don't want it to search through my code base, waste tokens. It's like, hey, here's what you need to do. Here are the files. Here are the endpoints. Just go run with it. Yeah, so here's some of the example of the phase checklist. You see phase one, it does the checklist and every phase it kind of pauses, right? There's like a safety checklist here, right no placeholder codes we want you to make stuff that could break things there's some error handling and then it kind of runs through this yeah and i want to pause for folks that are not watching on that checklist if you could go back to it it's really interesting every phase at the

18:50Terry Lin:end of the phase says you have to verify the paths make sure you're using real data no placeholder that you're handling errors well and so it's really interesting that you really again are following this pretty classic development pattern of like product manager, you know, defines requirements, breaks down the tasks, engineer executes, you do a QA phase, and then you move on to the next task. So you sort of structured your rules and your process to map to that. And then the only difference is you're using an AI PM and AI engineer and AI EM to sort of run QA to run through all this. Yeah. And I think 80 % of the time I'm working with the model, it's this last 20 % where the checklist is just burning through tasks for me too.

19:37So I think the biggest switch is probably just learning to work with the model, I think is where if you get good at that. I think after this stuff, it's just kind of table stakes. It's just executing.

19:47Terry Lin:And I'd love to understand, you've mentioned conserving tokens for a while, and I have to understand your motivation because I am just like, eat all the tokens you want. I just do not care. Tokens, everyone. You get a token, you get a token. Is this a cost sensitivity thing? Is it a performance of the tasks where, you know, too much context in the cursor context window actually degrades performance? What is your motivation for token conservation? probably my sanity if i could put a thought to that so the reason i say sanity is because llms are really good at just generating a bunch of code it could be very robust and so sometimes when you have very large files i noticed cursor when they go through your code based files they read in like 200 line chunks at a time and what i realized over time was that i had i had these files that were like a thousand a thousand fifteen hundred lines long and it was spending a lot of time just going through that.

20:46And then as it was going through tasks, it would just start getting tripped up a lot. And then that's when I kind of stumbled upon this concept of like, I call it vibe refactoring, where you basically have to take vibe coding. You have to like reorganize it into something that's cleaner. And then that way, I guess it's like good engineering principles. They do that too, right? You refactor code, you keep it clean. So that's something I do with another rule here where I kind of have the PRD rule, but it's not meant to create features, it's meant to refactor existing code bases. And so I essentially give it the same, you know, context docs.

21:17I give it the API endpoints and I give it different guidance on, hey, what do we need factoring? Why should we do this? How do we make sure it doesn't break? Like how will we QA this after it's done? And it kind of runs through the same process of where you're doing that PRD planning, but your goal is very different. It's more of a cleaning your apartment, making sure everything works fine. Okay.