Understanding what's possible, doable & scalable

29:29Yeah, that's so encouraging. Certainly, like I say, there's an element of workforce impact of AI that needs to be considered on the negative side. I guess if you want to consider it that way, but also there's a positive side and opportunities that are unlocked for sure. I have to say that tool will never exist if no one builds it, Daniel. And so this is just a challenge. Like I'm doing this personally and we can unpack how, but like, if you've been given gifts that overlap with like an expertise here, it's incumbent on each of us to think through, like, how can I make the world just a smidge better?

30:08Right. Because I didn't just ignore all the need that overlaps with my expertise. And it's not just about writing a check at Christmas time. Yeah, hunt that out. It's about, yeah, creating that outpost of goodness and beauty in what we create and not just like in your case, in your background with art and actual sort of fine art, but also in the technologies and the products and the tools that we're building. I love that. I'd love to maybe talk, maybe just highlight a couple of these solution archetypes because I think this is also kind of helpful maybe for those in our audience. Maybe they're on the business side and they would connect with them, but also maybe they're on the technical side and they're just struggling to connect certain ways in which they can communicate the potential of this technology to stakeholders.

30:58So I love the one that you have here of generating new ideas with AI. How do you think about that as a solution archetype? Well, the obvious way to do this that anyone would just think of on their own is you could go to something like ChatGPT and say, go a step beyond what a normal ho-hum user would do, which is to use it like Google, but to ask it for help with inspiration. And so anyone could just lean forward in their chair and open chat to you and do that right now. But the use case that we highlight in our solution archetypes documentation, there are two or three under that specific archetype.

31:45But the one that I'm most proud of and we'll talk about today is the first app I ever built. And this app was a persona named Andy. And so Andy is a roundtable discussion, like focus group style facilitator. So there's essentially, it's really just a system command plus some tuned hyperparameters that help Andy do his job well. But Andy's supported in Python with the ability to synthesize or spawn new personas. And so what Andy will do is listen to the transcript of a meeting. What I would love if there was like a low enough latency audio, you know, like the files that OpenAI is kind of working on right now.

32:23I'd love to be able to get Andy like in the whole group participating in real time. But for right now, the way it works is you can kind of give them a chunk of transcript. And what Andy will do is kind of look at the problem you're trying to solve and spin up a focus group. So Andy creates, I think it was six or eight, I think maybe eight additional personas. So now all that means is in the Python code, there's like eight new things that Andy just created system messages and some hyperparameter tuning. so that now in the code, the tool has the functionality to summon any of these additional personas to weigh in on the conversation.

33:02And so Andy, as a facilitator, can pick which one has a turn to talk. And so then Andy opens the group, and you can use the world knowledge for this bit, and I did, and he just understood how to run a focus group, which actually is cool, because I didn't know how to do that when I had the idea to do it. And so Andy will say, you know, like, okay, today we're going to talk about new product ideas for X industry. We happen to do this for the publishing company where I worked. So I was still working for them when I made this tool. And Andy could, I think it was like a fireman, a nurse, a web developer, a doctor, you know, around the table.

33:36And the problem this company had was their new product development team was struggling to come up with new products. And pretty much at the end of each meeting, it would just be the same product. And this was a problem. And so Andy understood my job is to think of new ideas, but then kick it to the group for discussion. And it was so fascinating to watch them talk about it. In this instance, the very first time we ever used it, someone in the group came up with an idea for something that was very close to the products that they had been making, but a little different. And then the nurse piped up and said, that's great, but that form factor would not work at all for my job.

34:17And if I wanted to bring that to work and benefit from it, it would need to be smaller so they could fit in my pocket or purse. And so then someone else built on it and said, you're describing a deck of cards and this could work because, and then the web developer kicked in behind and said, that's a great idea. If there was a QR code on the cards, you could then link it to an account that they have here. Let me write that right now. Of course, the code didn't fully work, but it was cool. You could see it try to write the code. And this, we were just letting it run, right? They loved the idea. It was the freshest, best idea they'd had in the seven-year existence of the new product development team.

34:55It was a brand new product idea. And it was the first app I'd written, and it was the first time we ran it. And that, for me, was a major moment. And that was on GPT-3. Yeah, and there's an element to this solutioning that you've been doing that makes me think about how quickly you're able to get through to a proof of concept or demonstrating the value that you had in mind with a certain AI use case. And I'm wondering, as you're doing this solution, I'm guessing that there's a balance or maybe potential objections here. You know, in this scenario, I could see how a company might object to putting in so much product information or IP information into one of these systems for privacy reasons.

35:43Or maybe they're thinking, oh, what you're describing is going to take so many, so many LLM calls. It's going to be slow or maybe costly, which maybe are relevant concerns. But also, you kind of need to know what is possible before you get to some of that optimization and also try to get to an end-to-end solution. So how do you think about, as you're solutioning, balancing this desire to move quickly and get to a solution and end-to-end thing versus privacy, security types of concerns and balancing those things that maybe are optimizations or scale issues or environment issues with the ability to get to an end-to-end solution?

36:28Yeah. Sitting across the table from maybe the best person in the world at this, the best I've ever met, you know, when it comes to like thoughtfully considering, you know, security and the risks associated with it, I will say like this is not something to ignore like ever. And we've learned to start with security when it comes to like forming relationships and assurances with a new client. And we need to be good at that, right?

36:56But you got to start somewhere. And sometimes what we do is we start with synthetic data. So it's kind of like, hey, pharma company, we're going to invent a synthetic drug and we're going to make all the synthetic assets that a company would have. and what's neat about that is these tools do that for you. So, I mean, if you have a persona, remember I said like the new paradigm is having a persona fully aligned. Well, that persona that I've taken the time to get up to speed on the project can just spit out all this stuff I need to prove to them that what they want to do is possible, right? There are so many trust me's floating around in this space, right?

37:35Like it can do it. We'll get you there in 36 months. You only need to pay us, you know, X million dollars to build out your and it's BS. Maybe that works for a big co who has a metabolism that is the same as a, you know, a 20 ton whale. But like, you know, for mid co, mid small that has a normal human metabolism and would like to see something before Christmas. Right. Like, you know, there are ways to get to POC proof of concept in sometimes hours. I have had a persuasive proof of concept scripts written before the end of meetings where the client is talking about a solution they want. And how do you do that?

38:23Because I probably went into the meeting with an aligned persona on the goals. I hear a few little things in the meeting and I have a bot that I've created called Persona Craft that I can just throw in a few things and whip, you know, bang, bang, bang. Oh, OK. Throw that in Python. Is this what you're talking about? Like, holy smokes. So, yeah.

38:51What's up, friends? I've got something exciting to share with you today. A sleep technology that's pushing the boundaries of what's possible in our bedrooms. Let me introduce you to 8 Sleep and their cutting edge Pod 4 Ultra. I haven't gotten mine yet, but it's on its way. I'm literally counting the days. So what exactly is the Pod 4 Ultra? Imagine a high-tech mattress cover that you can easily add to any bed. But this isn't just any cover. It is packed with sensors, heating, and cooling elements. and it's all controlled by sophisticated AI algorithms. It's like having a sleep lab, a smart thermostat, and a personal sleep coach all rolled into a single device.

39:32It uses a network of sensors to track a wide array of biometrics while you sleep, sleep stages, heart rate variability, respiratory rate, temperature, and more. It uses precision temperature control to regulate your body's sleep cycles. It can cool you down to a chilly 55 degrees Fahrenheit or warm you up to a good, nice solid temperature of 110 Fahrenheit. And it does this separately for each side of the bed. This means you and your partner can have your own ideal sleep temperatures. But the really cool part is that the pod uses AI and it uses machine learning to learn your sleep patterns over time.

40:12and it uses this data to automatically adjust the temperature of your bed throughout the night according to your body's preferences. Instead of just giving you some stats, it understands them and it does something about it. Your bed literally gets smarter as you sleep over time. And all this functionality is accessible through a comprehensive mobile app. You get sleep analytics, trends over time, and you even get a daily sleep fitness score. Now, I don't have mine yet. It is on its way. thanks to our friends over at 8sleep. And I'm literally counting the days I get it because I love this stuff.

40:46But if you're ready to take your sleep and your recovery to the next level, head over to 8sleep.com slash practical AI and use our code practical AI to get 350 bucks off your very own pod four ultra. And you can try it free for 30 days. I don't think you want to send it back, but you can if you want to. They're currently shipping to the US, Canada, United Kingdom, Europe, and Australia. Again, 8sleep.com slash practical AI.

41:33yeah it's certainly there's an element of this where you are acting as an architect and a solution developer getting to that proof of concept showing value. But I know one of the other things that you're particularly passionate about and have been exploring is actually finding kind of individuals in organizations that are champions or could be champions of AI. Maybe they're not from a kind of technical programming background. Maybe they are, but kind of teaching them how to utilize these tools effectively in a sort of crash course kind of way. So how do you think about that? And what are some of the things that you're taking people through to help them understand how to use these tools in an effective way?

42:21I just want to preface this bit by saying I am trying to become a master at what I'm about to describe, and I'm not there yet. This has been the hardest thing for me to navigate so far. Here's the reality. Companies need more than code. Like what they really, whether or not they realize it, when they tell me we want to become AI enabled, I think some part of it is they want their people to know and understand this stuff. And I think if you just kind of trust it to luck, that people who have a really demanding job are going to develop an accurate sense for what's going on under the hood, you're going to be disappointed.

43:05And so we have developed, the company that bought my small business happened to be a change management, like an instructional design company that they have helped BigCo for decades navigate massive change. And so it's just kind of lucky. I mean, that's actually part of why I liked the offer. But I have worked with their experts to develop a pretty well-defined approach with mixed results. Where I want to get better is I want to get better at identifying the people who are really going to become the super user rock stars. So we've developed this tool called a fit checker that like essentially is a quiz.

43:48It's like some sliders. They slide and it helps us kind of score. I don't even really know if that's going to work great or not. But what we are doing is letting them get a peek under the hood at what's going on with these tools? Because we want to demystify it, right? But they think it's magic or BS, you know, or both. Can I just kind of run you through the beginning of like what it would be like in one of our early sessions with them? So you can see, okay. So, and feel free to try this in your organization. If you're kind of trying to align with someone on like what's actually happening with these tools.

44:24And so what we'll do in the very beginning is I'll stand in front of the room or one of our other, you know, coaches will stand in front of the room and say, hey, everybody, I'm going to say something, and I'm going to say a few words, and what I want you to do, and you can maybe even do this in your car, or if you're listening to this at home, just blurt out what pops in your mind, right? And so I'm going to say some words, and then you're going to blurt out what pops in your mind. Are you ready? Ready. Here we go. Peanut butter and? Jelly. Jelly. It's jelly every time. Yeah. It's always jelly.

44:54And the whole room says jelly. No one says anything except jelly, right? And then what we do is we explain to them that they essentially provided a completion to our prompt. And jelly happens to be a token. And peanut butter, peanut is, if it's a capital P, is three tokens. Butter is a token and is a token. That's the GPT-4 tokenizer. And so then we can open up a tokenizer and we can show them those individual tokens. And we can click on the token IDs and we can show them that those are just numbers. and that there's this language model that is trained to spot weird patterns to what tends to come next as a number when you give it a string of numbers.

45:35Those were converted to bits or entire words, and then it just spit back jelly. And so when you understand that, and then we show them, we tell them like, and look, if it spits a sentence back, it's just kind of a token, token, getting longer, getting longer, getting longer, you know, and it's starting from the beginning. and they get a sense for the like, oh, it's just mimicking language. It's mimicking what a next word that a human might say when they heard a thing. When you understand that it's working that way, it helps you use it better. Okay. So then while I'm talking about all that, I might start talking about blues and rock and roll.

46:19I might mention Memphis, Cadillacs. Maybe I'll say something about Graceland. And I'm talking about all this. And then I'll say, let's try it again. Peanut butter and, you know, just like jelly, jelly. Someone yells banana. And behind me on the board, I take a thing off and we've written banana on the board, right? And then we just have this weird moment because some people in the room are young and they don't even hardly know who Elvis is, right? And I don't even know if you would have gotten that, Daniel. Yeah, yeah. I definitely would have gotten the Elvis, Graceland, et cetera, but I would have been somewhere there.

46:55Yeah. And so I think it's that Elvis like peanut butter and banana sandwiches or something. And so now we can just unpack the concept of context because by inserting some new ideas, we sort of shifted the probability of that index of potential tokens in the order that they would be. And we've shifted it enough so that a banana could potentially show up as worth picking. And someone did. and that just sort of demonstrates well most of you still said jelly but someone said banana i'm just telling you it has never not worked like it always works and so then we start talking about okay so these are tokens they're parts of words they're numbers they go together it can predict the next one to mimic human communication or thought and then there's only a certain amount of these that can fit in its brain or its memory its context window and we'll show them on a board We just put little hashes for like, this is a token, this is a token.

47:57Well, what happens when we get to the end? Because, Daniel, when I started working with these tools, we only had 2 ,000 to work with. And now it feels infinite, but not really. But because you learn early, right? Because you learn early, you learn to work within the limitations of the tool, and you learn elegant workflow then, right? Because we appreciate, even though it feels big, it's still scarce because they've got to pay for what we submit. And so we show them, these are how the tokens stack up. And then we show them, like, when you start getting toward the end, if we fill up this 2 ,000, what's it going to do?

48:27It's chopping off the ones in the beginning. And then we ask them, like, have you ever had an experience with chat GPT where it feels like I've already told you that? Oh, yeah, you see some, oh, oh, because it didn't forget it. It selectively, it deleted it, you know. And so we just run them through, you know, that's like the first 10 minutes. And we just build on that and build on that and build on that so that they build good AI usage habits and they can interact with the tool in a way that leverages the strength of the tool and with an awareness of what can go wrong. Yeah, I love this approach.

49:05And in my own workshops, when I start going through some of this of how a generation actually happens, how text is generated, I often see that a number of really important and interesting questions kind of naturally pop out out of that. Like, hey, if this is just producing these tokens in this way, how could it produce anything meaningful? And how does it seem so coherent? Like there's an intent behind it. Why would it produce anything actually valuable and connected to the real world? So, yeah, I love this. So because we get all those same questions, we have identified what we call layer one, layer two, and layer three knowledge.

49:48We only teach layer one knowledge in our boot camp. We will address layer two and layer three questions individually, but not as a group. Because the layer one knowledge is the strict information set that is required to use the tool well. And so we would never talk about prompt injection attack in the group setting. Because what we don't want to do is scare off the non-technical person. And in fact, we target the non-technical people. This is their moment. Like, you finally programmed computers with plain language, you know? Yeah, really good points. And on the data side, I think one of the interesting things is, like you say, not going into some of those details in that context, but definitely getting to a point where people can realize how data starts to become integrated with these AI systems in a meaningful way.

50:41An example I often give is, I go into a chat interface, I say, summarize this email for me. And of course, you haven't pasted in an email. The model doesn't know how to summarize an email that it's not given, but everyone knows what to do. They could paste in the email that they want summarized and ignoring the kind of potentially policy-related things around that. It immediately makes sense that people can bring the right data to the table, inject it into these existing models without having to do complicated fine-tuning and actually produce value. Can I tell you our five-step process to effective reasoning?

51:21So we have a slide on this and we teach it. I think I hope I can remember all five because I just told you there's five. What we teach when it comes to like reasoning over data or just processing information, it could really be almost any project. We teach to do it in this order. So we say what we like to do is teach them to start with end goal alignment. And so because we know that, I think it's known that these models tend to pay a little more attention to what's going on early and late in your context, we want to get the real end goal articulated very early. So a neat trick I have for end goal, and so this actually can be fun.

52:04So this is a way you can work from your hot tub. Maybe you should open the show with that clip. So this is the way you can work from your hot tub. I like to put in my AirPods and hit record on my phone and just talk about what I need to get done on this project. Just talk, talk, talk, talk, talk. Get it out. This way I don't have to sit in front of my computer at my desk. I might go on a walk or on a treadmill and just talk through. This is the who, the what, the when, the where, the why, right? And get it all out there. Okay. Now, what we do is we transcribe that. I use Mac Whisper. So I'm doing that locally.

52:42So it doesn't matter if this is something for a top secret project. I can transcribe that locally. And then I can inject that into essentially my first prompt. And it might be big. I've had those go on for three hours or as quick as 10 seconds. And what we tend to do is try to rush to work. But there's a bunch of steps before we start working. So step one is establishing the end goal. Step two would be to validate the end goal. So now we can go back to that same model and say, tell me what you think we're going to do. Like, just lay it all out. Give me bullet points, you know. And so it will go through and say, I'm expecting you to do this and you to do this.

53:27And then I'm going to do this and we're going to do this. And what the client wants is this. And so we're probably going to do that. And you can look at it and go, actually, this or that is wrong. now what i like to do instead of playing ai whack-a-mole at this point right so anyway ai nerd knows exactly what i'm talking about is i use this next these next few bits of conversation to identify where it misunderstands the end goal if it if it does at all and then what i do is i go back to my source material and i modify my source material to clarify and so what we do is we get rid of all those subsequent prompts for validation validation is for me it's not anything that needs to linger in the context.

54:07So I delete all that. So step two was validation of end goal alignment. Step three is we curate and ingest our assets, our, like you said, the email. Remember I said curate, and that is big. You don't always have to do it, but it's helpful. Remember I said I learned on 2000 tokens? This is why I make a habit of curating my content before I put it in there. What's the least? Now, there's a balancing act here. There's a trade-off, cost-benefit. You don't want to overdo this step, but it is helpful to trim the fat out of transcripts. You know, if you've got a transcript saved as VTTs and you look at and you open that up in Notepad, 80 % of the tokens are time code.

54:50So, you know, maybe we get a DocX that just has speaker names. And, you know, so we're just mindful of like, what are we throwing in next that isn't going to distract the model that's relevant? Okay, let's pretend like we've done all that work. What do we do now? Do we start working? No, we validate again. So we go back and we say, what do you understand that I just gave you? And how do you understand that it relates to our end goal? And we'll talk about it. Usually this part, it gets right if we've done a good job of end goal alignment and curation. But sometimes you'll spot, oh, it can't see the infographics.

55:26I assumed by attaching these PDFs that, okay, what color is the biggest slice on page 14 of the pie chart? That's what a validation thing might look like over your data, right? Okay, you can see that pie chart. Okay, you see the orange. Okay, you know what that is. How does that relate to our project? Okay, you understand. Okay, you validate all that. Now get rid of that because that doesn't need to persist, but I need to be comfortable. This model understands the goal. So it's aligned. Now it's finally time to start work. We've done all this work. And boy, can you burn through a project when you've done those steps?

56:02And this is the fun part, really, where we just start, okay. You can almost just say, okay, go at this point. And it starts to do the work. And then I think the fifth step, I haven't been numbering them on purpose. The fifth step is just letting it synthesize assets. And this can not only be like, if you're using ChatGPT, you can have it create spreadsheets or whatever, but you can also, if it understands, hey, I want to use UDO to make a song about that, or I want to use Ideogram 2 to create slides for that, or I want to use, and you can just have it start to create prompts for other things and go off and do it, and it's really fun.

56:39So we've done all that, and we've got that context persistent. The client sends us an email and complains about whatever. What do we do? Copy, paste, boom, done, right? It's just so little work. Yeah, that's super encouraging. I love this example because, you know, oftentimes you might be, you know, I might be in a meeting and you have to respond very quickly. But a lot of times it's those slow times where you're able to think about something for me, like on a walk around the block or, you know, on a, on a hike or, or something like that, where I'm really, you know, present and I'm maybe more calm and thinking through things in a, in a slower fashion.

57:18But I also want to capture that and I want to summarize it. I want to bring it into conversations in the midst of my workday. So I love that example. And I think it also illustrates how you've integrated this technology. You're practicing what you preach to some degree, kind of integrating this technology into your daily rhythms. And that brings me to maybe a last question here as we're starting to wrap up is as you look to the future and you're thinking about the new solutions that you're creating for people, the things that are on your mind, the things that you're architecting, what's most exciting for you now?

57:55What are you excited to explore? What are you excited to kind of prototype with the partners and the customers that you're working with? What excites you looking forward? Well, I will stand by what I said initially, that if we never got another innovation, these tools are good enough as they are to justify going down on the timeline of humanity as a massive technological revolution. And the tools that I wish for that are already sort of like they're almost done baking, right? The tools that I wish for are really low latency, in and out, audio, video, image agents. And so, you know, I've built so many tools that interact with people.

58:38One class is called like mind mappers, where we're essentially trying to convert tacit knowledge to documented process, right? And so like, you've got the single point of failure person in your organization, how do you map out like their subject matter expertise? So what would be awesome is if there was a low latency conversant model. So like what OpenAI kind of has, but it's walled off right now, like an API where I could give a person an option between an email chain, text messages, a phone call. And right now I can't do the phone call. And that bothers me. And then I think too, just as people start to see the value of the tools that maybe they're a little less weirded out and fearful of interacting with them.

59:28And so of course it makes sense that people feel nervous that they're gonna lose their job. The reality is they are gonna lose their job as it exists today, if there's knowledge work, they should, because that's like continuing to want to farm land with an ox and whatever the thing behind a plow, I guess, like, you know, like that's wishing for keeping that, you know, when the tractor dealership just showed up and like, no, we, we're not going to want to go back to like, you know, the days of like trudging through tedious, you know, knowledge work that could be automated. But it is going to be a process that requires empathy on the part of us, the AI engineers, the architects, the developers, the, you know, the leaders, and to appreciate like, this is a scary thing.

1:00:19And so another thing I wish for is just to get past this adolescent phase to where the tools feel less scary. And then I got to go back to like, I am very excited to see what it means for learners who maybe their teachers don't have time to go on every curiosity journey with them or explain, you know, every little lesson. And so that's really big. I could give you such a long list, so I'll just stop there. I am very excited for how this technology develops, but I will say the good guys have to have a hand on the wheel here, Daniel. And if you leave it up to BigCo, we won't have the stuff that really makes life better for a wide range of people.

1:01:04Yeah. Well, I think that's a great note to end on. Thank you. Thank you, Mike, so much for joining. And I would definitely encourage you to heed Mike's call, get your hands dirty, think about those solutions that are going to create goodness and create new paradigms of the way that we work. So yeah, thank you so much, Mike for joining. Yeah. Thank you.

1:01:54dot fm slash community thanks again to our partners at fly.io to our beat freaking residents breakmaster cylinder and to you for listening we appreciate you spending time with us that's all for now we'll talk to you again next time