#236: AI Answers - No Time for AI, AI Budgets, Vendor Terms & Data Risk, AI Disclosure & Vanishing Entry Level Roles

0:00Cathy McPhillips: Where does management come from? What do managers do in the future if we don't train entry-level employees? So if we stop hiring as many entry-level staff, then we decimate the future managers and leaders of the company.

0:14Paul Roetzer: Welcome to AI Answers, a special Q &A series from the Artificial Intelligence Show.

0:19Cathy McPhillips: I'm Paul Roetzer, founder and CEO of SmarterX and Marketing AI Institute. Every time we host our live virtual events and online classes, we get dozens of great questions from business leaders and practitioners who are navigating this fast-moving world of AI. But we never have enough time to get to all of them. So we created the AI Answers series to address more of these questions and share real-time insights into the topics and challenges professionals like you are facing. Whether you're just starting your AI journey or already putting it to work in your organization, these are the practical insights, use cases, and strategies you need to grow smarter.

0:57Cathy McPhillips: Let's explore AI together.

1:03Cathy McPhillips: Welcome to episode 236 of the Artificial Intelligence Show. I'm your host, Paul Reitzer, along with my co-host, Kathy McPhillips. Welcome, Kathy. Thank you. Kathy is our Chief Marketing Officer at SmarterX. If you are a regular listener to the podcast, you have heard Kathy many times before. So Kathy is not taking Mike's job. Mike and I are still doing the weekly every Tuesday. And Mike is the host of the AI Transformation Series that we've started. We've done three of those now. And then Kathy and I do these special editions of what we call the AI Answers episodes. So these episodes are based on questions that we receive during our monthly Intro to AI and Scaling AI classes, as well as some questions that we get from virtual events.

1:49Cathy McPhillips: So these are all community-driven questions, and we just kind of go through. There's usually, I don't know, Kathy, what do we have, 15, 12 to 15 of these things?

1:57Paul Roetzer: Yes, we do.

1:58Cathy McPhillips: Kathy prepares a brief that I look at as we start recording. It's a wonderful brief, but I just kind of dive in and we go. So, yeah, so today's 15 questions. They are based on the last two Intro to AI and Scaling AI classes that we did. The Intro to AI was July 29th and Scaling AI was July 9th. So these are both from late July. And, yeah, so the interesting thing, and we'll kind of get into this as we go through these questions, is the Intro class I have been teaching since fall of 2021. So we've had north of 60 ,000 people register for that series. And back in 2021, pre-ChatGPT, you can imagine we were getting some very foundational questions, just very basic, trying to understand what the technology was and where it was going and how to figure out use cases.

2:54Cathy McPhillips: And the intro questions have evolved. They have become much more advanced. So even the people that are attending the intro class generally have really, really good questions. And then the scaling AI we have now done for probably going on two years, Kathy, I would say we've been teaching that one.

3:10Paul Roetzer: Just about.

3:11Cathy McPhillips: Yeah, we've done 18 of those. And so that's probably had 10 to 12 ,000 people, I would guess, go through that series. And that one teaches our five-step framework for scaling AI in an organization. So those questions tend to be more organizationally driven, team department driven. They're thinking bigger picture about how to move beyond just pilot projects and actually scale it in the organization. So if you've never had a chance to attend either of those, we do continue to teach them. You can look in the show notes and there's links to sign up for the next ones. I think we have one coming up at the end of September, maybe.

3:44Cathy McPhillips: Does that sound right, Kathy?

3:45Paul Roetzer: We have September 10th is our next Scaling AI. And September 30th is our next Intro to AI.

3:52Cathy McPhillips: Okay. So apparently I'm teaching a class next week. All right. So September, you can join us for both of these if you'd like.

4:00Paul Roetzer: Actually, no, it's on Friday. Actually, it's on the 11th because you're not going to be in town on the 10th. So we have to shift that date. Yes. Okay.

4:07Cathy McPhillips: All right. So, okay. So today's episode is brought to us by Marketing AI Month. This is a brand new thing. And I am going to let Kathy kind of tee this up for us. And then we will get into the Q &A.

4:20Paul Roetzer: Also, do you think you ever need to say that people are worried I'm going to take Mike's job?

4:24Cathy McPhillips: I do because I think that these episodes with you, Kathy, are so popular. and people might dip in and just want to hear you and I talk and they don't realize there's this broader show. So I don't know. I think I never know when someone enters into the artificial intelligence show world and it might be their first experience with it is.

4:43Paul Roetzer: Fair.

4:44Cathy McPhillips: And AI answers on like a YouTube clip. I don't know. So yeah, I always try and set it up that way.

4:49Paul Roetzer: I'm not taking Mike's job. I don't want Mike's job. We are all good. I don't want Mike's job either. So thank you for being here. This episode is brought to us by Marketing AI Month. We launched that Tuesday of this week. So this month we are doing something really fun. We've got this amazing AI for Marketing course series as part of AI Academy. All month long, you can get it free. It's valued at$499. So you'll get the series. It's five expert-led sessions by Mike, practical tools, frameworks that we actually use as well, and a professional certificate when you are done. So that's free through September 30th.

5:28Paul Roetzer: You do not need to complete it by September 30th. You just need to enroll by that date. And to close out the month, which we're actually doing on October 1st, we're holding an AMA where similar to this format, I'll be peppering Mike and Paul with all of your questions. Anyone that has registered for that or enrolled in the course is eligible to attend. So if you only want the AMA, enroll anyways. But the course itself is amazing and I highly recommend it. I took it myself over the holidays and kind of kicked off my year really strongly. So I was really excited about that. So if you visit smarterx.ai slash marketing, you can enroll right on that page.

6:05Paul Roetzer: Anything to add on that, Paul?

6:06Cathy McPhillips: No, I take advantage of it. It's our most popular course series. AI Academy has, I think, 22 professional certificate course series right now. So the idea here was like, let's just accelerate marketing, understanding and adoption because our experience with a lot of our AI Academy business account customers is that marketing is often the tip of the spear. It's the department that's leading on AI adoption, but then also an essential part of communication and strategy to bring other departments in and drive AI adoption and transformation through other departments being communicated through the marketing team.

6:45Cathy McPhillips: So marketing to us is a compounding value for the industry. The more marketers we can get to understand this stuff and be adept at using it in a responsible way, the faster we can spread that throughout organizations. So if you're a listener and you're not a marketer, make sure to send that link to your marketing peers within your organization because it, again, is open to everyone and it's a great value for everybody.

7:11Paul Roetzer: For sure. Which takes us to our first question. Let's jump right in. Number one, what are the best ways to use AI in marketing?

7:19Cathy McPhillips: I might turn this one back to you and say, how are you using it, Kathy, as our chief marketing officer? I'll keep it simple. And then if you have something to add, Kathy, I'd love to hear your perspective on this one. I just start with what do you do? Like I would look at the tasks and the workflows that you are expected to perform every day. I would look at the goals that you're expected to achieve. And then I would go in and just ask your favorite AI assistant. You could use our jobs GPT tool if you want, but you could just give this prompt to any of your favorite assistants and say, I am a director of marketing.

7:49Cathy McPhillips: My job is this. Here's the KPIs I'm responsible for. Help me figure out like the top three to five ways that I could be using AI today. These are the tools I have access to. Here's the data I can connect to. Just give it the information and let it help you figure that out. But the answer is it's personal to you and your role in the company is how I kind of simplify that.

8:11Paul Roetzer: Agreed. I think I would add, yeah, I'm using it a lot for strategy. We are rolling things out on a regular basis. So me getting a head start on getting all of the strategy out of my head onto paper. It's repurposing strategies that have worked in the past. It's building this knowledge base of all the things that we're doing. It frequently helps me write because I do enjoy writing, but it does give me a gut check on some things. I'm using some of our paid advertising to help me figure out new ways to message. looking at data, looking at trends that I might not be able to surface with my non-analyst eyes.

8:49Paul Roetzer: So one thing we've done on the marketing team is we have these documented processes for virtually everything that we're doing. So we're trying to, on a quarterly basis or whatever cadence we decide, running it through saying, can we do something more? Can we do something more? So having the processes, having the use cases documented, and just because things are changing so rapidly, updating that regularly has been really helpful for our team.

9:09Cathy McPhillips: Yeah, one example, and we don't have to get into great detail on this, but the marketing AI month that we just talked about, we decided like last Wednesday that we were doing this. Like it was a very spur of the moment, hey, let's just give AI for marketing away. Like let's just try and accelerate awareness and understanding in the marketing industry leading up to our MECON event, to our big conference in October, which is for marketers and business leaders. And so Kathy, I don't even know how she did it all, but we launched that campaign three business days later. It was an entire thing. And I'm sure there was ways Kathy leveraged AI to help in the planning and the production of assets, things like that.

9:51Cathy McPhillips: But to launch a major campaign in a matter of three business days is wild. You think, what could we have done historically? Now, I'm sure Kathy worked really long hours and it wasn't like she just hit a button and boom, here's your campaign. But even that, there's just no way you do that. We have a small team. We don't have 20 marketers where Kathy is just like, all right, everybody go do your things. It's like, no, we get our hands dirty. We're actually doing this stuff. So that's an example of a very real, relevant thing that I'm sure we couldn't have pulled off without the support of AI.

10:26Paul Roetzer: Right. And people. I mean, Jeremy Zimmer, our director of marketing for Academy, he was obviously integral in all of this. May Sarah in social media, everyone. And then I built this brief and like, here's all of the things we could be doing with my knowledge, with AI's knowledge. And I told the team, I'm like, I know I'm missing something. So please, as humans, go in and help me figure this out. What are all the pieces, parts that I'm missing? But yes, AI definitely gave me a very good roadmap to get it all done. Okay, number two. I have a team that is mostly made up of AI beginners. They are curious and optimistic, but also time-starved.

11:00Paul Roetzer: The most common pushback I hear is, I believe that if I invest time in AI, it will eventually generate efficiencies in my day-to-day work. But how do I carve out the time for now? Sales targets and client deadlines won't pause while I do this. Are there any good tips to respond?

11:15Cathy McPhillips: I would do a forcing function of a workshop. It could be a one-hour workshop. It could be two hours. It could be half day, however you want to do it. But I would do an AI workshop where you teach them a framework So we have a use case model we use. It's free. You can go, I mean, it's in our book. We teach it online. Like there's plenty of ways to go learn it and you can apply that framework to your own company. But basically, let's say it's, let's just pick 90 minutes. We are all going to leave here in 90 minutes with one to two AI use cases that we can implement starting tomorrow morning. That's going to save us at least three hours a week.

11:50Cathy McPhillips: Just set some parameters because what it does is it forces you to look through your tasks and workflows that you do over any given week and say, which are the ones that take more than three hours? Okay, now you've narrowed that down to, let's say, there's five things on that list. Which ones have a repetitive process that we could automate elements of? Which ones require creation of strategy documents or analysis of data? You go look for the use cases that are a good fit, and then you just pick one. And you build a project. You build an agent. like you just build something. So make the first like, you know, 20 minutes setting the stage, make the next 25, 30 minutes brainstorming where people are prioritizing lists, make the last 30 minutes or so sharing what you've done or actually building the thing.

12:36Cathy McPhillips: So maybe extend it to two hours and now you can actually have a build session with it. So I would you have to force it like you have to have something that says we aren't leaving here today until everyone has built a thing that's going to save them time. And then if that works, do it every other week or do it once a month. And like, then you just stack those things over time. And all of a sudden, everybody's got five agents or apps that are, you know, saving them 10, 15 hours a week. Yeah.

13:01Paul Roetzer: And I think showing them how, you know, I think the jam sessions we're doing where people on our team are opening up their computer, sharing it on the TV. We're all in the same room. We're online. We're watching them click the buttons and doing the things because that is daunting. People are like, okay, I think I can do that. I probably have the brainpower and the capacity to do it. I just don't even know how, and I don't have time to really figure it out. So having someone show you how to do it is, I think, so valuable.

13:25Cathy McPhillips: Yeah. And I'll, I don't know, again, I haven't really looked at these questions, so I don't know if I'm going to get into this and I'm not even actually a hundred percent sure I want to share this yet, but like I started a project this morning where I'm basically going through and trying to look big picture at where agents can be infused into my role. And as I was doing it, I was like documenting journaling it like I normally would like, okay, I'm going in here, I'm creating this, I'm going to test this agent. And so I would go through and like this whole dialogue with myself, but then I realized, wait a second, maybe I'll just turn on screen recording and I'll just like record this in Zoom or Google or wherever, and then eventually take that and turn it into like demos for the team.

14:05Cathy McPhillips: But then I'm like documenting all the steps I'm going through, all the visualizations I'm seeing as well. So all of that is helpful. But I think you're right, Kathy, like just, you need to infuse whether they're workshops, build sessions, jam sessions, whatever you want to call them, those have to just become part of the routine. And then it gives people the time and the permission to do these kinds of things.

14:27Paul Roetzer: Right. The build sessions at Macon this year, not to plug Macon, but I will. I'm really excited about that because you bring your computer, you sit down and 45 minutes, you leave with something tangible. And it's just so helpful, even if it needs to be, you know, sussed out after you get home, but just being there and watching someone else do it is pretty cool. Okay. Number three, we are a small company. How should we think about using external consultants to help? The team is so busy doing day-to-day work that we are not making time to create agents and take us to the next level. We have lots of ideas, but need the time skills.

15:00Paul Roetzer: What thoughts do you have on that?

15:03Cathy McPhillips: Yeah. I mean, there's a decent chance you can find consultants who come in and help you analyze your workflows, look for ways to drive innovation, look for ways to drive, you know, automate things, build some agents.

15:18Cathy McPhillips: I think that that's a short-term fix, honestly. I just feel like this is going to be so critical to every organization's operational structure and the way they manage the company that you have to own it. Like, so I explained this a bit on the podcast, but we're building SmarterX Labs within our organization. We're a small company. We have 19 employees. I think of labs as the R &D unit, and their job is going to be to go to each team or department or business unit and work with them to say, where are the barriers right now to achieving your goals? What are the workflows that take the longest or what are the major things that we should be doing as a company that we just don't have someone in that seat right now.

16:06Cathy McPhillips: We can't do it. And so they're going to basically function as forward deployed engineers. So that's the concept we talk in the podcast, where you basically have people who go in, look for things to build, problems to solve, and then they build them for them. But I believe the reason we have to internalize this is because we have to maintain that IP. We We have to maintain that know-how within the organization because if we go in and we solve something with Kathy and her team on the marketing side, it may translate over to the sales side or the ops side. An agent we build here might just need a couple of tweaks and all of a sudden, it's a sales agent too.

16:44Cathy McPhillips: Because AI being infused into all these areas is so critical to what we're trying to build, I just see it as absolutely fundamental skill we have to build internally. So if you're going to rely on consultants in the early going, I think that's fine. But I would be training or developing people who can function in that role moving forward. And it could be a marketer who's just also pretty technical, very comfortable in the data and the systems that's trained to become an AI ops person. And you don't have to go hire software development people. You don't have to go hire AI engineers. They can be business people, knowledge workers with domain expertise who are just the most advanced users of the AI platforms you use.

17:29Cathy McPhillips: So that's how I would think about it.

17:31Paul Roetzer: And you think about subject matter expertise, being, like you said, domain expertise, institutional knowledge. A consultant can help you with most of that stuff. So do you start building notebooks and knowledge bases now if you're going to start using someone on the outside?

17:48Cathy McPhillips: I mean, if you're going to bring in a consultant, they're just going to come in initially and document processes, audit workflows. They're going to look at what the AI is going. Again, all could really be valuable. If you truly don't have the time to do this and the alternative is no one does it, then yeah, hire somebody to at least get this moving. But I don't know. It's one of those things I think is so critical to every company that I would be very aggressively looking at defining what that role is and seeing, do we have someone on our team who could, this is a career opportunity for them. They would work really well.

18:21Cathy McPhillips: We've seen this in companies, Cathy. You and I both know people who are in AI ops roles today who 12 months ago, 18 months ago, were marketers or they were project managers or whatever. So this to me is that whole creating, seeing an opportunity for a career path for yourself, defining it, and then going and building it.

18:39Paul Roetzer: And when we start this internally, I would like to be tribute and I would like you to tear apart my department and tell me all the things that I will swallow my pride. And I can't wait to see what comes out of it.

18:50Cathy McPhillips: I think it is just like a totally collaborative thing. I think it's more of like, you have a wishlist, like you have roles you'd love to fill. And I think this division is like, okay, Kathy, what are the five roles you would put on your team today if you had the approval to hire them? And they might look and say, I think we can get you like 50 % of the way there on three of these in like two weeks. It's not like you're not gonna be your FTE, but like we could probably build something that'll at least give you that functionality. That's how I'm thinking about it. And then you kind of like go down from there.

19:17Paul Roetzer: I'm ready. Okay. Number four, who should be the point person for AI transformation in the enterprise? In some organizations, it's the CFO with project manager who understands the systems and knows where the data sets are hidden. Has this been your experience?

19:32Cathy McPhillips: I don't know that there's a right answer for this one yet. I do think it depends so much on the skill set of this person. So I've seen one where it was like a chief legal officer who became like the head of transformation because that person was just also extremely qualified on the AI side and understood the business deeply. I've seen it being driven. We had our first AI transformation spotlight that Mike did with Good Karma Sports. that is a CEO-driven transformation. I mean, Ty, who did the interview, was telling he's a key, key driver of the whole thing, but that is being very much led from the top by Craig at that organization.

20:14Cathy McPhillips: And so this is a CEO who got into it, said, this is the future of the company, I'm all in. And he is like right there every week in every meeting about AI strategy. So in that case, I don't know that they have a chief AI officer, but it's probably Craig. Like he's the driver. I've definitely seen where the CIO does it if that CIO has deep business knowledge, like if they're not just thinking about it from a technical perspective. We've definitely seen instances where like Dan Slagan, who's speaking at Macon this year, marketer, CMO, and Dan just sort of emerged into being an AI transformation leader at a major company because he just became the most adept at the technology.

20:54Cathy McPhillips: So I don't think that there's a right answer. I really do think it is, where is your organization at? Where is the CEO in the organization at? Because I'm a believer that if the CEO is extremely AI forward, that is probably the person who should be at the top of the chain when it comes to spearheading the transformation. Obviously, it doesn't have the ability to be in every detail, but the CEO should be the driver. That's probably going to be the most successful organizations is when it is a top three priority for the CEO and they are actively involved in everything.

21:28Paul Roetzer: Yeah. It just opens up that permission and enablement when the CEO is in that role.

21:33Cathy McPhillips: Yeah. This is totally behind the curtain stuff. I actually messaged our team today because I was working on something pretty significant for our organization. And so I just messaged, I was like, hey, can you guys send me the excerpt from our AI policy regarding connectors? Because Tracy and Elizabeth, they're going to laugh when they hear this now. Because I may be instituting an R &D exemption. That's a term I just made up. I literally put in parentheses to them. And like approving things myself. So I'm not bypassing our approval channels in our AI policy,

22:11Paul Roetzer: but I am accelerating them.

22:15Cathy McPhillips: And so within like 10 minutes, I was copied on emails to our IT team and our lawyers saying that basically our CEO is planning to accelerate approvals, we'll call it, on our AI policy.

22:29Paul Roetzer: Now, let's think about companies. Think about companies we know where the CEO has gone rogue, Paul. Let's not. Yes.

22:35Cathy McPhillips: Oh, no. Tracy did her job. She was like, here's the 15 things that I'm basically like want to check before I like go back to Paul. But like, and they're all really smart things. It's like, yeah, I know. Like, I'm not going to just like hit the buttons. But yes, when the CEO is has an urgency like, hey, I'm doing a presentation on this in seven days and I need these connections in place right now, then things happen and you do it within the safest way possible. But you have to also like an example. Again, I don't want to spend our whole time on this, but like Tracy's thing is, okay, cool. You turn that on.

23:12Cathy McPhillips: Does everyone have access? Do we have to communicate with everybody not to use the access you just granted? So while I want to move fast, that is why I reached out to her first and said, I'm going to do this within our guardrails. We have to set that example. But we also have to find a balance here because I have no choice. Like, I have to do this in the next seven days. So how do we do this in the most responsible way possible related to our guidelines and policies? Yeah. So you may have some access in your ChatGPT account later today that doesn't look familiar.

23:49Paul Roetzer: Okay, number five, are there generic agents everyone can use? Or for agents to be beneficial for us, we need to build them on our own?

23:57Cathy McPhillips: It depends on what platform you're in. So this is actually the initiative I'm referring to. I'm working deeply on integration of agents into SmartRx. And I haven't talked about this internally. So again, it's not even a joke. People think I'm making this up maybe on the podcast. I'm like, yeah, no one on our team knows, literally no one on our team knows about some of the things I'm working on and they hear about it on the podcast. So a couple of weeks ago, I started a personal initiative to deeply infuse agents by the end of this calendar year and in essence, figure out what that looks like moving forward, all the connections, how we're going to structure them through the SmartRex Labs division, things like that.

24:37Cathy McPhillips: And so I have a forcing function that I have a presentation next week related to agentic enterprises. And then I'm teaching an AI executive workshop at Macon on October 13th, where I need to actually present a model for this. So I'm deep in the agent world at the moment. And what I'll say is like an example here would be if you're in ChatGPT, they have standard agent templates that you could go in and just start with those. like when it pops in, it's like a marketing strategy agent. They've pre-built the skills for it. They've built the system instructions for it. All you do is go in and give it a knowledge base or connect it to like HubSpot or whatever it is you want to connect it to if you're allowed to.

25:21Cathy McPhillips: So I guess the short answer is yes. There are generic agents if you're in Copilot or if you're in Google, ChatGPT, Claude, they've pre-built some agents that are more universal, But most likely, you're going to be building things that are specific to your role or to your organization because for them to be really valuable, they're going to need a knowledge base that is personalized to you or your company. And they're going to need connections to data sources that your administrators are going to have to approve.

25:56Paul Roetzer: Got it. Okay. Number six. I love this question. Can you set up checks within an agent that require it to pause for human approval before taking specific actions?

26:06Cathy McPhillips: I wouldn't build an agent that you couldn't do that with. So, yeah, it just depends on how technical you are, like where you're building these agents. But again, if you're working, I'm going to assume most people that listen to our podcast are probably not like advanced codex users, cursor users. Like they're probably not the, now I may be wrong. Again, we don't really, it's hard to know your podcast audience. It's just all these people. It's quite opaque. So I'm assuming that most of our listeners are more marketers, sales professionals, business leaders that are not deeply technical coders, software developers.

26:51Cathy McPhillips: So we, I'm just going to use the collective we here, are likely we're Salesforce users and we're going to be trying to figure out how to use the Salesforce agents that are available to us. We're going to be building co-pilots in Microsoft. We're going to be building chat GPT agents. there's instructions within there that where you're going to say, this is the checkpoint where you're going to do, or you're going to give it read only access where it can't actually go and send something or do something. So the vast majority of us are likely only working with and building agents where humans are in the loop or in the lead.

27:25Cathy McPhillips: And yes, you should know when an agent would be taking an action and you should be the one that giving it permission to do that. in low-risk scenarios, you may give it permission to do the actions without approval. Like, example there, send Kathy a summary report every Sunday night of marketing performance data the week prior. And it goes to HubSpot, it gets the data, it pulls it, it summarizes it, it sends it to her. So it's going to take the action because it's just between me and Kathy. There's no risk of it sending to our customer base, that kind of thing.

27:59Paul Roetzer: Okay, cool. Number seven, For organizations with many data sets containing personal and sensitive information, a complex enterprise architecture, legacy systems, shared services across very different departments, and pockets of AI-forward teams, how can agents be enabled for those teams without creating risk for the rest of the organization?

28:19Cathy McPhillips: With great care. That is like when you're talking about these agents that are accessing sensitive information, let's say like a healthcare system, a law firm, a bank, wealth management company, whatever. We all have sensitive information. We can even say SmarterX. We have customer data that we always obviously want to be extremely sensitive to. So anonymizing data before you put it into these systems, things like that. you have to have IT and legal involved. You can design what you want to have the outcome be. You can build prototypes of agents. You can do all of these things. But when it's going to do things that present risk to an organization, especially if you're in a heavily regulated industry, you have to know in advance the approval systems you have to go through.

29:04Cathy McPhillips: You have to deeply understand your AI policies. That's the example I was giving earlier with Tracy. Tracy just taught an entire course and we'll get to some AI policy stuff. I see it highlighted below. She just taught a whole AI Academy live course on our AI policy, which is a very comprehensive document. And if you're going to be building agents, you have to be familiar with that. You have to know what you're allowed to do and not allowed to do, what data you're allowed to put into it, what connections you're allowed to make. And so what we're trying to do as an organization is these sort of like, I'm just going to make up a term here, like horizontal approvals.

29:39Cathy McPhillips: Like, okay, if we want to connect HubSpot to ChatGPT, let's approve it once so that if Kathy or Tamara or Dia or anybody wants to do something between ChatGPT and HubSpot, they don't have to come ask for the connection permission every time. It's already been connected, and then it's approved for read-only access for all these different uses. So you do something once and then it permeates across the whole organization with permissions. And so I think you'd be in a situation like that. Like if you're in this kind of enterprise where you're going to be accessing personal sense of information, doing these complex architectures, try and solve for as many of those with a one-time permission and training process as you can so that like everybody's not coming and trying to get approvals for basically the same thing.

30:29Paul Roetzer: Right. Number eight. How should a company fund its AI investments over the next few years? And where should I expect the new spending to come from?

30:37Cathy McPhillips: Oh, boy.

30:41Cathy McPhillips: It's got to be dynamic. I don't know. Like, this is a really hard question to answer. Like, I'm going to just, like, I'm thinking out loud here. People is my first thing. So, like, when I'm thinking about SmarterX, I'll just take it to make it personal. So, for our company over the next three years, where am I going to put AI investment dollars? My first investment is the people to help me figure out the answer to that question. So the people who think 24-7 about our AI investments, that could be human capital, it could be token capital. It's like, where's our spend going to go? What platforms are we going to invest in?

31:15Cathy McPhillips: But first, you have to have the right people to make those decisions. Second is going to be my platforms. So what are our token budgets? How are we going to optimize those token budgets?

31:30Paul Roetzer: How do you even budget for token budgets?

31:32Cathy McPhillips: Yeah, like how do you teach people what that even means and how do you manage them? And then third would be probably more of an infrastructure thing. Like, do I need to be buying Mac minis? Do we need to be buying GPUs? Like, are there business cases from a hardware perspective where we actually need to be like thinking differently about the company to where we have access? And do we need to be building? I guess the fourth would be, is there going to be an increasing argument that we should be training our own model at some point? You know, I think that a lot of companies are going to head in that direction where they're going to get these open weight, more efficient models.

32:11Cathy McPhillips: And they're just going to, you know, do reinforcement learning, you know, training on them to be adept at their specific industry and business case. So, yeah, I guess that's four. And again, I've never actually thought deeply about this question at all. So I reserve the right to change my answer at some point.

32:31Paul Roetzer: But it is hard. I met with a group this morning, and they're on a three-year cycle where they're trying to plan out the next three years. How do you even do that? And as leadership, are you able to give your team permission to just as intelligently as you can make those decisions knowing that they're going to change?

32:49Cathy McPhillips: Yeah, I think three years is hard, but every enterprise obviously thinks in three to five-year roadmaps. They're all looking in that direction. I think you need to make decisions that are as flexible as possible and as evergreen as possible. So you're not locking... I don't know. One of the ways I think about this one is if we go really, really deep with ChatGPT as an example, and everything starts running through that and all the connections are for ChatGPT and then something happens to open AI or like their models or whatever, and it's down for two hours, two days, two weeks. What happens to the company?

33:30Cathy McPhillips: There's no redundancies, there's no backup plan. And so betting on a single provider of intelligence isn't a super scalable thing, probably. And so when you're looking at three years, a strategy would be like, we need intelligence redundancy. And that scales. That becomes a strategy that lives over three years and actually starts to affect the decisions you make because we've set a priority around redundancies.

34:03Paul Roetzer: Hold that thought because I do have another question later that I don't want you to answer right now. So let's end that right there.

34:10Cathy McPhillips: That's the end. Okay.

34:11Paul Roetzer: Number nine, I'm kind of going into this a little bit. Beyond subscription costs, how are companies actually budgeting and forecasting total AI spend, including training and education time, setup and integration, internal token usage and token usage driven by external user or customers? Is anyone doing this well enough to have a real forecasting model or is everyone still finding out when the invoice arrives? Are there any resources you would suggest?

34:36Cathy McPhillips: The best I've heard, I don't even know if best is the right term here, a model I have heard is literally just setting a cap each month. Like here's our budget for the month. There are definitely more and more organizations trying to figure out the token optimization and management. There was just something from Google, I think it was on Monday I saw this, where they're starting to offer that kind of capability within their systems to help you manage it. There's startups emerging that do token optimization and basically route you to the most efficient models. I saw something, I think it was this week, where OpenAI is now actually experimenting with outcome-based pricing.

35:18Cathy McPhillips: So part of this is we're developing systems for the way that AI is priced right now, but most of these labs and providers of the intelligence know that this is not a viable, scalable system. Per token, utility-based pricing works sort of for developers. But once you start getting into departments like marketing, sales, operations, HR, that's a really, really bizarre way to try and price stuff. If it's not seat-based with unlimited usage, or it's not outcome-based, if the outcome is very predictable and able to be attributed specifically to the model's work, then it gets kind of tricky. So yeah, I don't know.

36:06Cathy McPhillips: An awareness of how the tokens are being used, what the different models charge per token, it's very early. I don't have a great answer for like, go look at these three companies because they're doing it perfect. I don't know anybody who's actually solved this yet, including some of the the big tech companies that are responsible for building the models. I don't know that other than providing unlimited uses of tokens to their employees, but even meta just changed. Like they've made changes to the models that they're using for this reason because they couldn't manage the token budgets and they're a builder of their own models.

36:42Cathy McPhillips: Right. So.

36:44Paul Roetzer: And if you do know somebody, listener, reach out. We'll have them on the transformations.

36:49Cathy McPhillips: Yeah, totally.

36:51Paul Roetzer: Okay, the next three questions are from the Academy Live we had on Tuesday this week with Tracy, our COO, and then Samantha, our legal counsel. So these were, I thought these just fit in well with some of these discussions. So the first one is, what does a right-sized AI vendor approval process look like? How long should a review take? What happens to employees while they're waiting? And how do you catch it when a vendor update introduces new integrations or changes how an approved tool handles data? And I know that you are not doing, you know, that ops is doing this, but as the leader, what thoughts do you have on this?

37:26Cathy McPhillips: I would start early. I mean, we've been in situations with some customers of ours where they got excited about instituting, you know, different AI technologies into the organization. They'd made the business case for it. And then they go to procurement and they would be told there was like a multi-month wait time before procurement would even look at anything. So it starts with really just understanding your organization's procurement process and what kind of risk and security considerations need to be made as we start getting into the AI technology that might require access to certain data sets.

38:01Cathy McPhillips: And yeah, I think it really is about an understanding of the process. And then from there, it's going through and building that in. And so if you know that it's going to be three months before you can get the system you want or the approval for the use case you want or the connection you're looking for, you need a plan B about what that workflow looks like without those tools. So, yeah, I think, again, this one is very subjective to the organization and what their approval processes look like and how they're defined within the AI policy. So I know in our case, we have to submit now through form requests to the operations team when you want a new vendor, if you want to make a connection to a data source.

38:46Cathy McPhillips: And then I think the team says like within 48 hours or something, they'll at least give a preliminary, hey, we're going to have to look deeper into this. This is going to request kind of the example I gave with Tracy earlier. Like I put in a request, we'll call it. And then there's, you know, within a few minutes, there's an email to the legal team. There's an email to the IT team because for the ops team to make their decision, they need the expert perspective from those two bodies to make the informed decision around the vendor or the connection that's being requested. So, yeah, it's going to vary by company to company.

39:20Cathy McPhillips: But the best thing you can do is deeply understand that process before you get started so you don't run into that obstacle when you're ready to go. for sure.

39:28Paul Roetzer: And in one instance that we had internally, the request was put in, it took longer than normal because it was brand new, brand new tech. We were trying to figure out, well, ops was trying to figure all of it out with legal. And there were some redundancies because things were being done twice, one the old way and one trying to do it with a new tool. So there was a minute of double work, but in the long run, it's going to be so much better now that everything got approved the right way. Yep.

39:53Cathy McPhillips: And I will just say being on the other end of this, You know, we have our AI Academy, which is a technology platform. And so we have to go through procurement, especially for these bigger enterprises. And you have to meet some very stringent standards that are only getting stricter as we go. And so Tracy can attest, like, I mean, she spends a fair amount of her time getting through procurement systems. And she could probably sit here all day telling you about all the new requirements over the last, like, 18 months as a result of AI.

40:24Paul Roetzer: Yeah, there was one she did last week or two weeks ago that she was just like, this is the wildest form I've ever filled out.

40:30Cathy McPhillips: Yeah, they get crazy.

40:32Paul Roetzer: Yeah. Okay, number 11. When reviewing an AI vendor's terms and data policies, what should companies look for regarding what happens to their data if the vendor is acquired, goes bankrupt, or sells its assets? How can organizations manage or mitigate that risk?

40:47Cathy McPhillips: Or sells the data for training purposes. Um, I, I'm like this one. I just honestly feel like my best answer is you, you have to rely on your legal team. Like everybody's legal team is going to have different appetites for risk. They're going to look at it through different lenses. It is just simply not a decision that like a traditional business leader, um, department leader, you don't have the expertise and likely you don't have the permit in your company to make those decisions. You have to involve legal and whatever other channels are required. And hopefully, based on our previous question, that's all clearly documented in the AI policy.

41:29Cathy McPhillips: And if it's not, it needs to get documented so that you can follow these procedures. And then you know when it's a non-starter. So you're getting really excited about this new AI tool. You go in and pull up their terms of use. You throw it into the chat GPT and say, tell me how they handle our data. And immediately it pops up with like three red flags for you. Another way you could do it is take your AI policy related to vendor data usage, drop that into chat GPT thread or Gemini or wherever and say, hey, here's our AI policy or build a project that has that AI policy baked into it. I'm going to give you a contract for a new vendor.

42:09Cathy McPhillips: can you tell me how this relates to our terms of use requirements and flag for me what I should highlight for our attorneys. I do that kind of stuff all the time where I'm just like, here's this doc, here's this doc, compare them and tell me what I should ask the attorney.

42:23Paul Roetzer: For sure. And then, you know, just thinking about when they change the terms and conditions or when something, I mean, there's so many, it's not like approve it once and it's done. It's like set up some normal cadence of just reviewing everything to make sure it's still.

42:37Cathy McPhillips: Yeah, like when Anthropic came out with Fable 5, they changed their data retention policy where they kept everything for 30 days. And a lot of people were up in arms about that.

42:47Paul Roetzer: Right. Number 12. When should a company disclose externally that AI was used to create content or complete work? Is there a meaningful threshold or does it depend more on the nature of the work and its potential consequences?

43:00Cathy McPhillips: This gets in again to AI policies. Everybody needs to make these decisions for themselves. I feel like it's a bit of a sliding scale where more and more, it's just going to be kind of assumed and expected that AI played a role. And I think like tagging every single thing with like, AI helped here, AI did this. I don't think most people are going to care most of the time. So I think that there's some basic rules that at least I tried to live by. And I hope our company tries to live by is like when authenticity matters, you should be doing the work. It should be human in the lead. Like, it's okay if you're using AI to edit or verify ideas and things like that.

43:40Cathy McPhillips: But it should be you and you should stand behind that work. This came up on episode 234 or 35 of the podcast. We were talking about that one author who used it and got called out on it. He's like, of course I use it. I use it all the time. Like, what are you talking about? I generally like to default to if you're using it, it can be something as simple as like co-authored with Claude or Claude supported it in the editing and revisions of this document or Claude helped me with brainstorming, whatever. Just disclose it because it doesn't do any harm, but I don't think we're at the point where we need to make a huge deal of it anymore.

44:21Cathy McPhillips: Right. So yeah, again, you got to make these decisions in your ad policies and then provide that guidance to your team because everybody's wondering. It's like, I don't know if I'm supposed to tell somebody I didn't actually look at their email. My agent is now answering my emails for me. In that case, I would just say, yes, you should disclose it if your agent is answering your emails for you. But yeah, I think it's a very personal decision and it needs to be decided in the policies.

44:46Paul Roetzer: I was working on something last week and I can't even recall what it was, but I kind of went into it blind, not really knowing what I was going to be working, like what I was going to be doing. And I used AI a lot. And I put that in the, when I distributed it to the team, I was like, this is Claude written with Kathy supporting it. Like, I honestly, I don't know. So this is my first pass I'm learning. And just some people looking at this, like this doesn't sound like her. Um, it's so, it's so obvious when it's not, you know, but I just want people to know that where I was on that, in that process.

45:20Cathy McPhillips: Like one example, and you could attest to this Kathy, like sometimes we'll be working on a problem and I'll go into chat to people. Like, here's the exact problem. Here's the context you need. Here's what I'm trying to think through. build a brief for me. I just need to think about this. And it'll build a brief. And I will take 10 minutes. I'll read the brief. I'll give it some thought. I'm like, this is really good. I will send it to the team internally and say, this is completely unedited, but I have reviewed this and this is a really good starting point. And then I turn it over to the team.

45:47Cathy McPhillips: So they know I didn't do it, but they know I've signed off on like, this is a good direction. Come back to me when you all have made progress on this. So yeah, I just, I don't know. I don't think there's like rigid rules around this. I feel like it's just evolving as our use of the models changes.

46:03Paul Roetzer: Okay, number 13, entry-level staff have historically learned their trade by implementing task-based work assigned by their managers. If that work is not going to be assigned to AI, how do entry-level staff learn their trade and where do future managers come from?

46:18Cathy McPhillips: My theory is apprentice programs. I talked a little bit about this on the podcast this week. It was 235. I think it came up because there was that research that came out about apprenticeships were actually like the number one thing. And it just, that happens to be what I'm talking about for Mekon. My opening keynote is the architect, the orchestrator, and the apprentice. It's a theory and that it's very, just a theory on what the future of work looks like and what the different roles we're going to need within organizations are. My belief is that we can actually accelerate learning and expertise through apprenticeships with the support of guided learning from AI and personalized learning.

47:01Cathy McPhillips: But I'm not sure yet. I have six weeks to figure out what exactly that is. I've been thinking about this model since March of this year. I don't know. This has been my great debate. Where does management come from? What do managers do in the future if we don't train entry-level employees. So if we stop hiring as many entry-level staff, then we decimate the future managers and leaders of the company. And I don't know, again, kind of like the earlier question, I don't know of an organization that has this figured out. I've talked to lots of companies and there are lots of AI transformation leaders and HR leaders at big companies that are just starting to ask these questions themselves.

47:44Cathy McPhillips: And nobody seems to really know where to go with it yet.

47:48Paul Roetzer: Yeah. 41 days, in case you're wondering, tell my con. No pressure.

47:53Cathy McPhillips: 41. I only have three talks to create in that time period.

47:58Paul Roetzer: Number 14, you described the actual mechanics of an LLM practicing, predicting the next word as a bit of a black box. Is it really? And if so, isn't that a little ominous?

48:09Cathy McPhillips: It is really a black box. There's a field of study called mechanistic interpretability, where they try and understand why the models do what they do. The LLMs itself is making predictions based on proximity of words to each other within a graph. So this is a very oversimplified way to try and help people visualize this. Imagine all of human language as this three-dimensional cube. And within that cube is every word in the human language. and how close those words are to each other sort of shows a probability of them occurring next to each other. And so when ChatGPT starts writing something, it's in essence going through this dimensional graph.

48:53And it's like knowing that, you know, Kathy sat on the chair and it knows

48:59Cathy McPhillips: that chair is a likely word connected to where Kathy would sit because people sit on chairs, could be a bench, could be the table, could be all of these things. But in essence, like imagine And in that cube, those words all appear near each other, sat, chair, desk, whatever. That's kind of like how it works. The way to think about the language model, the closest proximity to it is the human brain. And that would be like saying, well, do we really not know why Paul picked the words he just picked when he was explaining this? Like, no, we actually don't. Like, you could put me under an MRI machine and it might see what neurons in my brain are firing when I'm talking, but it can't drill in and say, why did he choose the word he chose?

49:42Cathy McPhillips: It's likely just on my training and in my experience as a human of the words that appear near each other in the English language. So that's kind of it. Like, yeah, it's ominous. Yeah, it's weird. Like, we don't really understand this intelligence that we've created. All we know is that we give it more and better data and we give it more chips to train on that it seems to just get smarter and more capable of simulating human behavior and language. And that's kind of it. That's the basis for the first scaling law of AI that has now stayed true for the last decade, which is more data, more computing power.

50:22Cathy McPhillips: The bigger the model, the better it seems to get. There was this sort of ominous thing that Ilya Sutskova said back in the day at OpenAI, which was they just want to learn. These models just want more information, and they just get smarter.

50:38Paul Roetzer: Okay, number 15. The nice thing about these questions is that I get to decide the question. So I actually dropped in my own question. My husband's been traveling for a week and I've had a lot of free time. So these are the things I think about. Okay, when I worked with Joe Polizzi at the Content Marketing Institute for years, he always emphasized the importance of building on land you own. Your blog, your email list, your audience. We've seen what can happen when people go all in on rented land like LinkedIn, Twitter, or another platform. The algorithm changes, products, audiences. Audience becomes harder to reach, the platform disappears.

51:15Paul Roetzer: Now people are building products, audiences, and sometimes entire businesses on custom GPTs and other AI platforms. Are we watching the same story repeat itself with LLMs, and how should we think about the risk of building on rented land?

51:29Cathy McPhillips: That's a deep thought to have.

51:30Paul Roetzer: I know. A lot of free time. not really free not really free time actually alone time yeah i think it kind of relates back

51:38Cathy McPhillips: to the thing i mentioned earlier about you know if you build on a single platform you're sort of a prisoner to what happens to that platform i i look at this a lot now with like how transferable skills are between agents and platforms the system instructions that are used to build a gpt can very easily be copied and pasted into Claude, and you can build the same capability over in Claude. So I think that there's an inherent risk here. It's probably part of the reason why some organizations are now choosing to take open weight models and train their own models. It's just a more reliable system.

52:15Cathy McPhillips: The model lives on internal servers. You're not captive to a cloud service going down and you no longer have access to it. So I do think that more organizations will start to build redundancies and backup plans in. At a very surface level, you can start by making sure everything is documented. We've done this again internally at SmarterX, where we're now at least from a governance perspective, documenting when apps and agents are created by different people. Ideally, you include what the system instructions are, what the knowledge base they have access to is, so we can go back and audit it. And so let's say like worst case scenario, you build everything around ChatGPT, you've got 25 agents that are basically running the company, you've got 75 skills that those agents use to do what they do, and then ChatGPT goes down for five days, you could, worst case, go grab the skills, recreate them in Claude, or you just mirror everything in Claude.

53:14Cathy McPhillips: Someone's job literally might just be maintaining a mirrored image of the ChatGPT instance in a second platform so that if something goes down, you just flip a switch, everybody moves over to Claude and continues their work. I would imagine there are organizations that are actively doing what I just described. I haven't talked to them. It's just one of those things you can kind of make an assumption that someone is ahead of the game doing this because we are talking about entire companies running on this. Now you have companies laying off thousands of people under the assumption that agents can do a lot of this stuff.

53:46Cathy McPhillips: Well, if those agents aren't available, work doesn't happen. So there needs to be redundancies built.

53:52Paul Roetzer: Yeah. And I was talking to Claire, who is our producer of the podcast and works on our content. We were talking today about how Mike had actually done, talked about this in one of his courses for Academy about how to extract some of that from one tool to be able to upload it and put it in another tool or how to document it and what that process is. So it really is a critical step of all of this. All right. That is 15 questions.

54:19Cathy McPhillips: That's a good question. I feel like those are all different questions, too. I don't know if I've answered any of those. Maybe the marketing use cases has probably been answered at different times, but it changes. But the rest of these are relatively original. Really good.

54:31Paul Roetzer: You don't look at this beforehand, but Claire and I do.

54:34Cathy McPhillips: So she does a lot of

54:37Paul Roetzer: has this been asked 20 times already? Let's not ask it again. So yeah.

54:42Cathy McPhillips: Good stuff.

54:44Paul Roetzer: So thank you, Paul. You'll see Mike next Tuesday back on the podcast. If you're interested in learning more about Marketing AI Month or MECON, our big event coming up in October, please let us know. I'd love to help you get you to both of those things.

54:57Cathy McPhillips: Come to Cleveland. We've got, oh, do the pod because we have the pod 100, which gets you a VIP lunch with me and Mike's. We're doing on the final day, the October 15th, a VIP lunch for anybody who uses the pod 100 when they register. So yeah, MECON.AI. Get registered. We'd love to see you there.

55:13Paul Roetzer: Thanks, Paul. Thank you.

55:16Cathy McPhillips: Thanks for listening to AI Answers. To keep learning, visit smarterx.ai, where you'll find on-demand courses, upcoming classes, and practical resources to guide your AI journey. And if you've got a question for a future episode, we'd love to hear it. That's it for now. Continue exploring and keep asking great questions about AI.