326. Rethinks: The Secrets to Getting the Information You Need from AI

0:00Kian Gohar: Hi, Matt here. I wanted to remind you that for each episode, we provide English language learning support where we share useful information to help you develop your English skills. Check out these English language learning and other resources along with our newsletter and deep dive videos at faster smarter dot IO under resources. And now a word from our sponsors. Their support covers the cost of production, allowing us to bring you this episode free of charge. The real value of coaching is what it helps you see and what it helps you do differently. I have been coached and I have coached others.

0:35Kian Gohar: And for me, the value is clarity. Seeing what you're doing well, where you're getting in your own way, and what you can do differently. And if you're watching this, I'm guessing you're someone who wants to keep improving. That might mean communicating with more confidence, becoming a stronger leader, or getting clearer about your next career move. Strawberry.me is like therapy for your career. They match you with a coach who fits your goals and gives you practical, personalized guidance you can actually use. It's all designed to make coaching more financially accessible. And because it's online, you can use it from anywhere in the world.

1:12Kian Gohar: Visit strawberry.me slash smart to get 50 % off of your session and make sure you tell them Matt sent you.

1:23Kian Gohar: AI can generate answers in seconds, but the quality of those answers depends on the quality of the conversation. I'm Matt Abrahams, and I teach strategic communication at Stanford Graduate School of Business. Welcome to this Rethinks episode of Think Fast, Talk Smart, the podcast. We're revisiting my conversation with Jeremy Utley and Kian Gohar to explore how conversational AI can become a true thinking partner, helping individuals and teams solve problems more creatively and communicate more effectively. Enjoy this journey into our communication vault. The promise of AI is tremendous, yet most of us, if we're using it at all, are using it incorrectly.

2:10Kian Gohar: It's not about a transaction. It's about a conversation and interaction. I'm Matt Abrahams, and I teach strategic communication at Stanford Graduate School of Business. Welcome to Think Fast, Talk Smart, the podcast. Today, I am really excited to chat with Jeremy Utley and Kian Gohar. Jeremy is a repeat guest to Think Fast, Talk Smart. He's an adjunct professor specializing in creativity and entrepreneurship at Stanford and the author of Idea Flow, the only business metric that matters. Kian is founder of Geolab and former executive director at Singularity University. He's the best-selling author of Competing in the New World of Work.

2:55Kian Gohar: Welcome, Jeremy and Kian. Thanks for being here. Thanks for having me back. Such a joy to be here with you. All right. Shall we get going? Let's do it. To get us started, Kian, I'm curious, what motivated you and Jeremy to look at the impact of AI on creativity and problem solving? So in my business, we coach teams to achieve complex business goals through team transformation practices. Sometimes this involves tackling a difficult innovation project. Sometimes it's struggling with change management. But it always boils down to this one issue, which is how can we solve this problem? And that usually entails human ideation and prioritization.

3:34Kian Gohar: And I wanted to see if we can bring a new technology tool into the mix to get better ideas. And given that I've been teaching AI in Silicon Valley to executives for a long time, that was always my hunch. But it wasn't until ChatGPT became publicly available that I had the aha. And I didn't quite know how it could be useful to the issue of problem solving on teams, but I had a hunch. And so we designed the study together to learn from real world practice of how teams might use AI to get better ideas to solving problems. And what we did was we recruited hundreds of participants from many companies in Europe and the US.

4:14Kian Gohar: And we asked them to identify a particular pain point within their organization or a problem that they needed to solve for something that was real. And then we actually gave half of the participants in each of these problem solving ideation workshops had access to chat GPT and the other half didn't have access to it. So they were just thinking on their own as humans. And then the other half had humans plus AI. And we went through a problem solving ideation exercise. And at the end of it, we asked the problem owners or the executives who shared with us the particular pain point to grade all the various ideas from A, being spectacular all the way to D, not worth pursuing further.

4:55Kian Gohar: And so we ran this blind study to understand how generative AI can facilitate problem solving, ideation and collaboration with real world examples.

5:07Jeremy Utley: The other thing that we did was we asked participants before and after the session, how they felt about collaboration and problem solving and ideation, because we wanted to see what was the impact of just a standard brainstorming session on someone's attitude towards problem solving and collaboration and innovation? And what was the impact of a session that included generative AI? Did it have a differential impact on participant sentiment?

5:33Kian Gohar: And the research to me is fascinating. Jeremy, can you summarize the results of the research you and Kian did and relate it to the quality of idea generation and the feelings people have about the ideas that were created?

5:48Jeremy Utley: Very broadly speaking, we found that teams, if they want to outperform using AI, they need to adhere to certain practices. And sadly, most teams do not. So despite the potential to dramatically outperform, most teams leave the vast majority of their innovation potential on the table when they use AI. When you approach AI like an Oracle or like a search engine, just looking for the answer. It may feel like magic, but the data shows you end up underperforming. If you want to get world-class results and outperform, you can't approach it like an Oracle. You need to instead approach AI like a conversation partner.

6:26Jeremy Utley: And that doesn't feel nearly as magical. It feels more like work. And for that reason, the teams that outperform actually feel worse than the teams that underperform. It's very counterintuitive.

6:38Kian Gohar: I find that absolutely fascinating. And I also find that, interestingly, the ability to have a good conversation with AI is what makes the difference for creating valuable ideas and solving problems. Kian, can you go into a little more depth with the counterintuitive findings that Jeremy just shared? Why do you think it is that AI-assisted teams that delivered worse solutions actually felt better about their work? And the AI-assisted teams who delivered better outcomes actually felt worse related to their non-AI-assisted counterparts? Because rewiring our brains to act differently takes practice.

7:15Kian Gohar: And we professionals have decades of experience or certain ways of doing things. And we're asking them to work differently now with a different workflow. And that's hard. It's like going to the gym after not having exercised for a long time. It really sucks. And you don't particularly enjoy it at first, but it has long-term benefits. And so we found that teams that were able to use AI effectively, they actually had to rewire their workflow and how they went about to try to incorporate AI as a co-pilot on the team. And that's different. And it was exhausting. And they felt like it was just a lot of work, but ultimately they had better responses.

7:56Kian Gohar: Would you add something to that, Jeremy?

7:58Jeremy Utley: Part of the challenge is, I almost picture a cartoon strip with a thought bubble that's like, but I was promised superpowers. The premise and the promise of AI, I think it's intimidating and it maybe strikes fear into a lot of people's hearts. That's one thing. I don't think there's a reason to be afraid. I think there's a lot of reason for enthusiasm and fun and seeking fluency, obviously. But I think there's a premise of this is going to be like magic. But when you start there and you think, all I have to do is ask a question and then I get answers. Well, guess what? You do. You ask a question and you get three pages of call it B plus documentation.

8:33Jeremy Utley: And teams go, it is kind of magical. Well, let's go get coffee. Like we're kind of done, right? If you're expecting magic, you will get it, but you won't do that well. If you've gotten practice being a conversation partner to an AI co-pilot, what you realize is the first step that you get out isn't great unless you're really thoughtful about framing and providing context and digging deeper and pushing back and asking questions and treating it much more like a conversation. And then the more you treat it like a conversation, the less magical it feels. And the more it feels like AI is actually getting you to work and pulling your best thinking out of you.

9:12Jeremy Utley: And so I think part of it is just the mindset people have when they hear, oh, we get Shaggy BT. The people who go, oh, magic are almost always the underperformers. The people who roll up their sleeves and go, oh, now we have work to do. It's maybe a different kind of work to do, but I'm excited to do it. Those are the people who outperform.

9:30Kian Gohar: I would also add that there is a reason why ChatGPT gives you generic answers. And it's designed mathematically to give you the most likely answer that it thinks you want to hear. And so as a result, the first few suggestions it offers are supposed to be generic and average. And so unless you have this conversational back and forth with it, like you would with a colleague or a friend, to push it and to give it context, you're just going to get average answers and average ideas. then that's really not good enough if you're trying to solve complex problems. It comes down to mindset shift and work.

10:04Kian Gohar: There is no magic. It's not for free. If you really want to solve problems well and get creative, you actually have to have a conversation. So Jeremy, can you share your Fix-It methodology and describe the different components and how they work?

10:17Jeremy Utley: Absolutely. So Fix-It is a five-step methodology to turbocharge your collaboration. F stands for you have a focused problem, right? So you don't want to boil the ocean, but you want to be very focused in the kind of challenge that you are bringing to ChatGPT. So in this example, instead of saying what recommendations would you make for a human interfacing, I would say I'm joining a podcast and I'd like to give concrete, tangible suggestions to professionals who haven't had much experience with ChatGPT. if you had three or four, or maybe I'd start with 10 suggestions for someone who's unfamiliar, but wants to be more familiar, start by asking me three questions to better understand the kind of professional I'm describing, right?

11:03Jeremy Utley: That's what we mean by F kind of focused. So that's a very focused prompt, right? I is ideate individually. So before coming to the team, because remember our study was conducted in the context of teams trying to solve problems before coming to your team and even before coming to chat GPT, think for yourself about what do you know and what's your point of view. Again, this kind of hones the context that you bring to the conversation, right? It's critical. Research has shown that the best way to brainstorm is to alternate kind of individual ideation and group ideation. Well, the same is true actually in the context of collaborating with generative AI.

11:38Jeremy Utley: You want to alternate between individual personal thinking and thinking assisted and amplified by AI, right? So that's the I. X is context. So providing sufficient background context. A lot of times we recommend that folks upload documents, right? When we were conducting our study, we would have a problem owner actually do a dossier that we would upload to ChatGPT to provide context. If you don't know what your context is, here's an amazing hack. Ask ChatGPT to ask you for the context. So as an example, I've got a friend who's negotiating a lease, trying to build a gym. And he's trying to come to an agreement with the property owner.

12:17Jeremy Utley: And there's a gap between where they need to be. And I said, hey, why don't you ask ChatGPT for help? He said, well, how would I do that? I said, well, have ChatGPT interview you about your objectives and then interview you about the counterparty's objectives and then make some recommendations. Well, that interview effectively becomes a means by which you can provide context. So the X is make sure that you're giving ChatGPT context on the problem. The second I is interactive, iterative conversation, right? So you're never just taking the first response that ChatGPT gives you. You want to be having a back and forth and a dialogue.

12:54Jeremy Utley: So for example, if Matt and Kian and I are going to try to title this episode, we might say to ChatGPT, hey, we'd love 10 titles for an episode. And then ChatGPT comes back with 10, almost everyone's default is to think, which of these 10 do I like best? What Reed recommend is just immediately say, I'd like another 10. And then read all 20 and say, here are the ones that I like. Would you give me 10 more like these? Chances are you're not going to get as good. And then if you think, wait, why did I like number two? Oh, there was a funny alliteration. And number seven had a funny pun. And number nine made reference to, okay, use those as design principles for the next 10, right?

13:32Jeremy Utley: That's an iterative back and forth that yields, we'd probably find that the fourth tranche of 10 title suggestions would be radically, exponentially better than the first 10, right? But it's by going back and forth. F-I-X-I-T. T is for team incubation. So it's important to then bring the ideas that you've generated individually and with ChatGPT or whatever LLM you're using back to your team, and then importantly, commission some experiments. So this is where this work dovetails with traditional innovation methodology, but you never just want to select one idea and move forward. It's impossible for an LLM or for a human to a priori know which solution is going to be the best fit.

14:16Jeremy Utley: So as a team, you want to have a practice and a process around incubating or low resolution prototyping a handful of the high potential solutions that you've generated in order to determine which one actually solves the problem the best.

14:31Kian Gohar: This methodology, upon hearing it, makes a lot of intuitive sense, but I can definitely see the effort involved. I like that it involves individual work and collaborative work, not just with the ChatGPT, LLM, but also with others on the team. Kian, not surprisingly, I'd like to dive deeper into the fourth step that Jeremy introduced us to, interactive conversations. We've spent a lot of time on this podcast talking about how to have better conversations, but of course, focusing on humans. What specific advice can you provide to us for improving our conversations and our communication with LLMs to make sure we maximize the potential goodness that can come from collaborating with a tool like ChatGVT?

15:20Yeah.

15:20Kian Gohar: Yeah. So the first thing I'd say is to make sure you've downloaded a LLM app on your phone and interact with an LLM on the phone instead of doing it on the web browser. If you don't see a app on your phone, it's very unlikely that you'll use it on a consistent basis. And the more you use it, the more familiar you become with it, the more likely it'll actually be part of your workflow, whether it's individually or whether it's as a team. So that's really the first step. Download one of these apps on your phone. And part of that is because we have historically been for the last 20 plus years in the internet era and the browser era, we see a text box and we know exactly what to do with it.

16:01Kian Gohar: We type in a particular word or particular question. And now that we have large language models, the user interfaces look pretty much exactly the same, like the Google search engine box. And it's oftentimes difficult to figure out the right kind of question to ask it or to word it in the right way to get the right answer to suggestions. And so we actually think it's a lot better if you start interacting with these large language models through conversation, through spoken audio, rather than just trying to type it into a search engine box and trying to figure out the exact right prompt. The second thing is that you should be uploading your prompts with audio messages, voice messages, and talk to it just like you would talk to a friend.

16:41Kian Gohar: So whether you're talking to a friend about a particular problem or talking to a colleague about a particular problem, just record it literally on the phone on audio to text. And then the large language model will transcribe that and then offer out some suggestions. And then the third thing I'd say is think about using different models. There are several different kinds of large language models that you can use from ChatGPT to Claude to Bing and to others. And they all have their different flavors and different kinds of personalities. and you might want to use one of those models for a particular exercise or activity.

17:16Kian Gohar: And then maybe you should go talk to another model, just like you would talk to two or three different friends and get their opinions, or two or three different colleagues and get their opinions, because they're going to give you different kinds of nuance and different kinds of answers. I find that really helpful advice. I mean, we approach the communication with technology based on the interface that we have with it. And if we move to our phones, which we're used to communicating with very differently than we are, let's say, a browser in a text box, it can really change it. And it strikes me that it's not just the actual interaction interface, but it's also the curiosity we bring to it.

17:51Kian Gohar: When I go to enter information into a search engine, I just want to get the answer. I don't necessarily see it as a conversation and a dialogue. So that curiosity that I bring with the follow-up questions or the doubting and the exploration and expansion, I think is really important. And I like the advice to check with different LLMs, just like you would check with different friends. It gives you different ideas and inputs.

18:15Jeremy Utley: You can even feed different LLMs answers into one another, right? Oh my goodness. We can facilitate and broker a conversation amongst LLMs. Absolutely. I mean, I plug in Chad GPT's answer into Claude and say, what do you think of this all the time? And vice versa, take Claude's answer and plug it into Chad GPT. Ask Chad GPT you what it thinks of it. So this is why Googling is such a bad metaphor for what we're talking about here, because you'd never query Google about a Google query to get meta for a second, right? But after a sales call, I hop on my chat GPT voice app while I'm stretching for a run.

18:49Jeremy Utley: Instead of sitting at the screen, I'm stretching for a run. I say, Hey, I just talked to Matt and Kian about our research and a couple of follow-ups. Would you craft a quick memo that I could send to the team, just thanking them for the time today and how much I enjoyed the conversation, right? Well, it's going to do it. Well, then my next thing is I just look at it and I say, if I were to upload this memo to you and I were to ask you for advice on how to make sure that they read it and respond, what three changes would you recommend? And then immediately, I had this happen the other day. I was doing a voice vomit on a post sales call and I had ChatGPT write a memo.

19:24Jeremy Utley: And then I asked ChatGPT, how would you criticize this memo? And ChatGPT He said, it's far too long for today's busy professionals. No one's going to read this memo. And I said, would you please go ahead and shorten it to a point where you think that people actually read it? You can ask ChatGPT to evaluate its own work and it will do so dispassionately, right? And that's the fun, but that requires iteration and a back and forth.

19:46Kian Gohar: In many ways, the advice that you all are giving are advice that we give to people when they are engaged in empathetic dialogue and human to human interaction. It's about curiosity, questioning, about challenging in certain ways. And it changes the metaphor completely, as you mentioned, Jeremy. We'll be right back to finish our conversation. But first, we're going to take a quick break for a word from our sponsors. Their support covers the cost of production, so we can bring you this show for free. This podcast is sponsored by Squarespace. The best communicators don't just get their message across.

20:21Kian Gohar: They make it easy for people to know what to do next. your website can do the same thing. With Squarespace, you can build a polished online presence that clearly shows people who you are, what you offer, and how they can work with you. If you offer coaching, consulting, classes, or other services, Squarespace lets you showcase those offerings, take payments, and use acuity scheduling so people can book time with you directly. And with built-in analytics, you can see how people are finding and engaging with your site, helping you make smarter decisions about where to focus. So your online presence doesn't just look professional, it helps people take action.

20:58Kian Gohar: Go to squarespace.com slash thinkfast for a free trial. And when you're ready to launch, use offer code THINKFAST to save 10 % off your first purchase of a website or domain. Teams don't usually struggle because people aren't working hard. They struggle because everyone is working from a slightly different understanding. One person has the latest version, someone else remembers a decision differently. Before long, you're spending more time getting aligned than actually moving the work forward. That's why I find Notion's approach so interesting. Notion is the AI workspace where your team's knowledge, projects, and AI all come together in one place.

21:38Kian Gohar: When everyone is working from the same shared context, teams move faster, stay aligned, and spend less time searching for information. What I like most is the philosophy behind it. It feels less like AI replacing people and more like AI helping people communicate, collaborate, and make better decisions together. Learn more about how Notion can support your business at notion.com slash thinkfast. That's all lowercase letters, notion.com slash thinkfast to try Notion today. And when you use our link, you're supporting our show. So before we end, I like to ask my guests a series of questions. Two are similar to everybody and one that's very unique.

22:21Kian Gohar: So Jeremy, for those who haven't started with chat GPT and large language models, where should they start?

22:29Jeremy Utley: I think for a lot of people, if they say, where should I start? We've talked to so many professionals who say, I've been meaning to try chat GPT. I've been meaning to try generative AI. And that to us is a shame. Every single listener to this podcast could have at least 10 hours of chat GPT under their belt. And if you find yourself going, oh man, I'm behind. Well, don't worry. You can get off speed quickly. Here's a simple place to start. This is something that every single listener can do right now. Think of an emotional human decision you're trying to make in your life. Something that you would ordinarily ask a partner or a friend or a spouse or a colleague about.

23:11Jeremy Utley: Okay. Think about what that is. Go to ChatGPT and say, hey, I'm trying to make this decision. Will you please ask me three or four questions before giving me your recommendation? If every single listener will do that one simple activity, they're going to have what we call a personal epiphany that's going to have cascading impact. All of a sudden, they're going to start thinking, could ChatGPT do this? Could I ask ChatGPT about that? Right. But it must be emotional. It must be deeply personal. and less be the kind of thing that you would ordinarily ask another human being about. Get ChatGPT to ask you three or four questions about it before giving you advice and start the snowball rolling that way.

23:57Kian Gohar: Question number two and three. Kian, I'm going to ask you because Jeremy has previously answered these on an earlier episode and I encourage everybody to listen to that episode where Jeremy and I talk about his book, Idea Flow. So Kian, who's a communicator that you admire and why? And you can't say ChatGPT. I'm going to give you two answers. One is Peggy Newton, who is a columnist for the Wall Street Journal. And I so deeply admire her elegant prose and her personal anecdotes that make complex topics readily understandable. She was a speechwriter for George Bush Sr. And she developed the phrases that we became very familiar with during his presidency, like kinder, gentler nation, or read my lips, no new taxes.

24:46Kian Gohar: and the politics and the policy aside, the ability to translate big ideas into a few short words is just so masterful that I always enjoy reading her articles in the Wall Street Journal, regardless of the politics or the policy. And the second person I'd recommend is Sam Horn. She is an author of a book called Kung Fu and she is a complete master at teaching which words to lose and which words to use to convey meaning without tripping over yourself and creating unintended arguments. And these two are role models of how I think about communication. Thank you. Both of those recommendations, I hear you talking about words and the ideas that those words bring about.

25:36Kian Gohar: Both of those are individuals who have mastered that craft. Because words are really conversations that are the seeds of innovation and how we think about solving big problems. And if we don't get the words, then we won't be able to actually get to the meaning and connection of what we're trying to solve for. Final question for you, Kian. What are the first three ingredients that go into a successful communication recipe? The first ingredient is something that might seem counterfactual when we're talking about a communication recipe. And that is listening, listening to the environments, listening to the context and listening to what your counterparty might be thinking.

26:14Kian Gohar: And so trying to understand the context by listening is super important. The second one is to know your audience. Who are you speaking to? Who's on the other side of the table? Who's in the room? What are their priorities and what are their goals and so that you can think about crafting your message in a way that resonates with them. And the third thing is to know your end goal. How do you want the audience of your intended communication to feel after the communication is over? Whether that's in person, whether that's in a meeting, or whether that's online. How do you want the audience receiving the communication to feel.

26:54Kian Gohar: And then you work backwards from that to structure the flow and the necessary ingredients to make your communication easily relatable, understandable, and to land. We have certainly heard the notion of knowing your audience and being critical. I really appreciate and like this idea of backward mapping. Start from the outcome and then we have to build and do and say and emote to get to that outcome. Thank you for that. And thank you both, Kian and Jeremy, for the conversation. I find your work fascinating and super helpful as generative AI becomes more and more commonplace. Further, I love how the concepts of effective communication and conversation and critical thinking can help our partnering with AI to be better, more creative problem solvers.

27:48Kian Gohar: To learn more about the results from Kian and Jeremy's work, go to howtofixit.ai and check out their article in HBR. Thank you so much. Thanks for having us. Thank you for joining us for another episode of Think Fast, Talk Smart, the podcast. It was great learning from Jeremy and Kian. Among the many things they shared, the one that stands out most to me is that AI is most powerful when we treat it as a collaborative thinking partner rather than simply a tool for finding answers. To learn more, please be sure to dive into our library of past episodes wherever you get your podcasts. Our back catalog is full of helpful communication tips and tools to help you be a better communicator.

28:35Kian Gohar: This episode was produced by Katherine Reed, Ryan Campos, and me, Matt Abrahams. Our music is from Floyd Wonder with special thanks to Podium Podcast Company. Please find us on YouTube and wherever you get your podcasts. Be sure to subscribe and rate us. Also follow us on LinkedIn, TikTok, and Instagram. Before we wrap up, I just want to say thank you for listening. It really means a lot to hear how people all over the world are using these ideas in their own lives. It inspires me and the whole team that brings you this show. If you want more episodes and resources, feel free to follow, subscribe, and explore past conversations.

29:12Kian Gohar: We're grateful for your support of Think Fast, Talk Smart.