Pushing Boundaries: GPT-4's Remarkable Achievement in Original Creative Thinking

0:00What can 160 years of experience teach you about the future? When it comes to protecting what matters, Pacific Life provides life insurance, retirement income, and employee benefits for people and businesses building a more confident tomorrow. Strategies rooted in strength and backed by experience. Ask a financial professional how Pacific Life can help you today. Pacific Life Insurance Company, Omaha, Nebraska, and in New York. Pacific Life and Annuity, Phoenix, Arizona. There is an absolutely fascinating report that was just published in Science Daily that was reviewing the power of chat GPT-4 specifically and comparing it to GPT-3 and how creative it is.

0:42So on the podcast today, we're going to dive into what exactly they were able to find in the study and what the implications are for AI in general. So the study overall was led by Dr. Aaron Guzik, who's from the University of Montana, and he found that ChatGPT, using specifically GPT-4, matches the top 1 % of human creativity, which was gauged by a widely respected test. So the research adds an entire new dimension to AI's potential to transform industries, especially those that are really heavily reliant on creative output. it. So essentially, Dr. Guzik, when he was doing this whole, you know, this whole research report, he is an assistant clinical professor at UM's College of Business, along with, he had a bunch of different collaborators on the study.

1:29So he had Christian Glide, Christian Byerge, and essentially what they did is they used what is called the Torrance Test of Creative Thinking, or the TTCT. And this is a really esteemed measure of human creativity, and it's been in use for decades. So this is nothing, you know, new they invented. This is very standard. So in order to keep the study impartial, Scholastic Testing Services, which is the one that runs this test, they were completely oblivious to the fact that, you know, AI was being used in this test. So they did not know that GPT-4 was going to be used or was in this, and they just graded everyone the exact same, you know, keeping it essentially a blind study.

2:08So GPT-4 proved to be a very elite performer on the creativity scale. And it scored in the top percentile for fluency and originality, which I think, you know, demonstrates its capacity to generate a large number of new ideas. And it also performed a little bit, so it, like, it did a little bit less well on the, when it came to flexibility and the capacity to generate diverse kinds of ideas. And when I say less good. Like, it performed in the 97th percentile, so still absolutely incredible, but not the literal top percentile, which it literally performed in the top percentile for originality. So Dr.

2:50Guzik said that some of his UM students also were in the, you know, the elite category in there, so he's, you know, kind of bragging about, which is funny, but, you know, it's pretty unequivocal that GPT-4 managed to outdo a majority of all college students nationwide. So of course, there's some people that can still beat it. There always will be. There's always someone better at doing something. But overall, on average, this is in the top percentile for creativity and in the 97th percentile for flexibility and some other things. So I think this team overall, they released all of these reports at the Southern Oregon University of Creativity Conference in May.

3:32And this is what Guzik said. He said, we presented the results, without much interpretation, but we shared strong evidence that AI seems to be developing creative abilities on par with or even exceeding human abilities. So I think a really interesting aspect of this study was one question that Guzik kind of posed to ChatGPT. He said, you know, he asked what it would signify if it outperformed everyone else or if it performed really well on the TTCT and the AI, this is what Chad Chibut said, quote, it said, we may not fully understand human creativity, which I believe is correct. We may need more sophisticated assessment tools that can differentiate between human and AI generated ideas.

4:15I think that's really interesting. I think that kind of shows a really, like it reflects a lot of the nuances that are becoming increasingly essential as I believe AI evolves and kind of converges with our daily lives. But I also think that it emphasizes the need for us to understand the mechanisms of AI creativity better and also to kind of develop robust assessment tools for that as well. Dr. Guzik, who is a longtime enthusiast of creative thinking, he kind of got this study kicked off after he saw some really surprising and novel responses from ChatGPT when he was using it. And the TTCT's prompts designed to resemble real-world creative tasks were used to put, you know, essentially ChaiGPT to the test.

5:02But Guzik was, like, absolutely blown away by how well GPT-4 generated original ideas, which is essentially a trait really traditionally given to the human imagination, right? Before we saw all of these tools, people were saying, but humans are the only ones that can imagine or be creative or whatever. And now we're seeing, you know, AI doesn't do such a bad job of, you know, hitting some of these benchmarks in some of these tests as well. But ChatGPT's ability to really evolve was also shown in its performance improvements over its predecessor. So this isn't the first time that Guzik has tested an AI on this.

5:39Guzik actually did some research where he conducted the same experiment on GPT-3 a year ago. And what's interesting is GPT-3 did not even score as high as humans in tasks involving original thinking. So this more advanced GPT-4 now outperforms the majority of humans, whereas GPT-3 couldn't even get as creative as the average human. So I think this really just shows what rapid advancements in AI technology we're making right now. I believe this is a paradigm shift. A lot of people don't understand this. And I think just with the fact that GPT-4 is now going to be publicly available to everyone in the world through the API, like this thing is just opening up, I think people don't understand how much better quality AI responses are about to become in the next month or two, in the next week, honestly, as everyone that already has GPT-3 integrated into their applications is now going to be switching to GPT-3.

6:34before, because it's just a simple change of their API, I think we're going to start seeing some even more incredible responses from a lot of different tools. And this is what Guzik says specifically kind of on this whole thing. He says, for me, creativity is about doing things differently. One of the definitions of entrepreneurship I love is that to be an entrepreneur is to think different. So AI may help us to apply the world of creative thinking to business and the process of innovation. And that's just fascinating to me. So all of this research, I think, really paves the way for universities and businesses to explore and integrate AI into their curriculum and workflows.

7:07I think that's absolutely essential. And I believe that with the necessary precautions that we obviously have to look at ethical guidelines. But I think that Guzik's research really shows the promise of AI, not just for a tool, but also for something that can be a catalyst for creativity and innovation. I think as we step into the future, where AI becomes a very central part in a lot of tools and a lot of people's lives i think this is going to be an absolute game changer so i think this is a massive step forward for gpt4 for openai and chat gpt in general but also i believe like for coming up with creative ideas this is going to be absolutely incredible in the future and it really levels the playing field in a way that some people like some people don't but it is what it is and i think this is going to be really interesting to continue to follow in the future