First impressions of GPT-4o

0:05Welcome to Practical AI. If you work in artificial intelligence, aspire to, or are curious how AI-related tech is changing the world, this is the show for you. Thank you to our partners at Fly.io, the home of changelog.com. Fly transforms containers into micro VMs that run on their hardware in 30 plus regions on six continents. So you can launch your app near your users. Learn more at Fly.io.

0:42Hello and welcome to another fully connected episode of the Practical AI podcast. In these fully connected episodes, we keep you connected with everything that's happening in the AI world and help you find some resources to level up your machine learning game. My name is Daniel Whitenack. I'm founder and CEO at Prediction Guard, where we're safeguarding private AI models. And I'm joined, as always, by my co-host, Chris Benson, who is a principal AI research engineer at Lockheed Martin. How are you doing, Chris? I'm doing good today, Daniel. How's it going with you? It's all good. Yeah, I got the chance last week to visit Boston and see a bunch of cool stuff, tour a few labs around MIT, which was a lot of fun.

1:29I toured a couple labs where they're using AI to make proteins, like drug candidate proteins. Very cool. So the idea is one of the companies literally named AI Proteins. Hopefully we can have them on the show sometime. I requested their CEO while I was there. He was giving us the tour. But yeah, the idea being that you can use various AI driven methodologies to explore the space of proteins for drug candidates and they're kind of binding to certain stuff. I'm not a biologist or anything like that. But then they take those and then synthesize them in the lab and test them and eventually hope to get them into drug candidates and through FDA testing and all of that stuff.

2:22So it's pretty cool. I would love to have them on the show. So I've loosely followed that field over the last couple of years, largely because someone that I used to work for is a chemistry PhD from Harvard, but is very familiar with biotech. So he's kind of kept me up to date on some of that. It sounds fascinating. I know that drug discovery is really all about AI these days. I think that's where all the action's happening in that field. Yeah. And it's pretty amazing, at least from what I've heard from a couple of those companies, just the hopefully the speed, like the orders of magnitude faster that they'll be able to explore the solution space, I guess.

3:06So testing, you know, thousands and thousands of drug candidates very quickly rather than maybe, you know, a postdoc or a PhD testing only a handful over the course of many weeks or even years. They're able to do things much faster, which is really interesting. And of course, they're exploring that really useful application of that technology. But I guess this is one of the reasons that some people might have sort of ethical concerns with some of this stuff, because it's kind of like you can apply the technology in a really positive way and explore drug candidates. I'm sure you could also think about things that would be harmful to humans and even think about like biological weapons and that sort of thing and explore that solution space in the same way.

3:58And those sorts of things don't need FDA approval. So yeah, I imagine that there's people smarter than me that have thought more deeply about those concerns. And I know it was mentioned, I think, in our last round of interviews about Mozilla's report on AI this last year. We had an episode on that. But yeah, I was thinking about that while I was there. It sort of cuts both ways, I guess. It does. Just since you mentioned that, I know in kind of the defense and intelligence world, with AI capabilities being the great equalizer, the idea of malignant forces in the world deciding to focus on such things, which incidentally is very illegal under international law.

4:40Right. But certain places in the world don't care so much about that. And so we'll have to see. I've had a lot of conversations about the very good and the very bad about AI with folks lately and what an uncharted world we're moving into at this point. Yeah, yeah. Well, I'm very happy. At least the people that I've run across are quite ethical and moving towards things that will hopefully benefit us all. But speaking of benefit to many people, there was something rolled out this week that definitely caused a bit of a stir and also instantly appeared on a bunch of people's phones and devices. And that was the next GPT, GPT-40, standing for Omni, if I got that right.

5:33GPT-40 Omni. I don't know the full background of that naming, if it's meant to evoke omniscience or... I think the explanations I've seen have been about multimodality. Yes. You know, just the fact that it can photographic, video, voice, everything. and yes, it was quite a release. You know, everybody's been talking about upcoming, the expected release in the summer potentially of GPT-5. You know, interestingly enough, this came out and was quite, even though it's still part of that four family, it's had quite an impact. I know that in the last week, I will say that it's been the most open app on my phone pretty much around the clock.

6:15Yeah. Starting to feel like a family member because it's involved in all of our family decisions, trying to, things like getting a leak in the drywall and trying to use it to do something as mundane as figure out the plumbing concerns. And it seems to be, you know, when my wife and I are talking about household things, we now have ChatGPT as a third party and all those conversations. It seems to have supplanted my daughter. I'm not sure she likes it very much. Yeah. Well, since you have been getting hands-on and using GPT-4.0 quite a bit, for you either in the announcement or in your own use of it, what are those things that stand out as the things that have changed from, let's say, GPT-4 to GPT-4-0?

7:02Well, I don't have a list of things in front of me at this moment or anything, but things that I've certainly experienced is it is much faster than just GPT-4 had been. It's able to respond very quickly in any of the modalities that we're talking about. And when you say modalities, easier meaning sort of text, speech, image. That's correct. It seems much faster. I haven't measured it across the board, but I think the thing that's been notable in my own workflow of it is that I'm not having to wait around and kind of figure it out. And, you know, before this week, I'd kind of say, okay, I'm going to get onto GPT-4 and ask it a question.

7:40I kind of stop everything when I'm doing and do that. And I think the difference is the thing that's really impacted me the most is maybe the subtleness of no longer waiting around, being able to do it just by speaking and being spoken to. And it's no longer a stop and do something kind of activity. It's now as I'm doing it, as we're in the middle of conversation, it just becomes part of the conversation. I don't tell my wife, hold on one second. I'm going to check real quick on this question with GPT-4 and let's see what it says. And then we can, you know, take that into account as we talk. Now it's just like right there at the kitchen table, we just do it.

8:17So third party in the conversation. Dueling GPTs. That's right. That's right. Yeah. I think that some of the main features, if people haven't been following it quite as much in the news, which I'm sure a lot of our listeners have been following it quite closely. One thing that they focused on was speed. So in particular, with the voice response, when your voice is kind of recorded in, then they're talking about responding in milliseconds rather than I think before it was a few seconds, something like that, which, of course, is much faster. I think in general, it's a fast model, in my understanding, in terms of response and streaming across the modalities.

9:05Also, it's in terms of access, both account-wise and cost-wise, another drop in cost as far as the cost for the performance goes. So that's a trend that I think continues. And also, one of the things I was happy to see was most of the GPT models over time have penalized you basically in terms of token count for putting in languages other than English, because you would get higher token counts. And if you're charged by how many tokens you put in or generate out, and let's say you're putting in Korean or something like that, then it's actually more expensive to use the tool in those other languages.

9:50So I think they, at least in my understanding from what I've read, I don't know if that's fully equitable at this time. But there was an effort to kind of correct some of those issues as they came up. Have you used the video features much? I have a bit and it's very good compared to things that have come before, but sometimes it seems to get amazing context and occasionally it struggles a little bit. I think it depends on how much context it's able to get out of the imagery. Yeah, I've mostly used this sort of image related stuff versus video. So audio and text and image is kind of what I've done.

10:32But yeah, they show a good number of things in the demo videos related to video and also even kind of combining one version of this running with another version and having interactions between the two. and interview prep with the tool and all sorts of cool stuff. So if people haven't seen it, I definitely recommend that people go and check out the demos to kind of get a sense of the performance. But yeah, it's overall quite impressive. The subtlety of being able to do these things with that reduced time and across modalities, you know, while it might not be whatever giant jump that upcoming GPT-5 would be, the fact that it's changing our behaviors and the way that we're using it in this last week and enabling things that just weren't practical before.

11:24I think that really makes a difference going forward to the point where I work with here in the Atlanta area. I work with some of the local universities and their various computer science colleges and schools and such. And I was at one on Friday for kind of a day long strategic planning meeting on computer science and where they were going with it. And we were talking about this while we've been talking about AI's impact, obviously, at any kind of computer science program. This may change not only what you can do, but education as well in a pretty fundamental way in terms of teaching and being able to do it in real time and stuff.

12:00And we had quite a rich conversation around, it lasted quite a while, around how we might be able to utilize these new capabilities in the classroom going forward, and also how it might change curriculum. So I think we're really starting to get to a point where I think a lot of new capabilities in education are right around the corner.

12:31If you're anything like me, you have a certain tendency to put things off until the very last minute. Seeing the dentist, going to the doctor, home improvements, that never ending chore list of yours. And while most of the time it works out just fine, the one thing in life that you really could not afford to wait on is setting up term coverage life insurance. You've probably seen life insurance commercials on TV and thought, yeah, I'll look into that later. No, later doesn't come. This really isn't something you can wait on. Choose life insurance through a ladder today. Here's what we love about ladder and while we allow them as a sponsor.

13:09They are 100 % digital. No doctors, no needles, no paperwork. When you apply for$3 million in coverage or less, just answer a few questions about your health in an application. Ladder's customers rate them 4.8 out of 5 stars on Trustpilot. And they made Forbes Best Life Insurance 2021 list. You just need a few minutes and a phone or laptop to apply. Ladder's smart algorithm works in real time, so you'll find out if you're instantly approved. No hidden fees. You can came to any time, get a full refund if you change your mind in the first 30 days. Ladder policies are issued by insurers with long proven histories of paying claims.

13:48They're rated A and A plus by A.M. Best. Finally, since life insurance costs more as you age now. Yeah, right now. Now's the time to cross it off your list. So go to ladderlife.com slash practical AI today to see if you're instantly approved. Again, that's ladder.com slash practical AI, L-A-D-D-E-R, life.com slash practical AI.

14:35I'm having a discussion with my podcast co-host Chris about the new GPT-4.0 model. What do you think we should talk about? That sounds like a great topic for your podcast. You could discuss how GPT-4.0 compares to previous versions, especially in terms of its capabilities and improvements, delve into its applications across different industries, like healthcare, creative arts, or customer service, and consider exploring the ethical implications, including privacy concerns and bias in AI. It might also be interesting to discuss real-world case studies or predictions about how AI will evolve in the future.

15:13What do you think your listeners would be most interested in hearing about? Well, a lot of that seems pretty interesting. You mentioned healthcare. What were you thinking in relation to healthcare? Sorry, I'm having issues right now. Our systems are experiencing heavy load. Please try again later. Well, at least I'm not the only one having issues. At least I know that Chad GPT 4.0 has issues itself at times. There's something slightly satisfying about that, I must say. Yeah, well, it was doing pretty good there until it had volume issues or whatever they're experiencing, and I got cut off. I'm going to call it a mental blank.

15:52You know, I have those from time to time. You know, I'm just going to say it. So what do you, Chris, our friend over in the chat GPT world, chat GPT 4.0, suggested some things about privacy concerns as related to AI. I probed a little bit healthcare related things, but it wasn't able to give me an answer and got bogged down. But it also mentioned privacy concerns. Yeah, have you thought about that as you've obviously been using the system? What changes now in terms of privacy now that we have 4.0 and not 4? How is it different, if at all? I think it is. And this is a topic that has come up quite a bit this past week in various online forums.

16:39There was a particular LinkedIn post. I'll try to find it and include it in the show notes if I can, that brought it up. And with us now talking to it and receiving it back, how does that impact? Is this recording? Is it not recording? How does this qualify? under different state laws. When we were busy typing it in and getting our questions back, while there were privacy concerns, it wasn't extending now to audio recording of voices, which is covered under state laws of all states in the US at least, and I'm sure many countries out there. What do you think? I'm just curious. I know neither of us are attorneys, but now that we're leaving our phones open to chat GPT and capturing people, I'm sure I've done it in public places a bunch this week.

17:23So how do you think that impacts? Do we need to tell everyone we're doing it? Okay, everyone, quiet. Everyone, okay, I'm starting chat GPT-4. It's weird because it's some of the same feelings I think people had originally when they started bringing Alexa's or Google Homes into their home. And it was sort of always supposedly not listening, but it had to be listening at least to get the wake word, right? So there was this awkwardness there in terms of what's actually being recorded and that sort of thing. I think the difference here, you kind of almost got there when you were talking about how you were using it in your everyday life.

18:00I think people can see that this technology, because there's a quick response. So there's, as I was playing that, like you could tell the first response that I got from ChatGPT was pretty quick. I would say it's still not quite like you and me talking. It's not natural, right? But it's pretty quick. And so there's this tendency then to think, oh, well, I can leave this on at certain times. Or like you say, have it as part of the dinner table conversation. You kind of then bring in these devices like the meta AI glasses. And like, maybe I just have chat GPT watching what I'm watching through my meta AI glasses and telling me about this or that.

18:40And so you've got all of these modalities coming together. It's recording in your kind of physical space, not only your voice, but potentially images and videos from your physical space. And all of that data is going over an API to OpenAI or Microsoft or however the Microsoft OpenAI conglomeration, that's not a word, works these days. But yeah, it's that embedding, I think, of the technology in the physical world or the clear application of that within our sort of physical world. And like you say, not pausing to go and pull up a tab and talk to ChatGPT. It could be ubiquitous and embedded in our physical world, I guess would be a good way to summarize it.

19:26To extend that a little bit, Sam Altman, the OpenAI CEO, one of the comments he had made this week in an interview was somebody was saying, when should you use it, I believe? And he said, oh, you should just have it on all the time. Just listen. And I'm paraphrasing him. I'm not quoting him. But the gist was never have it off. I know that was one of those moments that the privacy notion. At least right now, I'm operating under the assumption that it's coming into play when I and the people around me are familiar with it. And we've kind of made that choice to do that. But I certainly, you know, going back to the Alexa notion and stuff, I think this is going to continue to be an issue here.

20:06The Alexa stuff, we have those as well. Oddly enough, I don't find myself paying much attention to them anymore. I guess I've just gotten so used to them being part of the environment and stuff. But we'll see. Yeah. Well, AI meeting the physical world is definitely, I think, going to become more and more a reality at the Boston Logan Airport. When I was flying out this last time I saw, they had, you know, normally they have little boots where there's a person that's like your helper at the airport. Like if you have some random question about where the bathrooms are, am I at the right gate or how do I catch this bus?

20:45There's a helper. They just didn't have anyone there at the thing and then just relabeled it virtual assistant and just had a screen that you could push and talk to. And I know there's a good number of companies that are working on sort of interactive virtual agents for retail environments, that sort of thing. And then you have this crossover with the glasses and Rabbit R1 and Humane AI pin and Meta AI glasses and all this stuff. So are you becoming a cyborg, Chris? Are you mostly just keeping it in your phone? I think I've accepted the fact that it's inevitable to do that. I say that half tongue in cheek, half not.

21:26To that point, actually, it makes me think, you know, this is penetrating so far beyond people like us in this space. And I have a very good friend who I don't think would identify as a technology person. And she brought up the fact that, and this isn't even specific to ChatGPT4 or anything, but it is to your effect there. She brought up that they had pulled in, she and her daughter had pulled into a Chick-fil-A. And they noticed a sign that said robot crossing. And they didn't really know what that meant. But then they actually saw a robot delivering food. And now that robot, I'm sure at this point, doesn't have very sophisticated AI capability for interactions.

22:08It's probably pretty basic. But in the conversation I pointed out, it's inevitable. You have with so many, you know, as we pointed out a week or so ago that we're over a million models already in Hugging Face. and with these kinds of profound releases each week, it's only a matter of a very short time before even the most mundane retail experience is going to have both robotics and AI in that. And so all of those things raise the privacy concerns that we were talking about before. And they also raise cultural and just folks getting used to it, frankly. And of course, that inevitably led to the concern over jobs and such as that as is often coming up.

22:50But I think this is maybe the first year that it's moving so fast in terms of these capabilities that even I am trying to, I'm even struggling to take them in as they come out. How about yourself, even though you're in that profession? Yeah, well, I have definitely, it's even out here in the prairie in Indiana. The prairie. It's becoming the Silicon Prairie with Intel building their big factory in Ohio and new semiconductors coming to Lafayette, Indiana. So plug for the Silicon Prairie. Anyone out there can hashtag Silicon Prairie on Twitter and all the places. But even out here at, well, I'm close to a university, close to Purdue, but they've had food delivery robots here six to eight years now where they've had food delivery robots around campus here, which is cool.

23:42You just made me feel very parochial. Yeah, but it's cool. I don't know. It'll be interesting to see also which brands kind of embrace this and how they embrace it and put it into their retail environments and how that's associated with specific brands and how they create new experiences for customers around these things. It's quite interesting.

24:21What's up, friends? Do you remember when chat GPT launched? I do. It felt like the LLM was this magical tool out of the box. However, the more you use it, the more you realize that's just not the case. The technology is brilliant. Don't get me wrong, but it's prone to issues like hallucination on its own. But there's hope. There is still hope. Feed the LLM reliable current data. ground it in the right data and context, then and only then can it make the right connections and give the right answers. The team at Neo4j has been exploring how to get results by pairing LLMs with knowledge graphs and vector search.

24:58Check out their podcast episode about LLMs and knowledge graphs throughout 2023 at graphstuff.fm. They share tips on retrieval methods, prompt engineering, and so much more. Don't miss it. Find a link in our show notes. Yes, check it out. Graphstuff.fm, episode 23.

25:38Hey, so this week I'm going to have to migrate some services out of AWS ECS into a Kubernetes cluster that's hybrid between my cloud environment and a data center. What should I keep in mind? That sounds like a big project. Here are a few things to keep in mind when migrating services from A to S ECS to a hybrid Kubernetes cluster. One, assessment and planning. Start by evaluating your current ECS configurations and services. This will help you understand the dependencies, data flows, and necessary resources. Mapping out how these will translate into Kubernetes is crucial. Two, Kubernetes environment setup.

26:16Setup. Yeah, that was getting kind of boring. So maybe just tell me what cool song I should listen to while I'm doing the migration. Uh-oh. Oh, boy. Are you still there? No dice. All right. Well, man. A simple entertainment question. the open AI, you had your chance live on the practical AI podcast. And I got skunked both times. So yeah, well, we didn't rehearse enough yet. They did have some pretty cool videos on the chat GPT four O release. You know, they had several different ones, but one of them, they had two phones taught, you know, with chat GPT talking to each other and they introduced them and they gave them little monikers to differentiate between them.

27:03But I will admit, I tried that at home right after it got released. I saw that video. I was like, I want to try that. And it did not, I will confess, it did not work well for my end either on rehearsed. So I guess those ChatGPT folks at OpenAI have the inside track on smooth conversations. I'm sure it worked at one point, as most demos do. But still impressive. Nonetheless, I have to say I gave it a pretty complicated question there, maybe one that I could definitely use some help with. So yeah, I think it did pretty good at answering, of course, and was responsive. I'm wondering, Chris, what you think about now that we have GPT-4.0, what is the future of all of these different physical AI device gadgets that have come out in recent times.

27:54So there's been the Rabbit R1, there's been the Humane AI Pen, there's been the Meta AI Glasses, and probably others that I'm not even aware of. What's your thought on how this influences these sort of AI gadgets? While this is also a golden age of AI startups, it's also the bar keeps getting raised very rapidly and unexpectedly. So you can go from super cool to obsolete overnight. You can be one announcement away from a tough moment there for your product or service. So, you know, for instance, now that the world has had a little time to try out the 4.0 version and it's changed the way we do it a little bit, that's set a new bar.