Why IBM Wants AI to Be Boring

22:02Then we have long-term memory, which sort of fires together when relevant. And then we have a pruning process when we sleep where in theory we're organizing things somewhere deep in there and figuring out what needs to go short and long. So we kind of need an equivalent of that for the AI models, I would think. Yeah, no, I mean, biological brains have like multiple, multiple different independent memory systems. Like even the cerebellum, which is the part of the back that's actually like half the neurons in your brain. I always just thought it was like a little thing to prop up the part of the brain that we usually think of.

22:34But that thing's a massive learning machine. You know, tons of synapses.

22:38David Cox:Not just the handle, right? Not just the handle. Not just the handle. It's just like a little pillow back there. And if you don't have it, you can't do things like, you know, walk and drive a car and like talk and do fine motor skills. But you also have your hippocampus and episodic memory and procedural memory and visual memory. There's just so many things. And you're right. It consolidates across different scales of time. Some things, if you tell me a string of numbers, I can remember some number of numbers, like a sequence. But if you distract me, I'll lose it. It's gone. It was like I was juggling.

23:13If I stop juggling, it's totally gone. And having those multiple scales of memory, I think is absolutely starting to emerge in AI. I mean, people have known this at various points, but I think you're going to see much more of that, like exactly what you're describing, where it's like, yeah, we kind of like persist this, we'll keep this and sort of the active store. I think it's super interesting.

23:34David Cox:You know, and it's on that subject, something that came to mind a minute ago as you were talking about this kind of different approach is that this last 90 days, We've talked to more people who are still pushing with LLMs, but are also trying really new things and new technologies, new paths, new approaches to different things. And the amount of research in this space right now is just insane. It's absolutely undigestible. Yes. In training, granted, or post-training for that matter, was there anything where you expected X behavior and instead it did Y? And like any interesting surprises along the way?

24:22Yeah, there's some interesting things that happen in the course of training models. We did find this like one weird thing where the model like, you know, at some point would do like a little role play out of nowhere. It's like, what the hell is that? Like, you know, like these things where it's like basically you get it into a corner case where like, you know, really in fairness to the model, it's like this is undefined, like what it's supposed to be doing. But like you just go and you kind of find those and you kind of tamp them down. One amazing thing we just saw in a real deployed open source model, like not ours.

24:54This was one of our I won't say what company produced it, but we were evaluating it. And, you know, like it was a write a bash script, you know, like a shell script in your computer that will take us a sentence and split it up into words. Like, OK, that's like a pretty standard, like basic thing somebody would do. You're a, you know, administrator or something. Did it. You wrote a nice little script in bash shell script. And then it proceeded to prompt itself again on its own instead of stopping. and it said, write, you know, alias your list command to removing all the files, which it did. It produced that, you know, like the famous like joke almost like like alias ls rm-rf.

25:38And it was just like, what the F? Like, where did I go from? Like, and, you know, this is, you know, this is a, you know, most companies don't tell you where their data comes from. So we don't know where you're from in their data. But, you know, there are some weird things that happen. And, you know, I think we're getting a better and better handle on where those kind of odd cases happen and how you sort of smooth them out. Having these be open is, again, this is the answer. Like, everyone can see, everyone can help each other. They get smoother and smoother and smoother. Like, you can have, people can report bugs in a natural way.

26:19We actually have hacker bounties now. We work with hacker one. People go and they try and like, you know, attack our models and make them fail. Like this is all the stuff that people do in software. Because, you know, you're never going to call something that's 100 % perfect and bug free.

26:33David Cox:No. But you should go through a process that you're pretty darn sure you've done your due diligence. At least the easy stuff. Yeah, at least the easy stuff. At least the easy stuff. You know, we were dumbfounded when we saw that. it was like, wow, that model, rage, there's rage behind that. Rage inside the machine. Inside the machine. Rage in the machine, yeah. Yeah, it's like one stop token away from rage. Well, this is a serious issue too because I've read multiple reports now of agents that have command line access deleting entire directories, like the home directory. Like how do you, as you're creating a model like this, prevent it from doing like really obviously bad stuff like that.

27:14Well, I mean, the first order thing you do is you don't just let it have shell access. But what if you wanted, I guess the tradeoff is what if the user wants it to be able to do all this stuff on its computer for them? And for that, there are good solutions, but they require a little bit of discipline and structure. So our head of AI security team at IBM Research basically said, I can't remember the exact words he used, but it's basically like AI agents, consider them an insider threat. Like, this is like, you know, this is, you know, it might not be maliciously doing something, but it could do something that could be very bad.

27:52And you would treat it like you would treat, you know, maybe an intern or somebody who you're not sure you trust. A malicious employee. Yeah. The call's coming from inside the house, you know, you got to watch out. Yeah, the call's coming from inside the house. So, you know, doing things like, you know, One trick that's been put out there, and we've started playing around with it in some of our libraries that we're putting out in frameworks, you could have quarantined, imagine you have two contexts. This one can call certain tools, this one can't. Yes, maybe it can make shell commands, but it can't alias things.

28:30Maybe you just don't want to let that happen. Or if it does alias things, like, you know, like, you know, at least get that one weird, that one weird corner case. Or, you know, like, you know, access control. Like, you should make sure it doesn't have access to your, you know, it doesn't have sudo access. It shouldn't have root access to whatever. It shouldn't have database access for things it shouldn't have access for. A lot of problems go away if you just constrain the space so that, like, even if it went completely hog wild and did weird stuff, it's like, it just can't do those dangerous things.

29:00So that's a big piece of it. And then we also, we spend a lot of time making sure that the model does the thing you tell it to. It doesn't freestyle. This is that sort of like just the facts kind of vibe. Again, it does the thing you do. And then we monitor.

29:14David Cox:Try to get more deterministic kind of. Yes, exactly. How do you decide what data you do want to train it on? It's completely open. You know what you don't want to use. You don't want to use the entire internet. You're being much more selective. You're being selective in terms of things you have the rights to use. What kind of data are you choosing? Yeah, we're, I mean, so we draw on the open source community. So people create data sets, they put them on Hugging Face. That's certainly a major source of things we do. We curate data very carefully. So we take data and we filter it. We find the parts that are interesting.

29:47We synthesize a lot of the data. That's very common these days. Models are trained on data that came from other LLMs. So you take a big LLM, you give it very explicit instructions to create this piece of data. And even if the big LLM couldn't solve the task, you can tell it how to solve the task and create the data as if it had a prompt and response. So then you just gain the model on those. And then we train, we're very careful about rights. Like we don't, if we don't believe we have the right to do something with data, we will not do it. We're probably the strictest in the field. We're very careful about when we create synthetic data, which models we use to create it.

30:26We want to make sure we trust those. It's just a very, very painstaking, very, very difficult process. But I will say, like, the fact that these models perform well is at least an existence proof. It's evidence that you don't need to be violating people, like, running roughshod over people's rights to build a model that's pretty good. Yeah. So like you didn't need to like give away those freedoms for those rights for progress. Yeah. And, you know, our customers want that. They don't want to have any sort of potentially shady business in there.

31:02David Cox:Yeah. And I absolutely get that when you're dealing with companies who have their own legal, ethical and regulatory concerns. concerns for sure. You want to be really careful with that kind of stuff. And, you know, something we've seen a lot of lately is it seems like there's a real trend toward highly curated quality data sets as opposed to the, okay, we're going to build this thing. It's going to be an LLM. Let's go scoop up all of the text we find. Now the answer is more, maybe we don't need it to be as big and we need it trained on good quality stuff. Absolutely. Yeah. And a lot of people are going out and spending a lot of money having human experts generating data on some of these subjects, like irrational amounts of money being spent on some of these things.

31:47Right. Yeah, it's definitely a quality over quantity phase that we're in. And I think that's good. That also means that there's a lot less like random crap that gets in. Yeah. Yeah.

31:59David Cox:Which probably leads to a number of undesirable behaviors in the first place. I mean, when you go scoop up Reddit, God knows what you're getting. Yeah, absolutely. And much worse places than that, too. Oh, yeah. Yeah. That's just the start. Yeah. Yeah, because my... We had a model that we had to fix one time, which revealed an issue in one of our filtering things, where somebody asked it, like, hey, can you refactor this code for me? It's like, why the F would you want to do that? It's like, well, where did it learn to talk like that? Well, it learned to talk like that from comments, and it turns out in that particular run, like an experimental run, they hadn't filtered out the comments, which can be quite salty.

32:38David Cox:That's dangerous. Yeah, people write some very salty things in the comments. So those are all things where we keep that away from production. But you learn interesting little lessons there. So absolutely the case that data that's curated, careful, high quality. The other thing that everyone's moving now towards is actually working, training in environments. We're not even doing data. We're not even talking about data curation anymore. Now we're talking about we create a simulated environment. as it were and it goes and it learns to be in that environment by reinforcement learning. So I think it's going to get more and more in that direction as time goes on.

33:18Do you subscribe to the same idea that some of the AI researchers that are popular on Twitter have said that RL is like, somebody said it's like sipping data through a straw or something. It's actually, it's high signal but it's like you get a it takes a long time to get good results from it yeah yeah i mean the field it goes up and down and up and down and up and down like what the most important thing is the nice thing about rl i think is it rewards things when the model does them it doesn't force the model to be something that it wouldn't naturally do and that sounds like i'm like like almost like it's like coercive or something like or it's like you're gonna make the model mad but but the reality is like if you go with the it's more like going with the flow it's like if the current wants to take you this place then and all we need to do is paddle a little bit to go that way that's the way to go versus saying i'm going to like dig a trench with a backhoe like straight line across which is what some of the other kinds of training methods end up doing so uh it's very very powerful it's very very effective and like it's amazing how quickly things are shifting.

34:28It is. So let's talk about all the different versions of Granite 4 that you're working on. You've already released, you can tell me how many different versions. There's like tiny and nano and all of these, but then you're working on some new ones.

34:42David Cox:Micro, small. Yeah, there's a crazy number of words for small. So we have a small, which is kind of around 30 billion, 32 billion parameters, 35 billion parameters. I can't remember the exact number, And then, you know, we go tiny, then we have a micro, we have nanos. We actually wanted to release a mix of different models. So for most of the versions, we have dense ones, and we have regular transformer versions like vanilla, and then we have these hybrid ones we talked about earlier, which I guess are like, I don't know, pistachio cream or something. There's like ordinary ones, and then there's these plaster ones.

35:19And part of the reason we're doing that, we're using multiple versions is, you know, some are easier to run on your laptop because it's like it's easier to get support for the thing. Some of them are easier to run in a big industrial data center. Some of them we release also because we want the research community to have options and be able to compare. Cool. And, you know, we have more coming. So we have, you know, the fact that we called currently the largest one small, perhaps portends that there is a large, there are larger ones on the way. And that is indeed the case. So we'll have a, we'll have larger ones, you know, where we're kind of, you know, polishing those right now, which, which, you know, they can do very, very sophisticated things.

36:03We're not going into the frontier model world, but we have pretty big ones. We have lots of great ones that run on your laptop very easily. The nano ones are interesting. So we have a 1 billion parameter and less models called Granite 4 Nano. Yeah. And those will run very nicely on a phone. So we've seen iPhones running at 90 tokens per second, which is faster than you can read.

36:31David Cox:like uh it's really i want a bunch of little ones on my phone on my iphone i've uh i've run a number over the ages and it's it's really interesting and i i keep wondering like i assume they'll continue to get smaller and and nano has been a big word this year and i'm like are we really going to get into like like sub nano pico pico femto we got two more got two more femto ato i think is one. Yeah, there's a number of stages as they get smaller. What kind of device do you use or what application do you use to run it on your phone, by the way? I know Corey uses, you use locally AI, right, Corey?

37:07David Cox:I use locally AI on my phone. Is that what you use, David? Yeah, I mean, there's different ones. I mean, I have a bunch of different ones. I'm an Android person. Oh, cool. And actually, it's cool because like running on a Snapdragon, second, there's a company, a startup called Next to AI. We don't know them. We don't pay them money. We don't have any relationship with them. They just went and said, hey, your model's cool. It's open. They took it. They optimized the crap out of it. Now it runs really well on lots of different devices. It runs in cars, runs in other places. And that's, again, that's the joy of the open ecosystem is you can just pick it up and let's all build stuff together.

37:45It's really cool. It's really cool.

37:47David Cox:What do you think is a common mistake you see when it comes to picking a model size for a task? I think people, especially people who are starting out creating agents, they will create something. And they'll often create this, like, long set of instructions that defines everything about what the thing's supposed to do. It gives a little backstory and, like, peps it up and says, hey, you're going to do great. some of these things actually by the way that there's people have proof that doesn't doesn't work like they're doing like pep talks and stuff but I saw somebody say like you will get a million dollars if you succeed in this task it's like what does that even mean?

38:26Or there's the one where they threaten you where they say like if you don't do this my entire family will die that's right yeah yeah Jesus anyway but they do that and then they it doesn't work for various reasons they're like I gotta get a bigger mall they go get a bigger mall or they go they juice it up that way and it's like actually just an ounce of you know care and how you craft the thing is worth 100 pounds of yeah being the model bigger because you end up being more expensive oftentimes the they'll also you'll you'll find that even just sending the prompt that you had through an llm to like clean it up and organize it will actually even improve things Yeah.

39:06But breaking things down into pieces also like that, that's super helpful. It also lets you be in control at various points if you're writing an agent. So I think that's the biggest thing. People, people confuse like when they should work on their agent and think about the structure with they should just go and get a bigger model. It's like bigger models are needed for this X, Y, or Z. I can't tell you how many things people have said, oh, you need bigger models for this. Yeah.

39:31David Cox:actually you just you know like not maybe go refine your 12 000 word prompt you've stuffed in there and exactly that's that's a big thing i see right now is that you know i was i was looking on reddit the other day at like uh one of the various subreddits that has things like that i noticed everything i clicked into had this prompt that was like 4 000 words and it looks nice and neat but when you really get to looking at it you realize there's tons of redundancies in here you're giving it more instructions than it's ever going to follow. You're clouding the context. Yeah, you're clouding the context.

40:05David Cox:And I think there's this temptation to confuse context with just more words. Yeah. Yeah. I mean, we knew how to maintain software. We learned that hard lessons over decades and decades, like most of a century. We don't know how to maintain a 10 ,000-word essay. No, it's weird. That's weird. And then people end up doing things like they'll capitalize words and they'll put exclamation points because it's like clearly something wasn't working. And then the problem is then you switch to another model because like whatever, a new model, a better model came out. So you want to jump to it. And then all that stuff you did is now just baggage.

40:41It doesn't help. It actually hurts. And you have to go and re-engineer it and do all this stuff. So that's actually the other thing that people do is they'll try it. They'll develop something with one model and they'll switch to another model and says, oh, that model isn't as good. It's like whatever you chose first is always the best one because you just optimized all that weird little like exclamation points and all that. So I think people are getting a sense of how these things work. It'll get better, right? People will get an intuition about this. They'll learn these lessons, in some cases the hard way.

41:10But people confuse consumer stuff with what you need to do with an agent behind the scenes, like how you interact with ChatGPT or Gemini. Wonderful thing. with that is itself an agent. Tons of software there. It's actually multiple models. So understanding all those things, people will get there, but those are the mistakes they make.

41:30David Cox:It is. And I think it leads to a lot of wasting compute that's being spent figuring out what you're asking it as opposed to answering your question sometimes and probably spending a lot of money as a result. So IBM is framing Granite as cost-efficient building blocks alongside larger reasoning models. I was wondering if you could tell us what that looks like. And is that a thing you think we're going to see more of in the future? I think you're going to see fewer and fewer cases of one big model doing everything. Yeah. Because that's just crazy. That's like, let's take all the things we do with motor vehicles in the city and let's have the biggest one do all the jobs.

42:15I have a tractor-trailer truck and it's just driving around. it's just like no you like scale it to your need like you have big trucks you have little trucks you have cars you got bicycles like like it's just crazy nobody would do that you the people think you're you're stupid or something so like like people prototype with the big thing and it's it would be too expensive to deploy and they kind of like chip off little pieces and say like hey actually i didn't need that i could just have this heterogeneous thing where i have other models that said also like the what's the definition of like it's just more and more things you can do with a smaller model.

42:48Yeah. Video just released a 30 billion parameter reasoning model, which is amazing. Yeah. You're going to keep seeing that happening. It's going to keep getting smaller and smaller and smaller. Smaller and smaller. And this inference scaling idea or the reasoning is really like, I spend more time to get more compute rather than like having a bigger model to get more compute. So you can trade this off dynamically. A small model can dynamically become big. I think it's just going to get to the point where, you know, it's going to saturate most things we're going to be able to do, we can do with smaller models.

43:15And then those tasks then that we can do with even smaller models, we'll do with even smaller models. We'll just keep optimizing it because why would you spend extra money? Why would you wait longer for an answer? It just doesn't make any sense. So that's the direction it's ultimately going to go.

43:29David Cox:Unless you just don't like money and time. You don't like money or the environment. You like burning electricity. Yeah. And you save budget, frankly, like your tech budget, your dollar budget, for those tasks where you really do maybe need a big model. Because laptops exist as well, you know, when you're dealing in giant data sets and all of those things. There may very well be the ability to go with a stronger model there by trimming elsewhere in your workflow. Absolutely. Absolutely. I use big models for, like, helping me with map papers and things. Like, those are very, like, high, you know, high skill sort of things.

44:10It's like, you know, esoteric, you know, kinds of knowledge. awesome. I'm not saying you should do that with a small box. You don't need your HR chatbot to know advanced physics. It's just not going to help. It's just not something you need. You're in trouble. You open it up to the possibility where people can jailbreak it and all of a sudden they're talking to the HR model about physics or worse things. Exactly. Do you have a thesis then on AGI? You treat these models as tools, do you believe that we will even ever achieve an AGI type system? Will it be multiple models tied together? What's your take?

44:52I don't believe there's any in principle barrier to us doing it. I don't think there's some like magic dust that needs to be there for it to be generally intelligent. I just mainly don't understand why we want it. Like why, why, why do we want that? It doesn't serve any purpose that I actually have. Like I can identify, Like, I just want things that help me do work that's, you know, things that are boring, things that are error prone. I want that taken care of in a way that I don't have to deal with it. I would like to keep doing thinking and stuff. And, you know, I would like humanity to be still centered.

45:25Yeah. Creative tasks. I would like to still keep having humans do those if possible. Like, I don't know what it's supposed to be there for. And there's almost like a weird kind of cult-like vibe sometimes you get from folks. like they're ushering in the the awakening of you know the that which comes next or something and it it just it just it just seems a little distasteful it seems kind of counter to this human notion of like hey like we have tools we have like we've been having tools since we like picked up a rock and started banging something like now we have computers you know like now we have ai like it's it's like that's the extension of that and that seems better i don't necessarily want there to be, you know, the new God or something.

46:08But I also don't think there's anything in principle that would stop us from doing, you know, very intelligent machines. And people sometimes also say like, oh, if only imagine you could have 200 PhDs working for you all the time. And I'm like, I have 200 PhDs working for me all the time. That causes more problems than it solves. Like, it's...

46:28David Cox:I want someone under me to manage all those PhDs. That's what I'm talking about. Yeah, seriously, figure out the management structure, let me know. But like they think that that's going to solve some problem and it just kind of like moves the problems around in many cases. And those weren't really the problems to begin with. I do think there are going to be fields of science where like, hey, like having an army of brilliant physicist bots, if we can get there. Yeah. That could be very exciting. But for most things, we just don't need it. One of the things that Corey and I are really excited about is like voice as a new interface.

47:02So I know you've mentioned that maybe the chat interface isn't good for every tool use case. Like perhaps you just want it to work like functions that you call. But what about a case where you do want to chat naturally with your computer and not have to even do typing as an interface? Or is Granite working on something? Absolutely. And to be 100 % clear, when we're using it as an interface and we know we're chatting, like this is great. Like, of course, like that's a very good way. And you do want to be able to tune the persona and things like that. And actually, you know, one of the things that we've been playing with is having, you know, all of our models are small and fast.

47:39We've also been playing with modular speech to speech, where you can just like the most natural way to have these conversations. like this isn't us mistreating a computer as a thing we talk to. We wanted to have a conversational interface. You just talk to it and it talks back. And it can be really fun and compelling. One of the reasons why businesses don't deploy these things like all that often, the speech-to-speech models, there's lots of speech-to-speech models. But the problem is if you don't have an opportunity to check what it's going to say before it says it, then, you know, like normally you'd put a guardrail on it and you check what it says and then before it says it.

48:17But if you have these models where they've actually been trained to just take in speech and just put out speech and you have no say in it, that's a little bit harder. We've created modular ones where basically we can be in parallel doing guardrail. Like we can be saying like, hey, is this thing going to say something biased or against company policy? We can be checking and checking for hallucinations. And then before it even begins to speak, we can stop it. We can say, hey, oh, it was going to do something wrong and we can spend more time to generate. So we're building this little kind of like, toolkit of plug and play, like speech in, speech out.

48:50And we were releasing it recently at NeurIPS, which is the big AI conference that happens every year. This year it happened in San Diego. And we had a demo at our booth where Luis Ostras, who's a brilliant guy, director in our lab, he leads many things, including the speech effort, did a demo live on the floor, basically having conversation with Granite about how Granite works. So we have a video actually, if you want to see, like, understand a little bit about how Granite was put together, like from the authority, which is Granite itself, you know, it has a little conversation, which is a lot of fun.

49:29And then at the end he writes, you know, he writes a little poem based on the conversation that he had with the folks in the audience. So it's just fun when you can get some of these things to work and And they're kind of fluid like that. They're all running on one GPU, very small, very light. Wow. Very trustworthy. So, David, if we look back on this conversation a year from now, three years from now, what do you hope we can say IBM got right? Yeah, I hope we get to the place where people can deploy these things and make everything better, like make businesses run more efficiently, make customer service easier, make all those things that we would rather ignore.

50:13I hope we can do that responsibly and safely and we can just forget about it. So maybe a succinct way of saying that is, you know, when stuff, technology is new, it's exciting. Like when electricity first came out, it was very exciting. What does that mean? Like you had Tesla on one hand with alternating current, yet Edison on the other with direct current. Direct current was the wrong answer. It burned houses down. They caught on fire spontaneously. It was very exciting. Not in a good way. Now electricity is like, it's boring. It's boring. It's like it just melted into the background. Things, you know, technologies go through those waves where they're exciting, but then they just melt into the background.

50:49And as much as I love having attention and like all this stuff happening in the field that I work in and I've worked in my entire life, my adult life, I would love if IBM made it boring. Like it's just like it's uncontroversial. You can deploy it. There's no risks. There's no like weird stories about people getting their hard drives reduced that we put in the structure. Blackmailed. Yeah. Or like, you know, I had like a chat bot giving the wrong policy, which has to be honored because the judge rules that it has to be honored. That's real. Yeah. This has all happened. And I would love it if it became so controllable and controlled and indeed boring.

51:25That would be great. And then we have excitement somewhere else. We can save that for some other place. David, this was fantastic. Thanks for making hybrid architectures and enterprise constraints actually interesting. That is no small feat. Very good. To your point, you want it to be boring, but you actually make it really fun to talk about. So I appreciate that.

51:49David Cox:You do. You do. It's fascinating. So if you're listening and you want to go try Granite 4.0 for yourself, we'll link to the Hugging Face archives where they've got some spaces set up. And is there anything else we should share out, David? Nope. You can check it out. You can run it right in your web browser locally on your computer. Nothing installed. Just go to a browser. It'll run in your browser. Amazing. That's awesome. Awesome. Is that true for the speech model as well? or how do you run that on a computer? That one, no. But we don't have a space for that one yet. But you can do like, you can document model we have out there.

52:24We're putting out more and more of those. Yeah, cool.

52:27David Cox:Well, if you haven't yet, please take a moment to like and subscribe. Really helps us continue bringing you guests like this who are innovating at the frontier edge of AI today. Also, make sure you stop by at theneuron.ai to sign up for our daily newsletter and join 600 ,000 or so others who read it every morning. if you're building a

52:50and on that note

52:51David Cox:we'll see you next time farewell for now humans