#321 Nick Frosst: Why Cohere Is Betting on Enterprise AI, Not AGI

0:00I first got introduced to neural networks when I was at the University of Toronto in undergrad. Famously, OpenAI jumped on the Transformers, scaled it up, the GPTs. They're working on neuroevolution, just came out with this book. Are you looking at, does Cohere explore these new, or not necessarily new, it's not new, but these other schools of AI to see how they might be applied? This episode is brought to you by Tasty Trade. On Eye on AI, we talk a lot about how artificial intelligence is changing how people analyze information, spot patterns, and make more informed decisions. Markets are no different.

0:45The edge increasingly comes from having the right tools, the right data, and the ability to understand risk clearly. That's one of the reasons I like what Tasty Trade is building. With TastyTrade, you can trade stocks, options, futures, and crypto all in one platform with low commissions, including zero commissions on stocks and crypto, so you keep more of what you earn. The platform is packed with advanced charting tools, backtesting, strategy selection, and risk analysis tools that help you think in probabilities rather than guesses. They've also introduced an AI-powered search feature that can help you discover symbols aligned with your interests, which is a smart way to explore markets more intentionally.

1:40For active traders, there are tools like Active Trader Mode, One-Click Trading, and Smart Order Tracking. And if you're still learning, Tasty Trade offers dozens of free educational courses, plus live support from their trade desk reps during trading hours. If you're serious about trading in a world increasingly shaped by technology, check out Tasty Trade. visit tastytrade.com to start your trading journey today i'm going to myself tasty trading is a registered broker dealer and member of finla nfa and sipc i mean you have a really interesting background you were jeff hinton's first hire is that right at google brain when he was over there in toronto yeah oh in toronto yeah and he very generously did it is to explain neural networks to me and gave you know i was completely um this is 2017 so that that really woke me up to what's going on uh and then you know since then i i started of the podcast and i've i've talked to a lot of people um but uh you know the the the big companies um are focused on agi you guys are not you're focused on the more practical problems of getting a ai to work in the enterprise so um yeah why don't you introduce yourself give your background how you ended up with jeff at google brain uh and then how you joined coher or founded coher with aiden i don't know how many of you were there at the founding and what coher has been doing for the last couple of years and then we'll talk about everything that's in the air these days okay yeah happy to yeah so i'm nick frost i'm a co-founder of Cohere.

3:59Prior to this, I was a researcher. I had the pleasure of working with Jeff Hinton at the Toronto Google Brain Group for a few years, working on foundational research. I first got introduced to neural networks when I was at the University of Toronto in undergrad. I was pretty excited about the idea of AI at the time and learned about neural networks in cognitive science department actually and uh thought they were they made a whole lot of sense and were really exciting i remember learning about them in i think 2012 or something which was shortly after alex net which was the the paper that really showed yeah that that neural networks were the best form of machine learning i think at that time right um and i remember learning about that in 2012 and thinking wow if only i had been here a little bit sooner kind of missed it at the time That turned out to, of course, not be true.

4:50And neural networks have remained exciting since then. And I've been working in and around them, yeah, since learning about them back then. Yeah. And yeah, go ahead. On founding Coher, how did you meet Aiden? Yeah, so I knew Ivan, the other co-founder from University of Toronto. We were both there at the same time. Aiden and I were also there at the same time, but we didn't really meet. So I properly met him when he was working in Google Brain as well. And he came up to the Toronto office for a summer and then later came back and told me about Transformers after having helped write the paper that introduced the transformer architecture.

5:30And we were all very excited about that and realized that there was an opportunity and a need for a company to make Transformers useful for the enterprise. And we've been doing that since. Yeah. And famously, OpenAI jumped on the Transformers, the Transformer algorithm and scaled it up to the GPTs. Was that already happening when you guys started Coher or were you kind of at the same starting line? hmm um so we start uh we started quite a while after opening i started uh we were founded in like beginning of beginning of 2020 end of 2019 um so were we starting at the same starting line no i think we we we uh yeah we were a little while later um but they and they had been working on scaling up transformers we we founded cohere after the gp after the first gpt paper but before the second one i see uh but were you guys the first model that you built was it uh a similar sort of architecture to what open ai built yeah i mean all the architectures are the same architectures which hilariously, the models that we're using now are still transformers, which is the paper that was introduced in 2017.

7:02So we're almost a decade into pretty much the same architecture. There's been changes in scale and changes in how you train them. And now we're seeing changes in having multiple models together and then using some switching mechanism to choose which one to use based on a query. But all of the models themselves are still transformer models. they're still the exact same architecture that was introduced by those authors back in 2017. Yeah, I've been talking to people about post transformer architectures and world models. And there's interesting things happening, but most of the post transformer architectures, I mean, like Mamba or things like that.

7:46Yeah, they, they're really based on transformers. I mean, And they have these different tricks and things, but ultimately it's still transformer based. Yeah, I think the transformer is, it happens to be very stable and easy to scale. And scale is what has been making these language models suddenly useful. I mean, I myself worked on a variety of other architectures. architectures when i was working with jeff we were working on a few different forms of capsule networks yeah which are very cool and had really interesting properties but did not scale well at all and so are not relevant from a from a practical aspect um they're not really used they're not used in like industry at all today because mostly because they don't scale well yeah yeah yeah i remember when i think my the last time i talked to him was about capsule networks.

8:45Yeah. And I think transformers have some of the nice properties of capsule networks, but are way simpler, way easier to scale, line up way better with the hardware that we have. And so it can be much, much, much more useful. Yeah. So you've also worked quite a bit on distillation, which I remember Jeff was working on last couple of times I talked to him. So So can you talk about, so Coher famously has sort of eschewed the AGI goal. I mean, that's, you're not your stated goal as it is for many of the companies coming out of the transformer revolution. can you talk about what Cohere's focus is and how distillation, for example, plays into it or what other aspects?

9:43Yeah, let me handle those questions separately. I don't think they have a ton to do with each other. So to address the goal of Cohere and as it relates to AGI or not AGI. So when people talk about AGI, I mean, we've been talking about it for many years now, but there's still ambiguity about what they actually mean. When someone says AGI, the way that I interpret it is a computer that you treat like a person. But for all intents and purposes, you expect it to behave as a person. You ask it the things you would expect a person capable of doing, and it does those. When you're not asking it to do something, it continues to behave as a person.

10:23Like, that's kind of what I mean. And then when people say a SI, like artificial super intelligence, they mean that, but instead of a person, a God, like that's kind of, yeah, that's what I mean. So I don't think we've built AGI and I don't think we're going to anytime soon. I don't think the transformer behaves like a person. And I don't think making better transformers makes them more like people. yeah i the analogy i use which i think was first introduced by fineman is the is the flight artificial flight analysis analysis or analogy sorry um and that's to say you know flight is a property there are biological systems that can fly like birds um there are artificial systems that can fly like planes and they fly in completely different ways right like a bird flies by generating lift by flapping its wings down a plane flies by generating forward propulsion and then having a wing shape that, you know, results in a difference of pressure between the top and the bottom of the wing, and that creates upwards lift.

11:29They share some things. The wing designs are kind of similar in that a plane can glide in the same way that a bird can glide, but they don't generate upwards lift in the same way at all. And that means that while planes are super useful and have, you know, completely changed the economy and even our understanding of the world and social dynamics and all kinds of things um they there's lots of things that they can't do that birds can't right like we've made nothing as efficient as an albatross we've made nothing as reactive as a bat we've made nothing that can hover like you know can turn on a dime like a um a hummingbird um we've made nothing that can do that the flip side is you know planes can carry insane amounts of of weight They're way larger.

12:16They can go way faster than birds. Like we've done all kinds of things that an artificial bird machine, like an ornithopter, wouldn't be able to do. So I think we're at a very similar place with intelligence, right? Like we've definitely made artificial intelligence. It's just not the way humans do intelligence. And subsequently, it can do lots of things that people can't do. Like a transformer at this stage can summarize documents better than me, can find answers to questions in mountains of unstructured data better than me. It can figure out tool chains better than me. It can do mathematical reasoning better than me in many cases.

13:00But it can't understand nuance. It can't understand cultural dynamics. It can't work on its own in a goal-directed way, in a way that a person can. And I think anybody who uses an LLM is aware of those limitations and they figure out how to make LLMs useful and they are really useful. But understanding the pretty huge difference between working with a person and working with an LLM. So all of that was was a preamble to say that I don't I don't think we're going to make AGI. And when I say we, I mean humanity. So I don't think anybody's going to make AGI. And I think that that used to be a controversial opinion.

13:36Like two years ago, I would say that and people would be like, no, look at what we're getting. At this stage, that's the widely held opinion outside of the very small Silicon Valley rooms. That's the widely held belief in all universities. That's the widely held belief for anybody who's kind of paying attention, who isn't doing that within a small room in Silicon Valley. So that's not our goal because I don't think it's possible right now. And I don't think we need to do that in order to make something super useful. I think LLMs are massively transformative, massively useful, and they can still be adding so much more value to the world than what they're adding right now.

14:18So our goal is to make these useful for the enterprise, useful for people working with them. And we've made a lot of progress in that. And there's more progress to go. We have more work to do. And so that's what we as a company have been focusing on since we started. Yeah. And what are the practical steps or practical directions that you've taken Coher to address that? I mean, that's why I mentioned distillation. distillation i know jeff was talking a lot about distillation at the time that you were working with him uh and it seems i mean i know that that your thesis is um you know providing ai that's capitally efficient i think it's a term that you guys use for the enterprise in other words you don't need these massive models, you're more concerned with trust and reliability and capital efficiency and working with regulated industries.

15:30Can you talk about all of that? Where you take Transformers to get to that? Yeah, that's exactly what we've been focusing on. So when I say we're making useful for the enterprise, I mean, we're making it an efficient solution so that it can actually, you know, it doesn't just burn through money and instead is useful in providing value above and beyond what you're paying to use it. So we make very efficient models that can be deployed on a small number of GPUs. We make models that can be privately deployable, right, so that it can access secret private data without leaking that data to your competition.

16:06We make models that can be customized. So we work with some of our partners to make models for particular areas, particular languages, or particular industries, so that they're really good at the things that they want, that the companies want those models to do, as opposed to being really great at generating images or, you know, doing mathematical reasoning, which turns out is not the majority of the work that's done out there in the world, right? So we make things that are useful for our customers. We also work a lot on frameworks for using those models. So it's one thing to have a model. It's another thing to have a system that your employees can use to help them with their work.

16:43So we work a lot on things like that. Yeah, you mentioned distillation. So just distillation is a little bit of an overloaded word in these days. I mean, when Jeff first introduced it, he was calling it dark knowledge, which was a very, very exciting term. And what he meant at that time was distillation being like logit distillation. I don't know if your audience is mostly machine learning engineers. I assume it's not. So we could break down what logit distillation means. Sure. Yeah. So a neural network and even a transformer, we think about it as outputting a word, like one word at a time, or people think about it as writing a sentence.

17:23It's not really what they do. What they do is give you a probability distribution over what words are likely to come next. So when you give a prompt into a model, the very first output is a probability distribution over all the words that could come next. And then you pick one word based on, you can base it on the probabilities themselves, or you can just say, I'm going to take the most likely next word. That's the very sampling techniques. People talked about that a lot in the beginning of this. At this stage, we kind of figured it out. You can use a pretty standard sampling technique and that works.

18:03Logit distillation is when you train one model based on the output of those probabilities from another model. So that and the dark knowledge that when Jeff first coined the term, he was able to show that you could take a really big model trained on image classification. And then you could train another model on the, a small model on the output of those probabilities from that big model. And it would learn a bunch of things much faster. Like hidden in, like if you're imagining, you know, imagine you're trying to recognize handwritten digits. This is kind of the canonical machine learning thing for a while.

18:38And you would have one model that was really good. And so it's, it's output probability distribution would say, Hey, that three looks a lot. It's a three. It's definitely a three, but it looks a little bit like an eight. And like if you trained a model on that probability, the small model would do a lot better than if you trained it with just saying, hey, it's a three. So that was distillation. That's not really done much these days in the world of machine learning, of transformers. In part, that's because it's a really computationally intensive process. And if ever you want to train a small model from the logit distribution of a big model, you got to be running that big model all the time.

19:15And like that, that, that takes up a lot of time. Also the, the probability distribution, when you're recognizing handwritten digits, it's, you know, either a zero or, you know, all the way through nine, but like those are the digits. When you're looking at the probability distribution from a transformer, it's like every word. So it's a huge probability distribution. And so it's just not as effective. So we don't, we're not really doing anything like that, but your question is, comes from a, the right place, which is that we're interested in making efficient models. And one way we could be doing that is with distillation.

19:46We're not, but it makes sense. It's a good thought. So how are you doing it? Yeah, a lot of it comes from just going at that angle specifically and figuring out what do we need, what do our customers want the models to do and what is kind of cool, but not really relevant to enterprise real world use cases. So generating images is a good example. We don't make a model. You can't generate a meme with our model. And there's a really good consumer application for that. I like using generative image to make jokes. It's fun. I like it. It's fun. It's fun to put up your picture and see it as a cartoon, like whatever.

20:29That's a good time. Turns out it's not really relevant for a business. There just aren't that many use cases, certainly not in the industries and the knowledge workers who we work with, just not really a lot of usage. And so if you just don't train your model to do that, you can save a bunch of parameters. You can make the model a lot smaller and still be really great. Now, the flip side of that is there's lots of things that enterprises do really care about, like tool use, like enterprise search, like being able to answer a question based on a really complicated technical document with image input and graphs and schematics and things like that.

21:05And so we train the model to be really good at that stuff. And that's how we're able to make models that are efficient and subsequently much easier to deploy in real world environments. Just to call that out, like the model that we released recently is called Command A Reasoning. It's a reasoning model trained on enterprise reasoning. It requires two GPUs to run. Deep Seek models, they take about 16 GPUs. Lots of the other models take significantly more. So there's models that we outperform that were like, you know, eight times as easy to deploy from a hardware perspective. Yeah. And are you starting with a base model and then training it on highly curated data for a specific use case?

21:53Are you working with open weight models and fine tuning them? No. So how do you get to the small model? So you could do that. You could take an open source model and then try to fine tune it. But the reality is most of the underlining capabilities of a model come from the pre-training phase. So we train our own models from scratch. There's about 10, call it 10 companies in the world that make foundational models from scratch. We are one of them. And we are unique in that group in our singular focus on the enterprise. Yeah. And the models, how do you make a smaller model, a more capital efficient model, if not through distillation, if not through pruning, if not through data curation?

22:45Well, there's some techniques that we do that I won't tell you about. But the short of it is you just choose an architecture that works. You're principled about the data you put into it. You train it for as long as you possibly can. And you focus on the things that give real ROI to businesses as opposed to cool, flashy tricks for consumers. Yeah. And that focus is primarily through the quality and scope of the data that you're training it on? Yeah. Yeah. Data curation and a handful of other techniques. Yeah. Yeah. So you train a model to be good at, I don't know, what would be a small model? A really common use case is I'm going to ask, I have a model, I have a bunch of documents and a bunch of tools, things that my company uses, like my HR software or my sales software or something.

23:43I want to ask the model to figure out, answer a question and write a document about it, cross-referencing all of this various data and tools. And then I want it to update another tool based on the output of that document. Like you can imagine a lot of your work, the work that I've just like that generic description is a lot of what people do behind a computer. And so we make a model that's really good at thinking through that, those kinds of things. Yeah. And, and the, the the um training data its training set is focused on on business problems i mean how so we still train on we train on the open web on the text that's available for training from the web as a starting point and then further refine it on stuff like i've just described right and add that into the pre-training and how many models have you guys deployed so far how many models we we've been releasing about the new the whole industry all of us have been releasing about a new family of models once a year it kind of seems right but but your models are they domain specific or are they just more efficient and more yeah they're more efficient they're easier to deploy and they outperform on the things that our enterprise customers care about Yeah.

25:01Yeah. Can you give us a few use cases? And I know that you're, what I've read, you're focused on regulated industries. So these models are small enough to deploy on premise or in a virtual private cloud or something. That's exactly right. It turns out that an LLM, like a language model, is really only as useful as the data it has access to. For a real world application, it's really only as useful as the data it has access to. And a lot of the useful data is, for very good reason, private secret data. It's data that you can't just send over the wire to some server somewhere. You can't have a a company train on that for various legal reasons, for very strategic reasons.

25:48And so when we work with a customer, we don't ask them to bring their data to us. Rather, we bring our model to them. And when we deploy, we deploy in a customer's environment, whether that's the cloud that they're managing or a virtual private cloud or a GPU that they have on a shelf somewhere. And so we'll deploy wherever we need to deploy in order to be useful for that customer. That has opened up a lot of real world applications that you would not be able to use an LLM on otherwise. What are some of the applications? Yeah, so a large customer of ours that's using our models and our agentic platform for them is the Royal Bank of Canada.

26:33So RBC, they're one of Canada's largest companies. And they're using our models across the company. for things like doing analysis on quarterly earnings reports. That's a good example. Or working through documentation on a bunch of different things or something. But it's mostly being used by the people who work in that company to do their job behind a computer faster, easier, more effectively. Yeah. Did you go to NURPS this year? I went. Yeah, I wasn't there this year, but I have been in previous years. actually i missed this yeah yeah uh no i met a bunch of guys from uh uh my wife's japanese i'm gonna say it wrong i say sakana but i think it's pronounced uh sakana what's that yeah in japanese i think sakana it's fine yeah yeah uh and and um they're working on uh evolutionary neuroevolution just came out with this book yeah are you looking at those coherer explore these new or not necessarily new it's not new but these other uh schools of ai uh to see how they might be applied um so we we are not those those guys are great i i got a lot of uh they got a lot of respect for some of the research that that they uh they did um and they continue to do cool stuff we are not working on evolutionary things.

28:11I think it's a promising, cool research direction. And I think one day some useful stuff might come out of it. It's so far not particularly relevant to business, but really cool. And one day might be, but for now, no. Yeah. Yeah. but i mean more generally are you you guys have kind of your heads down working on on enterprise uh applications and models that apply to the enterprise or is are you also or at the same time exploring new architecture still or are you is there enough work ahead of you uh that that you're not that's for somebody else to do. We've definitely done some work on new architectures and we continue to do a lot of research.

29:00Right. Like we published more than 100 papers at this point. Our people within Cohere have been collaborators on more than 100. We released the weights of our model into the research community. Our open science initiative, Cohere Labs continues to collaborate like across the industry with people and publish huge amounts of papers. some of their work has been on new architectures. Some of it has been on efficiency or the impacts of AI or various other training paradigms or the consequences of evaluation and stuff like that. They're kind of working all over the place. We have a big commitment to open science.

29:37And I think that's the way that this industry will continue to evolve. but our focus is always on making useful stuff. Everybody here is motivated by making something useful. And I think sometimes people feel like that's at odds with the idea of research. And they'll try to say, oh, there's two different things. You're either doing research, which is inherently useless, or you're building something useful. You're doing enterprise stuff. Those things are very similar to me. We're solving problems. We're answering questions. Those questions are relevant and we think the answers to those questions are going to be impactful.

30:15We're not pontificating, but we publish our findings all the time and collaborate with people and solve novel problems. It's just that those novel problems are targeted towards something grounded in reality and useful. Yeah. And that's something I've wondered. Do you build two problems or do you build? Yeah. I mean, you're obviously very involved with the enterprise writ large, you know. And do you see problems that you know you can solve and build to that? Or do you build generally useful models and then companies come to you and you figure out how those models apply to their problems? I mean, it's a chicken and the egg.

31:09Yeah, we generally, so we build models that we release. And we again, we release those weights available for research use. So and all of the models that we've ever made, you can find the weights of those models on, on, on hugging face. And then we work with those models and our customers to further refine them to be particularly useful for whatever they're interested in. Sometimes we do that if the problem requires it, like sometimes people have come to us and said, Hey, we need it to be slightly better at Japanese or something. or we needed to be better at this field. And we'll train the models a little more to make them particularly good at that.

31:45And other times, a lot of what they want to do, actually, the model out of the box is great at it. So we'll work with that. But it depends on the problem. But we start and release new foundational models every year. Yeah, yeah. And how large are the models? I mean, you're saying they're... 111 billion parameters. That's the largest model that we have released so far is 111 billion parameters. Wow. Yeah. And it can compete with the behemoths on specific tasks. How do you evaluate? Yeah. It's fraught. The space of evaluating large language models is fraught and a huge amount of the evaluation is misaligned with the way people actually want to use it.

32:33our suggestion to companies is to say don't like, don't, don't look at whatever is the latest eval that you think is exciting. Like, I mean, people these days are talking about ARK AGI as an eval, but it's a weird pixel matching game. Like it's a weird pixel reasoning game. I've never seen anybody in my life have that task as a job. Right. And if you work under the assumption that these are not building towards AGI, as again, most people think they're instead building something useful, but very different than a person. you shouldn't evaluating it on a random pixel matching game isn't a particularly good measure of how useful they're going to be so instead we say look just figure out what you want the llms to do and write like i don't know 10 10 or 20 here you know however many as you can examples of that problem and then ask the llm see if it gets the right answer and if it does like that's your evaluation and that's going to be a better eval set and looking at you know i've been in this industry long enough to see a new eval come out every year.

33:31And it's exciting. And then people realize it's no longer relevant. I mean, when we started, the eval was called LM1B. And that was an eval. I'm not even sure if you've ever heard of that. That was an eval that was about like news sources from 2011. And at the time we wrote an article being like, this is a very bad eval. And I feel like we could have written a paper every single year talking about why the eval is bad. And it fundamentally comes from people think that they're evaluating something super general purpose. And these LLMs are general purpose, but their deployment is more targeted. Their deployment is more like, hey, I need it to help me with summarizing the week's emails to prepare a slideshow to tell my boss of it.