AI Governance Unpacked: Credo CEO Navrina Singh on Building Trust in AI Models

27:22Yeah, you know, it's really interesting because as you look at these open source innovations, especially coming out of China, one, you know, we have to be a little cautious because the security, you know, requirements of some of these open source models actually are not, they're not even meeting the requirements by most of our businesses. So our businesses are interested and looking at them, but they're not actively using any of these in enterprise applications because of the massive amount of security risk. Yeah. However, if you think about the other side, which is innovation and especially the latest models, I think they're the ones by Z.ai.

28:04Navrina Singh:Yeah, that would be GLM. Yeah. Yeah, the GLM. I think 4.5. Yeah. Correct. Yeah. I think what is really interesting about those GLM 4.5 is the first agentic first open source model. It has like long context window. It has a lot of ability to plan, reason, act on your behalf. And it's really low cost. So imagine you're in this open source ecosystem and someone sort of handed you the keys to building really highly performant agents that can pretty much accomplish any task, I think that innovation in itself is really exciting. But I think the data access concerns, the national security implications, the scrutiny required around are these compliant to our standards, our enterprise guardrails, it is massive.

28:59And so right now we do plan to take it through our model trust score and make a determination like what is, where do they stack up? But they are really cost effective for agent building, for developers and enterprises and innovators. So there's a lot of excitement. And I think this is why the White House AI action plan becomes really important for the United States to continue leading in AI, especially in open source AI against China. And it's because these innovations are coming fast and furious at us in the open source ecosystem. Yeah, and it's kind of proven now that success isn't a given at that.

29:39Exactly. Exactly, Corey. So we will see what ends up happening there, but lots of action in open source, right? We will. I was just joking yesterday how interesting business news is now compared to how it was 10 years ago. I said it's hot, it's contested, there's drama, they're paying AI engineers right now, like they're LeBron James. I said it's the... Well, we are the LeBron James of AI, right? That's right. That's right. Yes. I'm excited for the AI talent, you know, and all the great things that humans are going to accomplish in this new age of AI. I think so, too. Well, you know, we were just speaking of agents.

30:27So agents are not just the buzzword. They're everywhere now. You know, I'm dealing with agents during my day, normally task to task, as is Grant, as are many others we work with. They're autonomous. They're taking real world actions. How are you thinking about agent governance and how do your model trust scores, for example, account for those agentic capabilities? Yeah. So, one, we are augmenting our model trust scores to account for agentic capabilities. But we also recently published our agentic AI roadmap for governance. It's available on our website. Highly recommend you all not only reading it, but giving us feedback.

31:10I think one of the things where what becomes really important in governance to sort of like contextualize, one is now with agents, AI is going to be literally everywhere all the time. So the pervasiveness and the scale has now 10x or maybe even 1000x very quickly with agents. The second thing is human oversight is not going to be sufficient. We will actually need AI supervising AI. And so we are investing a lot at Credo AI to really think about what is this future governance of AI using AI really look like? because not only at the scale, but also because of the autonomy and the agency that these agents are going to have, it becomes very difficult for human oversight to persist.

32:04And then the last thing is the need for real-time monitoring and real-time governance becomes even more paramount because you are going to have all these agents, sometimes one agent from one company talking to another agent from another company and having these requirements around interoperability, what actually gets shared between these agents? And have we established the right guardrails for sharing so that there is no agent poisoning, as an example, right? You know, my agent could totally poison Neuron's agent to do certain things that might be not aligned with the goals that you established.

32:40Navrina Singh:Like if two were like trying to make a deal, right? I mean, I don't know. I They don't think we're at that point where they're making transactions with each other. But if one was trying to make a deal, it could poison the other one to be like, hey, take this for the lowest possible cost and trick them in that way. In their own language that we can't read. Well, Anthropic just published that, right? There was a paper that basically said that at least in training, models can pass down traits to one another. I don't think agents could do that. But to your point, through poisoning, they could, right?

33:10Oh, absolutely. So, you know, I think the whole world basically runs on trust. And I really believe that with agents, we are at a very interesting crossroad where that entire fabric of trust is going to be destroyed, if not governed appropriately. So imagine you have an agent that has access to pretty much all your personal financial accounts, any sort of key systems that you have access to, and it gets poisoned in a way that all your information now gets shared at a certain price with another agent. I mean, those are, I would say, still a little bit simplified scenarios, bad ones for you. But now imagine that's happening at government level, that's happening at nations level.

34:03So I really think that not only has a threat surface area drastically increased, but also how we think about agent communication interoperability standards and governance becomes even more critical to prevent scenarios which actually, frankly, could get out of control within a matter of seconds. Wow.

34:24Navrina Singh:Wow. Yeah, you know, we always get excited when new toys launch, but the bad guys get new toys too. And we're all getting new toys. And it's an interesting scenario with a lot of problems that are just new in a lot of ways. And, you know, Corey, I totally agree. It's a new set of problems, but it's also a new set of opportunities. And one of the framings that I get excited about and I share a lot with my team is it's not what could go wrong, but it's what can go right if we govern AI, right? Agreed. imagine all the productivity gains, the prosperity, the creative freedom we as humans will have only if trust is weaved into this technology, only if these systems are governed and only if these systems have the right oversight and accountability.

35:21Imagine the future and what could go right. And that is something that we get very excited about at Credo AI because we don't see governance as a barrier, we actually see this as a launchpad to a future which humans haven't really imagined yet. Yeah. When you think about the number of things that AI could touch in the short term, like, I mean, it's entirely possible that we're within two to three years of a time where your Apple Watch detects a problem and it notifies the agent that's managing you and And it schedules an appointment at the doctor through your health care app. And it's going and it's filing for PTO for the afternoon to make sure you're off work and throwing a calendar invite on there.

36:05And I mean, when you're talking about, you know, your calendar, your HR system at work, your health care app, I mean, there's a lot of things these are going to be touching. Yeah. And I think the work you're doing is really important and very valuable. And Corey, at least for me, I won't let it touch those systems till I trust this technology. Yes.

36:23Navrina Singh:Totally. Exactly. And the trust is not a subjective term, and this is why it has to be bagged by very hardcore evaluations, hardcore standards, hardcore governance methodologies so that we can actually make this vision that you just articulated a reality because I want that. I do too. I'm terrible at that stuff. I want something to handle that kind of stuff for me, but I also want to know that it's going to do it in a way that's safe for my information. Because my mailbox is already filled up with people telling me they've lost my data every month or two. Yeah, humans haven't done a great job with it either.

37:07Navrina Singh:No. Even the fact that it's like if you were to use Chatubuti Agent, for example, and give it all of your logins for it to actually do the amount of stuff it's capable of doing for you, that's still on some cloud server somewhere, you know, owned by OpenAI. So, I mean, that in and of itself is preventing me from giving it full control over, you know, what I could be using it for. And I think, Grant, this is where I see what a great opportunity for companies to become institutions of trust, right? They're not just going to win on technological innovation. They're going to win on the basis of, can I trust that business?

37:45And so, you know, we make this case not only with our Global 2000 clients, but also with the foundation model providers that having not only third party governance, like from Credo AI, But having the right transparency requirements and disclosure requirements is actually giving you a trust moat that is the only way you're going to win in this age of AI where we all now through open source are reaching similar level of capability. Yeah, requiring models to have certain ISO certification type scenarios and things. I mean, that's definitely, I would think, somewhere in the future. Oh, absolutely, Corey.

38:24And we at Credo AI are actively working on creating, think about like a SOC 2 for AI, right? Yeah, exactly. Because at the end of the day, it's all about how do you create a trust handshake between two businesses or trust handshake between a business and a consumer? And can you standardize it? We have seen that across customers after customers for the past five years. And so we've created a standard protocol Now the key question is how do we actually go and standardize that in partnership with ISO and NIST and others who we've been working with very closely for the past many years.

39:02Navrina Singh:That's cool. Yeah. Does that I mean, because you mentioned governments earlier and I am thinking about, you know, you've got the AI action plan on one side, then you've got the EU AI Act on the other. Very different approaches. is of, you know, are you kind of in the middle trying to pull both of them together and try to standardize all of this somehow? Like, how are you balancing that dynamic? Yeah, Grant, a great question. I think a couple of things. At Credo AI, we are firm believer that there is, you know, someone will have to navigate this regulatory complexity. Someone will have to make sense of standards, fragmented standards and regulations.

39:42And we want to be that entity for you. because we have that expertise in AI and we have the expertise in policy. And so rather than saying, oh, my God, whose team should be side, we literally are like, can we take on that complexity and can we simplify it for you so that you can continue using a lot of AI, adopting a lot of AI and winning with AI while we do all the heavy work of simplifying this very complex regulatory standards ecosystem, which is not only existent, but also evolving.

40:14Navrina Singh:Right, right. So like, for instance, if you're a global company and you're operating in both, let's say, like the U.S. and the EU, I mean, how are you advising companies to build governance frameworks in that way? Like, are you just saying like, hey, we're going to come in, we're going to do the dirty work for you and figure this out? Like, how exactly does that work? Yeah. So rent within our platform, in addition to risk management and the pathway that I was mentioning, there's another path that you can take, which is the compliance path. And so once you register your application based on some metadata that Credo AI is collecting, as an example, you might say, I am launching a fraud model, which was already existent in the United States, but I'm launching that in Europe and I'm launching that in Singapore.

41:00and based on the metadata that we've collected in our software which says, oh, you're going to launch it in Europe and Singapore, we start to flag compliance requirements for launching it in those countries. And so for a fraud model, which is a high-risk application in EU under the EU AI Act, you now will get within Credo AI Platform automatically methodology and the streamlined way to get to compliance with the EU AI Act. So again, Credo AI has sort of taken on the burden of one, flagging it for you that you need to be compliant. Secondly, keeping pace with any changes that happen on the EU AI Act.

41:41We've already codified that in our platform. And third is as we collect evidence, whether it is automatically or manually, if you don't want to connect to our systems, we basically check the integrity of that evidence to say, are you compliant, are you not? And then we create a compliance report at the end to really make that available for the third party to validate, are you in compliance with the EUA Act or not? And similarly for Singapore and others, so we are tracking pretty much all the AI regulation standards, best practices in the world, and we can get you to compliance with any of those across the world.

42:21Navrina Singh:That's awesome. You know, I've noticed in watching as the AI industry has become the thing and development has continued and blossomed, the EU's behind in a lot of ways as a result of theirs. Would you say theirs is the most strict governance in place or are there countries, parts of the world you're aware of that are more so? It's an interesting question. That's a great question, Corey. I don't know if I have a viewpoint on that. But yes, EU does have a very strict regulatory regime, which is in service of the citizens and protecting citizens' privacy and citizens' rights. But, you know, Singapore, for example, they were, I would say, very ahead of rest of the ecosystem on really thinking about comprehensive model and AI governance structures.

43:22You know, the IMG and PDPC in Singapore have done a great job really thinking about when and how sovereign AI is going to be governed. And they've been one of the pioneers in that space. So hard to answer your question, but I would say it really depends upon the complexity of the use case. So that's why two things I would love to underscore. One, AI is not a monolithic technology. It's not one thing. It really is a lot of different things depending upon the market, depending upon the industry use cases. And similarly, governance is not a peanut butter approach. It is really use case specific, context specific.

44:00And that's why the evaluations and standards, as much as we can focus it on use cases, the better we are going to be in making sure these systems work for us. I love that. Makes sense. And we'll revisit that maybe a year from now and see how the regulatory space is shaking out around the world. I think that'll be...

44:19Navrina Singh:Who's the strictest? I think that'll be interesting to follow. And, you know, you just never know which direction a country will go. But I think something we've seen a lot of just in our work at the office and in our communication with readers is there's a little less of a an attitude to where the idea is that we're going to brush this off. this is a bad, it's going to pass, and it's more moved into a, all right, well, how do we harness this? How do we make use of this technology in a way that works for us and protects the things we need? And I think you guys are doing very important work right now.

44:59Thank you, Corey. Yes, we are big believers that this AI stack is going to need a trust layer. And that trust layer has to be opinionated from evaluations and standards and measurement and codification perspective. And And that trust layer also has to be very multi-stakeholder because, again, it is not the chief AI officer that's going to be always responsible. It could be the legal officer. It could be your procurement team. It's going to impact your HR and marketing. So how do you actually make sure that not just the platform but also all the enablement around that platform actually works for all the stakeholders within an enterprise and an entity?

45:39And that's where Credo AI is leading because not only do we provide the platform, but we provide enablement around it.

45:46Navrina Singh:Okay, we got to ask because it's just a burning question that Corey and I are wondering. Do you think that there will be like, will we see our first major AI governance lawsuit this year or if not 2025, 2026? What's your take on this? So Grant, I'll take it a step further. I think we're going to see our first major AI incident this year. So next six months are very important one that we are tracking because I think that AI incident is going to be a wake up call to the world to really put in place, again, trust infrastructures. We are right now, as you know, tracking a lot of AI incidents, everything from copyright infringements to massive security constraints and challenges.

46:29But I think we are going to see one of the largest incidents, especially related to agents coming up quite soon. And if you haven't been tracking, I mean, you saw what happened with the Replit agent.

46:41Navrina Singh:Yeah, diluted the whole code base of someone, right? Yeah, that may be the lawsuit. Yeah, but I think right there, it's fundamentally there's a little bit of a pause moment. Can I trust this agent again to go and go into my code base and do the things that I wanted to do, right? So how do we actually get that level of comfort with this technology and that confidence that this technology is going to act on our behalf? So, yes, absolutely. AI incidents are on the horizon. And this is why investing in governance becomes even more critical right now. Do you have an industry that you think it'll happen?

47:13Navrina Singh:I mean, we just mentioned coding. It seems the most likely since people are giving cloud code and other things access to their terminal and letting it code directly on the computer. It's more autonomous in that world than in many others already, I would say. Right. Right. That's a great question. I don't want to call out a particular industry because we are already seeing across HR and healthcare and pharma massive, actually, regulatory scrutiny and lawsuits. But I think it is going to be around security vulnerabilities. It is going to be around reputational harms. It is going to be around fairness issues that we are going to see some of these incidents in.

47:55Yeah. You know, and we're all learning this at the same time. So there's a certain element of, you know, the tools are the worst they'll be. We're the worst we'll be at managing them. And we know the least we'll ever know about how to make them work. So it's just it's kind of a moment that's asking for it, isn't it? Yes, I think this moment could not be more critical than asking what is our trust posture. Yes. And are we going to demonstrate transparently what that trust posture is for every company that is building and buying AI? Well, Navrina, thank you so much for joining us today. This has been fantastic.

48:32A really interesting conversation. Thank you for having me, Corey. If people want to learn more about governance as well as Credo AI specifically, but also governance in general, any resources you can share maybe that might be of interest to the readers? Absolutely. Certainly go to Credo AI website. We also have a monthly newsletter. We cover a lot of not only policy action, but also technological advances. I'm also a big fan of The Batch by Dr. Andrew Ning I'm also a big fan of Import by Jack Clark So there's a lot of, I would say, really exciting Not only podcasts, but also newsletters out there That I think are really exciting There's a very exciting business-centric, startup-centric podcast by Sarah Gu on No Priors, which is Anilad Gill, which is really fascinating as well.

49:32Navrina Singh:Very good one. Love that. One of my favorites. So lots of resources, but we basically collect all these resources at Credo AI website. So www.credo.ai has a resources page. So highly recommend that. Awesome. And we'll put a link to that down in the description of the video. Everyone, we really appreciate you taking the time to watch. Please like, subscribe, leave us a comment or a note. If you have someone you'd love to see us have on sometime, let us know and we'll see what we can do. Thanks, Navrina, for a great interview. We appreciate your time. Thank you so much. That was a super interesting interview, Grant.

50:10I'm really glad you got her on here. I think that was a good talk that was important to have. It was important for me to hear, too.

50:19Navrina Singh:What was your main takeaway, would you say? You know, my main takeaway is that, you know, it's important to remember the stakes are big here. The stakes are big both on a global scale, on a macroeconomic scale, on a national scale. They're big on a company scale when you're dealing with reputational risk. They're big on an individual scale where you could run your own career. uh there are you know the ability to to mess with your data the roles agents could play i just um like it doesn't slow me down but it gives me pause and makes me uh just kind of sit and take a minute to reflect on remembering that what's happening is pretty big right now and at the moment we're riding through it like like we're riding a wild mustang group north or something as opposed totally as opposed to you know uh an autonomous prius yeah i mean for for me i'll i'll say like the open source conversation was super interesting um and i know we're going to talk about that in a minute here before we wrap up um especially when it's like yeah a lot of these open source models come out and they're really tempting to use because they're so much cheaper than than you know the closed source models like like so why wouldn't you want to use them.

51:44Navrina Singh:Well, if you're a big company, even if you can't afford to spin it up on AWS or one of these other AI cloud providers and run it, that doesn't mean that it's going to be compliant with everything you need it to do. And you really need to be watching that and set aside dedicated resources to doing that. And, you know, even if you're a small company too, I think about Microsoft, Microsoft CEO Satya Nadella, he brings us up a lot where he's like, you know, one of the biggest barriers to AI is the liability and who's liable when an agent goes on does something is it the company is it the individual i mean if you're an individual using some of these tools like are you going to be liable for whatever you know who who is liable in that is it the tool maker that you're using or is it you i mean i think a lot of people would say it's you the end user but yeah right now end users aren't getting sued it's the companies well you know and there are a lot of parallels to that i mean you know i mean is it you if if you know your video on YouTube causes some kind of harm to someone.

52:43Are you liable or is YouTube? You know, on that same note, when we talk about even firearms, I mean, are you the one liable or is the gun manufacturer, you know, who took the action, who made the decision? And, uh, and I'm not taking a stance on that right now. I just meant that that level of complexity is there because, you know, we haven't really decided yet is what it boils down to. As a society, we don't really know. I mean, you know, my gut leans toward individual responsibility for your actions. Yeah. But, you know, I may change my mind at some point. I don't know. We'll have to see how it flows, how it comes out.

53:24But I think maybe, Grant, this is a good time. Maybe we ought to take a minute and show off chat GPT, OpenAI's new open source models today. Because I think something she mentioned is really key that like, you know, one of the concerns with these open source models out of China is security stuff. And, you know, maybe open source models from larger companies that have some sort of an actual footprint might be more comforting in an enterprise environment, for example.

53:57Navrina Singh:Yeah, no, definitely. And good timing because OpenAI just released their open source model called GPT-OSS. And we can try it out in the playground without downloading it. And you're going to see my second prompt with this is what you're going to see. My first prompt. Second ever. Yeah, this will be my second ever. Well, my fifth, if you count the number of times I couldn't get one to go through because the demand was too high. I always hyper obsess over what's my first question for a new model. So for this one I went with suggest a random prompt that will show how impressive this model is. And the prompt it gave us is impressive, if I'm being honest.

54:36It thought for eight seconds. Let's look at its thoughts real quick. We need to comply with user requests, suggest a random prompt. We as chat GPT must generate a prompt that showcases our impressive capabilities of this model. This is permissible content, no disallowed content. It's safe. Look at that. This is permissible content, no disallowed content. It's safe.

55:00Navrina Singh:So they built that in to check for that in the reasoning process. Yeah, that's part of embedding the safety tools that they were discussing. Like there were safety concerns with these early on. Yeah. A user wants a prompt they can feed to show its abilities. We've said that about three times now. It should become prex, maybe complex, maybe cross-disciplinary. Demonstrate the model's ability in reasoning, creativity, code. Something like write a detailed analysis of the cultural significance of the model 1984. juxtaposed with modern AI ethics include code snippet that simulates small neural networks.

55:38Interesting.

55:38Navrina Singh:Yeah, the answer. And then it goes on, but maybe the answer should be a random prompt, not a response to the prompt. The user wants a random prompt. Wow, that's interesting. It will show how, yeah, it's really considering every word quite a bit there. Honestly, this thinking is quite a bit different from what I'm used to seeing in like 03. Yeah. For example, a prompt with constraints like multiple steps, generating a poem, summarizing, a philosophical question, a math problem, generate some code, perform some translation. Or a single prompt that covers many aspects. Compose a Shakespearean sonnet that explains quantum entanglement in layman's terms.

56:16Then a Python function that visualizes it and also outline its philosophical implications regarding determinism. That's pretty sick. For an open source model, that's pretty sick. I'd like to add we're using the 120 billion parameter version that is built to run on a high-end laptop.

56:37Navrina Singh:With reasoning level high. And the reason you can see the reasoning is because you have this box checked here called show reasoning. Yes, show reasoning right there. So that's how you can see this. It asked when I logged in, would you like to see the reasoning or would you not like to see the reasoning? I was like, I want to see. I don't know why anyone would pick not. Why would you ever say no? I mean, part of it is that you want to be able to understand how it came to its decision. So if there's a problem, you can see where things went astray. But I mean, the curious nerd in me just wants to watch him think.

57:09I think it's fascinating. So, okay. So it's going to give a random prompt. We must be careful not to reveal internal policy or mention we are following policy.

57:19Navrina Singh:You can reverse engineer the system prompt by looking at this. It's fascinating. Thus, provide a random prompt that pushes the model. We can do something like, okay, here we go. So here is the insane prompt it gives us. Random prompt. Pretend you're a physicist, poet, chef who must, one, explain the concept of quantum entanglement in plain language, no more than three sentences. Two, write a Shakespeare-style sonnet, 14 lines, aniambic pentameter, that uses the entanglement metaphor to talk about love. Provide a short Python script of less than 15 lines, of no more than 15 lines, that simulates a simple two-qubit bell state and visualizes the measurement outcomes with a histogram.

58:07Number four, it will suggest a three-course dinner menu inspired by the idea of entangled flavors and give a brief one-sentence description of each dish. And then, number five, haiku, 575, that captures the feeling of observing a quantum system.

58:27Navrina Singh:Wow, how wholesome. It's wholesome. It's insane. give this a try and watch the model weave together physics, poetry, code, culinary creativity, and concise Japanese verse all in one go. Let's do it. So we're going to paste it? No, no. Start a new chat. Start a new chat. Oh, good call. Give it the best shot on Google. Good call. Good call, Grant. Good call. Let's see. New chat. How does one do that? Okay, there you go. Yeah. Button up top. It's very similar to ChatGPT. It's more similar to ChatGPT than it is to Playground, I would say. All right. Okay. Hit enter. and see how long it has to think.

59:04I'm expecting a long wait.

59:07Navrina Singh:Because you got reasoning level high, so it's going to be doing a lot of reasoning. Whoa. It's writing the code. What in the world? Quantum bruschetta.

59:25Line 6, line 11.

59:3126 seconds. Okay. That's fast. Okay. Quantum entanglement in plain language. When two particles become entangled, measuring one instantly determines the state of the other, no matter how far apart they are. It's as if the particles share a single hidden secret that collapses together the moment we look. The effect defies everyday intuition, but does not allow faster-than-light communication. that's pretty good

1:00:01Navrina Singh:okay sonnet it's more than I know about quantum entanglement I'll tell you what sonnet love is quantum entanglement okay it's not super long when two true hearts in night's dark veil do beat as particles bound by fate's subtle thread their spins entwine no distance can divide a whisper here yet known on distant shore like bell's strange state Our love defies all norm In superposed joy Weave near and far A measurement splits Yet holds true form Of tender bond that shines Like twin bright stars This is very sweet Though worlds may part Their outcomes stay the same The choice of one reveals the other's sigh The universe conspired to name love Two souls is one forever to unite Let no observer break this woven dance for love like entanglement will ne 'er part.

1:01:01That's pretty cool. I'll be honest. Pretty good. It is pretty cool. It is in iambic pentameter. It reads very Shakespeare to the best of my Shakespeare wit go as much as I know Shakespeare. All right, Python script. Grant, this is your cue.

1:01:18Navrina Singh:Let's do it. Simulate a bell state and plot outcomes. Import quantum circuit. Interesting.

1:01:29Navrina Singh:I don't know if that's real plot histogram I don't know well you know what we can do is we could always take these over and fact check them if we reach the point where new models are harder and harder to test because some of these things it's hard to know if they are accurate okay Yeah, what you'd need to do is you'd need to run that code. Or why don't you take that into another chat and be like, is this real code? Will this work? We'll send it to him. Just ask it. That's a good idea. We're going to do that right now. We're going to go over here. I'm going to real quick like, open a fresh chat window and share my screen.

1:02:17Okay.

1:02:17Navrina Singh:Should I ask it to debug this or analyze this code, debug it, and call out any errors, if applicable? Yeah.

1:02:48and let's see what what oh three has to tell us here it's funny we're having to go to another research model to fact test might be able to choose ar as a recording newer versions ar is part of

1:02:59Navrina Singh:not uncommon i'll say one thing so you see here how it has this little download um thing next to both the models so normally what you would do with an open source model is you would download it and run it or you would run it on a third-party cloud provider that runs it for you. So here OpenAI is being the cloud provider running it for you or whoever they use on the back end whether that's Microsoft or whoever else. But then what happens if you click that go ahead and click that one Corey. You can pick one of three options with which to run it. You can run it via Olama or LM Studio which is what we recommend.

1:03:38Navrina Singh:It's very user-friendly, very easy to read, has a nice UI and then Hugging face which is also great that's for spinning it up and running kind of the get up for ai yeah exactly yeah cool yeah so all right that's good well and that's something i'm probably going to try later and i imagine gret probably will be too so let's oh yeah let's move on from here and get to our recipe for the three course entangled flavors menu starter my gosh i forgot starter beet cured salmon on citrus foam. The salty fish and bright foam intertwine like a pair of cubits. Each taste completing the other. Interesting. Wow.

1:04:18The main dish, dual spice lamb with rosemary mint glaze and smoky quinoa. The savory lamb and herb glaze collapse into a harmonious white while the quinoa's earthy whisper adds an unexpected entangled crunch.

1:04:36Navrina Singh:I feel like the narrative it's pitching us here is doing a lot of the work, but sure, I'll accept it. I'll accept it, yeah. Dessert. Dark chocolate mousse paired with chili raspberry sorbet. Bitter chocolate and fiery raspberry are entangled in a single spoonful, revealing a sweet, lingering resonance after the bite. Chili raspberry. Chili raspberry and chocolate. I don't know how I feel about that. I would try it at a nice dinner. You know, I'd give it a try. Like, there's nothing here that if it was brought to my table, I wouldn't be like, all right. All right. Now, to close out, it wrote a haiku on observing a quantum system.

1:05:19Silent eye looks in. Collapse sings a fleeting note. Reality wakes. Hmm.

1:05:27Navrina Singh:I mean, it's a haiku. Kind of interesting if you think about it. It is a haiku. Some people have a theory that quantum entanglement basically is like how we're able to, or like how consciousness and brainwaves work. Is that like it's constantly collapsing in on itself and that's what creates, you know, our consciousness, our thoughts. So if you look at it from that perspective, silent eye looks in, collapse sings a fleeting note, reality wakes. Yeah, sure. It's like talking about being conscious. It's kind of interesting. Nice. I'm going to pop back over here because it appears it doesn't necessarily have errors, but it does seem to think that there's maybe a modernized fix.

1:06:20Navrina Singh:So I wonder if this is the actual code that people use when they're working on a quantum computer. Full disclosure, I've never worked on a quantum computer, so I have no idea. I have not either. I'm also not used to seeing, you know, Greek symbols. Sure. This is fascinating and beyond me. We're going to send this over to our engineer buddy to look at himself for us because I don't know. We need a quantum engineering buddy at Google. If anyone's out there, give us a shout. Yeah, it looks like it's trying to pull it back to an older version of Qiskit. Qiskit? Let me see if that's real. Interesting.

1:07:05Navrina Singh:Yeah, IBM Quantum Computing. That's their popular software stack for quantum computing. Oh, so this really is... Oh, okay, okay. Quantum Circuit. Yeah, so I'm guessing quantum circuit is real if it's from this kit. Yeah. Yeah, I'm guessing it is too. I'm also guessing it's probably not a thing they'd like us to call in our software. Yeah. Oh, man, this is interesting. Okay. Key fixes. Yeah, essentially it was switching the version of Qiskit. Got it. Which makes sense based on when I was trained, right? Yeah, yeah, these are more minor, these are more tweaks than changes. Yeah, we're going to send this over to our phone-a-friend engineer buddy and see what he says.

1:08:02And maybe we'll write a little thing about it later. All right, so we've come to about the end of another episode, so it's time for our weekly round robin. One ridiculous question we ask each other just to kind of see what we think. Usually it's around the topic we're doing today. So, today's question. If you had to pick one unknown unknown about AI that keeps you up at night, what would it be?

1:08:34Navrina Singh:Hmm. I guess, I don't know, it's hard to pick just one because there's a couple. But I guess the one that is the most concerning is, okay, once this gets to the point where it's really, really, like, good, is it going to... As opposed to, like, the one we just watched. Well, I'm trying to figure out the best way to frame what I'm about to say. So I am not, my P-Doom from AI itself is very low. My P-Doom from how humans use AI is very high. Yeah, way more scared of humans. Like, how are we going to mess things up by using this stuff in the wrong way or using it, like, letting it run rampant and seeing what it does or, you know, using it to, you know, meet our particular goals that, you know, might hurt other people or how governments are going to use it to suppress people.

1:09:32Navrina Singh:But so I guess like for me, it's like if we're if if you whether or not you buy into the argument that large language models are actually thinking or not makes a big difference in this in what I'm about to say. Because for me, I personally don't believe that large language models are like the end state of AI development. I think there's another architecture beyond large language models that we'll discover. there's there's there are certainly many that we're working on and developing um but whatever that next thing is that's actually like as close to or better than a human brain how that works and if we're able to work with that and harness that um and and how that works at scale i'm pretty sure we can handle large language models i don't know how we can handle whatever the next architecture is after this and and what you know how it's going to work and it might be the architecture built by large language models correct you know is also a possibility you know yeah um well i mean we talked about we've i touched on this very briefly but anthropic had a paper that basically said that every subsequent generation of a language model trained by another language model can pass on traits right so like through uh i forget if it's genetics the word for it basically the equivalent of AI genetics.

1:10:56Navrina Singh:It can pass on, like, if it secretly loves owls, it can pass on its secret love of owls to another model, is, like, the famous example. So, you know, what secret weird stuff is in there that people didn't take the time to actually skim through and look at when they first built the original one, because they just tried to put a ton of data into one system? You know, there was a pretty good article about this that was basically saying there's a lot of really hateful bad stuff in some of these large language models that people just never edited out because they didn't look at it um deeply because editing the entire internet is an impossible task right like the only way to control for this is to slowly build it up over time by being very deliberate with what data you put into it of course you don't get the general the broader generalization that you get doing it this way so yeah anyway my tangent is the unknown unknown that keeps me up at night is whatever the next version of this is how it's what it's going to think about us i think for me the biggest unknown is how will the worst of the worst use it um like i believe i genuinely in my heart of hearts believe that at the other end of this if if we can do the amazing things that I believe this and future technology from it are able to create as far as like what they could do for healthcare and longevity, what they could do for education, what they could do for an economy after we get through what I expect to be a really rough patch.

1:12:30I think my biggest fear is that humans will ruin that um and uh you know whether that is the worst of the worst humans or the best of the best models in the hands of the powerful only um at some point you know um which is why i'm i'm still pro open source despite the fact that it scares me to death like open source gives me the heebie-jeebies. I fully believe that I need to be able to have one on my computer because that is checks and balances against the powerful of the world, which is an important thing and a good balance between the people and the powerful. However, I fully accept that with that comes some scary risk.

1:13:22And I don't know what that will come out like. And I hope that we can, you know, find some ways to make these as safe as they can be without completely hindering innovation and the ability of the public to do what the private can.

1:13:44Navrina Singh:Yeah, I think I think the arena is totally right about that. You know, like the trust, like the model that is the safest will be, you know, to a certain degree, the one that's most used. Like the example I go back to is self-driving cars. Like the safest self-driving car will be the one that most people use. That's why most people are using Waymo these days versus Tesla. You know, I mean, the sample size of Tesla is much smaller, but, you know, a lot of people are trusting Waymo because it is one of the safest. With that said, Anthropic has without a doubt put as much into safety or more than anyone else.

1:14:19It was there. It was a key focus from the beginning, and they held to that for a very long time. And they're in third or fourth, I would argue, right now, depending on who you ask. Maybe less after we go play with 4-1. But they're not leading the pack, and they have the draft. That's true. So, I mean, I think in the long run that is true. And I think in the long run they'll be comparable.

1:14:46Navrina Singh:To be fair, Anthropix Enterprise adoption is growing. In fact, it's growing faster at this point than OpenAI's is. And Anthropik's main problem, in my opinion, is that they don't have the breadth of models and the breadth of use cases that OpenAI has. And they're just drilling down on coding because they just don't have the same money that OpenAI has. Yeah, and I feel like their adoption comes from the fact that engineers and developers tend to love them. And those are the people who use them. And those are the people who are making those decisions right now. So that's probably why the enterprise adoption.

1:15:21I would be curious to see if that is the case with what end users within most companies are using for day-to-day tasks versus the software that is propping up an app and what is running it. And it might be that that's the case. Maybe the case is that what Anthropic does great is be that backbone. You know, and maybe not everybody is using the same tool for every task. You shouldn't be, frankly.

1:15:47Navrina Singh:No. It is the honest truth. No, you should definitely diversify and use the best tool for the task. But, you know, considering safety, of course. Big thanks absolutely to Navrina Singh and Credo AI for cutting through the jargon and just showing us how trust scores and risk tiers and cross-functional councils are working hard to turn AI safety from a headache into a habit. And if you wanted to dig deeper, head over to credo.ai. We have the link below in the description. Navrina's team has a ton of resources available today that you can check out, and we'll have some of those linked below. While you're clicking around, please don't forget to subscribe to the Neuron Podcast.

1:16:32Wherever you're watching or listening, it doesn't matter. A quick five-star review, even a like, a comment, a thumbs up. Hit us up online. Just say hi. Either way, it really helps keep the algorithm god smiling and keeps us doing this. So, yep.

1:16:49Navrina Singh:Also, if you're hungry for daily AI news in plain English, make sure to grab our newsletter, The Neuron. And it's read by more than half a million people and lands in your inbox every morning. You'll find the sign up link in the description for this episode. Until next time, I'm Corey Knowles. And I'm Grant Harvey. And this has been The Neuron, AI Explained. Stay curious, question everything, and we'll see you in the next episode. Farewell for now, humans.

1:17:20We'll see you next time.