Less about Models; More about Architecture

0:02Welcome to the Practical AI Podcast, where we break down the real-world applications of artificial intelligence and how it's shaping the way we live, work, and create. Our goal is to help make AI technology practical, productive, and accessible to everyone. Whether you're a developer, business leader, or just curious about the tech behind the buzz, you're in the right place. Be sure to connect with us on LinkedIn, X, or Blue Sky to stay up to date with episode drops, behind-the-scenes content, and AI insights. You can learn more at practicalai.fm. Now, on to the show.

0:41Daniel Whitenack:Welcome to another episode of the Practical AI Podcast. This is Daniel Whitenack. I am CEO at Prediction Guard, and I'm joined as always by my co-host, Chris Benson, who is a principal AI and autonomy research engineer at Lockheed Martin. How are you doing, Chris? Hey, doing great today, Daniel. Looking forward to a conversation here. Yeah, yeah. Excited to chat about all sorts of things, both in terms of background and current work with Chaitan Gupta today, who is chief AI officer at Rackspace. Well, welcome, Chaitan. How are you doing? Hey, I'm doing good. Thanks, Daniel and Chris, for having me.

1:17Quite excited about the conversation. Let's start. Yeah, yeah.

1:22Daniel Whitenack:Well, like I say, we had bonded even yesterday when we were chatting about our backgrounds in physics and mathematics. I know you started out in mathematics and spent a bunch of time at Hitachi. Do you want to give us just an idea of a little bit of your background and what you've been involved with over the years? Sure, sure. So my background is actually quite diverse. It's sort of atypical of people in my role. I started off with a PhD in mathematics. Then I joined Hewlett-Packard Labs as a research scientist, working on data mining, machine learning. Then I joined Hitachi as a principal researcher, if I remember correctly.

2:02And then I sort of, you know, through the management ladder was, when I left Hitachi in 2026, I was leading all of AI research at Hitachi globally. It was a very strong team. And that's what I did. And we were focused a lot. We started focusing. We started with focus on industrial AI, as we defined it at that time. And as the sort of industry matured, we sort of started looking at a broader spectrum of things. And that's what I was doing. And then I joined Rackspace. And I've been here for four months. So it's been sort of a very, very fascinating journey. And in some sense, if I, prior to this podcast, I was thinking about my career.

2:46Typically, you don't think about these things. And I realized that I have a few, in some sense, I have followed the trajectory of the whole community at a broad level, right? I mean, if you remember, guys, we were all doing machine learning, making small models, right? Solving specific problems. And there was a community in Bay Area that was focused primarily on set accommodation systems. And as Chris, you would know, in companies like Lockheed Martin or Hitachi, we focus industrial problem, right? And so that's what I was doing for the first half of my career. And then more than the half. And then obviously, deep learning became important.

3:24We started looking at vision models, language models that blossomed in total as language models. And in some sense, a lot of the traditional machine learning problems today are much more solvable with automated tools, with sort of wipe coding and so on and so forth. And the new challenge now is, yes, you can do all this in machine learning and AI, but how do you make it accessible to more people? How do you make it actionable? How do you make it much more safe? And that's where rack space comes in. So in some sense, if you look at the trajectory of problems, that's how I have traveled in some sense, like looking for trouble, so to speak.

4:01I love that. So I've got it. I want to go back for a second. I'm going to drag you back as you kind of went through your timeline for a second, because as you were talking about kind of coming up through the ranks, the managerial ranks at Hitachi, you know, and you look at, you know, with with kind of the AI world exploding, you know, in terms of volume and importance and budgets and all that stuff. And so many times organizations will go to, you know, kind of an outside expert and bring them in to fill a particular key position and stuff. And yet you kind of came through the ranks at Hitachi.

4:38And I'm wondering if you have any thoughts about like what, what, you know, because there are other people out there that are watching this and listening to this right now that are in their careers, and they are aspiring to, to, to move up through the ranks themselves. And what were some of the things that you brought to bear that made you able to kind of move up through that and take on the leadership role at Hitachi before you were able to come over to Rackspace, just from a kind of a growth learning standpoint, if you wouldn't mind sharing? Chris, that's a hard one. I was more prepared to answer the I-related question.

5:13I'm sorry. But I think that's actually a very, very interesting question. I think one thing that I think helped in my career Hitachi was we decided to take a bit on industrial AI. This was 2016, 2017. And not very many people were talking about it. So we started a small research lab in North America. And I said, given where Hitachi is, it's a giant industrial conglomerate. It makes a lot of sense. But there wasn't a lot of background around it. And it was a risky bet. We were in Bay Area. We were competing with the likes of Facebook, Google for top talent. But we took that risk. We were sort of forward-looking in the sense that we figured that industrial AI, physical AI will become important.

5:58And we tried to succeed. And we did succeed to a large extent. And I think that sort of paved the way in some sense for management, for our leadership, from the CEO downwards to have confidence and internal sort of talent to take Hitachi to the next level. And I think if I look back, I would say look forward, see what's sort of around the corner. You know, it's difficult, but I think if you spend time thinking about it, not just reading about it, but thinking about it as well, gestating what's happening in the industry around you and take a risk, take a bet on something different, something new that is aligned with your company's direction as well, obviously.

6:43Then I think that sort of shows your leadership in terms of looking ahead and ability to take risks and then to execute. I think that's maybe that's what worked for me. I would say.

6:56Daniel Whitenack:Yeah, I love that answer because even, you know, Chris, I don't know if you remember, but the beginning of this year, I think we said, Chetan, we do a kind of forward looking episode usually at the beginning of each year. And one of the things we talked about, about kind of coming into its own was this idea of physical AI, I think, if I'm remembering right. And so you were very much ahead of that, as you mentioned, you know, you took maybe a gamble or a risk. looking towards that direction. I'm wondering for the listeners who might not be as familiar with industrial AI, physical AI, these sorts of terms, if you could just help us understand kind of maybe what that meant when you started getting into it and what that means now, if those things are different, you know, in any sort of way.

7:44That's a very good question, Daniel. And I think it certainly doesn't mean the same thing. Nothing means the same thing anymore. but when we first started out we were trying to solve industrial problems right so if you look at the industrial value chain from design all the way to manufacturing and this is not just manufacturing but you think about power plants you think about rail systems you think about any industrial system you could sort of organize the challenges in say maintenance design production you can look at these verticals. And if you took a step back, you realize that although every vertical is different, the data itself is different, but there are a lot of commonalities in terms of the problems that you could solve and in terms of the techniques that you could bring to bear on them.

8:31So initially, it was much more around a prediction, like your failure predict. That was a classical predictive maintenance problem. And then can you recommend the right repair to a technician and whether that technician is an automotive technician or sort of in some other domain, doesn't really matter. But the math behind it was somewhat similar. And that's how it started out. And those are the kinds of problems we were solving around maintenance, around quality, how do you sort of predict if a part is going to fail, and so forth. And then as the industry evolved, both the physical industries, meaning companies like, say, Hitachi, they started collecting more data.

9:13And sort of the AI industry also was evolving towards deep learning, right? And as the technology was becoming more mature, then computer vision-related problems became much more important, right? Can you detect defects on a surface? Can you count number of cars in a parking lot kind of a problem, right? And then as we got into large language models, then there was further expansion of what you could do in the industrial world itself. So one big move, obviously, is what people call physical AI now, which is sort of contested definition, but the idea that you could do robotics at scale. So part of the work that we've done on automation with traditionally reinforcement learning now could sort of be done in a much more formal, larger basis with sort of around robotics, right?

10:00Can you make multiple robots work together? Or even a robot can work in this environment or not. And also the traditional sort of industrial AI kind of problems also became on a larger scale. So people started expecting much better answers, a way for humans to communicate. Metaverse sort of becomes important where you could construct these sort of virtual worlds for training, for problem resolution, right? So in some sense, the field has moved along with the maturity of the AI technology as well. So that's it. And then this is a very exciting moment to be in that space as well, I would say, because a lot of innovation will come there.

10:41Yeah, I know that you've run labs both in North America and Japan. And as we're talking about physical AI at this point, it occurs to me as I'm listening to you that we have a global audience here and people from different parts of the world have different experiences in terms of this move into physical AI and some of these processes. And I would imagine, I don't want to put words in your mouth, but I would imagine your time in Japan, there's a lot more robotics out there. And as we're all sitting here in the United States, there's a little bit less exposure to most people out there in terms of physically and robotics.

11:24I think we're kind of, in my view, we're kind of lagging other parts of the world in that capacity. I'm kind of curious as you're looking at this, kind of what your experiences were about different parts of the globe and how that's moving the notion of physical AI forward in these various contexts you're talking about. And maybe, you know, with the advent of kind of embodied intelligence coming more and more into play with robotics and edge computing and stuff, what your thoughts are around that, you know, with your experiences. That's an interesting sort of cultural dimension to all of this. So what I relearned, and I should not make a cultural generalization of people, but I learned that at least folks in Japan are much more open to robots than folks in North America.

12:16I think that is something cultural. I can't explain why. And that's why even very many years ago, we were thinking about robots for elderly support. Even if they could not move, at least they could talk and understand and be empathetic. So getting the empathy right was a very hard problem. And just the way a machine would talk to a human, how do you introduce the right language, the empathy, the warmth that typically humans have for each other? and so in that sense um so for use cases like elderly care i would say yes so the united states was sort of behind uh other sort of asian countries uh not all the asian countries but like the leading countries in japan china and so forth and then when it comes to industrial robotics as well you're right the center of gravity is not in north america like the center if you think about large language models the center of gravity is north america but in sort of in for robots for the industrial world, the center of gravity is sort of in China.

13:18Today, you've seen sort of the robots to do multiple things. And, but I think we are catching up. We have some excellent startups that are sort of trying very many new ways for this, the robots to sort of work in the physical world, right? Whether they are for industrial use or whether they are for sort of more commercial use. And I would say the commercial use robots would be the first one to have an impact. Because a lot of industrial processes are already, in some sense, robotized. Because they're task-specific. They don't need to be that general. More general robotics are needed, more for sort of human environment.

13:55Whereas a lot of factories are quite automated already. Right? And I think that's where the next sort of battle lines are. And I think in that there is, I mean, it's anyone's game right now, I would say. because some of the underlying math is where we are better than everyone else. Some of the underlying mechanics maybe other countries are better than us. But yes, but culturally you are spot on, Chris, right? So there was a broader acceptance and maybe that is still there of robots for day-to-day interaction in Japan compared to St. North America. Does that answer your question or did I sort of take it down?

14:34That was a great answer. I appreciate it.

14:39Daniel Whitenack:I hope that you're finding this episode practical and helpful. As the name suggests, we want this podcast to be practical, not just hype. And that's one of the reasons why we've partnered with the Midwest AI Summit, which is happening October 15th in Indianapolis. This summit is more than just a bunch of hyped talks. There's really practical things there, like an AI engineering lounge, where you can sit down with experienced AI practitioners like myself and others and work through your architecture, your tool questions, your roadmap, your strategy, et cetera. And that way you can actually leave with a huge value from the event.

15:20Daniel Whitenack:Again, the event is happening October 15th in Indianapolis. Some amazing speakers, some practical advice. Don't miss this event. Make sure you're there October 15th in Indianapolis. us, you can check us out at midwestaisummit.com and you can use Practical AI 20 to get 20 % off registration. Again, Midwest AI Summit and you can use Practical AI 20 to get 20 % off registration. So Chetan, I want to circle back to a comment that you made as you were talking about the arc of your career and landing at Rackspace. You mentioned something to the effect of going with the technology, but now moving to a place where you could make it more accessible and safe and et cetera and operationalize it, if I'm understanding you right.

16:16Daniel Whitenack:Could you help us understand that dynamic a little bit more around kind of the technology? Now it almost seems like with AI, everything is feasible, but not everything is easy, right? Or not everything is, you can't operationalize everything, you can't make it accessible. Could you help us understand your thought process around maybe that transition then from that world of industrial AI to now where when you're thinking a lot about these types of issues around operationalizing, scaling, making accessible these technologies in an actual useful and safe way? So, Daniel, when we were working with our industrial partners, both internal and external, we were a team of highly trained researchers, engineers, PhDs and all of that.

17:09And so, we will take up a customer problem, we'll understand the data, we'll build the model, and then we'll work with the customer to deploy it. Right. So and that deployment just not meant sort of throwing the model over the wall, but you sort of work with the customer, you talk to them, you understand exact business pain point, you understand the workflows, and then you integrate your model in the workflow. And typically, that model was small enough that you could have it in their own environment. Right. So and this is what we did again and again. But if you now look, and if you look back, like maybe even five years, most of us were not using AI directly in a day-to-day world, right?

17:47So AI was always mediated through maybe a solution that was developed, deployed, and operated by someone, right? So they took that responsibility. Now, with generative AI, AI has become democratized, right? Everyone has access to AI. Every enterprise wants to use AI. And you don't have this team of PhDs and researchers everywhere who can sort of build a model and deploy it. First of all, it's cost prohibitive, right? These large language models are typically very difficult to build. They're very expensive to build, right? So the whole model of how you would bring AI machine learning to an enterprise changes with generative AI.

18:28So that's the challenge, right? So at a high level. But what does that mean in practical terms, right? The number of things, right? So you ask a question, right? I use, for example, some large language model. Now, I could simply ask, what's the capital of, say, Norway? So for that, I don't need to go to, say, chat GPT, where my tokens are expensive. If I have a local model, I should ask it that question. It will be much more cheaper. And, right, so that's Sambhava. Number two is, this is an issue raised by Satya Nadella and even, I think, Jensen. Whenever you, as an enterprise, say, now you want to say, I want to operationalize AI, AI is accessible.

19:07I want my staff and everyone to use AI. Now, every time you ask a question, say to a large language model, in some sense, you are sending your data over to them. So your data sovereignty is not guaranteed, right? So the way Satya Nadella sort of said it, you're losing your alpha, right? That's your IP. So that's another problem. Then an AI in some sense today is jagged. So meaning that there are some tasks for which AI is really good at, right? So I can really write some very good sort of software program using AI, right? I can code with AI. But if you try to write an email with AI, you realize that it's not all that great, right?

19:48It's quite, right? So I get AI emails and they sometimes annoy me because they're sort of very verbose and, you know, they're very cliched in some sense, right? So although we thought that writing is the strength of AI, but it is turning out that it's good at it, but not that great at it. Humans can still do better at writing emails maybe than AI can. So that means in terms of capabilities of where AI is good or bad, it's jagged, right? So for an enterprise, they need to know, right? So this is where I should use AI. and then to my first point earlier, whether I should use a local model because that's more expensive, should I use a larger model and then how do I preserve my alpha, right?

20:33So these are only three questions that are today difficult to answer. Now, the jargon of AI also is in terms of how do you guarantee that it behaves in a responsible manner, right? So we all heard that Enthropic sort of the latest model hacked a bugging face website, right? So now, if you're going to deploy your own model, suppose you say, I want to develop my own model within the enterprise on my own data, and you deploy it, how do you guarantee that it behaves in a way that is safe? And also, if you're using someone else's AI, like you're using a large LLM, how do you ensure that it behaves within guardrails?

21:12It behaves in a way that is suitable for your enterprise. So the problem of governance and assurance becomes very important. The problem of how do you maximize the value of your AI through tokenomics becomes important. The value of preserving your IP becomes important. And the problem of having the right architecture so that you can cater to the multiple needs within the enterprise, from say programming to writing emails to summarizing to research, can be done in sort of a safe, guaranteed manner so that you can sort of interchange the models if needed. you can sort of, right? So because the model technology is changing very rapidly.

21:55So all of that requires a very systematic way of thinking about your AI architectures. And I think this is the next frontier. This is the next challenge, right? So now how do we go from simply sort of chatting with an AI through an interface to working in the enterprise where there are people, processes and all of the complications. It's a very long-winded answer, but I hope sort of it addressed the question. Not only was it good, I'd like to actually get you to extend it a little bit by throwing a couple of extra logs on the fire. One of the challenges that we see in industry right now, and it's been evolving over the past, especially over the past year, is where open weights or open source models are available from.

22:42And there's so many external considerations that get brought into bear as, you know, while, you know, to your point earlier that the center of gravity for model development, you know, for LLM development may be still in the U.S. A lot of those are closed. The number of available open weights models kind of shrunk a little bit with, you know, Meta's kind of going away from that. And NVIDIA started stepping up a little bit more because the rest of the commercial industry in the U.S. was reducing. And we're seeing an explosion of capability in terms of new models that are kind of rapidly catching up.

23:22We're very close followers or potentially equal from China. and overlaying all this, you have all these countries have their various exports concerns, import concerns, what you're allowed to use. And that makes it quite complicated as an ecosystem for companies in various businesses to try to figure out what makes sense for me. You talked about governance. You talked about data sovereignty. How do you navigate with some of these big issues? that go beyond the technology of AI and the implementation of AI and can affect management concerns all the way up to the CEO and the board of directors. How do you start, if you're a company now, it's late 2026, and all this has rapidly developed, how do you look at all these and make decisions for strategic interests in your organization going forward?

24:19Because it's quite the quagmire at this point. Yeah, no, and it's quagmire and the changes at a dizzying pace as well, right? It does. So there is no easy answer to that. I just, I do want to address the question of open models though, right? And I know you said that, you know, we are somewhat behind as, in terms of open models, but I think the beauty of United States is that the right incentive, we really step up, right? So now we realize, maybe the AI community realizes as well that, look, with open source models, maybe they're from China today. Obviously, NVIDIA is doing a great job in it, right?

24:56There is a lot of merit in that. And from what I know of people I talk to and friends I talk to, I think it's a matter of time before our open source models will beat the thought of everyone else. That's number one, right? On number two, like, how do enterprises get started? I think they have to sort of fix few things, I would say, right? Fix few things, meaning they need to sort of understand how much of their workload is sensitive today. Meaning if you are say doing an HR query, maybe it's okay to go to an LLM. So one thing you sort of figure out what's data is something you really want to work around.

25:31And there ideally you should sort of think about local models, open-weight models in your own environment that you control. So I would say that's one North Star you should try to fix. Don't marry into any model family because they will swap in and out both for commercial reasons geopolitical reasons right so so all of that will sort of evolve change so don't marry into any sort of uh but marry into an architectural way of thinking right meaning that these are the workloads that i can push to the x to uh to an llm outside llm this is these are workloads that i need to have on prem or in a government environment and what are my governance and assurance layer that I want to have.

26:17So what are the properties that are important to my enterprise, right? And that I really need to enforce in the way I work with AI, right? So I think if you have some of those principles pinned down, it becomes an easier way to get started. And I think most, and then a year or so, I would have said, right, pick one problem up and then do it well all the way and then pick the second one up, then do it all the way because at least six months two years ago there were so many pilots and like not as much impact on the bottom line or top line of a corporation but i think people are learning that lesson so i'd say but but yeah but that's the other thing right pick one or two set of problems that are meaningful uh in terms of impact if you're not started yet and start with there but for most enterprises which are already somewhere in the journey that have started i would say, stop thinking models and start thinking architectures, right?