#199 Shub Bhowmick: How Gen AI is Transforming Modern Data Science and Management

0:00The most powerful tool that's available to CEOs today is AI. If AI is embraced and made useful, companies can fundamentally become more profitable, improve customer experience, grow faster, and so on. So in order to be future ready, CEOs understand that they need to figure out how to make AI a part of its DNA. Hi, I'm Craig Smith, and this is Eye on AI. In this episode, I talk to Shubh Bomek, CEO and co-founder of Tredence, a leader in data science and AI solutions, helping enterprises leverage advanced analytics to drive growth and innovation. Shubh explains how AI is reshaping industries and talks about the challenges companies face in AI adoption and Tredence's vision for the future of AI in business.

0:54I hope you find the conversation as informative as I did. Hi. Good tech solves problems that you've thought about. Great tech solves problems that you haven't even thought of. What can the commerce platform trusted by millions of merchants do for you? It's time for Shopify, the commerce platform revolutionizing millions of businesses worldwide. Whether you're a garage entrepreneur or IPO ready, Shopify is the only tool you need to start, run, and grow your business without the struggle. Shopify puts you in control of every sales channel. So whether you're selling satin sheets from Shopify's in-person point of sale system or offering organic olive oil on Shopify's all-in-one e-commerce platform, you're covered.

1:42Shopify powers 10 % of all e-commerce in the United States. And Shopify is truly a global force, powering Allbirds, Rothy's, and Brooklyn, and millions of other entrepreneurs of every size across over 170 countries. Plus, Shopify's award-winning help is there to support your success every step of the way. So sign up for a$1 a month trial period at shopify.com slash IonAI. That's shopify, S-H-O-P-I-F-Y dot com slash IonAI. That's E-Y-E-O-N-A-I, all lowercase, all run together. Go to shopify.com slash IonAI to take your business to the next level today. Give them a try. They support us, so let's support them.

2:37So why don't you go ahead just introduce yourself, Shoup. Sure, sure. So first of all, thanks for having me, Craig. Really appreciate the opportunity. So Tredence is, you know, we are about 2 ,500 team of data scientists and data engineers. Fundamentally, we are curious-minded technologists. We love to innovate. What we do is we supercharge our clients with AI capabilities in a custom way. So we are not an out-of-the-box product. And we help them minimize the noise, focusing on real value creation and unlocking value from relevant data by driving business outcomes. Today, we're working with close to 25 Fortune 200 clients, including the largest companies in retail, CPG, as in consumer packaged goods, travel hospitality, telecom, financial services, healthcare, and so on.

3:48helping these companies improve their customer analytics and driving better experience, make their marketing campaigns more targeted, more measurable, and reducing cost of acquisition, augmenting revenue and pricing algorithms with more modern AI and improving their supply chain. As far as I'm concerned, prior to starting this company, along with two co-founders, Sumit and Shashank. I was in tech services, but the info is in management consulting with PwC and private equity. Let me take a pause there, Craig. Sure. Yeah, go ahead. Yeah, well, you focus on consumer packaged goods, or that's one of your focuses.

4:40why CPG and are there particular challenges to leveraging AI in that industry? Yeah. See, the most powerful tool that's available to CEOs today is AI. If AI is embraced and made useful, companies can fundamentally become more profitable, improve customer experience, grow faster, and so on. right so in order to be future ready uh ceos understand that they need to figure out how to make ai a part of its dna this is literally true for any business including all retail and cpg companies and why we are strong in this sector we have designed our evolution that way we started with retail and CPG, but now over the last 10 years, we have actually built strength in a few other verticals as well, including travel, hospitality, tech and telecom, and now increasingly in financial services and healthcare as well.

5:49But coming back to CPG, yes, you are absolutely right. We are working with 80 % of the largest retailers and CPG companies of the world. We have a significant specialization in these sectors. It's more than 50 % of our business today. And we have now become the go-to provider in data and AI in this space. We help them solve the most complicated and hairiest challenges. Do you want me to give you an example on this? Yeah, exactly. That's what I was going to ask. So this is a very interesting transformation work we're doing for one of the world's largest retailers. And the way it happened is last year in NRF, NRF being one of the largest retail conferences in the world, we met one of their C-suite executives.

6:46This is one of the largest convenience store operators, more than 500 stores globally, and they're ranked as a top 10 global retailer. So this executive, he was struggling with challenges that I'm sure you've heard elsewhere. where lots of data sitting in silos, recent acquisitions, large legacy footprint, redundant systems, hence leading to data quality challenges, lack of certified data assets that their stakeholders could rely on, significant lag in the data, to take multiple days for information to become available for analysis and insights and so on and so forth. So we started, we met there.

7:40You know, this gentleman, he liked the story we presented and the capabilities we demonstrated during the initial meetings. So, and so we started. We started on the engagement and over the last one and a half years or so, we've been working together. We have gone through a significant evolution. You know, these engagements, Craig, typically starts with helping the executive develop the strategy, right? These are blueprints based on conversations we have with their executive team, with their other technology leaders, and then bring our expertise into the mix and put the roadmap and the blueprint together, right?

8:26and then it's the work starts happening one phase after another phase uh typically we would start with helping them do the data foundation right so help them uh you know move close to 90 percent of the disparate data that i just mentioned to a standardized unified data model with standard definitions right so how did this help it helped them improve the trust in the data and unlock of previously hidden insights. Then it moves on to sort of setting up the governance, right? How do you keep this data robust and high quality over a period of time? So you set up an enterprise data governance console, you implement the cataloging tool sometimes, and the data stewards from this console are meant to define and align and back up these definitions that we just talked about, right?

9:23and and i can give you a lot of examples how the definition of similar metrics may vary what one person is thinking versus another person is thinking for example in retail you know retail sales per store per day a metric as simple as that there are many different ways to calculate it you know does it include returns or it doesn't does it include field sales or it doesn't right So getting alignment on all these basic definitions while you're setting up this new lake house is super important, right? Once you're done with the foundation and the governance structure, then you start working on the reports, right?

10:05How do you rationalize the reporting environment and get rid of all the redundancy and the wastage, right? With reports that no one looks at, right? And this is where we bring our accelerators. We have something called T-ingester, which is meant to rationalize a bunch of these duplicate stuff. And we have something called a medallion architecture. I'm not getting into the details of the technology, but it's meant to kind of drive this consolidation and one unique view of the data and the business. I'll give you one example here. by, so in the process of doing this, we were able to significantly reduce the latency, right?

10:50So we moved them from a batch-based architecture to a real-time architecture using structured streaming, and that actually reduced the time of doing, going through this consolidation process by almost 15 hours, right? So as an exercise, we did something interesting. We, so they had a store in the lobby of the headquarters. So we made a purchase there and then we walked up to the room of the CEO and he was able to see the purchase within five minutes. It was tacked to his loyalty card, right? So we made the process a lot more real time. And beyond that, I'll just say one last thing then I'll stop after that.

11:35So you've set up the data foundation, you've set up the analytics and reporting layer, now starts your AI and data science journey, right? So this is when we bring together all our AI and ML accelerators, you know, first of which was demand forecasting. So this particular retailer had a lot of spoilage in their P &L, right? They're making a bunch of perishable food and throwing out a significant percentage of that because the store managers did not have a good measure as to how much should they make, right? Again, leveraging the accuracy of our accelerator, we were able to beat the previous accuracy of the model by almost 500 basis points, right?

12:20Leading to a significant reduction in the spoilage. And these kinds of benefits are typically certified and validated by the CFO because it hits the bottom line directly of the company, right? And over a period of time, this journey continues to evolve. We are now building more and more additional AI and data science and data engineering based products for this customer and driving business outcomes across a variety of functions in the organization. Yeah. How much, what flavors of AI are you using in this? I mean, I'm imagining that you use sort of predictive analytics, for example, for the spoilage of some just simple, not simple, but supervised learning.

13:22but are you using generative AI now in report generation, for example? This particular project, we have now started to go there. So far, it has been more classical machine learning and deep learning mechanisms and modeling techniques. but now we are starting to explore, including aspects of generative AI, looking at unstructured data and so on and so forth. So why don't I share with you sort of our Gen AI journey, and then I'll be able to come back to the question you're asking. Sure. Yeah. So that's been a fascinating last 18 months, to say the least. Craig, I'm sure you are talking to a lot of leaders who are in this space and they have been sharing examples with you.

14:18We have come about our own journey in the last 18 months, as you can imagine, building these Gen AI based enterprise applications. And I'm going to divide this into four or five phases. So phase one was the introduction of foundational models. This is when we were also starting to explore, Chad, you know, GPT 3.5 when it first came out, November, 2022, and we were starting to explore, we were exploring what is the art of the possible, right? Intelligence search, you know, so we started experimenting internally as well as some customers. And very quickly we started seeing a deluge of models, right?

15:04A lot of foundational models, and that was adding to the complexity of when to use, what to use, and so on. So phase two, then I would say, which started probably about four or six months into this, we started to contextualize the models because the foundation models by themselves were a little too generic. They were wide-ranging, which is powerful for generic use cases, but unable to create specific accuracy. right and models need that context this is when we started looking at techniques such as rag and prompt engineering and how to basically how to make these models more effective by adding more contextualization vector data augmentation right and this is when the focus also started shifting from training to influencing right see in traditional ai what do you do in traditional classical machine machine learning craig like you start with the base model where you build the base model and then you experiment with it right you tinker with the weights and biases but in case of large language models or generative ai foundation models the model is already massively trained right you're starting with a very strong foundation so your job is to not start from scratch but take what you have and fine-tune it.

16:29Retraining it only if necessary, which is very expensive. So I would qualify this as phase two. Then moving on to phase three, we started going deeper with the customers, specific problems in smart manufacturing, in product innovation, right? Things needed to be done in a domain-specific way. So domain thinking, domain content, which is our strength, by the way, because we go to market by the verticals that we work in. So bringing all that context into the work that we were doing with our customers. Moving on, phase four, which I would say has been more in the last six to eight months. we pivoted and started focusing more on operationalization, right?

17:23Industrialization of all the experiments that were happening in the lab, in the POC phase so far, right? Moving past the incubation phase and focusing more on, you know, driving the value into production and hence driving business outcomes, right? And then phase five, which is where I would say we are now, are kind of getting started you know so as the solutions and these models continue to mature now we are focusing more on adoption right enterprise adoption which also by the way is going to be a very interesting journey you know traditional ai took more than you know you know it took a long long time to get to where it got to so with gen ai now you know exploring so while you do this also explore new features like multi-agents right but the emphasis is how do you make it simple and easy for organizations right uh basically we are now starting to see a lot of conversations around intelligence in different forms right uh every application uh craig in the world has to be reimagined as an ai application right and and the whole the the dance between humans and AI where, you know, humans are being augmented with AI and working together as co-pilots.

18:45It's our job to ensure this dance happens and it works in unison, in tandem, right? So, and I believe it's not a zero-sum game and a people versus AI, it's actually a compliment. So we are very excited to make this unison work in the most productive and effective way. So if, in what parts of, I mean, can you give me an example of what parts of an industry or of a company do you focus on, are you proposing that they use generative AI as opposed to more classical AI? Yeah, so the applications of generative AI, right, it's everywhere, right? In all functions of the business, pretty much all functions of the business, whether it's marketing or supply chain or customer analytics, wherever you have data, obviously, you have to start with setting up a high quality data strategy.

20:03But once you have that, you can use Gen AI to leverage and find more value off of that, right? Why do I say that, right? Because, you know, one of the data, you know, I'm sure you've heard this, close to, I'm trying to remember, I think, close to 80 % of business relevant information is still in an unstructured format, primarily text. So which makes it a prime candidate for generative AI. For example, in an organization, if only 20 % of the data you have been leveraging so far in structured data analytics and models on top of that, now you have access to that additional 80 % that has been sitting in unstructured formats, right?

20:58Now, the question is, I think the more, I think what you're getting to, Craig, is how do we do this, right? So typically we, again, follow a three-phased approach, right? We start with, so we have an offering called Foundry to Factory. So it starts with what we call Vision Quest. These are basically discovery and design exercises conducted by workshops and interviews where we help our clients identify the most optimal use cases to drive business value. The ones that are realistic and adequately worthwhile for both the CFO and the CEO. And then moving on, we set up a sandbox environment and an ecosystem to start the incubation and experimentation.

21:47We call this the foundry phase. This is when we bring all our accelerators, such as text to SQL, and I'll talk about what it means in a second, other domain centric large language models and so on. Right. And then the third phase, which where we call the factory phase, this is about, again, operationalization. Right. Which is in a plumbing with the various large language model options that exist out there, both open source, you know, Lama 3s of the world, as well as closed source options like Gemini and GPT and so on. And then running the processes securely, ensuring there's no biases, ensuring there is no hallucination.

22:38See, one very important thing to think about is in chat GPT, when we ask that agent a question, and I'm sure everybody does that nowadays, including our kids. If it doesn't know the answer, it sometimes tends to make it up, right? Which is what we call hallucination. But in enterprise world, that's not okay. You cannot make it up. It's okay. The model should say, I don't know the answer to this question, rather than making something up, right? So how do you ensure these things? You know, another important problem that we always are looking to solve is disinformation and deep fakes. How do you eliminate all that?

23:23So all of these things is packaged into something called responsible AI in a box. So while we do the help of our clients with the operationalization, we have to handle these technical challenges, but also these other challenges, which we call responsible AI. yeah well i was going to ask you know i had read that you had launched a gen ai as a service platform uh for enterprises can you talk about that this show is sponsored by better help comparison is the thief of joy and it's easy to envy other people's lives it might look like they have it all together on their Instagram, but in reality, they probably don't.

24:11Therapy can help you focus on what you want instead of what others have, so you can start living your best life. I've benefited from therapy. Many of my friends have benefited from therapy. It's no longer something that people are ashamed of. If you're thinking of starting therapy, give BetterHelp a try. It's entirely online, designed to be convenient, flexible, and suited to your schedule. Just fill out a brief questionnaire and get matched with a licensed therapist, and you can switch therapists anytime for no additional charge. Stop comparing and start focusing with BetterHelp. That's BetterHelp, H-E-L-P.com.

25:01Visit BetterHelp.com slash IonAI today to get 10 % off your first month. That's BetterHelp, B-E-T-T-E-R, H-E-L-P.com slash IonAI. Eye on AI all run together, E-Y-E-O-N-A-I. So visit betterhelp.com slash eye on AI for 10 % off your first month of therapy. You'll enjoy it. Sure. Yeah, it's a kind of an enablement. It's an orchestration platform to do a bunch of things we just talked about in the last 30 minutes or so. It offers a range of features, including deployment of the Gen AI and AI solutions, infrastructure optimization, accelerators to build these models, and responsible AI governance and monitoring.

26:08And what it's meant to do is help enterprises navigate the complexities of generative AI solution adoptions. So what do we do as part of this service platform? Step one is typically building the data foundation. Again, breaking the data silos, integrating the data in real time in the lake house environment that we talked about. step two is democratizing the data like once the foundation is built how do you drive self-service access to this information democratize the data to business stakeholders to various business functions in the organization step three is governance you know creating that one version of truth when you look at a certain business metric everybody is calculating it the same way they know how it's calculated step four is the platform itself support and operations 100 percent and uptime, and our ML Ops processes to ensure that the models are staying in tune.

27:10Step five is the value unlock layer, meant to drive analytics consumption, report rationalization to move from legacy to more modern self-serve reporting platforms. And then step six is the accelerator layer to answer all the common business questions and business challenges, anomaly detection engines to find patterns that are not favorable to the business at scale. Conversational AI engines, again, a very, very popular tool that we are leveraging nowadays to take advantage of all the integrated data to drive, for example, recommended persona-based insights to the business, right? Answer commonly asked business questions, right, in that context.

27:57So we are implementing this platform for clients across various industries, including retail, CPG, banking, healthcare, et cetera, and helping them modernize their data infrastructure and deploy AI-driven solutions in a secure way and with an emphasis on speed to value. Because gone are the days when we start a transformation journey and we wait to see the results two years down the line. And nowadays, you know, CFOs and CEOs expect those results within months, if not weeks in certain situations. So we have to be extremely agile and high quality while we are helping our clients through these journeys, Greg.

28:41yeah and just so i understand so a fortune you know 500 company comes to you they're uh a retailer or or consumer packaged goods manufacturer and uh or or some other sector and uh you take this gen ai as a service and you embed it in there excuse me no worries if you want get a glass of water please no no that's fine uh you take this uh gen ai as a service platform and embed it in their systems uh and train them on how to use it and connect it to their data excuse me or is uh is this a self-service platform that your clients and have access to and go onto the Treedance platform to build their Gen.AI tools.

29:44How does that work? Yeah, so we are not a software company, Craig. So this is how we do things, right? So we are a services company. We are a tech services company which specializes in data and AI. and what makes us unique is our accelerator portfolio. So it is neither of the two things that you mentioned, right? So here's the thing, right? So clients have two choices today. You go and buy a software as a service product. You know, you're going to get a faster product and speed to insights because it is something that's ready-made, something that is off the shelf. But the downside is that you're buying something that is off the shelf.

30:32is meant to work for everyone but in reality it doesn't work for anyone right plus all the plumbing work that you need to do to to feed the product with the data that it needs to be useful that usually takes a long time and the whole value proposition of speed to insights unfortunately goes for a toss flip side or the other option is that you start from scratch right you build exactly what you want your exact business processes your way of looking at the data your data model et cetera. Challenge is that it's a very time consuming process. It's got a long development lead cycle and you typically will end up missing the business opportunity.

31:13So at Traden's, through our accelerator portfolio, you get the best of both worlds. Leveraging our accelerator ecosystem, similar to the platform that we just talked about, which is by the way, certified by are hyperscaler partners. They're meant to help you with your data migrations, all the other components, data governance, cataloging, AI, more than 150 pre-built models that are already sitting in that platform. So you get a head start to a large extent, about 50 to 60 % along the way, but the remaining 30, 40 % customization that you need to do, especially which is super important for large companies in fortune 500 uh in amongst fortune 500 companies you get to do that customization so you get a head start and you are able to quickly get to that uh finish line uh leveraging the pre-built capabilities in our platforms am i making sense uh craig yeah yeah and so if i'm a ceo and i and i don't know where to start uh i go to treedance you guys sort of analyze my business case and propose places where AI or gen AI can improve things.

32:37And then do you have, is it like a consulting firm where you have consultants sort of embedded in the company, helping the CEO through this journey using the various tools that Treatance has in its portfolio until it's set up, and then you turn it over. Is that how it works? That is accurate, but if you allow me, I'm not a big fan of the word consultants because consultants typically work on PowerPoint. But in our case, it's about driving the business outcomes. It's about writing the code, the technology, developing the technology that is actually meant to drive meaningful value. So yes, there are certain engagements where we would start with a very quick initial discovery to tell you what needs to be done, but then we will also help you through the execution journey, which is typically 95 % of what we do.

33:41Right, right. And I had read somewhere that you're investing a lot in developing your Gen AI capabilities across sectors. Is that right? Yeah, we are. We are. I mean, I think this is a big part of our future. And this is where the industry is going to spend a lot. And there is a lot of meaningful value. I'm so sorry. The sun is pushing into my face. So we are investing a lot into this area.

34:24So, you know, I had read somewhere, Craig, and I'm sure you have seen similar stats. You know, AI is supposed to contribute more than$15 trillion to the global economy by 2030, right? And generative AI and AI in general is a central priority for most Fortune 500 companies right now, right?

34:51So hence we are making it a big priority for ourselves. And here's the thing, when you are trying to help your clients with innovation, when you're trying to help your clients with, and giving them the headstart that I talked about in terms of accelerators, you have to invest, right? You have to invest in developing capabilities that are domain focused. Now, what we don't try to do, Craig, it's important to know this, we are not trying to become one of the underlying technology hyperscaler platforms, because that job is already being done by Microsoft and OpenAI and NVIDIA and Google and Databricks and Snowflake and AWS, right?

35:33So we're not trying to replicate what they're doing, right? Let me take a step back. It might be better for the sun. What we are trying to do is create the application layer on top of that, right? The domain contextualization on top of that, right? And doing the last mile plumbing and combining all these foundational capabilities and driving business outcomes, working very closely with our customers, right? So that's how we differentiate. And that's how we sort of create value for our clients. Yeah. Do you publish, does Tredans publish research that it's doing or that's coming out of its experience?