Highlighting Data Intelligence with Databricks & Bonbon’s Reward Innovation | E2020

28:21And so I can get most of the work done with English. But then there's gonna be that long tail of stuff that needs to be really precise. And I think there's just no shortcut. You've gotta be really precise. I mean, it's like a legal document. It's the same thing. Yes, it's written in English, but it's not really English. It's very precise language that's meant to describe something, you know, and it could be English, but you know, it's still got to be precise. Does that make legalese then a very high level object oriented programming language? It absolutely is. No, it totally is. Right. Actually, there've been attempts to make it into code where I have strict syntactical requirements and things, which actually enforces potentially correctness and verifiability.

29:02You know, this is a programming languages topic that's been around forever. You know, I suppose, though, if we do then use AI to automate the legal industry out of most of its employment, we can then just take lawyers and turn them into developers because they'll already be familiar with the style of communication. There we go. Problem solved. Labor market fixed, everybody. You heard it here first. I want to talk about the enterprise, though. And one reason why is I am a little bit muddled about agentic AI. And I know you guys have talked about recently the Mosaic AI agent framework and agent evaluation services.

29:36Sierra by Brett Taylor is working on agents inside the enterprise. To me, it's become a bit of a catch all term, if that makes sense. And so I'm kind of kind of curious to me, how do you slash Databricks define agentic AI? And then I want to dig into that a little bit. But let's start there. Yeah, you're right. It is a catch-all term. It became the new cool thing to say, and it's not well-defined as to what it really... I mean, I would say how the term came about was building something that can work through a set of steps, some sort of a workflow, break a problem down and actually work through multiple steps to get there.

30:12So we see this all the time with RPA, Robotic Process Automation. Yeah. Okay, everybody. You know I love newsletters. I love sharing knowledge and what I'm up to through my newsletter. And if you've got a newsletter, you got to check out Beehive. It's spelled B-E-E-H-I-I-V. I use it for the Twist ticker and our Twist 500 newsletter. My team is raving about Beehive because it is an all-in-one platform that not only powers our newsletter, but it's got all these incredible platform features that are helping us grow our subscriber base, which is what these newsletters are about. Listen to this. Beehive's co-founder is the same person who helped Morning Brew reach millions of subscribers.

30:59In other words, they took all those tactics and they put it into a platform. They've got a great feature I just want to tell you about. It's called the AI Post Builder. This makes writing easy. That's a great writer. Sometimes I need inspiration. Well, they will get inputs for your ideas and then shape and optimize your content for the maximum impact. It's perfect for busy founders, right? And they're available 24 hours a day. They also have a referral program that turns your audience into ambassadors. It works great. Plus, hey, if you want to monetize, they got an ad network. And you know what?

31:30It's super affordable starting just at$39 a month. So here's a great call to action. 30-day free trial plus 20 % off your first three months. Go to beehive.com slash twist, beehive.com slash twist, B-E-E-H-I-I-V.com. What a great product. Yeah. So we want to say like, here's a common pattern I do over and over again. Maybe it's like, I have a bunch of data in one file. I cut and paste that. I drop it into an Excel format. Then I go and apply this formula. Like it's a common pattern we see over and over again, but it's just very hard to automate. So I think that's what the RPA stuff is all about.

32:09And agents were kind of a way to say, well, maybe I can have it observe what a person does and then learn. That's where this started. I think we're not quite there from the perspective of I can just turn it loose and be 100 % correct. But what we want to work toward is doing that within Databricks or other data platforms, because these are the kind of workloads that dominate inside of Databricks. People do ingest of data, they do some sort of transformation, then some sort of dashboarding, and they can make some kind of decision. Can we start to automate some pieces here? Really, that's the overall vision.

32:41We have a few different ways that this looks today. Okay. I want to pause and talk about this because it sounds like when you describe breaking up a problem into smaller pieces, taking steps, honestly, that reminds me of going back one step, what I see with O1 when I asked to show me its logic. So to me, it feels like the agentic AI you're describing is kind of like what that model is already doing. And I presume here that I'm confused and I'm oversimplifying things. So can you just set me straight? No, conceptually, you're right. And I think what the big difference is between that and what we're doing is the idea that I want something that's really truly bespoke and built for my process, my data, built from my data and the learnings that I have within my company.

33:24O1 is great. It's a concept and it works well under the conditions where they tested it. But you can break it pretty fast. I mean, if you just go on Twitter and look, there are lots of people that broke it and made it do stupid things with really simple questions. And I think this shows that to make these kinds of systems work, you do need specificity. You need to work from your data. Your data is the ground truth of what should really happen. It's not a general problem. And you know, if this is not a foreign concept, if we look at like humans, the same problem, like if I got generic smart kid out of college and said, go solve this problem that I have 20 years of institutional experience around in my company, they're going to fall apart too.

34:02They're not going to get the question right. Maybe if they're really smart over time, over the next couple of years, you could train them to be really good at it. So I think this is the same thing we're seeing here is that we see the value in building things that are bespoke and experts within a particular domain. We call this concept data intelligence, where you can use your data to create intelligence, to actually automate things and make it useful for your business. Yeah. And I actually, I watched a clip you guys sent over about the difference between general intelligence, which is essentially a large model that can do cool stuff.

34:35And then data intelligence, which is that model applied to your existing corporate data set. And this, by the way, is always why Databricks, Data Lakehouse, plus Mosaic ML made sense to me, because why not have a brain on top of your bucket of data? But at the same time, in that same post about the Mosaic AI agent framework, you said, or your company said, sorry, while building a proof of concept for your Gen A application is relatively straightforward, delivering a high quality app has proven to be challenging for a large number of customers. And I'm curious if that issue is around the data they need to have perhaps cleaned and ready to be put into the model, or if it's something else that's making it difficult to turn a bucket of cool data, cool models into an actual product people want to use inside their company day to day.

35:18Yeah, I mean, there's a lot of things to pick on here. So data quality, data cleaning, all of that. Absolutely. You need to get your data in very good shape, because if you're going to start calling this data the ground truth, you need to make sure that's really true. so you know bad data leads to bad outcomes that's true for humans too if you give someone the wrong information they're going to do the wrong thing um sometimes you give them the right information they did the wrong thing or the wrong information the right thing humans are bro humans we could talk about that a lot too i know let's not get let's not get bogged down sorry it's an election year so i have a lot of thoughts about that but let's sorry back to you navine uh but humans are actually still quite amazing at understanding causation and how to get to an end goal, which we haven't quite cracked yet in this field.

36:03When we look at some of the problems I've seen from different companies is, you know, I would characterize the second half of 2023 and the first half of 2024 as every Fortune 500 is like, oh my God, what's our JII strategy, right? Everybody had to have this. I've talked to boards of large companies and they all said they're asking their management, you know, what is your strategy? And the strategy was go get an open AI account. Right. And I think, you know, people that really know. That was my strategy too, Naveen. I mean, their defense. And that's fine. You know, I think it's a place to start, but really you need to understand what success is.

36:39This has to have a business impact. Like just putting a new feature on your banking app may or may not have any effect on your business. It has to be something that your users want, that they get value from, and actually use your platform more or do more banking or do more loans or whatever it is. So actually thinking through that is still a business strategy question. It's really not an AI problem at that point. It's really, is this a good thing for my customers? Perhaps at some point, AI will be able to tell me that. Today, that's not really true. Okay, so now we can say, well, I've identified a problem.

37:11Great. Now I want to think through what is success criteria. So I can take components that are in Databricks and I can start to assemble them i can get to a working demo pretty fast as you said and that demo might have some kind of performance uh attributes to it it's going to break in certain places oftentimes we find that this looks cool from a demo perspective but if you actually go and uh try to roll it out you'll see all kinds of degenerate use cases all kinds of places where it breaks okay so let's characterize that let's build an evaluation and give it a score this is a 37 correct for my for my application or is 37 % useful for my users.

37:47Like start to score this stuff and evaluate it. Once you score it, now you're in a place where you can actually start to improve it. We have all kinds of tools that can say, okay, well, let's try this to see how it moves the score. But you need to get to that point. And I think this is actually the blocker for most of these applications. Beyond all the plumbing stuff of security and governance, you need to get all that right. So I think there's a lot of steps that are kind of in feeding progress for enterprises. And we're trying to break down those steps and make it systematic for our users. So I'm familiar with the CEO of a company called Skyflow, Anshu Sharma.

38:23He's a friend of mine and likes to explain things to me. And one thing that he was early on was helping corporations keep PII safe. And then the Gen AI boom came and everyone was talking about putting the data to work. And I presume that there is the same kind of tensions between people wanting to keep their data properly permissioned and only the right people can see it and i can't ask about naveen's work data and so forth has databricks and other vendors solved that issue or are we still working out how enterprises can bring their data safely into an ai context today yeah and that's a that's a great question it's actually a big part of what we do at databricks is well this we call this data governance yeah the cool thing about databricks is that we already had a uh framework for this we call the unity catalog and this is basically a universal source of truth of of permissions and lineage and you know logs of what happened if you're a regulated industry you're a bank or a health care company you need that you can't actually go willy-nilly and say okay here's some data i'm just going to throw into a model and you know hope for the best like it doesn't work that way right that's a one-way ticket to sitting before congress being told that you're a bad person Yes.

39:33And actually not just sitting in front of Congress, like massive lawsuits and huge damage to your business. I mean, what's really interesting is that banks and other regulated industries, they work on trust. Their whole business is based on trust. I'm trusting that that bank is doing the right thing with my money when I put it in there. Otherwise, I wouldn't give them my money, right? So they have to be kind of conservative in this regard. And, you know, that's been a real big blocker to get this stuff out to their customers. What we're seeing today is they're thinking about a blast radius. Like, okay, I can try building a solution.

40:03Maybe I can expose it to internal users as a tool or an aid. And start there and then maybe work your way up to something that goes external. That will happen as we start to trust these solutions and be able to engineer them through all of the edge use cases. But really, this is a huge blocker. So what we've done is inside of Databricks, we've integrated a bunch of stuff that came from Mosaic as well as a lot of things that have been developed here under the umbrella of Unity Catalog. So, for instance, if you fine tune a model, that model is fine tuned on some data. That data had some attributes associated with it.

40:37Maybe it had certain access controls. Those access controls need to be inherited by the model because that model actually, you know, is a function of that data to some degree. Or when you use a vector database and you do a pattern like RAG or retrieval augmented generation, there may be data in that vector database that can be viewed by certain users and not others. When a user hits that endpoint, the vector database needs to be aware that, oh, this user can't see that data, so don't pull that data. So these things, they seem simple. They're conceptually not difficult problems, but to manage it at scale gets very, very complicated.

41:11And so we were able to leverage this actually quite mature framework, which we just open source by the way it was open sourced uh at the data and ai summit in june and uh on stage zahari who's the cto and co-founder of uh databricks actually hit the button on stage that open sourced it i thought it was kind of cool no no i mean i i'm always here for pageantry and uh why not push the button live was it open source on to uh where did you guys put the code online i think it's on a github repo cool i'll throw a link to that in the uh show notes if anyone wants to check it out absolutely so anyway i think the point is that we had a quite mature way of thinking about this in terms of data governance and we've applied that now to gen ai so different components like fine-tuning embedding you know vector databases function calling all of these things live under the guise of this unity catalog which sort of takes away that security and governance plumbing side of things to unblock some of these use cases now getting to correct models that are useful models all of this stuff is the next uh frontier i'll call it okay but listening to you talk about this and thinking about the issues that have come up and that have been uh fixed or or at least worked on by databricks and other companies to get enterprises from we have an open ai account to we're using our own data in production for a feature that's either internal or external it seems that a lot of the pathway has been smoothed so it has not helped a lot of companies go from we have a gen ai strategy which is we're tinkering we're playing so we don't fall behind, but we're not really in production yet.

42:41Two, this is something that we've delivered both to customers and also our employees. Yeah, they're getting there, right? This is still so new. I mean, remember, ChatGPT still was not even two years ago. We're coming up on it in a month. It was November of 2022. So this is actually quite unprecedented how quickly the industry is moving, right? Two years is amazing to actually even get a POC going on this stuff. Well, that is, yes, that is enterprise speed. If you're curious why it took two years and that's fast to get a proof of concept, go work for a private equity-owned company for a while and you'll find that two years to do anything is fantastic.

43:18So I hear you. It is. It is. So we are starting to see that now. And I'm having conversations almost daily with enterprises about how to do this and how to do it well and what are some of the best practices we've seen. Having these governance tools out there, we launched this stuff this year. these are these are within six months we've launched a lot a lot of these features and you know you need to be a trusted partner like i know there's a lot of startups building cool stuff and absolutely continue building cool stuff there's no reason not to but you know it's hard for a big company like uh i don't know but name your big favorite bank you know they they can't take and use favorite bank i mean pick your favorite super villain yeah who charges you atm fees chase i guess okay anyways good but yeah take a chase like for them to use a startup it's quite hard right i mean they're just just getting a tool approved maybe they can get some pocs going but they need trust in their vendors and you know databricks has already built this trust so putting these features in on top of a trusted platform they already have it has actually been quite transformative our business is growing very fast um and you know i can't talk about the precise numbers, but we're seeing a lot of uptake.

44:31I've written a lot about Databricks' growth over time, and this is actually a beautiful segue to what I wanted to ask, which is before Gen.ai became a household name, the thing that everyone talks about in the world of technology, Databricks was growing very quickly by itself, in the pre-Mosaic ML days, if you will. Now, of course, when you pull up Databricks' website, it is the data and AI company, clearly an emphasis across the branding and really, let's be honest, the industry. so now when new customers are coming in to the databricks world are they coming in because they're more interested in the lake house product data storage and bi or is the ai now a leading vector by which you guys land new logos and accounts uh i would say it's a combination of the two honestly because uh we still see a lot of companies out there who are struggling with uh migrations of their data platform i mean again going back to enterprise speed i know it sounds like, you know, things that were like 10 years ago, but these are still problems from 10 years ago.

45:28Like these legacy systems. I actually talked to a company, not in the U S but, uh, uh, in Asia that was literally still running on, you know, old IBM mainframes from 20 years ago. And they're like, how do I modernize? I mean, I know this sounds silly, but like, this is how these industries work. Right. And that's how COBOL programmers have a hundred percent employment right there. Exactly. And actually they're, they're getting paid a lot. days because that's actually that's my backup plan to journalism by the way is if this all falls apart i'm going to go buy a couple of books on cobalt and just nestle myself deep in someone's basement it's gonna be great yeah take a year go learn it really well i think you'd have a you know great career prospects for the next 10 years yeah uh so i think uh we are seeing a combination of companies who want to leverage their data for ai and this is this is squarely the vector that we we are winning on right now is that we are the combination of those two things and frankly it's not something that's new to databricks in fact databricks started there spark started there it was actually the uh netflix challenge uh if you remember this thing i think from 2012 or 2011 where netflix published a bunch of data and they said okay go build something that can match you know preferences of our users to movies they might want to watch and they published a big data set so uh matey and others actually made spark in response to that because they want to scale it out over a whole bunch of cpus so uh you know ai has been something that's been front and center for databricks in fact many people use databricks for ai and then use other data platforms uh oh interesting yeah okay i want to touch on that because one thing we've heard so much about uh at an industry level is the need for more compute to avoid being gpu poor how well do you know, Jensen, will he send you H100s?

47:15But you know, prepping for today's show, I'm seeing Databricks on GCP, Databricks on Azure, Databricks on AWS. And you guys, you know, you'd mentioned you're not working on DBRX2. So does this mean that Databricks doesn't need to build a hyperscale network of data centers around the world? And if so, is that a risk to the company? Well, we do not build data centers. So we never have. We run on top of the cloud, we're a software company. So being GPU poor, we actually do still a lot of fundamental research. So we still have a sizable footprint of GPUs. But we're not building those open source models.

47:55It doesn't mean we're not doing other work. That's on the internally facing side. Externally facing, we're running lots of inference. We're seeing inference kind of take off for these models. I mean, as we get richer use cases, things like agents that we've been talking about doing multi-step like there's a ton of pressure on inference making inference really fast making it really cheap making it very high quality uh all of that takes gpu compute so i don't see the world where we are not doing a lot of stuff on gpus we're just going to have more and more of this going forward and you know we're we're expanding into multiple geos we're not having everything in one place now so i think the the pattern of deployments of those gpus is maybe different instead of having one giant monolithic cluster we're breaking things apart but the numbers i don't see going down anytime i worry about platform risk because you guys are dependent in a way on these major scaled cloud platforms and at the same time microsoft is making its own ai models it has its own bi tools they don't want to share right now they'll be besties with you because it's convenient for them but it just you guys have such an ability right now to raise capital at whatever price you want i mean so to me i'm just kind of curious why not offer a databricks cloud alternative to these big tech companies who i don't think ultimately want you guys to win yeah i mean uh i would say you should ask my boss about that one but tell him to text me back i mean geez come on ollie no i mean i i think uh all possibilities are there in the future we'll see how things go i mean what we have done is establish a particular point in the stack of trust.

49:34So we do own these abstractions for data. Now we're owning a lot of these abstractions on how to take that data and make models that are really great and how to deploy them. So when you have that abstraction, you can move down the stack, typically. Moving up a stack becomes very hard. So you see this. If you're a hardware company, moving to cloud becomes hard. If you're a cloud company, perhaps moving down to build your own hardware is less hard when we see this every cloud company is building their own inference chips now yes so you know how is it going to run in the future that does not include databricks just to be clear correct we are not doing hardware i mean we don't do the cloud so we're not doing hardware i'm just making sure that when you talk about these other companies it's not you guys you're not making your google is microsoft is meta is everybody else okay cool that's right exactly so uh how are things going to shake out in 10 years?

50:25I don't know. But right now, I mean, there's a good reason for the clouds to work with us. We drive a lot of incremental revenue to them. We store a ton of data and now we have the tabular acquisition. Lots of data in the cloud is on our formats. So that gives us a pretty good control point. And the clouds are incentivized to work with us. At some point, maybe that's not true. Well, I think what I'll do is I'll just go start buying small servers and then plugging them into the back of your guys's hq and then slowly build up a cluster that you guys can then offer as as pass to me i don't know i just one of my things that i'm most scared of right now is that the largest tech companies the ones that are the hardest to knock off that have the most market weight are also the companies that own the major cloud platforms and that to me makes them so monolithic and hard to kill that i'm worried that we're going to see less innovation over a long time frame over many different you know technology cycles but it just makes me a little scared and i know there's a lot of money to build a hyper skilled cloud and so there's only so many companies that can do it and so the fact that you guys currently don't plan to just me as an individual that bums me out although i understand from a business perspective how it would be frankly distracting to you guys right now i just i hope that in five years you're showing me around, you know, Dita Bricks, Dita Center number nine, or whatever it is.

51:52That'd be great. Yeah. But I mean, if you think about it on a long time horizon, one of the largest incumbency events in tech happened in the late 70s. What are the two largest companies in the world today? Apple and Microsoft. They both started then and they both started with the PC. They've leveraged those, that advantage over and over again through multiple tech transitions and uh you know it hasn't really slowed innovation right they've been on they've been maybe behind but they buy buy into it or you know buy a startup that's that's doing something innovative so i don't know i mean i like i like that argument i i just wish that instead of the magnificent seven it was the magnificent 27 you know like just yeah more competition i don't disagree but i mean it's sort of like how these ecosystems evolve you have to have certain capital to have that incumbency.

52:41And they have it and it keeps going, right? It keeps rolling forward. Yeah. It's great for you guys because you can work with all of them at once. And therefore, no matter where your customers are, you are as well. So I get that. There's a lot of talk about small models versus large models. And my impression going back to the models you guys made at Mosaic is that quite often we can see a 7 billion parameter model be as good as a larger one if it's well-tuned. Just what is the Mosaic slash Databricks perspective on corporations using larger versus smaller models today? Yeah, I mean, I think it's about building the right thing for your application.

53:14Do the thing that matters. If latency really matters, go small. If quality really matters, maybe go bigger. But one thing you'll see as a trend is literally no one has built a model bigger than GPT-4. That happened, I don't know, 16 months ago, something like that. And that model is still the biggest one out there. And even OpenAI has been building smaller and smaller models. You have GPT-4.0 and 4.0 Mini. 4.0 Mini is probably less than 10 billion parameters. I don't know with specificity, but it's very fast. And we can look at that as kind of a metric for how big it is. And I think we're seeing that they're able to extract higher levels of quality from a smaller model.

53:51So the science is by no means done here. There's a lot more work to do. There's a lot more we can wring out of smaller models. And then chaining these smaller models together as in these compound AI systems is the way forward, right? that's how we're going to do this economically. That's how we're going to do it in a modularity fashion where we can very independently verify each module and build end-to-end systems that are very reliable. Well, then do large models then exist as a way to help create smaller derivative models? Yeah, that's, that kind of had to happen. It seems like we built this giant model and then we said, oh, well, we can actually take the outputs of that model and actually start to modify smaller models to have better performance.

54:30And so, you know, Microsoft did a bunch of work with the five models that they published. Those are like one in 1.3 billion parameters, and they're very high quality. So we're seeing that trend over and over again where I can, it's not distilling. I'm using synthetic data generation from a bigger model to basically bootstrap and train a smaller model. And it works. So maybe we need to go that route of building something really big to go and build something really small that's high quality. Okay. On the synthetic data point, I wasn't going to bring this up, but you brought it up. so I can blame you if we go over a little bit.

55:02Synthetic data, to me, makes a lot of sense, but always, just based on the way my brain thinks, considers it always to be of slightly lesser quality than non-synthetic data. Is that a bias I need to get rid of in my thinking? I wouldn't say get rid of it yet. It's kind of, I don't know if you remember that movie, Multiplicity, where the guy makes a copy of himself to do more work. It's actually kind of a funny movie. Basically, he makes a copy of himself to have one at work and one at home, when they make another copy and a copy of a copy michael keaton getting worse michael keaton that's right yeah i've not i've not seen this movie but the the uh it looks hilarious so that's going on the family list thank you very much yeah so anyway it's a little bit like that i mean you you you start to become more uh uh unbound from the real distributions as you make more copies and more synthetic generations of it it's like you think of it as making mashups some of those mashups are not accurate reflections of the real world.

55:55We call this concept model collapse. If you train a model on a bunch of these synthetic objects, maybe you get something that actually doesn't give you anything useful. So far, we're seeing it as a good way to augment models and actually allow them to explore the distributions of the real world. If you can keep tabs on keeping quality and keeping it grounded. So I think it's a useful technique. It's not something that's going to solve everybody's problems. I still think we have some major issues in the field of AI. We've gone very brute force, you know, 20 trillion tokens of data to train a model.

56:29That's 20 ,000 human lifetimes of text. This is not how humans work. I mean, humans and animals actually use a lot less data to get much higher quality causal models. So we have a lot of science, I still think, to do to build to build these things uh economically and you know very well power your brain runs on 20 watts of energy a mouse runs on 800 milliwatts and they do amazing things no that's one of the most encouraging things that i've seen i've seen the people showing like the the brain watts versus data center watts and how many you know equivalent flops you can get and so forth do you think that eventually and this is completely off topic so this might be a stupid question to be but like there was a there was a an effort i read about this in like business week like 10-15 years ago to replicate the human brain thinking that that is the neural network that is so efficient can do so much we should do it we seem to have kind of moved away from that in the transformer revolution do we bend back towards biology over time so okay those are like the human brain project and stuff like that i think that you're referring to i mean i'm a neuroscientist and i thought those were like very poor facsimiles of what what was going to lead to intelligence.

57:40It's because you're trying to replicate something without understanding why that thing exists. It actually becomes, you know, you get one little parameter wrong, the whole system falls apart. These are delicate systems. So I do think going down the path of trying to extract principles makes sense. Now, I do think that looking at biology still is potentially a useful path forward. There is a different dimension to more data that we are missing currently in these systems. They're not trying to understand how to make better decisions and self-critique maybe we're getting there now with this kind of like this this chaining idea uh but you know our brains have multiple systems that actually um interact in very interesting and precise ways so you may have read uh daniel kahneman's book thinking fast and slow yeah very you know this is this is a great book if anyone hasn't read it um it actually has roots in neurobiology there's there are systems that are very old in our brain that kind of make learning without parameters, without a model.

58:38Then there's these other systems which actually try to simulate different realities and then make good decisions. We use both of these things in concert. And systems we build today, like the AI systems, don't really do this. They don't have a good grounding of, this is a real representation of the world that I can verify. They're just sort of like almost pattern matching against things that we're seeing in the training data. So am I saying we're on the wrong path? Not necessarily, but I think there's a lot of work to do that we can take some some inspiration from biology without direct mimicry i think the biggest takeaway from our conversation today for me is the idea that there's still so much improvement coming in our ability to build intelligent software systems like the orchestration point from 01 what you just said about this i mean this to me feels like a we have 10 years of incredibly quick development ahead of us and we're going to end up with i mean something truly marvelous at the end.

59:34Absolutely. Look at the internet, right? I mean, I was a freshman in college when really the first web browser came out in 1993. All the basic concepts of what the web was going to look like sort of existed then. But look at what we have now. We have cloud computing. We have ways to do live updates and applications that run within a browser. None of that stuff was really contemplated at that time. And it was, again, went from this big monolithic thing that sort of worked till now we can engineer modules, make things work really interactively. And we created a set of systems that are highly reliable.

1:00:10Companies like Google basically figured out how to build resilient infrastructure. This needed to happen to have like, you know, nine nines up time. You know, all of this engineering has to happen. And we're just at the beginning of this chat. GPT was two years ago. You know, we've got 10 years of innovation left ahead of us to make these things really, really amazing. I thought you should point to that sign that says chat gpt came out less than two years ago every time someone complains that ai hasn't solved world pc i'd be like all right it's been 22 months everybody calm down actually i just realized that my my my first child is basically as old as chat gpt huh that's gonna be a really weird milestone to have in my life okay uh one last question before i let you go intervene a lot of startups out there working on ai drive solutions services lots of people building cool stuff in AI.

1:00:59Databricks has been acquisitive. It's how you joined. Tabular, of course, was a big deal. I'm curious about two things. One, how often are AI founders calling you up very politely? And also just how interested is Databricks today in doing more, perhaps smaller tuck-in acquisitions of AI-focused startups? Yeah. I mean, I do angel invest and I typically try to pick companies or founders that I feel I could be useful to them. As an angel investor, I kind of see myself as a de facto advisor or helper whenever I can. And things that I find interesting, if it's a new, cool technology, all right, let's give it a go.

1:01:38I am really interested in places where we can apply AI to actually solve vertical problems. Healthcare is actually a very important one to me because I I think it's something so fundamental to what it means to human. And we have the pieces to actually go make healthcare accessible to many people, but it still hasn't happened. There's a lot of structural reasons, things like that. So let's get to that. So I like to invest in these kinds of things. So yeah, I do get called up quite a lot. And I'm happy to talk to any founders who are building something cool. So there's that. Sorry, what was the second part of the question?

1:02:11And just how acquisitive is Databricks going to be, say, in the next 12 to 18 months? A lot of companies have been built. many of them probably aren't going to make it to the next level. So how many do you guys want to tuck in? Yeah, I mean, we're always open to these tuck ins. We've done a few of them. We're always looking, we have a really good corporate development team that's basically scanning, finding places that fill in holes that we have. So we're always looking at this, but we're very thoughtful about it. We don't want to buy something just because, you know, it looks cool. We're going to look at it like, okay, how is this going to work strategically with what we're building?

1:02:42Does it solve a problem that we don't have a good solution for today? Does it have a customer base that we want access to? Something like that. There's got to be some reason. So we're pretty thoughtful about how we assess it. And when we bring these companies in, I mean, some of our agent evaluation stuff has come from a tuck-in that we had, a company called Lilac AI. They actually created a really nice UI for embedding data and visualizing it. We've actually morphed that into something that is going to be much bigger and very impactful for our customers. So we are absolutely interested in more tuck-ins.

1:03:13And we're growing fast. we're growing, our revenue is growing more than 60 % year on year. And so I think we are going to be one of the major players for these companies to go find an exit toward and, you know, keep in touch with us, build a relationship with us. You literally just threw the we're growing fast, 60 % growth. I was not going to bring up you guys going public. But I mean, dear God, you just you're demanding it. So I know the answer is we'll do it eventually. Not yet, blah, blah, blah. But like H1 2025, like if you guys don't do it, then I think I'm going to submit like my tiers in a vial, like, come on, give me the filing.

1:03:47So just so you know, that's coming next summer, if you guys don't pull the trigger. But Naveen, thank you so much for coming on. We appreciate it. We'll have you back in another 250 episodes or whatever it was the first time. And where can people find you online just before we go? Yeah, I'm on Twitter pretty actively. So Naveen G. Rao, that's N-A-V-E-E-N-G-R-A-O is my handle there. I'm also on LinkedIn, of course, so you can find my name there. But yeah, Twitter is usually where a lot of the AI stuff happens and where we have a lot of cool discussions. So follow me. Absolutely. For everyone else, more interviews, more live news, more twists coming your way.

1:04:22I'm Alex. I'll see you soon. Bye. Okay, everybody. Welcome back. It's time for a Jam with JCal session. What's a Jam with JCal session? This is where a startup founder tells me what they're working on, who their customers are, what their product does, what they're trying to achieve. essentially their vision for how they're going to change the world. And then I asked them, hey, what's your biggest challenge? Are there things you're struggling with? And then using my experience, having invested in 400 companies and taken well over 10 ,000 pitches from founders and having done 2000 of these episodes of this week in startups, sometimes you know what, I know where some of the bumps in the road are and we have a dialogue about how to solve problems which is what startups are all about so with me today is a gentleman named elliot easterling he's the ceo of a company called bonbon.tech and that is our partner today.tech is a great domain name i use it for founder fridays.tech many people out there are using dot tech domain names because they're awesome they let people know, hey, you're a technology company.

1:05:33And so thanks to our friends at dot tech for supporting this segment of the show. We pick all the companies that come on. Welcome to the show, Elliot. Thanks, Jake. Thanks for having me. Appreciate it. I just want to walk you through a little bit of the business. I'll take you through a deck. And then I'd like to jump into some questions I have about about some of our go to market. Review the deck. I like reviewing the deck. Let's see how you did. Great. Awesome. Well, I'm the CEO, as you mentioned, of Bon Bon technologies, and we are a rewards platform for publishers, allow publishers to reward anything.

1:06:07And that ultimately drives a lot more engagement and much higher registration rates. So let's just get started. Really, the pain that we're trying to solve is for ad-focused publishers, the 99 % of publishers that are ad-focused. And they've been really suffering from wave after wave of big tech changes, things like cookie deprecation, which started in Safari and now is moving to Chrome. Search results pages referring less and less traffic out. Social media algorithms referring less and less traffic out. And then now AI, which is going to have a massive impact as big tech platforms use AI to drive more engagement on their platforms and ultimately refer less traffic.

1:06:44So what we're seeing is publishers are looking right now for solutions. They're planning for a smaller user footprint, but they also want to drive a lot deeper, more profitable customer relationships from those users that are on their platform. And so what we built basically is a fix by rewarding engagement. So we built the first publisher rewards platform that gives consumers relevant rewards, access to unique content, simple and transparent data and privacy controls, and ultimately a better user experience. Publishers get logins, and logins re-enable cookies and lost IDs. They also allow the publisher to build direct relationships, which they're starved for in the world where they're mediated by these big tech platforms.

1:07:25we also give the publishers gamified engagement a points program that drives things like visitors page views watching videos and ultimately that's all delivers five times more monetization per user all right you got me so far it's interesting like i'm i'm i find myself nodding as a publisher just so we we check in here as you run me through the deck yeah publishers do have these problems and if their consumers are logged in and you know a little bit about them you can monetize them better. And people like gamification. So let's see how this works. Cool. Yeah. And you're a media maven. So I think I knew you'd really understand the business pretty quickly.

1:08:01So we've cracked the code on sculpting consumer behavior with this rewards program. So we built an optimization engine that drives outcomes like 300 % higher registration rates, 100 % more engagement, 250 % higher ad rates. And then we're finding that 54 % of people after they log in will actually complete their data profiles and provide even a richer understanding to the publishers or who's visiting their site. And this is all at a moment where publishers need this most. And so let me just walk you through really quickly, kind of like a sample of the tech. Finally, I want to see the product. Okay, here we go.

1:08:35And so on the left, you see a little icon, a little tab. That's us. We control that. And that's all the user's navigation that can access the rewards program. But on the right-hand side here, what you see is that at any time through the process, we can allow a publisher to trigger a rewards window either inline or as a pop like this that essentially runs eight or nine offers simultaneously to figure out through machine learning what do the users of that site care about most that will make them willing to register and ultimately this technology delivers like a 3x higher reg rate from anything they've been doing prior okay so for the audience that was listening there's a little arrow on the side like a little chip on your phone if you were to click it it pops up says hey would you like to win this television set or whatever some kind of sweepstakes if you log in if you register you automatically get updated yeah yeah you get entered into the contest i love it yep and then after someone enters or registers when we verify their email we actually gamify the registration process like we say hey tell us more a little more about yourself give us your name 94 of people will give us name for more points.

1:09:40Zip code, 91 % give us zip. Gender, 89 % give us gender. And even they verify their phone number with us at 54%. And we're just getting started. The other things we do is we give people points for reading. Every article they read, they earn more points and that drives 100 % more engagement. And just high, high level, we provide them with what we call three parts of the platform. Open identity manager, which allows them to collect and manage first-party data. This rewards engine, which runs hundreds of offers to try and give publishers the ability reward anything. And then we have a bunch of front end tools that consumers interact with that basically sort of deliver the product.

1:10:14And we also have an API where publishers can call it to issue the rewards on their own. Very nicely done. Have you started to deploy this with any publishers yet? Where are you at in terms of building this business? Yeah, we are live on 27 websites. We have 60 ,000 BombBomb members have actually registered. The one thing is when a user logs into the publisher, they become a Bombard Awards member and also a publisher's first-party data. So we've been able to build our file up to 60 ,000 as of last week. And we're running about 60 million monthly pages across our publisher network. Great. Publishers are looking for tools like this.

1:10:53And if you can help them build their profiles and give them those gamification tools, I could see a number of them wanting to participate in this. The challenge, of course, is you're trying to solve a problem for publishers who are really struggling, which means they're not high growth. And so they might have very small budgets, right? And so that is an issue is that they're constrained. So has that come up? And how are you picking your ideal customer profiles when, hey, you know, the publications I used to run, whether it was Engadget or Autoblog or the ones that Vox created themselves like The Verge, etc.

1:11:34A lot of those publications are having challenges these days, right? And so how do you think about that? Because you're picking publishers, which are a group of people whose businesses might be flat. They might be contracting. They might be slow growth. And they may not have budgets. Yeah, so we offer two solutions for enterprise publishers. We have a SaaS platform where publishers can pay a SaaS fee, and we also have a free with ads version. Publishers have to have a minimum size to be able to qualify for free with ads, but we basically inject ads into all our modals, and that essentially pays for the full program, including the rewards.

1:12:10Got it. So there's nothing to lose for those publishers by using this tool. And on the enterprise version, I assume I can opt out of like you owning the profiles and they're just for me if I pay you enough money. No, the whole program is across publisher rewards program that's sort of backed by Bon Bon. And one of the things we do is we issue basically we issue the rewards out across publishers. OK, that's clever. So hold on. That's an important thing to pause on there. if you're going to give away this oled tv that costs five thousand dollars it that cost of five thousand dollars we'll get it abstracted across 10 publishers so it's really net net five hundred dollars each or if it was a hundred publishers it would be fifty dollars each which is super clever okay i i understand yes we have the benefit of that in that we basically can amortize the cost across publisher number two is sort of we put the the privacy guarantee we allow the users to come into our system and opt out of any publisher they want.

1:13:14We know, generally speaking, consumers don't like to do that. They just don't want to mess with the settings, but it is a really key important part of our value proposition. One of the other nice things here is, you know, the user can participate or not participate. So what I like about what you're doing is, you might have somebody like me who looks at it, you know, as a 53-year-old now, and I'm like, I don't want anything for free. I don't want to be part of any sweepstakes. But I might want you to know a little bit more about me as a publisher because I want to get certain mailings, etc. So I have an email for just when I'm shopping that I use for my shopping websites.

1:13:53And I don't mind them knowing about me because I would rather get sales for men than women. Or you get the idea. or I want tickets to Knicks games, not, you know, I don't know, Miami Heat games or Chicago Bulls games. So that kind of personalization is a benefit because you don't waste my time. And then when I was younger, I might, if I couldn't afford the OLED TV, that's$5 ,000. I might very much want to join that sweepstakes. Thank you very much. So I think that's a very interesting approach too, is this data will help us personalize content for you. And then there's, hey, this data might let you get into a sweepstakes.

1:14:34And you didn't mention the gamification, but that one was a highlight for me as well. So do you have an example of gamification yet, or that's on the roadmap? Yeah, so one of the things we do is after people register, we sort of say, hey, and we deliver this to the email campaigns, we sort of send them a weekly email that basically says, hey, here are some articles you could read that are personalized to you that earn you extra points. And so we're essentially starting to sort of like enable gamification through our newsletter program. And we do also offer point bonuses for doing things that go to this one publisher and play one of their games.

1:15:10And that will earn you 100 points. So we're early on in the gamification process. But I do think that that's going to be some of the critical things. As sort of my promise of publishers is like, hey, not only can I get you a lot more registered users than you could on your own, but these users are hyper engaged because through gamification, we can sort of sculpt traffic directing back to your site to spend time on the things that you care about and ultimately reward them for that behavior and that activity. One thing that's tried and true is inviting a friend or a friend member bring a member kind of gamification.

1:15:44So you might find some interesting, you know, engagement there from publishers. If you could say, hey, I'm already a member. I think my brother should know about this content. then, you know, who else should know about this story? I enter somebody's email, it emails them, they click, they register, I get points. So that's tried and true. You see that in Robinhood where you gift a stock, you get a stock. Uber, you give a ride, you get a ride. Dropbox, you give some storage, you get some storage. So I think you got a really interesting tool-based and network-based business here, two ways to win.

1:16:19And so what's your question for me? Any challenges right now in the business? Yeah, so, you know, in today's fundraising environment, And, you know, ever since the market turned, rather VCs sort of, they behave similarly. And they'll sort of, the mantra now is like, I want to see revenue. And all they care about is revenue, revenue, revenue. Now this is a network effect business. And we win by building distribution. Right now we're building users at zero cap. And when you think about most rewards businesses, they're paying, you know, five bucks, 15 bucks per user. We're not paying for users. And so our go-to-market really should be 100 % focused on building distribution, getting more users, but at the same time, VCs and the market wants to see revenue, revenue, revenue.

1:17:01Me putting pressure on publishers to essentially pay us is going to slow my business down. So how would you navigate that? Yeah, sure. This is a great question. Network-based businesses do have a carve-out in our industry when it comes to monetization. So if you're a SaaS business, if you're a superhuman and you charge a dollar a day for an email product or slack and you charge eight to thirty five dollars a month depending on which plan you are per user yes of course people want to see the number go up in your business however if you could prove that if you you have these 60 000 members and if you said hey we're giving away the iphone 16 and we just want you to take a survey about mobile phones and visit these three reviews of you know we're going to present you with 10 pages every time you visit one of these 10 pages in our network and uh you scroll to the bottom and you know click on the iphone 16 you get entered in now you'll get a bunch of sweepstakes jerks who you know kind of just do this you know scammy kind of way but you also if it's good content might get people to actually who are interested in iphone 16s and technology to visit 10 pages that's good for the publishers and it costs you fifteen hundred dollars to set up the sweepstakes and give it away now you could prove to people if you could prove to people that when you do that the publishers get page views and uh the cost is spread for the iphone 16 across 30 publishers so it's only 50 bucks a publisher and you just run those experiments without having um the publishers tell you i'll pay for it you just you're running a 1500 experiment you say look we sent of the 60 000 people three percent of them you know went and actually did something imagine if we had six million people and three percent did something now we'd really have a business so that's up to you to prove it to me the investor that you could do these little experiments and whoa look at the interest that this created and they had to click on an email to confirm they were in it and if they put in their friend's email they got you know ten dollar they got uh ten more entries and their friend got double the entries so there are little experiments like that where you could show growth in the user base and the page where you're driving and be able to correlate it so that that's really up to you to run those small tests and prove to us you can grow you know five to ten percent week over week that would be what viral growth would look like in your business now for sales and for sas products you show me a sas business growing ten percent a month i'm interested you show me a business like yours growing ten percent a month i probably would think not very interesting so So you got to get to that 5 % to 10 % week over week.

1:20:00That's a lot of tests to run. But you could actually do that with very low dollar amounts, yeah? Yeah, yeah. I think what you're sort of pointing as a sort of a unit economic story that we need to tell to investors like you to basically say, hey, we're getting these users for free. And we're running these experiments that actually show that we're getting revenue traction, the revenue traction is building or the activity traction is building on a per unit basis. Engagement. Yeah. Engagement is the name of the game. So you could say to me, you know, if you came to me with these 60 ,000 and said, okay, it cost us, you know, six months.

1:20:32And, you know, we burned a half a million, we burned$100 ,000 in six months, getting to 60 ,000. So we're going to need to spend a million dollars to get to, you know, what we think is 3 million, because we're getting better at it. It's a fixed cost business, whatever. But if we did this many prizes, we would get to, you know, 10 million. and then every week or every day we're going to run an experiment with a million of those 10 million so we don't burn them out but every 10 days they're going to get some sort of offer to engage them and then we'll start charging the publishers because they'll be addicted to it but you know you have to get the flywheel started right and so getting the publishers to agree getting the vcs to agree all this stuff takes work so you may have to just invest small dollar amounts to show it on a micro basis yeah and then say imagine we add two zeros yeah and here's what it costs to add two zeros to the velocity i'm showing yeah yeah makes tons of sense pretty straightforward yeah and that's like your investment but i the good news is i think you can show this for very small amounts of money how many people working on the team i'm curious how many developers you have yeah so we have a product person a couple engineers who are really really good or kind of working kind of core on the product.

1:21:44Outsourced engineers or full-time employees? Offshore? Offshore, part-time, basically is what the team looks like, yeah. So you're in year one. You're an early stage startup, I take it. We're pretty early, yeah. We're sub two years, yeah. Okay, yeah. You raise... How much did you raise? 1.4, basically. Oh, okay, wow. So you raise a decent amount of money. Yeah. You should be able to get that network effect going, just invest in giving away whatever the product of the moment is that aligns with this i would also start thinking about you know the virality of social media and tiktok you went after publishers which is where attention is going away so they need your tool obviously they really need it but their businesses are contracting and you may be selling like uh you know deck chairs to the titanics now what business is growing tick tock shorts video podcasts there's a bunch of things that are growing in the world so i'd also think about these tools and say hey if we were to get 20 if we had 60 000 tick tockers and people making shorts and they were engaged in the network what might this look like because sometimes you know a surfer is only as good as the waves presented to them but when you go big wave surfing right yeah you're you've got these little tiny waves three four five foot ones okay fine you can surf them but it's not like you're going to be going to the north shore and do it you know hitting a 20 30 foot wave where like whoa it's impressive if you can catch one of those waves so i encourage you to really think deeply about the beach you're surfing at the beach you're surfing at it doesn't the time might be going out faster than you can build a business i i love looking at you know and i've i've done this myself right i caught the blog i caught the magazine zine wave in the 90s i caught the blog wave i caught the podcasting wave 14 years ago when i started still riding that wave and caught the angel and seed investing an incubator wave sometimes there's waves that are bigger than you and if you catch them right man that's why i'm i'm doing more uh thinking about live and shorts um you know tiktok shorts uh or well tiktoks aren't called shorts i think youtube calls them shorts but anyway i've been thinking about other alternative formats outside of podcasting and i've been thinking about live a whole bunch so i encourage you to think about that as well you have an interesting business i mean i would have if you had presented this business to me to come to our accelerator i'd be like yeah let's do it but i think you're in a weird position having raised a bunch of money not having violent product market fit or market pull yet so you really got to get some market pull and i think it might be that you you might need to find another beach where the waves are a little bit bigger so i would think about that e-commerce is another one direct to consumer the people in direct to consumer were getting slaughtered yeah slaughtered we're trying to get facebook to work and facebook worked for a long time then it stopped working yeah the ones who went to tiktok and social media and podcasts did make it work so sometimes it's just like it's not you it's the environment you're applying your skill in so i think you're very talented yeah yeah and i we are i can't disclose what we're doing but i think some of the things we indicated our areas where we're testing right now doing some experimentation good yeah i'll leave it at that i don't want to tip your cards yeah great job And for everybody listening who wants to learn more, give us your URL again.

1:25:16I know that you're a.tech, but remind everybody of your domain name. Yeah, bonbon.tech. What a great domain name. Bonbon.tech. If you would like to get one of these great.tech domain names, just go to get.tech and get one of those great.tech domain names. Tell them your boy, J. Cal sent you. We'll see you all next time on Jam with Jake Allen this week in Starnux.