Aligning Learning and Endogenous Decision-Making

0:00Okay, let's unpack this. Imagine you're trying to make a really important decision. for your business, let's say. But the very act of trying to figure out the best choice actually changes the information you're looking at. Sounds about like something out of a sci-fi movie, right? But this is a very real and actually an incredibly costly challenge for AI systems in the business world today. It really is. And what's fascinating here is how often the observations we collect, you know, the data we use are inherently biased by our own choices for so long. Our AI models have sort of operated as if the world just stands still waiting for us to decide.

0:34But that's rarely the case in reality, is it? So this deep dive is all about a groundbreaking paper that tackles this exact problem head on. It's titled Aligning Learning and Endogenous Decision Making by Christian, Harsha, Paracas, and Quants. Right. And our mission today is basically to cut through some of that dense academic language and pull out the most important nuggets from this research. We want to give you a powerful shortcut to understanding how AI can make smarter, much more reliable decisions in those complex, real-world scenarios you probably face every day. And at the heart of this whole challenge is something they call endogenous uncertainty.

1:13Think of it this way. Your decision itself directly influences the outcome you observe, and that creates this massive blind spot. You simply don't get what they call counterfactual information, meaning you never truly know what would have happened if you'd chosen differently. It's like only ever seeing the results of the path you actually took, never the paths you didn't. Okay, so here's where it gets really interesting and I think incredibly relevant for business. Imagine you're a retailer trying to set prices for your products. The price you choose directly impacts the demand you see for that product, right?

1:49If you set a high price, you might see low demand, low price, maybe high demand. But the demand you observe is always tied to the price you chose. And this paper calls that endogenous uncertainty. That's it, precisely. And the traditional approach, often called to stage or predict then optimize, it completely misses this critical feedback loop. It's like separating the learning stage, say, predicting that demand from the actual decision making, like optimizing your price for profit. Okay. And the real bombshell here, the critical flaw, is that models train this way. Well, despite looking good on paper, maybe having low error, they can lead you completely astray when applied to situations far out of sample.

2:31What does that mean, far out of sample? It means if you decide to set a price that's very different from anything the model saw during its training, well, its prediction for demand at that new price might be wildly off. And that leads to actual outcomes, your actual revenue or costs being significantly worse and potentially much more expensive than what the model predicted. Yeah. To give you a sort of mental image of this, the paper uses a great visual in their figure one. It really illustrates the problem perfectly. Imagine you have two different demand prediction models. On the surface, they might look equally good.

3:04Maybe they have the same, say, mean squared error when compared to the true demand data you have. Okay, seem equally accurate. Right. But when you actually use them to make a decision, like setting a price to maximize revenue, which is your final task-based objective, right? One model might lead you to a price that results in terrible revenue, while the other leads to a much better outcome. The point is, a model that's good at just raw prediction can be absolutely terrible at helping you make the right decision for your actual goal. And this N10 approach aims to fix this to identify the model that performs better on that final business objective.

3:41Okay, so what does this all mean for you, the listener? The core problem here is that a traditional AI model isn't really aware of how its prediction actually affects the final decision or how its decisions influence the very data it's learning from. It's like, I don't know, predicting the path of a ball without accounting for how your kick changes its trajectory. Precisely. And this contrasts really sharply with what's called exogenous uncertainty. That's where outcomes are completely independent of your decision. For example, the overall online demand for a product might be largely independent of how you decide to allocate inventory across your different warehouses.

4:16Right. The demand is just out there. Exactly. Demand predictions are still crucial for those inventory decisions, obviously. But your allocation actions don't fundamentally change the demand itself. But in this endogenous setting we're talking about, where your action does affect the observed outcome, you simply don't have access to that perfect what-if information, the counterfactuals. And that makes the traditional separated approach deeply problematic for practical application. Yeah, that sounds like a massive blind spot for AI, especially in these dynamic business environments we see everywhere.

4:47So this paper introduces a, well, you called it groundbreaking, solution they call a cost-aware end-to-end method. Tell us more about that. Yes, absolutely. The paper's end-to-end approach directly addresses these limitations. It does this by integrating the learning and decision-making stages into a single, cohesive, trainable system. The core idea really is to train an AI model to be aware of its downstream optimization step. Aware of the decision it's feeding into. Exactly. So instead of just trying to guess the average outcome of some uncertain variable, which, by the way, often leads to suboptimal results when your costs aren't linear, like a simple straight line, this new method teaches the AI to directly optimize for the final business goal.

5:31It's kind of like training a chef not just to measure ingredients perfectly, but to actually create the most delicious dish overall. The model learns to make predictions that directly maximize the value of the decisions taken, whether that's, you know, higher revenue or lower operational costs. And it sounds like they've really gone the extra mile here. They haven't just created a theoretical framework, but they've also provided actionable ways to implement this. Like ways that work whether you need extreme precision for really critical decisions or maybe something faster for larger data sets?

6:01Absolutely. That's definitely one of the paper's true strengths. They've essentially given us the blueprints to put this into practice. They offer both these highly accurate methods using things like mixed integer optimization, which can be computationally intensive but give exact solutions for maybe smaller critical data sets. And they also provide faster, more efficient sampling based approaches for larger scale challenges. So it makes it practical for, you know, virtually any business context, really. Now, this does raise another important question. Given that we often have limited real world data, there are always going to be some errors in any function an AI learns, right?

6:39That seems perfect. So how do you make decisions that are robust to these inevitable errors, especially when you might have multiple different models that all seem to fit the data pretty well but could behave wildly differently in situations the AI hasn't seen before. Ah, right, the what-if factor. Dealing with that uncertainty in the model itself, this is where their robust optimization variant comes in, isn't it? It sounds like building in some kind of safety net. Exactly, and this is, I think, a real game changer for practical implementation. What the robust approach does is it constructs an uncertainty set.

7:10Basically, think of it as a carefully defined range of possible models. All these models in the set are consistent with the data, meaning they have a low error on the observations you've actually seen. Okay, so a family of plausible models. Precisely. Then, it optimizes your actions, your decisions, to protect against the worst-case predictions from any model within that set. What this gives you is a guaranteed minimum level of performance. It's a safety net, like you said, even if your data is imperfect or the world throws you a curveball you didn't quite expect. So instead of just picking one best model and kind of hoping for the best, they consider a whole range of good enough models and make a decision that protects against the worst predictions coming from that entire group.

7:53That sounds incredibly smart for actual real-world risk management. It truly is. And the evidence they show from their experiments, particularly if you look at figures 2 and 3 in the paper, it's really compelling. It shows that by embracing this robustness, by using this uncertainty set, not only do you get a safer, maybe more conservative prediction, but in their real-world pricing scenarios, this robust approach actually outperformed the other methods by over 30 % when they dialed up the level of caution, the robustness parameter they call epsilon. Wow, 30 % is significant. It is. And importantly, even when they set that parameter to zero, essentially dialing back the robustness, it still performed comparably to the traditional baseline methods.

8:35So you don't lose much by having the capability. And furthermore, their analysis in Figure 4 reveals something really impressive about its stability. As the complexity of the data increased, they represented this with a parameter prowess, the robust method consistently outperformed a standard two-stage approach. And it maintained remarkable stability without needing lots of extensive fine-tuning for each scenario. So this tells us it's not just powerful in theory, but it's also practical and reliable across diverse, messy, real-world conditions. Okay, so building on that foundation of making robust decisions with the data we have, the paper then takes us into what feels like an entirely new frontier.

9:13So far, we've explored how to make smarter decisions based on observations. But what if your decision itself was about how to get better observations? This deep dive introduces a really novel idea they call information gathering. Yes, this is a significant leap, I think. They frame this as a two-stage stochastic optimization problem. It sounds complex, but the idea is quite intuitive. In the first stage, you make a decision specifically designed to poll or survey some random variable, basically, to gather some new information. Then in the second stage, you use this newly acquired information to make more informed predictions and, crucially, a better final decision down the line.

9:54Okay. Can you give us an example? Sure. Let's use their electricity scheduling problem. It's a great detailed example, and importantly, they use real-world PJM electricity data. That's actual energy grid demand data from a major U.S. power market, so these findings are highly relevant to large-scale operations. Right, real data. So the goal is to generate an electricity generation schedule for an entire 24-hour period, trying to match supply with demand efficiently. In stage one, you make an initial forecast for the whole day, and you also decide on a specific time, let's call it W, when you'll update your schedule.

10:27Then at that chosen time W, you get to observe the true electricity demand up to that point in the day. Then in stage two, you use that new actual demand information to regenerate your forecast and your schedule for the rest of the day. So it's this fascinating balancing act then. If you wait longer to update, say wait until later in the day, you get more accurate future demand data because you've seen more of the day actually unfold. But the downside is you've been operating on your maybe worse initial forecast for a longer period. Exactly. Or you could update early. You use a potentially less accurate forecast for the remaining part of the day, but you only operated on the initial bad forecast for a shorter time before you got to course correct.

11:08What's really fascinating here is you're not just predicting demand, you're deciding when to predict better. You're actively managing the flow of information itself. That's the key insight. This problem differs fundamentally from traditional multi-stage problems. Why? Because the decision you make that update time to directly affects your knowledge of the random variable, the demand. Even though the demand itself is exogenous, you can't control it, your decision about when to learn more affects everything that follows. And the endogenous end-to-end framework is uniquely suited for this. It explicitly accounts for how that initial decision to gather information impacts the accuracy of subsequent predictions and ultimately the final operational cost.

11:47And did it work in their tests? It worked remarkably well. Their experiments using that real PJM electricity data showed fantastic success. Their method, which they called endogenous E2E, consistently minimized the operational costs. The results, shown in figure 8, indicate its cost distribution most closely aligned with the theoretical optimal cost distribution you could possibly achieve if you had perfect foresight. And table 1 shows its superior average performance. It was about 7.5 % better than another sophisticated method called cost learner and less than 8 % worse than the theoretical optimum on average.

12:23That's incredibly close for such a complex problem. Wow, okay. So we've talked smart pricing, optimizing massive electricity grids. What about something maybe more common for many businesses, like assortment optimization for retailers? Yes, assortment optimization is another really powerful application they explored in the paper. Here, the problem is deciding the ideal quantity of multiple different products, maybe hundreds or thousands, to stock in a store or warehouse. The real challenge is that the demand for one product often depends not just on its own stock levels, but also on the availability of other products.

12:55Right, like complementary items or substitutes. Exactly. Think shampoo and conditioner, or maybe two competing brands of soda. These interdependent size make it complex, so this problem typically involves many products making it high-dimensional. And it usually has a complicated non-linear cost function. You have costs for holding inventory, costs for back orders if you run out, procurement costs, and so on. And importantly, they tested this using realistic data generation methods designed to mimic real-world retail challenges, things like skewed demand distributions and the fact that you often only have limited historical sales observations.

13:29So how did the end-to-end approach fare in that messy, realistic setting? Well, the performance they observed, shown in Figure 5, was a really clear testament to the end-to-end approach's power. Their method, whether they used the exact mixed integer formulation or the faster sampling-based variant, required significantly less data to achieve much better performance, meaning lower overall costs compared to the traditional predict-then-optimize methods. It even outperforms simpler but sometimes effective approaches like k-nearest neighbors. And again, this just underscores the point that simply predicting the average demand for each item isn't going to cut it when your real-world costs are nonlinear and interconnected like this.

14:08Makes sense. And they also revisited the robust approach for assortment in Figure 6. They show that their proposed method for solving the inner worst-case problem within the robust framework consistently outperformed a more standard regularization technique often used for robustness. So it proves the value of their specific robust formulation in these practical, messy data environments too. It really seems like this whole end-to-end philosophy, especially when you add that layer of robustness and then consider this novel extension to information gathering, feels like a true game changer for tackling these really complex decision-making problems across a huge variety of different domains.

14:45It's really about moving from just passively predicting what might happen to actively making smarter, more reliable choices that account for the system's dynamics. So wrapping up, what does this all mean for you, the listener? This deep dive into aligning learning and endogenous decision making really does highlight, I think, the next frontier in how AI can help us all make better, more impactful choices. I agree. It's really about moving beyond simple, isolated predictions towards truly intelligent decision making. where the AI system actually understands the full impact of its forecasts on your final business objective, your bottom line, even when the world itself changes based on the very decisions you're making using the AI.

15:25It provides a robust, resilient framework to handle that inherent uncertainty and those complex feedback loops that are just so common in almost all real-world systems we deal with. Yeah, and imagine applying these principles more broadly, not just to pricing or energy grids or retail, But think about healthcare, where a diagnostic decision influences patient monitoring and treatment, which in turn generates new data. Or logistics, where the delivery routes you choose today might affect traffic patterns or even customer demand patterns tomorrow. The implications feel vast, potentially impacting everything from how we manage global supply chains to how we allocate critical resources in emergencies.

16:03And maybe this raises an important final question for you to think about. as these AI systems become even more deeply intertwined with our decision-making processes. And as we start to really embrace these more sophisticated frameworks like endogenous uncertainty and information gathering, how might this fundamentally change the very nature of strategy, planning, and problem solving across pretty much every industry you can think of?