Can AI Agents Learn From Expert Corrections?

22:39Grant:It's sort of like the next generation or next evolution of that feedback loop with the software development cycle, where you're almost letting the product itself get the feedback from the user. Because if I'm understanding this correctly, at the point where it's at now, the people who are using it, the accountants at these firms, they are giving it feedback and it's improving kind of autonomously, right? Is that more or less what's happening at this point? so i would say like so they are giving feedback and the feedback is triage so like when there is a new feature request it's definitely triage across the others because on the other spectrum you don't want to directly authorize that you know when someone asks for a request it automatically become becomes a new feature you know that could be interesting for that very specific person but that actually could be a pain for the rest of the users using it so this is also something so there is kind of a human review at some point in that early feedback phase afterwards you know in that iteration loop this one you know can be self-improving but on the feedback side from the user this is definitely important because one thing that's very interesting here is that on in one way you want to give you know to make the product as smooth and as easy to use as you know Arthur was describing before but on the other hand you don't want to allow any kind of feature to be added because you You know, in some way, if you create this, you know, you have a tax expert that instead of, you know, spending eight hours by manually reviewing the files and, you know, for example, switching to like reducing it to 30 minutes on the platform.

24:12If you start by adding some tools that, you know, for example, a game to play while the data is being extracted or a chatbot to click through without any intention, for example, It might mean that the person, instead of doing 30 minutes there, will just end up spending two hours for something that's helpful. So I think this idea of making sure that you stay focused on what is actually truly useful for the user and for the end task is quite important.

24:41Grant:You basically don't want them to spend eight hours just messing with the tool instead of actually being able to serve more clients. That makes sense. Yeah. How does it separate?

24:51Corey:And I guess this is where the human comes in as well. But I'm curious as to separating between, you know, a true system error of sorts and normal workflow noise, like just the difference from tax return to, I say return, from tax filing to tax filing. I assume big ones aren't getting returns usually. But I'm curious because there's a certain amount of uniqueness to every filing, even though the forms are the same. How does it recognize this return is different from this return, but not in a way that means something is broken?

25:28Grant:So the uniqueness. Yeah.

25:30Corey:I hope that makes sense. No, it definitely does. So I think in our problem space in particular, there are a few things that give a bit of structure that helps sift through these different things. I mean, there's still a certain amount of ambiguity, which might lead to not the rights. As I mentioned before, around like maybe it's a completely new type of document that was misclassified and things like this. But there are a few things that basically make some of the focus for the test. So classification is one of them. Also, the mechanism at which we deploy this product, we don't support the entire surface area of filing on all of the states and all of the aspects of federal tax and entity tax.

26:13So the way we deploy the product was very targeted. So we started even with the simplest of forms. And of course, when you do this initially, the probability of this being more correct is higher. So that kind of gets us the right shape for understanding exactly how to capture things. And we don't have the loop at that point. But also the tax engine gives us some guardrails around the fields that are filled in there. So the tax engine that the firms use, it kind of already encodes a lot of opinions on how you put in a value. So it's different, for example, if you ask Codex around your return in general, like Codex is going to be taking care of parsing the file and also doing the calculations and things like this.

27:01But the tax engine, for good and for bad, like it provides like a lot of guardrails around like exactly like what to expect. Why is that field filled in a specific way? And that gives us basically a way to more strongly measure errors. So, for example, the error is going to be anchored on a few fields, for example. And then we start seeing like the repetition of like a few tax returns, like are making mistakes on like these specific fields. It does create some problems around grading the task correctly because some of the fields might have more of a human preference. And like some of the fields like an accounting firm might do that slightly different than the other.

27:41So the way that we encode some of these things is by going through these initial interactions with the practitioners. So we get some of these things encoded in our grader. So we only get to the more self-improving loop once we get some of these things right.

27:58Grant:I guess if I understand this correctly, it would be like the tool that the companies use, it already kind of has details of like, this is how we report this type of revenue or this type of situation here. And it's able to parse that to some degree. So it's like already there's like, this is how our company does it. Tell us a little bit about what that is like and if this process of what you're doing now would be possible without forward deployed engineers or how integral that is to being able to work this closely with the clients. I think it's very interesting because it's a combination of someone who is a software engineer, but also is potentially a product manager, has a concept of architecture and is able to be ingrained with the customer.

28:48And so instead of, you know, having the people who actually go with the customer, separated, fully separated from the people who are designing the software, the idea is that by being very close to the customer, you're able to understand the intricacies of, you know, what you need to crack to basically allow a new workflow to be solvable. And so in the case of OpenAI, what is really interesting is that you are very close to research you're very close to engineering in order to shape how the models are going to improve, you know, where they are currently lacking in terms of results and where they are not able to do some tasks.

29:24You're also close to engineering because you work directly on the product that engineering is working on. You help them go in certain direction. You add features when there is a need to. But then also, you know, you're making sure that what the customer is trying to achieve, you're able to, you know, get them through. And so you work on very specific programs which are, you know, and so and which basically you try and go to see from zero to one if it's actually.

29:51Corey:I'd say that that shows up, too, from just the proof points in this project itself. I see you guys did like 7000 returns processed, like a third of prep time saved, roughly 97 percent draft accuracy. That's a huge number. And I wonder where is that the most meaningful to a practitioner and where is it most to an engineer? Do you mean like in terms of the value we give the engineers that we work with and the practitioners? Yeah, I think the way I meant that and I realized it was worded really poorly is like, is that high accuracy the most important point to the end user, I assume, when it comes down to it?

30:35Corey:97 % strong, but I assume there's always a desire to go farther. Yeah, so I think that is an interesting number to anchor on because if you think about from the perspective of a practitioner, and also we talked about the iterative, like basically expanding the perimeter of the product more strategically, so starting very small. And I think this is very important because if we supported a lot of coverage, So we covered 100%, but our accuracy was very poor. This is a complete failure, right? Because they will be spending a lot of their time. They will have no trust on the product. But if we deploy something where it will cover some of their work that they will have to do, like things that we classify that we can't support, like you still have to go through these manually, and that's completely fine.

31:23But if you support those with strong accuracy, I think that's a huge win. So we really focus on trying to expand the perimeter strategically and not really surfacing a lot of errors for people to fix. Because we imagine this would just generate a lot of friction to practitioners. And we think that in problems that you can really measure the outcome, this is a good way to deploy. Like in a lot of fields, sometimes like you build, for example, like a Go dataset. set but when you're when you're working through this with with an expert you're basically making them commit extra time to basically build this with you but if you can deploy something that is immediately valuable and can basically give them like a like a basically it helps them with something that is like lower complexity or mid complexity there's already a huge win so we i don't think we should we should ever track like getting to 100 because this is a world that's always going to have a lot of nuance and some ambiguity so the point that john made like some time ago on surfacing the the citations back to like what the numbers mean we think that's always going to be an important capability of the product because in it would you to understand precisely i would say it's kind of a model receipt it gives you a receipt on exactly like here is your bill it costs you$100.

32:49But here is the breakdown. So it's, I think it works in a similar way. So you can understand exactly where.

32:56Grant:As you approach this process, you know, from an engineering standpoint, from product standpoint, how did you think about the cognitive load of the expert? Like, how did you balance like how much they could actually process at a given time? Because I imagine with some of these accounts that are more complex, and you're offloading a good amount of the thinking to Codex in this case or TaxAri, like how do you still keep them in the loop so that they can still keep track of all of it in their head? Yeah. So basically what we, in order to kind of, you know, make sure that you, yeah, you need to calibrate the product to make such that what, you know, the preparer does and see, you know, it doesn't overload them.

33:37But then like my feeling is that this is a lot about also calibration and making sure that when you iterate and when you develop the tool, you stay very close to them. So here, the Thrive team and us, we also, you know, spend quite some time with the tax preparer and close to them. So I think there is a first point of making sure that you understand very well how they are doing things currently, what they are to process on a day-to-day basis. And there is this thing where you don't want to, you know, disrupt totally the new flow by saying, you know hey look just drop everything in there and then you know you trust it and then you submit it you know like there yeah it could be way higher they could you know process with many packages but it doesn't work right and you want you know you want to make sure that you um keep the expertise uh where it is in you know like account on that uh all expertise and so staying close to them by you know making sure that when the you know like having this closed loop iteration and you know delivering and shipping quite quickly to make sure that every time you do something you're not disrupting their flow and it's still in that capacity and it makes the their expertise and their knowledge to go into the the most interesting tasks basically is what we is definitely important um also maybe taking a step back on you know the kind of you know achieving so basically when we started this workflow there was not a goal of you know trying to reach 98 percent of accuracy for those packages you know i think what this is you know it's like how far can we you know push the taxes workflow and how best can we make it across individual taxes entity taxes and there is no real limits you know the same way you would say it's done for models in evals where basically you know you might say that you know two years ago you had some way more simplistic eval compared to now where it was okay you need to solve all those math math problem you need to solve those you know computer science problem now the evals have been saturated and so you focus on much more long-running tasks i would say there is you know some meta point that's a bit similar here where it's you know you have at some point solved or almost solved the w2 form then you start focusing on the more complex one but then you know once you have solved most of those then you know how can you help even out of the way you know and how can you take a step back and you know make sure that you're starting a non-running task even more.

35:59Grant:It's all about changing the frame. Like there was a great article that Dan Shipper released that was about like, you know, evals are all about changing the frame. Once you change the frame, you like start from zero again. And it's like, now you have a whole new thing you can work on. I have a question for both of you. So after working on this project, in your personal opinion, do you think taxes in general are a verifiable, solvable, like quantifiable task? or is it more of an art? I guess I'm asking, is there more art to taxes now that you've been working on this or is it more like something that at some point we'll be able to fully solve?

36:36I think for our problem in particular, we were solving a bounded part of it, which is essentially just the data entry part of things. There is something that I imagine there's more nuance around ambiguity around tax rules, which I think that's probably a space which requires more of the expertise. And also some of the expertise in our case is already encoded in the tax engine in a way, which provides some of the structure and some of the things related to this. But I think the art part, which is also what Holdings is trying to give the companies by giving them more time to spend on strategic work is how can you provide more tailored feedback throughout the year, for example, so the clients can basically better optimize based on prior information, how the next year is going to go.

37:37And I think that's an area that has a lot of ambiguity, a lot of context-dependent things where that's where I think the art is and that's probably the more strategic work that the moment you unlock experts from doing some of the things that they definitely don't want to do around just churning through documents, they can focus more on their time around that strategic work. And that's where the expertise shows the client management, the previous client relationship. I think that's where it shows, and that's the human part. That's the art part, I would say. You know, in this, I remember OpenAI and Thrive in the blog post,

38:17Corey:I believe it was, mentioning that this is a pattern that could apply to bookkeeping or audits or IT help desks or even other operational workflows. What workflows? What has to be workflows, sorry, or other operational workflows? What has to be true about a domain for this kind of agent loop to work? So the interesting thing is that so right now what we did with Stripe currently is focused on taxes. But so Thrive Holdings has other verticals on which you're focusing on. And another one is IT services. So basically where you want to solve IT tickets or help engineers solve the IT ticketing that they have.

39:01More generally, the loop that we have is usable on, you know, I would say many kinds of problems. And, you know, currently I wouldn't see a limit. I would say whenever you have some experts, you know, that know the field or that are doing something, when you have a product that is being developed where you are actually able to track, basically track what the true value is or what the correct value is, is a perfect setup for this group. And it's great. So in our setup, we made it specific to the taxes workflow, but it's totally expandable because things are centered around codecs, the way you structure basically your software and all the components and things that are saved throughout time.

39:44this can be replaced with the ground truth, the evals. The evals is something which can also be generic. And so what you end up seeing, like what you want at the end is you want to make sure that there is some users using a product where the ground truth that they have can be used beneficially in the long term. And this kind of setup, I would say, applies to an infinite possibilities of sectors because anyone who is basically working part-time from their computer, doing something can have, you know, on a certain product, can have this group that's there.

40:15Grant:So how would you, if you were advising a company to try to build its first, like, domain agent, kind of like following the same roadmap, what would be the first thing that you would tell them to do or set them set up so that they can get that, like, kind of same process going? I think for us as FDs and OpenAI and maybe for other people doing these applied applications, it might be uh it might be pretty straightforward what i'm gonna say but uh you need to be able to measure exactly what you're working against so basically building the the evals and they are very specific to uh to the domain you're working on so i think that's the the the first place to start so as when we go to a new new project we have the the scoping and all of this but we always start from evaluating first because it's basically the first building block that you need to leverage.

41:12It also helps you think about exactly like what is the right hill to climb. So I think, and that's very domain specific. So that's the first thing you have to answer, I think.

41:22Grant:How do you approach the evals? Is it like a goal? Do you have like a golden set where you're like, this is what a perfect version of this task looks like? Or is it a little bit more broad? like what's your advice on just like how to set up a good email so i i think a gold set is definitely very helpful in the especially in the initial discovery but in our domain we have the finalized returns done by the experts so in a way we already have the we already have the the the gold set like there is some complexity as i mentioned around making sure that we remove some of the like things that might be filed in different places and things like this.

42:03But the structure of the text engine helps us a little bit provide initial structure and comparison compared to if you didn't have this, you just had the end documents. But I would say depending on the domain, you might need to go the direction of doing more of the manual curation with an expert. You might have no other way to get through it. Or in other cases, basically you have a lot of data being generated on the field already that you can leverage.

42:28Corey:Much like in mathematics recently where you all discovered – I say you all, not you specifically – discovered an error in a known problem. I suspect we'll see that approach at some point too where it's finding problems in human-produced returns. And we'll probably see that in every field I would assume at some point. Do you have any thoughts on AI? And this is maybe a little off topic. Do you have any thoughts on AI in that role, in catching mistakes we've made versus just us trying to catch mistakes from it, which has been the approach so far? Yeah, I think it's a very good – like the math problem you mentioned is a very interesting example because it – I mean it relates to something which we in our case for the tax case, we all have also observed.

43:19is that for some of the cases, as Arthur mentioned, when you're building your golden set, you basically are using all the data that has been submitted from last year to the IRS. And so the quick part is you know this is true and it should be true. It's supposed to be true, right?

43:36Grant:Yeah. You might be able to find some more flaws. The IRS accepted this. Yeah. Does the IRS have a whistleblower fee? You might want to hit them up. And so, yeah, this is definitely a great luxury because, you know, like not, as Arthur mentioned, like not every use case does have this. And so in our cases, what we sometimes saw is that the model, like this loop, basically the software was extracting some fields. And so we would see some prediction when we were rerunning the evals. What you often need to do is deep dive into specific things that has been extracted to understand and try to see, okay, is it good or not to understand a bit the errors.

44:14And so we noticed that it had extracted some data and that it was actually true. but it was marking it as false because in the ground truth from the IRS, it had this value. And so diving into this, we realized that the actual submission that had been done was off by almost not a lot. So it was not very important in the end, but it was a very interesting case of, hey, look, the model found something that it's actually the real ground truth and the prediction, like the thing that we thought was a ground truth was not. One thing that's interesting there is that our take on this is basically you have you know when you have an agentic loop that's running it's going to be up and you know in the software it's going to be up with the same energy all the time and so there is with humans you can have you know energy and concentration and ability to do things that you know is you know can be more modular or you know some you can have some lows and so in the end there is this idea of you want to make sure that you concentrate what people are best at in some specific case so in In the case of like a huge tax return, you want to, you know, if they are spending most of their time on the more complex stuff that's more interesting and maybe even more funny to do in some way.

45:22And, you know, removing that part of, you know, just copy pasting a W2 from a box to another one. Then, you know, you potentially can decrease those errors. And that's what we hope as well to see in this case. It makes sense because in the in the AI discussion, a thing that I know, especially people who who aren't as as micro analyzing it as as we are, there's this temptation to believe that if a human's done

45:48Corey:it, that means it's perfect. And and the truth is, we are not perfect. You know, we are very fallible. And I love the idea of a backstop or a safety net behind me to to catch my own errors. I think that's a huge power that we overlook, and I love how this works. I think this is a really, really cool approach to a problem that everyone here dislikes. Like even for accountants, this is a tedious thing. Like it makes a lot of sense because with such a complex tax code, with so many variables that come into play in taxes themselves, themselves that this feels like a thing that if it can knock this down flawlessly, there are a lot of other things it's going to do really well with just because.

46:35Also, I think this is one of the type of tasks, and I think there is a bunch of tasks that are similar to this, which is like producing the extraction into the text engine requires a certain amount of effort. Reviewing requires a different amount of effort. So if you get sufficient accuracy on the things one-shotting and you have to fix a few things, reviewing is way less effort than you would do that job yourself. So the overall accuracy of the system increases overall because a lot of the work, as John mentioned, they'll be consuming a lot of your attention, which is maybe the causes of some of the errors and also like fat fingering or or doing like some of some of these mistakes

47:22Grant:yeah just the amount of stuff you have to review and then you get lost in the details and you miss like the most important thing because you were too busy like i had to double check all of the you know we're so distractible yeah well i had two two more things before we wrap up here i'll make them quick the first one is this almost makes me think like i wonder if this will lead to sort of like a adversarial game where the IRS will have their own AI, tax AI, and then the firms will have their tax AI. And then they'll be, you know, trying to one-up each other, you know, kind of like how today's accountants sort of are trying to do that with the government.

47:57Grant:I wonder if you have any thoughts about how that might play out, which is interesting. Yeah, that's very interesting. I actually never thought about that case, but that, yeah, you know, in some, what you could have with this is maybe you would end up having a conversation between someone's tax software and the IRS and each of the software AI could post a challenge, like I said on the channel saying, okay, this is true, no, this is true because of this. And then the other one saying, I agree, what I thought was wrong. And then the other one I think you need to have some framework if you want this to be working but you know in the future you could have something like this and i think it would definitely help solve some use cases where sometimes you know you might notice a mistake that was easy to fix and if you would have you know such a direct interaction you might fix it way earlier than you know having to spend way much more time months afterwards i would think so yeah this would be yeah because a lot of people in in the u.s are often say well why why do we have to you know figure this out on our own right at the end why can't they just tell us how much we owe

49:07Corey:they know the tax code you know why are we trying to do their own returns in the u.s is a thing too yeah we're not we're not tax law experts yeah exactly so it would be kind of nice if you could

49:19Grant:just like call them up and be like hey i think this is correct but is this correct and your

49:23Corey:agent could just kind of chat it through codex talks to their agent and uh and i just find out when it's finished yeah yeah exactly i think some of the problems as well are the different reporting mechanisms so your receipt you'll be leveraging like your accounting firm to do that like i think for some basically maybe you have the same fact being reported like you're guaranteed to have the same fact like your employer but i think for some facts it's slightly harder like like charitable contributions and and other things like this so i think that's where like i think things gets like more complicated around like what are the facts the IRS holds and what are the facts that you hold and make you sure like that those reconcile correctly.

50:08Corey:Yeah.

50:08Grant:Yeah. And then my last question to both of you is, you know, what do you want to see, you know, OpenAI? How do you want to see OpenAI improve its models from here? Like what do you really want to see from the next generation of models? What could really improve this use case and really help you with what you're trying to do? I mean, I think for like on this specific use case, I think what we see generally is that the ability to do a long running task definitely helps on a lot of other things. Because usually, you know, like if you're able to do more long running tasks, it means that the model is able to, you know, look up in some of the data.

50:45In our case, it would be, you know, the evals, the traces, also being able to do some tool coding. you know imagine at some point being able to do some part of the tax calculations itself in a in a way better way this would be interesting as well i mean i think generally speaking like the more it becomes able to do things that are related to you know the economy in general is better because i think in taxes you have a lot of concepts which are human invented and you know which are very potentially hard to grasp for a model. And so, you know, the more it understands the economy, and so I think we have, for example, evals that are well-known on this, such as, you know, GDP evals, which are more broad.

51:27I think expanding into those domains means that you have taxes that improve directly, but then also has consequences of other things improving. And so, yeah, this is quite exciting and will be useful for the next model version.

51:41Corey:It absolutely will. John, Arthur, thank you so much for taking the time to join us today. It's been really interesting. Thanks a lot. We're really excited to be here. Excellent. That's awesome, yeah. Great conversation. We'll share the link to the blog post about Tax.ai as well and make sure that it's there for anyone who wants to check that out and learn more and see what's going on. Please take a moment, if you haven't yet, to subscribe to the channel so you can check out all of our interviews with the people building and impacting AI every day. Also, please pop by the Neuron.ai to subscribe to our daily newsletter.

52:15Corey:And that's all we have for today, ladies and gentlemen, but we will be back soon with more. Farewell for now, humans.