Jev: 8 real use cases for the fastest, cheapest model I’ve ever used | John Lindquist

0:00In some ways, managers are really at risk. This concept of like org design and role design, you can put those skills to use when crafting your agents, which is why I am currently running, no joke, 40 GropBots right now.

0:13John Lindquist:Whenever you have constrained inputs that are interacting with apps, Jev is a good thing to reach for. This one is a favorite of mine because it represents like a workplace where you have multiple agents that could all have their own tasks and avoiding collisions so that they never end up on the same space. At each step, it's going to be able to control these different agents and make sure that they get to their specific task as quickly as possible and that they never collide or interfere with the other agents. This one's really fun to me because it's a presentation coach. I click the microphone, I'd start talking.

0:45John Lindquist:I'd have a list of bullet points in either a podcast or something that I want to make sure I cover or an interview or anything. It listens through everything I say and then starts checking the boxes to make sure that I covered everything. This model, which is basically free and incredibly fast, allows you to do discovery over data in a way that feels like it's opening up my opportunities and allowing me to look at things that I thought weren't high ROI before.

1:15Welcome back to How I AI. I'm Clara Vaux, product leader and AI obsessive here on a mission to help you build better with these new tools. Today, I have repeat guest John Lindquist, who has done one of our most popular episodes to date on the podcast. And here he is joining us on Jev Week at How I AI to talk about our favorite new model and all the amazing product-facing use cases that he's discovered testing Jev from TypeSafe AI. Let's get to it. John, welcome back to How I AI. I have to say this is Jev Week on How I AI. And the reason it's Jev Week is a lot of models have come out in the past seven days.

1:58You and I do a lot of AI things. And yet, when we're DMing back and forth, we're talking about one thing and one thing only, which is Jev. So, you know, folks have maybe listened to my episode earlier this week about why I'm hyped on Jev. I would love to hear why you're so excited. You know, I'm looking up here. You have 23 demos. we could probably look at. So you were going deep, Jev, as well. What about this model, this framework, this way of thinking about building has gotten you excited?

2:31John Lindquist:Fast and free are both amazing. If you've ever been tired sending a basic request to an LLM and waiting around for a bit just to have it call a function, this is now essentially instant. And I think of all the time I've spent on this and the hundreds of demos I made, I think I've spent 73 cents on it. And I love the one quote someone said, I think I'm going to pass this inheritance of Java onto my children once I die. Yeah, I like the one where it was like, TypeSafe is really taking off. They have to have made double digits in revenue by now. So yes, fast and free. I also have had that experience where it ran over like i don't know 20 000 records and was like i have charged you 0.4 cents right and and it it exposes those scenarios where i thought like i had five gigabytes of jason i'm like i had never passed this into an llm i just never like i don't have that budget on and on i'm like let's see what happens and it was it was 40 cents you know and i i wasted it on a really stupid thing, like try and organize this or whatever.

3:44John Lindquist:But five gigabytes of JSON is a lot of text. And it went through and did the whole thing in probably a couple of minutes. That's been my experience too, which is I'm looking at corpuses of data that I would never just dump into an LLM, both from a cost perspective and honestly, from a time perspective. It's like, this is not worth my time to even see if something's here. But then this model, which is is basically free and incredibly fast allows you to do discovery over data in a way that feels like it's opening up my opportunities and allowing me to like look at things that I thought weren't high ROI before.

4:27John Lindquist:Yeah, absolutely. Same. It's just opening doors that weren't closed before. Yeah. So just quick reminder for folks, Jev is not like a, the LLMs that you're used to and love. It does not output text. It outputs basically like decisions and scores and yes, no probabilities. And so you kind of put information in and then you get out very structured, type safe, very structured, limited set of options. But the ways you can use this are almost unlimited. And so John's going to walk us through some of his favorite demos of how he's using it. Hopefully a real-time, fast, cheap model can build cool things and a little insight of like, what else is around this?

5:11Because what I found is I see a lot of these like shiny demos on the timeline. And then you're like, you're hiding your, you know, your embeddings and you're hiding like how this is actually working. And so I think if you can lift the veil for us on how some of this works and where it's Jev alone, where it's Jev plus, that would be awesome.

5:29John Lindquist:Yeah, I would just, just to add to your definition of jev of it being type safe i think of llms being unstructured to unstructured where you put text in you get text out and this one is similar you put in unstructured sentences and data but then you get structured data out and the power there is it's humans talking to machines and they're the scenarios where you have functions and apis and everything set up, but I want to communicate with them using unstructured data. And we're so used to chatbots, and it's just not a chatbot anymore. You need services, you need functions, you need things you want to throw data at first, but you can talk to it however you want, which is incredible.

6:14Yep. Well, let's talk about talking to it. All right.

6:17John Lindquist:Yeah. So let's go ahead and try this out right away. So traditional to-do app, We have a list of to-dos. And the fascinating thing here, which I'll try right away, is I'll just say something like, book dentist appointment, remove. Buy oat milk, complete. Review pull request, low priority.

6:45John Lindquist:Now the speed there was all done through JEV. it does a few passes of looking through the text where it ensures that it lines up. If there's any like dictation mistakes and you say it doesn't get oatmeal quite right, it'll go through and see, does this quite match up with something or do I need to do a different, do I need to categorize this into something that exists like a confidence score? Like it's all a lot of confidence scores there. It matches it with the thing in the list and then it matches it with the function that it's going to call right based on based on the way that the operation that you want to perform on top of that data.

7:23So it's it's multiple layers of analyze the dictation itself to see if that's

7:28John Lindquist:like a valid dictation then analyze if it matches up with a task and then analyze if it matches up with the operation you want to perform on the task and it can do that all live and it even while you're dictating it classifies if you've spoken to a point where it can take an action. So as the words come in, it's like, is now a time where I can, where I have enough data to make a function call, where I can reliably keep on talking? Because you notice that entire time I was talking, I never paused, I never hit enter. It was able to classify, is this a moment where I have enough information to take action?

8:04John Lindquist:So it's just showing all these pieces of JEV fitting together to give you this real-time, like, give me kind of streaming data in, like, unstructured streaming data in, and I'll start putting it in all the places it needs to go. I have a question for folks that have just have no idea how to actually put this into place and want more of the tactics, which is, are those, you know, are those like sequential JAV calls? Are you like, yes, no, it's time to analyze. Yes, no, it's this is a good transcript. to score, here's the top task that relates to it, and score, this is the top tool call that relates to it?

8:42You know, is this like, because people are, I'm sure, trying to put in their mental model, like a tool call where you get all that structured data back. Like, you know, how have you set this up so people can kind of understand how they chain these job tools together?

8:56John Lindquist:My mental model is, remember back when we used to write code by hand, there is no conditions if if else switch statements all these things that were the moment of classification and if you just think of a a line-by-line program where things execute you think of where do I need those conditions and so as you're breaking the problem apart whatever the thing is that you're building just think of all the conditions that if else but the moments where it either needs to be put into this bucket or that bucket. And so as I was going through this, I built a lot of it backwards because I started with what I was building, marking the to-do as either done or removing it or heading a new one and such.

9:40John Lindquist:And I'm like, oh, wait, now I want the dictation to be able to select it. And so I just kind of built it backwards. And it was so easy with Opus 5.5 recently just to kind of wing it and like take it different directions and go through scenarios and see see the cases that that were exposed because I had that final idea of I just want to if you've ever used a Siri or a Google Assistant or any of those and you try to manage like add to my to-do list and they're all terrible and like how can I build like does Jeff enable this to actually be good and I think I think it does like it's one of the most common things like managing you could integrate your calendar into this.

10:20You could integrate it because they're all behind function

10:22John Lindquist:calls, right? Or MCPs or however you want to build it out. You could integrate anything with this and just have this top level dictation live inference running. And it's something anyone could build cheap and free. Like it's amazing. Amazing. Yeah. Okay. So you and I both did live voice apps. Mine was much more emo, which was just talking to the screen and getting a color and an inspirational quote. Yours is much more practical, which maybe reflects our different personalities. Let's see what other use cases you have. And what I like about what you're showing us is I talked a lot about internal data analysis and clustering and all this stuff.

10:59You have such great inspiration for how you can use this inside products or to do things that were previously really tedious. So I'd love to see another one of your favorite demos.

11:09John Lindquist:Yeah, let's dive in. So this one kind of breaks it down into its smallest pieces. if you think of a grocery store and if you think of, I have a function called like get cart, if we remember programming, and you type in something like what's in my cart. And that's how, if you think of classification of regular text to functions, like this is just demoing back. If the previous demo didn't make enough sense, think of like plain English language mapped to a function name. And that's one of the core foundational pieces of what Jeff does. That's that demo. Classifying documents. Sounds like kind of covered that.

11:49John Lindquist:This one I think is incredible because if you've ever run a scenario in, I use Google Contacts and often you put in the same contact twice and you've probably gone through the process of like merging them together. Very common scenario where you have messy data of someone typed in something wrong and they hit submit and now you have two company entries in your database or whatever. This can go through and go through all of the records and kind of merge those records together. So let's click on that. And it can go through and say this one wants to merge. So Cedar Grove Office Products obviously matches with Cedar Grove Office.

12:30John Lindquist:Ridgeway Data is Ridgeway Analytics. And it can go through and apply those merges for you by matching together and classifying. Is this one close enough to that one? And again, it's huge data sets merged together in milliseconds. This is like my favorite use case of Jev, which is like pairwise comparison of a lot of data to create grouping and clusters. And so in the episode that I just did, I did that over PRs. I were like, were these PRs on the same product area or the same thing or the same theme? But you could do data reconciliation, I think, is a really useful one. I went through recently and I had like 2 ,000 passwords in my 1Password.

13:14And you know how like whenever you do your 1Password, it's like update, save, make new. And I'm just like all over the place, had a whole bunch of different ones. And like, I was just thinking how much faster it would have been to use Jev and some of the metadata to just pair those up and merge them.

13:29John Lindquist:Absolutely. And again, it's not like you're giving an example where there's like six, things. You're talking about when you have 60 ,000 or 600 ,000 things, those pairs feel very expensive until you use something like this. And something we haven't shown is that there is a confidence score that comes back. So if there is a North Star Clinic and there's North Star Clinic services and they don't quite line up, like if you want them only to merge if you're like 99 % or more confident, you can set those parameters in there. So that is a knob you can turn as you go through a few checks. And I think like you mentioned before is after you do this pass, you can send a smarter LLM against the result and say, we were, the data did look like this.

14:19John Lindquist:Please, you know, either random sampling or the entire data set, ensure that this was done correctly. And then you can gain more and more confidence over time as you run more and more, you know, large data runs. One other thing you can do on that, in addition to validating the data pairs, is you can describe them. And so what I've done is do the pairs and then run. It doesn't even have to be an expensive model. But say, like, we've decided these two are the same. Explain to me why. And it can give a short kind of like, it's the same because they both say services and, you know, whatever it is.

14:53And so you can get sort of a quantitative matching and then a like qualitative explanation with two relatively cheap models once you've narrowed that down. Yeah.

15:06John Lindquist:Yeah. Great stuff. OK, great stuff. We are just psyched about it. I was like, I can watch all of these. This one's fun. It's kind of hard to show off. But essentially, if you think of having your entire application and so you have go back going back to the to do list and say that's one part of your much larger app. You can build additional abstractions around that so that if on your landing page, someone comes up and either has a command bar like a command K or other demos or sorry, other other parts, Omni bar, any place to put in text. and they start typing in the dictate or voice or something.

15:52John Lindquist:They can't remember what the to-do app was called. And you notice this is to-do and I typed in to-do. It's able to go in and match against the tool. And then you could type additional things where you'd say, go to the to-do app and then take this action. Like I would know in my head, oh yeah, there was that pull request. and then take all the pull requests and mark those as low priority. And so if you think of this multi-step classification where you can build your smallest tools and then build another abstraction of JEV around them where it can pick which of the smallest tools to use, this kind of goes back, if people have set up MCPs where there's abstractions around them, I think Executor and others where it picks which MCP to use, there's the tool itself and then there's a layer where you're picking which tool to use and you can build that as as many layers of that as you want that that's what this is kind of talking about it's it's difficult to show off but just imagine saying go to the to do app do this and yeah what i would say is like jev is a router is maybe like what's helpful a very very fast router.

17:07And so, you know, this use case, you're saying like, as I have text route to the right tool. And then if I can infer from the text, the job to do in the tool, I might as well infer that and like route as far down the user journey as I can infer from these like structured options. Other practical use cases I'm saying, I'm seeing a lot of people start to build these like JEV powered coding harnesses where it's like, given the first input, let's route to the right model, let's pick the right tool, all this kind of stuff. And so you can imagine very fast configuration, very fast routing, very fast, like I don't just have to take the first step and then think and take the next step, I can actually just build the chain and then execute it.

17:51And so I think these, these decision models like Jeff, if they remain fast and cheap, can be a new way to think about navigation of your user experience, whether that's a kind of like front end experience or a dev tool or whatever it might be. Yeah.

18:09John Lindquist:Even search to a certain extent of search for a long time has been has had a lot of really expensive options out there. And I don't know how far you could push this into replacing certain search engines. But yeah. You know, I gave an example when I did this, like I had 4000 YouTube comments and I was trying to do live search through this. And I want to demystify some stuff for folks because it actually isn't super fast to like search over all of these results and say like, yes, no, this is a good result and rank it. But you can like cluster like groups of like 30 or 50 or 60 and score them and rank them.

18:50And that is a much faster way to build quote unquote real time search with Jev. So, you know, for folks, again, I just want to like give people a little bit more specifics about what you're seeing on the timeline, which is you do have to think about how you're going to architect around JEV. It is fast. It is efficient, but it's not like completely latency free. And so you may need to go look around, see how people have done search. And then if you find one you like, go into GitHub, figure out how they actually made that search super performant. Because I've seen a couple of different strategies out there to how

19:25John Lindquist:to do this yeah and caching and all that stuff all this all the hacking yeah it's easy when everything's locally cached and just go over the the full corpus let's talk about games because this is something all right i haven't covered yet but i think is a really cool demonstration of some of the ideas inside jev yeah i built this one because i wanted to show off jev versus a traditional model. I picked a free model on open router. This is like a low reasoning model. I think it's a Kimi or something. I'm not sure. And for people that aren't watching, it's playing chess. Yes. This is playing a game of chess and JEV are white pieces.

20:05John Lindquist:The LLM are the black pieces. And here, this is set so that JEV is able to think through all the possible moves and then it ranks the highest three moves and then it thinks through all the next possible moves there and based on that two-step reasoning picks the best next move and so kind of like routering it's kind of like um spidering into the potential best moves um and then coming back to what the next best move would be and it's still doing this in sub one second times and using ai and inference whereas if you pass the same data set to an llm the amount of reasoning it takes this this is playing blitz chess so there's a one minute timer jev would be able to complete this entire game and beat it and the llms would eventually get there they'd be much more expensive and they might not even perform better because in scenarios like this where it's focused on routing and the next best thing to happen the like the amount of data that you can crawl into and find the the next best move can kind of balance out between these two unless you're getting like really super high level like creative chess where you're trying to stump someone by fainting certain moves or whatever but i'm not it i'm decent at chess um i can beat my kids perfect for now sometimes for now um and if you look at the um it has some benchmarks over here that jev was 10 times faster in the average move and it was four times cheaper and this was a very space money alphas on uh low reasoning so with very very tiny context windows yeah what what i think this demonstrates is one there are scenarios in which speed is a is an advantage um this is this is putting in the context of a game but um it's very obvious how much i mean 10x faster is not incremental and so it really does unlock this concept of real time and i do think about going back to when I was a young product manager and did a lot of conversion testing and growth work.

22:29And speed did matter in a lot of user experiences, especially consumer on what won. And so I do think, we hear a lot on the coding side about model optimization from a cost perspective, but I think we're going to start to see latency wars really heat up as well, which is like the faster the response the more like more magic you can unlock that feels like an if else statement the the better and so i think this is showing one where speed really matters i think the other thing it's demonstrating is this like multi-step routing that we talked about where you can think through turn one turn two turn three and then because it's 10 times faster you could apply another model on top at that point, a more intelligent model at that point, with kind of like a narrowed set of strategies and probably make a stronger decision faster.

23:25Right.

23:26John Lindquist:And I think this also demonstrates the, again, to the concept of router, there are a limited set of options in chess and other games. There's a lot of options, but it's a limited set. And you can pick from the most likely set and narrow it down and optimize cost and everything. data structures and your apis are all the same like they all have a limited set of things you can do even though it might seem infinite to you because you know you never know what somebody's going to type or do in your application uh this handles those when you have those scenarios it's it's time to look for a jev first i would say whereas the llms are much more um i don't i have no idea what i want the action to be or i don't even know if i want it to take an action or things where it's much more brainstorming and creative and finding what you want to do.