RxJS with Ben Lesh

0:01RxJS is an open source library for composing asynchronous and event based programs. It provides powerful operators for transforming, filtering, combining, and managing streams of data, from user input and web requests to real-time updates. Ben Lesch is the creator of RxJS. He joins Josh Goldberg to talk about his path into engineering and the RxJS library. This episode is hosted by Josh Goldberg, an independent full-time open-source developer. Josh works on projects in the TypeScript ecosystem, most notably TypeScript ES Slint, the tooling that enables ES Slint and Prettier to run on TypeScript code.

0:41Josh is also the author of the O 'Reilly Learning TypeScript book, a Microsoft MVP for developer technologies, and a live code streamer on Twitch. Find Josh on Blue Sky, Mastodon, Twitter, Twitch, YouTube, and.com as Joshua K. Goldberg.

1:10ben lesh welcome to software engineering daily hey thanks for having me i haven't seen you in a while yeah it's been a bit i'm excited to talk to you you've been doing a lot of really interesting stuff around observables and rxjs but before we get into all that can you tell us how did you get decoding? Well, let's see. I had been doing like little basic programming stuff when I was a kid, but then I went to art school, dropped out of art school after about three years and was working doing graphic design work at Ohio State University working there. I was not going to school there. There was a situation where someone was working on a website or whatever, and I was curious about it and I thought I could fix something or work on it.

1:51And basically I learned how to to do like some basic html and javascript stuff and i had a friend that was like dude you know how much more money you can make doing this and i was like really so at the time there was no google there was no stack overflow none of that i basically loitered in barnes and noble and read books because that was the only way to get recent information like they weren't at the library or anything you know practice doing stuff on my little compact presario eventually I got my first programming gig it was there was an ad they were trying to hire somebody for it was like 80 bucks an hour or something like that for four hours a day I had to call them over and over and over again because they're not calling me back because I you know wasn't qualified for what they were looking for and they finally brought me in they were like they asked me some questions I got probably half of them right and they were like so you know what we're gonna ask is we like I was like do you think you're qualified for this and I was like well no but but I really think I can do what you need done and I'm willing to learn and I'll work for$10 an hour.

2:52So if I work for two days and you don't like me, then you've only paid me for what you're going to pay somebody else for one hour. And you could probably even still hire that person if you wanted. They liked that answer and I got the job and it was a one month contract, which is I think why it was open for so long. And they kept me for like three months or something like that. And they gave me a letter of recommendation because I asked for one. And after that, it was you know, I got other programming jobs and the pay was higher and everything. So. That's a fascinating interview technique to offer to undersell.

3:24Yeah. Yeah. I mean, you can get away with it when you're, how old was I? I was 20, I think, 19, 20. So. So what tech stack was that? And I, I'm just curious. The first gig actually was a lot of just HTML and JavaScript. So the JavaScript was funny. The very first thing I had to write was they wanted, people don't remember this probably, but there's a thing. It still exists. If you get into any browser and you say window.status, there's just an empty string there. And there used to be this property in window you could set called window.status and you set it to any string. And you know, when you hover over a link, modern times, it shows like a little, like the URL underneath, like on the bottom of the whole page, that was the status bar.

4:05And you used to be able to set that manually with window.status. And they stopped allowing people to do that because it was a way that you could be like, aha, you're going to click on this link and it's not actually the link because someone was setting it. It was like a security hole. But what I was tasked with doing was making like a little ticker that played in that window.status of all the deals that were going on in this e-commerce site that day. So I learned about set interval there. That was the first time I ever used set interval and I had to slice a string and make it tick along and continuously wrap.

4:41And that was the first, that was the first professional problem I ever solved. Just getting a ticker at the bottom of the page with a now deprecated API. Right. Yeah. But you can get the string off of it if you want. That's the empty string is still there. That's, that's kind of representative of what you're known for in the web community. Now you're doing some sort of timing, some sort of streaming data shenanigan in JavaScript with bizarre APIs. How did you go from that to working on libraries like RxJS? Oh, boy. It's a long path. After that job, I was doing visual basic programming and ASP, classic they would call it now, but at the time it was just ASP.

5:20Got into doing.NET development, did that for 10 years. The reason I got into JavaScript development was around the time Angular was new, we were trying to build... I was working at a company, They were doing pharmaceutical robotics and they were trying to build like this client that was very rich in features. There was this belief amongst software engineers at the time that JavaScript was a joke. It was a joke language. You could never make good money doing that. It was not the work that you wanted to land on. And I was like, all right, I'm the senior. At the time, I was the senior engineer on that team.

5:58And I'm like, that's fine. I'll do this work. I don't care. and so and you know this was back when you had to like you had to like concatenate your javascript files yourself and all these other things there was like you had to basically build your own build tools so that's kind of how i got into it and i had to answer a bunch of questions on stack overflow about angular to learn things about angular so i'd find ones i didn't know the answer to figure out the answer and answer it it just so happens that that meant that i was one of the top answers on Stack Overflow. And someone at Netflix was building something with Angular, trying to build something with Angular.

6:31And they saw my name over and over and over again. And they said, hey, why don't we see if we can get this guy to come work for us? And I thought it was a joke. I got this email from Netflix. And I thought it was one of my friends screwing with me, like, that Netflix was like, hey, we're just randomly reaching out to you to see if you're interested in working for us kind of thing, right? Like, I didn't apply. And so I went and I was like, all right, I'm going to go to this interview. I'm going to have a phone conversation. That's what I thought. And I had a phone conversation with the person. I was very honest with them about my level of ability or whatever.

7:00And they were like, well, great. We'll have you come out. And I'm like, okay, cool. So I've never been to California. I'm living in Pennsylvania at the time. I mostly spent time in Ohio, Michigan, Pennsylvania. I was like, okay, so I get a free trip to California. And that's probably all this is going to amount to. I'll get a little tour of Netflix. That'll be awesome. Then the next thing you know, one of the, I think the VP of engineering or somebody was walking me out to the front door, like talking positive. I'm like, okay, so I'm going to get an offer, but there's no way it's going to be enough to get my family to move to the most expensive place in America to live.

7:31And then it was, so I ended up at Netflix and at Netflix, this is a really long story. I told you, like we're going to get to the RXJS thing at Netflix was my first introduction to RXJS. I'd never seen it before. And I started using it for some things as I started to understand it better and was working on real-time streaming dashboards. So Netflix, the RxJS really helped with memory management and this really busy streaming dashboards of charts and things. And of course, the updating streams and all of that. There was a feature that we needed. It was a re, I think retry when. So I added that feature through open source.

8:09And I'd also done a lot of work in open source on the Angular code base at the time too. even though I was working in Ember at Netflix, which is funny. Yeah. So since I had a lot of open source experience, I was approached by Jafar Hussein, who worked at Netflix at the time, and Eric Meyer, the guy who invented observables, and this guy named Ben Christensen, who worked on RxJava, who worked down the aisle for me. And they came over and they said, hey, we want to talk to you. And they said they wanted me to work on RxJS, to rewrite it. And I told them I was not qualified. And I I told them they should give them a list of other people they should pick.

8:44And they were insistent because of my open source experience. And so the rest is stuff that people know, I think. But yeah, that was 11 years ago almost now. So yeah, wow. On the reactive-extensions-rxjs-github, we see issue number 482 and pull request number 486. And yeah, over a decade ago. That's incredible. I also love that you've now used another interesting technique, stack overflow as a form of development, that a lot of this was started in part by you finding questions you did not know the answer to and answering them. That's a powerful technique. Yeah. Well, I mean, what I tell people, you know, I don't know that you could still use stack overflow for that now.

9:26But like what people should do if they want to make it somewhere is be conspicuously helpful. and right now the climate's pretty weird right now i feel like all of the really popular developers are these like kind of loud bombastic people on youtube or whatever like they're not i wouldn't classify them as being helpful all the time like sometimes they're the opposite of that i think but there are some of the information they give out is helpful and giving information to people in general is helpful i guess so but i think that being conspicuously helpful either in your own community, giving talks, writing blog articles, like whatever you can do, or just answering people's questions and being kind and nice and that sort of thing, like is really a pathway to success.

10:10Like you're going to be a lot luckier in your career if everyone knows that you know things and everyone thinks that you're useful and helpful than if you just sit and be quiet and keep your head down and that sort of thing. So, and Stack Overflow, that was, it wasn't intentional on my part. It was more just like, I want to be helpful and I want to learn the answer to this question. So therefore I'm picking out interesting questions and seeing if I can figure out the answer. We're going to talk about RxJS, but I have to agree with you for a brief moment that there's this kind of growing area of developers who are technical, who create content, you know, content creators.

10:46And it's a very different thing to be a great content creator versus a great, let's say, contributor. A lot of overlap, a lot of people who do both, but it is two similar areas, similar to how front end and full stack are similar areas. And it can be hard, I think, for people to understand when someone is speaking as a content creator, just to spread information versus educating, helping best practices get spread or even evolve. There's a monetary, a heavy monetary component to the content creation thing too, right? Like there's, it's not just people out there, like I'm curious and I want to be helpful.

11:19Therefore, I'm out there doing things. And, you know, ideally I get rewarded in some way by my getting a better job or something at some point. But like, these are folks literally like making money off of the number of views they get for things, which is, it's a different kind of incentive. Like it's, it's not, there's probably less incentive to be helpful anyways. Like unless the helpfulness is the reason people are showing up to your channel, I suppose. But I'm a little bit cynical about it. I have children. I've watched my children's brains rot watching weird short things on YouTube. I hear the word YouTube and I'm just like, I shudder a little bit.

11:56But it's just because I'm, I think I'm just getting old and cranky when it comes to that sort of thing. So yeah. Old and cranky, he says with a happy smile and laughter. Let's simulate stack overflow for a bit. The heyday of stack overflow. I'd like to ask you a few questions. To start, let's say that you find someone has asked, what is RxJS and why would I want to use it? How would you answer a question like that? Oh boy. Well, I mean, first of all, I know the answer to that question, but like I would probably just start off with you look at that thing and you're like, all right, so most people don't want a whole like book or they do want the book, but they need the summary.

12:37So you start off with some bullet points like RxJS is about streaming data. RxJS is the opposite of iterables. RxJS is the cancelable asynchronous type for zero to n values. And you can just start off with those simple bullet points and then dive deeper into what each one of those things actually means. What is asynchronous data? How is it the opposite of iterables? Why is cancellation important? Those sorts of things. you can come up with something that might take somebody a couple hours to read if you really wanted to get into the whole details of it. But the important thing is that people get the very, very high level bits of it and why they would reach for that particular type or push versus pull and some of these other things.

13:23Yeah, that's a great segue. The next question is, I've heard of these things that RxJS sounds like, but I don't understand the difference. What is the difference between an iterable and something that's provided by RxJS? Oh, okay. Well, so an iterable, we use those all the time. So if you use a for of loop, that's using an iterable. So you've got this, and I know you know this, but this is obviously simulated. But yeah, an iterable, if you use a for of loop, what it's doing is it's taking the object on the right side of that of and it's saying, hey, give me your iterator. And it calls the symbol.iterator thing.

13:58This iterator comes out and all the iterator is is just an object basically with a next method on it. And for every loop through, it calls next and it gets back a result that says it has a value and says whether or not it's done. If it's done, the loop stops. If it's not done, it takes the value and goes into the loop. And so what you're doing then is you're pulling values out when you iterate. So you as the consumer, the person that wants the data is saying, oh, I'm going to do something. Okay, give me the next one. You pull it out. And then you say, I want some data. And you say, okay, give me the next one.

14:29You pull it out. Now, an observable is exactly the opposite of that, where you subscribe and you give it a function that is your next. And it calls your function when it has the next value. So instead of you, the producer saying, or the consumer saying, I'm going to do something. I'm ready. give me the next value, you're saying, I'm waiting for you to give me a value and I will do something with it when you give it to me. So that's more like an event. So any sort of event that you might have, like a click events or mouse movements, or even getting a HTTP back or something like that, those are all models.

15:03You can be modeled as an observable. So that's, that's the, that's the main difference is one is you're pulling values out and the other one's pushing values at you. That push versus pull description makes a lot of sense, I think. And for anyone who has extra time, I think you gave a fantastic talk on this at RevoJS 2023 on pushing and pulling in JavaScript. Another topic that comes up a lot in the context of pushing and pulling is observables. Now you've been working on observables. Can you just define for us what does it mean to work on an observable or even what an observable is? Okay. Well, observables were the original like kind of team that created this was at Microsoft under Eric Meyer.

15:42And literally they went at it from an academic standpoint and were like, what if we took an iterable and pulled it inside out, like made the dual of it, if you will, and made this pull-based type, the iterable into this push-based type observable. That's kind of how it started. Or another way to go about it is if you've ever seen the observer pattern, people don't realize, but they use it all the time. So the observer pattern, there's two things. There's a subject and there's an observer. And the subject is shaped like add observer, remove observer and notify. Those are like the three things on a subject.

16:13And it does what you would think where the observer then has like a notify on it. That's all it has. And you take this, you take this observer, you give it to the subject. And then the subject when by adding it or whatever, the subject, whenever is notified, will notify all the observers of that subject. Right. And so it starts with that. But there's kind of a missing piece in here where you can kind of functionally chain any subject to any observer by like kind of chaining them together. And the way you do that actually is with observables. Like people don't often think about it. But an observable really what it is is you have this type that when you subscribe to it, it executes a function internally and says, hey, create a producer.

16:57here is this observer for you to next values into or call next on whenever you get a value or call error complete or whatever but we'll just focus on the next part of it and what happens is when you subscribe it calls that function creates the producer starts that producer starts nexting values and then when you unsubscribe there's some teardown logic that you've registered in there that will say oh i know how to tear this down with it's a web socket you might close it. If it's a mutation observer or whatever, you might disconnect it, like that sort of thing. So it's a really, really, really simplified type, the observable itself in its raw form.

17:35It exists in every single language that exists as far as I'm aware of. Well, every Turing complete programming language anyway, there's not one in like CSS or something, but it's a really, really, really simplified type and it's used for a great number of things, which is why it's being moved or been put through standards bodies, and it's being now added to the browser. That's a fascinating process that I think would be interesting to walk through, because for a very long time, as you said, this was just a common practice in a lot of libraries. And RxJS is, to many understanding, the most common, the very popular way to do observables.

18:15APIs are the foundation of reliable AI, and reliable APIs start with Postman, Trusted by 98 % of the Fortune 500, Postman is the platform that helps over 40 million developers build and scale the APIs behind their most critical business workflows. With Postman, teams get centralized access to the latest LLMs and APIs, MCP support, and no-code workflows all in one platform. Quickly integrate critical tools and build multi-step agents without writing a single line of code. Start building smarter, more reliable agents today. Visit postman.com.sed to learn more. so what is it about observables that you think has made it so that people not just want to do it but want to standardize it in the web altogether i think the most powerful thing about it there's several powerful things about it but the one of the things that's interesting for for folks is if you've got an iterable uh an iterable is a set of things as we know it's like an array can be an iterable a number of different things can be an iterable so an iterable is a set of things and if you have an observable it's also a set of things but it's a set of things over time.

19:18But what's true about sets of things is let's just say you have a basket. A basket can contain a set of things. If you have a container for a set of things like an observable or an iterable or a basket, you can take a basket of apples and create a basket of sliced apples by putting it through some mapping process of slicing it, right? You can take a basket of apples and make a basket of not rotten apples by filtering out all of the rotten apples. you can do the same thing there's obviously arrays and intervals have map and filter right flat map and these other operations all of those same operations exist on observables but the difference is the observables are events so now all of a sudden you can take events and coordinate them together with these same sort of operations you would use on you know these these static synchronous sets so that's the biggest thing i think that attracts people to it originally like initially.

20:09Now, what I also know is, and this is one of the reasons we liked it at Netflix was it also has this very deterministic teardown feature where when you're not subscribing, it tears everything down. Like if you chain a whole bunch of other stuff together, it will automatically tear everything down. If you have a retry, it will tear everything down and stand it all back up again. Like, and you still have this deterministic kind of memory management that gets really, really hard when you're manually coding events together where you might have to be like, oh, if I get this, then I have to subscribe to this.

20:41I have to start this web socket. And when I get this message, I have to send this HTTP request or whatever. All of those things, if they're mid-flight, become very difficult to make sure that you get all of those things torn down appropriately if you're manually writing it, where if they're all wrapped in observables, it's almost like having a finely block in every bit of your code to make sure that things are being torn down. Yeah, I can see why that would make for a much cleaner code, both to write and read. Using the example you brought up of your very first experience, little ticker at the bottom, that kind of brings up a lot of the concepts of observables and piping through, right?

21:16Where you need to start something, you need to update on an interval, you have constant actions, and then at some point you might want to dispose of it. Or start it back over again, like repeated or whatever. Yeah. Now, easier to read, that's in the eye of the beholder as with all code. But so RCS does have a bad name to some folks. And the reason is, honestly, async programming is really hard. Like there's no way around it. Like you're taking like synchronous programming with iterables or anything else you're doing, if this, then this, if this, then this, and you can read it top to bottom, left to right.

21:47No problem. Asynchronous programming, even with for a wait, you go down, you get to an await statement and it goes off into the universe. Like it's not on your page of code anymore. It's, it could be doing literally anything at that point. You could be awaiting a message coming from Saturn. Like there's who knows what that's doing and it could fail or not fail. So every single bit of asynchronous code, it goes off to some Schrodinger's cat problem and then comes back and then you continue on your way. And you can even do something where you do asynchronous code, but it makes a call out. And since you've left that synchronous code block, you can become reentrant where that call out actually comes back into the top of the same function.

22:27So if you've got any shared state, you've got an issue. And that's not a problem that's unique to RxJS. However, RxJS makes asynchronous work so easy that people create those problems, I think, easier because they're doing more asynchronous stuff, a smaller amount of space. And then they build more complex asynchronous things. and yeah, it becomes difficult. And then the other thing is, if you don't understand what a flat map is, you're gonna have a hard time. It's something, I hear fewer complaints about people who don't do or don't understand the operators anymore because it's been around so long.

23:06And I think a lot of people have exposure to it now. But that's not to say that I haven't seen some absolutely abhorrent code where people were using RxJS for things that they should not be using it for. You know, the completely synchronous things. They'll be like, yeah, let's just wrap this in observables. And you're like, no, dude, this is just, you're going over arrays. Just do arrays stuff. You'll be fine. You've made one request to one HTTP endpoint. It's okay if you don't cancel it when you're no longer using it. It'll just come back. And yeah, it'll eat some resources for a second, but it's probably not the end of the world.

23:39There's a give and take there. How do I know whether the app I'm working on is a suitable candidate for updating to RxJS or upgrading? So observables, and let's just talk about observable itself and not all the operators, just observables. Observables are good candidates for anything where you get more than one value or maybe no values, synchronous or asynchronous, it doesn't matter. So they're a good candidate for things where you could get no values, you could get multiple values, asynchronous usually, ideally. And then the other thing they're good for is cancellation. So promises have no cancellation.

24:11If you have a promise in flight, it must resolve or reject. You can't just be like, oh, this is done now. And if you have it resolved with null, so it's not annoying, then you've got to handle that null every single place that you could be using that promise. Therefore, you have to reject it. But now you have this error that you have to handle and you have to look and be like, is this an abort error or is it like an actual error? And then you have to handle it appropriately. So promises, they're not truly cancelable without a lot of annoying steps. and observables don't have that problem. Observables are more like, this is a push space type.

24:50You cancel it. It's done. Like it just silence. It tears everything down, finalizes itself or whatever it needs to do and carries on with life. So anything where you need some sort of cancellation, observables are really, really good for that. And of course, you know, things with zero to end values. Now, then you get into the operators and stuff. And if you're doing a lot of coordination of events, like say you've got two streams of data coming in and you have to make sure that they line up. Observables are really, really good at that because we have these operators where they can like RxJS has zip.