TypeScript with Jake Bailey

0:00TypeScript is a statically typed superset of JavaScript that adds optional type annotations and modern language features to improve developer productivity and code safety. The TypeScript compiler performs type checking at compile time, catching errors before code is run, and also transforms TypeScript code into clean, standards-compliant JavaScript. Jake Bailey is a senior software engineer at Microsoft, where he works on TypeScript, and has made major contributions to the TypeScript compiler. Jake joins the podcast with Josh Goldberg to talk about TypeScript and his work. This episode is hosted by Josh Goldberg, an independent full-time open-source developer.

0:40Josh works on projects in the TypeScript ecosystem, most notably TypeScript ES Slint, the tooling that enables ES Slint and Prettier to run on TypeScript code. Josh is also the author of the O 'Reilly Learning TypeScript book, a Microsoft MVP for developer technologies, and a live code streamer on Twitch. Find Josh on Blue Sky, Mastodon, Twitter, Twitch, YouTube, and.com as Joshua K. Goldberg.

1:19Jake Bailey, welcome to Software Engineering Daily. Hello. How's it going? It's good. I'm so excited to talk to you, Jake. We've done so much work and you've done so much work on TypeScript and its tooling. Don't be bashful. What is your blue sky description? Your profile say? I think my profile says, according to GitBlam, I wrote the TypeScript compiler, which is not true, but it is true if you don't bother to ignore all the migrations I made. So, yes. Then I guess if you look at the new repo, then it'll probably also say something similar, but... There we go. Less so. The one and only author of TypeScript.

1:56No, no, no, no. Absolutely not. I'm new, okay? Well, we're going to get into that. But before we go through all the tooling stuff you've done with TypeScript and the work on TypeScript in Go, which is incredibly exciting, let's start with you. How did you get into coding, Jake? Ooh, a long time ago, I was a child in elementary school and I had a TI-73. And by some coincidence, the teacher assigned an assignment, which was basically, I want you to tell the class about a button on the calculator. And I'm pretty sure I got assigned the program button. And so I had TI-Basic. And so my first Spriter language was TI Basic in like, I don't know, the third grade, fourth grade.

2:44I don't know. One of those. But a lot of time, obviously. And then I just it's one of those things where it's like I kind of think I knew what I was going to do from the beginning. And I think like in middle school, I'm like, I'm going to check out the book on C. And I just learned C. And then I was like, man, I'd really like to work on RuneScape private servers. So that I'm just like modifying Java and like trying to convince my friends to install Hamachi so they can play on my private server and never succeeding. And then, you know, bubbles from there all the way up. Right. So, yeah. And then I went to Illinois for computer science.

3:20Did that. It was great. And lots of work otherwise. And I've been at Microsoft for, oh, my gosh, what, six years now? Something like that. I don't remember how old I am most days. So remembering when I did something is always a challenge. Do you remember what it was that drew you to programming in the first place? I have no idea, honestly. I had just always been the basement computer guy, I think. Like quite literally, that's where the computer that I had was located. So I just spent all the end of my days sitting, futzing around on the computer doing stuff. So when you went into Microsoft, it wasn't originally for TypeScript, was it?

3:54No, actually not at all. In fact, it's funny. I'd worked there the year before as an intern doing something completely different. Everything I did beforehand was like SRE work. I'm not sure if you're familiar with what an SRE is. Let's say I'm not. It's like a site reliability engineer. So it's kind of like people that work on infrastructure and like support other teams and do all that kind of stuff. And I did that at like, you know, basically three different companies at that point. But then I was at Microsoft and then I was like, you know, I'd like to try to interview to see if I want to move over to DevDiv to do some programming language stuff.

4:26Because at that point, I was in the middle of my grad degree and I was taking so many programming language classes. And I was like, this is awesome. And I'm like, let me go see if they'll hire me. And then I'm going to tell a longer story, the origin of this. But like they interviewed me and it was terrible because I got put through all the rounds. And it was such a long day. I was so tired at the end of the day. And then I found out afterwards that because I did SRE work, my title was service engineer, not software engineer. So they thought I had no programming experience. And so they put me through extra steps that I didn't need to do.

5:02And then they apologized afterwards. And they're like, oh, we're sorry. We didn't realize you were in a software engineering role. And I'm like, yeah, yes, I am. What the heck? So then I got hired and I was actually going to be on the C++ team working on the C++ compiler. And the last minute I get an email and says, hey, you're going to work on Python. And so I'm like, okay. And so I spent the first couple of years working on Python. So working on Python language server, editor stuff, later on PyLance, PyWrite. So PyWrite is a type checker for Python. And I worked on that. And then eventually I switched over to TypeScript, trying something new.

5:37And I've been on that for the past three years, something like that. I don't know. I don't have a calendar in front of me to tell me what I did. But I think maybe three years now. One of the nice things about open source is that it's all in the public. After this, we can go and see exactly when you started. Yeah, yeah, yeah, absolutely. Absolutely. Do you remember what you were doing when you first joined the TypeScript team? Ryan, our engineering manager, he assigned me a bunch of bugs. This is pretty common when people join the team is that he'll assign them a bunch of bugs that are sort of like, get you started.

6:07And I had joined and Pyrite, if you're not aware, Pyrite is a typewriter for Python. And Pyrite was inspired by the design of TypeScript. It improves on it in some ways, It doesn't improve it on some ways as well. And so I was actually felt really familiar when I stepped over the line right to the other side and I started working on stuff. And so I think my first bug fixes were like fixing something with omit, with like the helper type to do with like private properties and protected properties. That was one of the things. And then working on parser bugs. So I think one of the first parser bugs that took me like three PR iterations was the parsing of arrow functions mixed with conditional expressions mixed with type annotations where if you think about it it's like arrows but then there's also colons but there's also type annotations and it's like really ambiguous like what you're supposed to do with these and i that took a lot of time and it's been the same ever since i've modified it like three years ago and so i'm assuming it's correct now but i always get anxious when someone reports something or i saw i think i saw nicolo from babble post something about these conditional expressions i'm like oh oh my God, please don't make me fix this another time.

7:19Because I was such a pain the last time to get it correct. Just like you take one TypeScript expression, you shove it in a JavaScript, and it does something completely different than what you expect. You're like, ah, right, the languages are different, right? And so that was one of the first things that I fixed. And then soon after I worked on the whole module migration stuff, that was like pretty soon after I started on the team, actually. Like a couple of months in is when I said, hey, Ryan, can I work on that? And I took over the work and we switched the whole code base to modules, which was the first time that I modified every line in the code base, right?

7:54Let's pause there, actually. Not all of our listeners are deeply familiar, as you and I unfortunately are, with CommonJS versus ESM. When you say convert TypeTube to modules, first of all, what does that even mean? What's the context here? So back in the days, in the past, there was not ES modules. There was no specific syntax blessed by the committee that was like, here's what's in the language and this was you parse. And so there are lots of competing formats for modules out there. I'll put out some acronyms, AMD, system.js, blah, blah, blah. And the one that Node used was something called the common.js.

8:30I say did use, it still uses, called CommonJS, where you basically build your module, the file up, but you put all your exports onto one object, and that's what you export and what everyone else consumes, right? And that's CGS. CommonJS is like import, export the syntax written out. It's different, works differently, has different considerations. And it turns out that TypeScript used none of these. TypeScript predates ESM for sure. it predates many things. And so TypeScript was written in terms of namespaces, which, by the way, were never called namespaces at the start. They were called modules.

9:09And in fact, they were called internal modules. Internal modules meaning that you wrote your code inside of modules, which are namespace-y, kind of like C Sharp or something. And you wrote your code in those. And then the TypeScript compiler would just take all those things and jam them together into one file. And that was the output. It was like that for like a decade, something like that, like eight years or something. But what that meant was that we were stuck on old syntax. Things turned out to be a lot slower because of it. And we had no idea actually. And we wanted to change over to modules, which required a big changeover.

9:44We had to like change everything that was implicit into an explicit import, take all the code unindent it. We had to rewrite a whole bunch of stuff, make it work, then add a bundler because now we aren't using TypeScript to put everything together again, stuff like that. So that was basically the big transform that I had done in that step, which was multi-month process to finally one day run the script that does it because it was fully automated and just rip through it and then send a line, a PR that's like 3 million lines of code or something. Right. And it worked out. So I actually want to ask you a question on this.

10:16in the general context and framing of migrations of code or doing code mods and changes? Because there are a lot of teams out there who have changes like this, where their code base was written in year X with module or some style Y, and then X plus five or 10 or even 20 years later, Y is outdated and now they want to move it to Z. What are the kind of strategies or mental framings that you use when considering how to or whether to migrate a code base in that way? So step zero, the thing to consider the best is just, let's speak JavaScript. JavaScript is changing so much all the time. It's like, that's the thing, right?

10:52And so the best way to solve this is to be preemptive and like, unfortunately, keep up with it and like try to make modifications in time as fast as you can and keep up to date so that you don't have to do a big migration. But that's obviously not possible from everyone, right? You need to have people dedicated to doing that. And so I think that the most important thing is to try and write tooling to do it, because if you don't, you're going to have a really hard time. And so the migration that I did, that was entirely read using TS Morph, which is a project by David Sherratt that wraps around TypeScript and allows you to modify code and turn it from one step into another.

11:29And so I basically was able to automate every single step of the entire process. And then if TypeScript changes, I would rebase the code base on top of the old one. Right. And so the automation is really the reason that it was possible. The only way is to do brute force, which that works for me sometimes, but not when I had to redo the whole thing over and over and over again, trying to keep up with it. There's tons of interesting stuff inside that whole process, but I'm not sure you want the little tiny details of that whole process. There's a whole talk about that somewhere. I think there's a lot of utility and understanding, maybe the high level concepts.

12:02You mentioned TS morph, and because you work on TypeScript, you're well-equipped to answer. What does it mean to morph TS? Or I guess in this context, what is an AST? AST is abstract syntax tree. Every language has this. It's like, here's your code, it's parsed out, and it turns into a tree of like, okay, the file is the top node. And then all the different statements inside the file are the next nodes down the list. And then you set up those, you say, ah, well, this is an if statement. And then you walk down and you have more stuff inside of those. It's like, oh, here's the condition. Here's the body of the if statement.

12:35Here's a call to another function. Here's the name that you're calling. Here's the parameters for that call. Oh, those parameters, those are also calls. It's a big tree that goes downward. And so when you're trying to write code mods, TS, morph, all these different bits, transformations, everyone has a different name for the same thing. You are taking the AST and you're just turning it into something else. You're doing like a rewrite from one thing into a different thing and making the code work the way you want it to work. And this is such a powerful technique for application migrations, right?

13:04If you have whatever, thousands, hundreds of thousands of lines or files, there's no way you're going to be able to brute force a large migration such as CJS to ESM. This seems like the only real way to do it oftentimes. Depends how much resolve you have. But yes, I would say automation. Sometimes, you know, you spend so much time doing the automation, it's not worth it. But usually not. So you accomplished one of the very fun migrations of TypeScript from CJS to ESM. What were the benefits or what were the results of that? Well, the headlining thing was that the whole thing got 30 to 40 % faster.

13:37That comes down to really weird details to do with JavaScript. In short, because of those namespaces, everything was inside of an object. So if one file wanted to talk to another file's contents, it was actually a property access. It was doing a full object access to do those things. And so the entire time, even though everything looked like it was local, you didn't wrote those out by hand. You actually were spending 30 % of the time just doing that. And so we ran the transformation. And then I ran the code. I'm like, whoa, what the heck? This thing is way faster. That doesn't make any sense. Oh, wait, no.

14:11Actually, this makes a lot of sense. Crap. And so once we turned it into modules now, yes, they're separate files. They import each other. But a thing like ESBuild or Name Your Bundler can understand that. So when they put that into a single file, it rewrites all the references to reference things locally. And then the engine is just like, well, that's just a static lookup. That's right here in scope. And so that's really fast, right? So we gained a whole bunch of performance that way. So that's performance. That's what matters, I think, to the end user, right? But for us, it was kind of like, well, instead of running a full 30-second compile or something just to get an output JavaScript file, we could run an ES build and that took like 100 milliseconds right so now anytime we start our test it was like instant versus waiting you know 20 to 30 seconds to actually get it to start that and other tooling that we were able to use that we weren't able to use before because no one used namespaces except for us basically no one no one used the weird features that allowed tsc to merge the files together like tsc secretly was a bundler the whole time and no one actually use it except for us.

15:13And so we dropped that feature straight out of like 5.0 or something because we're like, yeah, no, this is an anti-goal for us. We only preserved it for our own use. And now we absolutely do not. We need it, right? This portends things to come in two ways. One, the native code speedup and also the improvements to TypeScript internally enabling end user benefits such as performance, such as dropping deprecated features. But I want to switch topics ever so slightly for a bit before we go back to TypeScript, because TypeScript isn't the only large repo managed by the TypeScript team. What is definitely typed?

15:45Definitely typed is a collection of type definitions for libraries that don't have them. So if you npm install, I don't know, React, great package, right? It doesn't include any type definitions at all. TypeScript cannot read that and go, oh yeah, there's an export called use state, right? It doesn't have that ability. It can't analyze into the JavaScript code, look past their bundling and midification to figure that out. And if it did, it wouldn't really be very useful. If React were written in TypeScript, they would emit DTS files and they'd have it, but they're not written in TypeScript, right?

16:21And so definitely typed contains like some 8 ,000 packages worth of declarations that say, hey, this package exists. It has these exports. They have these types. And this is how you can use them, right? So when you install at type slash react, suddenly you have all the types that make React work. And so you can say import React. and now you are off to the races, right? You can see everything in there. And how is this set up so that the packages are defined and then published? Well, since its inception, which is, again, also 10 years ago, it was like thousands of individual folders with TS configs and basically a script that would scan them and see if anything changed and then package them up and publish them off to NPM.

17:09I'm assuming what you're getting at here is the new form that I introduced maybe a year or two ago, which is maybe more familiar to people that have been worked around in JavaScript. And so instead of having a whole bunch of loose things with like a couple of package JSON files and then like really weird TS config options to sort of map them together, which then broke things in other weird, subtle ways. I made an effort to convert the entire thing to a PNPM monorepo. And so now all the packages link to each other as though they're actual packages, right? And so you PMPM install and it takes a while in the current state, but you'll link 8 ,000 packages together or whatever subset you want.

17:49And so now they're all linking to each other, declaring dependencies properly. We used to scan the files and figure out like, well, you mentioned Node, so we're going to put Node as a dependency of you, even though that maybe wasn't required at all. And so the new form is basically a bunch of packages. And then we take out the files that we want to publish, declaration files, put everything else behind, generate a package and publish it up to NPM whenever it changes. Let's pause there a little bit. I'd like to go through a similar exercise as the migration before. Let's say that I'm on a team that has a large existing app that's not set up as a strict modern monorepo.

18:22And let's say that I really did want to migrate it over. What are the kind of techniques or tips you would give me in trying to migrate my existing app to a monorepo? I think it really depends. The main thing is that it depends on your repo layout. There's a lot of people that have what is effectively a monorepo, but it's not broken apart strictly. And so they have imports that are like relative paths that are really long. And they're actually supposed to split them out. You can want to split them out into different packages, right? Right. And so if you have the structure of different folders, you can create package JSON files in there.

18:54And then most of the package manager out there are totally fine figuring out where to find those things and map them together. I'll use PMPM as an example since that's what DT does. You just make package JSON files and you put them in your glob and then it finds them and puts them together. And the real challenge is just breaking apart things so that the imports are no longer relative to somewhere else. You're actually working in a world where all their packages are kind of like they exist on npm and they're already there and so you want to refer to everything by their public names because that's what people also let people are going to you know import them as or if it's an internal app it doesn't matter quite as much but now you can split up your packages into multiple pieces build only certain parts of them and that's like a whole different monorepo setup configuration thing with typescripts or something right but splitting them up and let you declare the dependencies and you can better see what's and used by different parts of your application.

19:44It's mainly the big transform is just the import stuff. That's the hardest bit, right? If you get them in the right folders as is, you can pretty much figure it out. And for TypeScript, that's a different thing where like we can pretty much figure it out no matter what you're doing, as long as you have the configuration file set up. So we're happy to do that. But there's like a whole bunch of stuff talked about there in terms of like project references and bits and bobs, but not sure what. Can we actually talk about that? Could you give us a primer what are TypeScript project references since you bring it up?

20:13Sure. So in TypeScript, everything is in a project. So we'll start off with that. You make a TS config, you say, hey, here's the files in this project. When you call TSC with no arguments, it automatically finds TS config.json, but you can pass a flag that says, here's the project. And so it used to be that you'd have TS configs, you'd build them. If you split your program apart, you could do that. but it sort of doesn't scale really well for like, I want to modify one part of my repo and then not rebuild all the other ones at the same time. And so a while ago, project references were added to TypeScript where you can split your application up into different blocks that define like, okay, here is the front end, which depends on the common bit or like, here's my application.

21:06It might depend on the component library, but also the API types. And also those can be different bits and pieces across where when I rebuild my actual front end, I don't need to rebuild all the stuff to do with the API. I mean, that's just always there, right? Or maybe I don't need to rebuild the components if I'm just really building the main front end or something. And so references allow you to split up your code into multiple pieces that declare different sets of files. And then you can say TSC-B, so build mode, and then it will remember, oh, well, this DSConfig doesn't need to be updated.

21:39It already built it and it will just skip it. And so that's a speedup in one way, but also it allows TypeScript to figure out like, oh, when you go to def on something, how do I map that to the actual source file and not declaration output files or something? It's all about like granularity and sort of editor use makes things better. Although there's always a limit where like some people will try to make like 2000 project references and then you have a real problem because it doesn't scale that well that way direction. but they're pretty good for breaking apart your code base into different bits and pieces.

22:09Yeah, when people think of TypeScript performance, very often they think of the type system because it's incredibly rich. Someone got Doom running in it. Congratulations again to Dimitri from Michigan. But really, another aspect of TypeScript performance is splitting things up and setting up your project references so that it only has to rebuild some of the application parts that have changed when you save. I'm curious, do you find in production that when people come to you with a team with performance issues, it tends to be, let's say, more in the type system or more with the project configuration side of things?

22:38Honestly, it's mostly configuration. There are people out there who have real type performance problems, whether or not they are just doing something wrong or if they're intentionally trying to do something that is really bad for performance. Lots of really crazy type libraries out there that try to parse everything out of a string or something like that, which don't work very well in terms of performance. sometimes. But most of the time, it's kind of like, oh, whoops, I accidentally didn't configure the types property of my tsconfig. And therefore, it tried to load 1000 at types packages, because unfortunately, the default behavior of TypeScript is to load all those by default.

23:17And like, oops, that fixed like 90 % of our problem. That's a real example that happened. I'm talking to a team about that. There's other places where it's like, we have one project, and it's massive and we just compile with once. Oh, you should probably try using references and see if that helps because it makes things incremental. There's other teams that they have gone so far with references, they have 2 ,000 of them. And so now the fixed cost of having a reference is the dominator on the amount of time you're taking. And so it's like, maybe have fewer projects. Do you really need one TS config per React component?

23:53No, you don't need to do that hard. It's usually stuff like that or they'll misconfigure stuff with like paths or they'll have a big paths array with like path remappings. That's like a hundred long lines long. And then we're like, oh man, we never thought anyone would ever have a hundred of these things. And so the code base is just this linear loop that loops over everything, trying to find them for every single file. Right. And so it's like, oops, did I make a little N squared or whatever? Then, you know, whatever terrible algorithm, oops, that wasn't intentional. Right. Let's like push it on us, but also like, well, maybe don't use paths or don't use the remappings or something.

24:27Like there's other better features to that same thing. A lot of the cases are just like the TS configs were completely different between all of our projects and therefore types couldn't reuse ASTs. Whoops. Didn't mean to do that. Like that's not good. Okay. We shouldn't even them out. Use a, you know, a shared base config or something between some of our projects to help with that. Other bits, like there's lots of, there's lots of stuff. Like this isn't a big contributor, but people will set their target to like ES5. Okay, so you're just spending a whole bunch of your mid-time transforming your code into term of iterators or something.