Free-threaded Python (Interview)

27:14So if you have slower single-threaded performance, you have loss of features in terms of it being more complexity, potentially, because your loss of a single lock, Now you need fine-grained locks and all kinds of other things you can do, which is problematic. And a huge lift by many developers over how long you guys been working on this? This has been many years in the making. Many years in the making. Yes. Okay, add all those together. Worth it? Well, we don't know. We'll see. We'll see. We cannot answer that. We cannot answer that. We don't know. So we're in the stage of, so actually this is an important thing, by the way, which I don't think we have mentioned.

27:57in 3.13, indeed, there is a build of Python that will not have the global interpreted log, but you don't get it by default. You need to manually compile Python. I mean, there's people that are redistributing Python with this, so probably you don't need to manually do it. You can fetch it from your distribution probably. Somebody will build it. Yes, exactly. But it's not the normal one. If you install Python 3.13, it has the guild. It's not like a flag that you give when you run Python. It's a separate executable. Yes, you need to, it's actually called Python 3.13 T. T. Or free threading. This is whatever.

28:27It's just the way it is. Sure. It's free-threaded. Yeah, exactly. So you need to run that. And with that build, you can actually select a runtime only with that one. You can select a runtime if you want the build or not. And this is very useful in case you want to try the performance difference or because you want to, like imagine, for instance, you are doing one of these libraries and you want to test if your library works with and without, but you don't want to run your CI with two builds or whatever. So you can do it. There are consequences of this because, for instance, In Python over the years, we have added a lot of new optimizations.

29:00Like, for instance, one of the things that I think Lucas was missing on the reason Python is low is that apart from this old new direction, and it's actually part of the reason, but is that it's dynamic, right? So this is very important. If you are adding two variables, X and Y, the interpreter is going to say, what is X? Oh, X is an integer. Okay, let me fetch the code for adding integers. Okay, what is Y? Oh, it's an integer. Okay, let me fetch the code. Okay, and then the next iteration of the loop is going to say the same thing. What is X? So it's very annoying compared to just It's like, you know, here's a bunch of bytes, and I'm going to add this to this bunch of bytes because I know what they are, right?

29:31So in this particular case, precisely to avoid this dynamism, we have added a lot of optimizations that try to find this pattern. So they say, okay, you're adding these two variables together, but, you know, the last 3 ,000 times I have seen this addition, they're where in there. So, you know, it's kind of a good chance that they are integers the next time. So I'm going to create this, like, super performance code that is going to just reach into the internals of the integer object in Python. and it's going to grab the real integer and it's going to add it at C speed. And it's like, you know, it's going to create a result without asking all these stupid questions and then it's going to box it again into a Python object.

30:05And it's much faster. You know, you can still have a float suddenly and it will de-optimize and whatnot, but like that is good problem. All those optimizations don't work without the kill because they are not thread safe. So if you use the single threaded version of the free threaded version, which is a bit of a mouthful, so you use Python 13T, but you don't have the GIL activated, you also don't have the optimizations activated because we don't know if they're safe or not. So, you know, what does it mean? It means that the free-threaded version is slower in single-threaded. So you only run one thread and you're using the Python 3.13 T version is slower because they don't have those optimizations activated.

30:46Slower for now or slower for good? Right. For now. Great question. So let's maybe take a step back and explain the kind of strategy that we took with this particular project, which is a huge undertaking. But at the same time, we have a release cadence that releases Pythons every year. Predictably in October, you're going to get a new version of Python. So there's no way you can show a big transition like this within 12 months. So what we're doing instead, essentially what you're getting is a state of the project as we are in right now. So there are things that we still need to do that are still not done.

31:23Most importantly, this split for the main interpreter still having the global interpreter lock and so on, and an experimental one that you have to build yourself and then you can run without a global interpreter lock, that is deliberate because with all those costs that we described, we really need to reach out to the community saying like, look, this is now not an idea. you can actually grab this Python and check whether this helps you. Is it better? Is it worthwhile, the scalability for this extra complexity within the implementation and maybe this cognitive complexity that is going to be sort of passed on to the user?

32:03Is that worth it? So the community will have to now step in and try out this new Python and see, okay, this is slower than the non-sorted version, but that's hopefully temporary. but the question to answer is, does the scalability that this enables help, right? So we really need them to give us two thumbs up, like, okay, keep actually working on this. Since this specialization that Pablo described and the JIT compilation that follows from that and so on and so on are obviously features that we really want to have, including in the free-threaded version. And there are ways to go around it, but they are very complex.

32:48It is, in fact, like on the bleeding edge of computer science as we know it. There are plenty of known strategies for improving performance of interpreted languages that are single threaded. For free threaded interpreted languages, this is really where the science sort of is still unclear, right? Because JavaScript, there has been so much investment in that, like making that fast with V8, right? Like in Chrome and so on and so on. Well, that is single-threaded for the intents and purposes of our understanding of the interpreter. With a free-threaded Python, how to make that fast is more of a challenge.

33:30Literally, members of our team are going to be the people writing papers about this later. So this is going to take time. longer than one release, longer than two releases, but at the same time we want to release what we already have so that it's no longer just a pipe dream. It's no longer just a plan or idea. Users can actually experiment with this and tell us, okay, this is actually awesome. This enables a use case we couldn't even try before because that's our chicken and egg problem. Before we actually enable this, we don't see use cases that are now impossible. So we cannot tell whether there's an untapped area of Python usage that would be now enabled with free threading.

34:14So this is what we're doing with 3.13. This is essentially this current state of the project. Now you can experiment whether you can see whether your web framework is going to be faster now, whether now you're going to be able to process data in parallel, and so on and so on. This is essentially one step of a longer project. Right. And it's out there in Release Canada. It's probably out there as we speak, at least in the official 3.13 experimental, as you guys pointed out. Are there early indicators? Are there people who have been just pining for this and they've already tried it in the RC and are saying, wow, we're seeing this, or no?

34:53Actually, that's a very good point. So one of the things that actually surprised me a lot, because we didn't know, one of the unknowns here was how the community was going to adopt this. The reason being that it's just not that Python needs to be multi-threaded. Now if you compile C extensions, which are really, really popular in Python, that's one of the reasons so many people use Python in the language because all the machine learning doesn't happen in pure Python. It happens on compiled code and QAn and GPUs, but Python is the driver or the glue. So all that code needs to be changed or adapted at least because like, you know, now you don't have the lock protecting you and like, who knows what happens now, right?

35:32So you need to, which is really challenging because right now you need to go through the entire code base, like the huge thing and sanitize the code base through this particular problem, which, and there is not a manual for that if you think about it. Like, so you need to reason about the entire thing and we were a bit scared about like how people will approach this because, or like what will happen in the sense that, you know, If you don't do this or you don't do this correctly, is this going to crash all the time? Or is it going to crash just once every blue moon or whatever? So we don't know.

36:02So one of the things that we have seen, and it's really impressive, is that the community has jumped this incredible. People are building two versions of every of these extensions, one for the normal build, one for the free thread of the build, which is already huge. This doubles your CI time, doubles your storage for the extensions. So it's a lot of work, actually. and there's a lot of people that are so excited to jump to these extensions and they try to fix it and try to try these little locks all around. So all those people have been trying it and one of the things I have been super, super surprised here is that once you add those extensions and you have a final application, like a machine learning, whatever it is, or an LLM or whatever you are thinking, that uses a lot of these things.

36:49It's just not one, like five, six, seven, really complex one plus the interpreters. plus maybe a web server, something like that. And then you run it through the free-threaded version, it just works. I mean, again, who knows? Maybe every Saturday you need your entire SRE team looking at it. But right now, it's not mega crashing. Because what we thought is that this was going to immediately crash when you add enough complexity to this. Because it's just a statistics thing. The chances that there is some bad interaction when you put out the threads and the amount of code that is here is inevitable.

37:22and still from time to time we find very funky in an interpreter when like you know you call a method and sometimes like another method is called because there was something like a memory reordering problem which is a very technical deep thing and debugging that particular thing is just halved but like that is very rare and obviously we fix it immediately and we are really surprised about first how many people have jumped at this and they're super excited and they want all of those extensions and working and second how quote unquote stable this is meaning that how little bugs we have seen in the wild.

37:56We expected many, many, many more, right? And we are seeing very little. So right now it's super promising. Yeah, what does help is that we do have the people behind it, like some girls who actually did the prototype of 3.9 and then 3.12, now who is working on making this for 3.13 and forwards, is being helped by other engineers who are really into this and are able to find thread safety issues and data races and so on, we were able to set up thread sanitizer, which is very complex to set up in an interpreter environment like Python. Turns out it does help us with finding those issues. Some of them are pretty funny, like figuring out that for this entire time, the thread handle in Python was not thread safe itself, or the threading lock was not thread safe itself.

38:50So once you actually did the operations of locking, yes, those were thread safe. But the Python object that you used was not because it didn't have to be. And now all of those things have to be actually fixed. So they are probably formally correct now. So yeah, there's been a lot of work and a lot of work is still to be done. But our hearts are warmed by how little, you know, terrible and unsolvable issues we've seen so far. Some of them are like, you know, complex to debug and complex to diagnose, but we've been successful in, you know, improving the situation so far as we go. So for your average programmer out there with their average Python app, whether it be a web app or a desktop app or whatever, should they be hopping on this experimental and giving it a try?

39:43or is it a thing where it's like, if you don't know that your code is thread safe, it's probably not? Or is it the other way around? Like, what's the water feel like? Is it warm? Is it, are there alligators in there? Well, it depends, right? Like, if they are paid to make sure that the thing is stable, the recommendation is no. Like, because like, I mean, try it out, sure. You can get very excited, by the way. Like, one of the things that we have seen, all the people, me, myself, right? Like, for instance, the company I work, Bloomberg, is that we have tried, like, to see what will happen. even if it crashes every three months, whatever it is, but just to see a sense of what is the improvement here because if then down the line you're going to justify any effort on your site to adapt your thing to the free-threader version, it's really good that you know what you're going to gain.

40:28So that for sure. Trying, we really want people to try and tell us what they see and what doesn't work. Is this a production ready workload? Absolutely not. Not only because we know it's going to crash or anything, but because we don't know the real status of the thing, right? Like, we take, you know, stability very, very seriously in the core team. So we are not comfortable right now. That's why the thing is called experimental because we are not comfortable right now like recommending people, you know, like especially when you don't only have the interpreter but all these other things, right?

41:01Like, and, you know, still we need to figure out like good APIs for those extensions to, you know, hook into interpreter and things like that. That is not there right now. So the chances that we can look at these things and say, yeah, this can run in production, no problem, it's almost zero right now. But as we release Python 3.14 and Python 15, we will move this experimental into supported, which means that you can run it in production, but there still will be two versions. And eventually, when we gain enough confidence that this is the future we all wanted, it will be only one Python, which will be Python without the free-threaded version.

41:38But until we call it stable and supported, that's the keyword, supported, I wouldn't recommend people to actually use it in production environments. Yeah, production environments. However, I would be a little more optimistic about this, saying that if you are an end user, probably the amusement park is still being built for you. So this is not something that is ready for end consumption. Unless you are really just trying to just see how pure Python works without all the libraries that you depend on. However, if you are a maintainer of a library that a lot of people depend on, you might be interested in checking it right now.

42:20So we already see plenty of interest from the data science libraries and web frameworks and so on and so on. And that's exactly the right moment for them to identify whether there's any blocking issues or whether the gains are not enough for us to justify the change or maybe the opposite or maybe they're amazed at how better things are right now. And like funnily enough, even things that were not ever designed with free threading in mind, like AsyncIO, like you run the test suite of AsyncIO and it's just faster on free threading because it just can magically utilize the fact that it is now truly parallel.

43:01So there is promise there. Definitely, we expect that once the library authors actually go through some of those hoops that are required for their things to be supported by the free-threaded version, we're going to see uptake on more libraries and then more users and so on. So now's the time for the library maintainers to check it out. But for end users, it is a little soon. That makes a lot of sense. Somebody should start a list of libraries which have taken that initiative. There is a list like that right now. There is a website. I forget the name right now. We might add this to the metadata of this episode so that you can check this out.

43:41Just as in the Python 2 to 3 transition, we had a list like this. There is a list. It is very fresh, so I forget the website name right now. But it's there. Cool. Very cool. Well, it's definitely exciting times. I'm even excited. I don't use Python on the regular. but I think that's early indicators of something that's potentially a huge success. So congrats to you guys and the whole team for the effort at least to make it no longer theoretical and make it an actual piece of software, bleeding edge piece of software that's going to produce even some papers. Should we move on to the JIT? Lucas you mentioned the JIT is another feature, 3.13 or is there more to say about the GIL?

44:22I'm assuming these things will probably interact as topics. Right, so the JIT is still like one of those other topics that I remember being this big, you know, pipe dream of CPython for as long as I can remember, right? We grew a few attempts to have a JIT in forks of Python or entirely new Python interpreters. Most of those efforts failed. There's one holdout that is actually doing very well, that's PyPy, and that is actually JITed, which proves that this language can successfully use this, I don't know, performance strategy for good results. As a clarification also for the people that don't know, ByPy is not a JIT for CPython, it's an entirely different interpreter.

45:10Yes. So it's like the same language, but it's just a different interpreter written actually in Python, which is JIT then, but it's a separate thing. Yes, it is an entirely separate technological stack. it is attempting to be bug-to-bug compatible with CPython. You know, people adopting it have seen, you know, your mileage may vary, sort of, you know, results where it might be really a drop-in replacement in some cases and might be more tricky to actually adopt it. Otherwise, plus the approach of JITing they're using has some characteristics with memory usage being larger and so on. So it depends on your use case.

45:50but obviously the most popular version of Python is CPython is the version that we are working on so this is an idea that just comes back and comes back and comes back like when will Python have a just-in-time compiler so to like a large approximation if you squint really hard this specialization that Pablo already mentioned is sort of slowly becoming a form of what the JITs are doing which is to remove all this indirection But the last step that they did not take so far, those optimizations, is to actually replace running this C code with this C code with an actual compilation to machine code as we go dynamically, which is what the JIT is doing.

46:33And Brandt Booker, one of our core team members, found out this paper about how you can automatically generate a JIT for the Lua interpreter and how it is slower than a handwritten JIT, but not by that much as you would expect. and it is much more maintainable. And using this approach with this additional machinery that is needed for the just-time compiler that tries to compile during our build time, like the simple opcodes that are later glued together to this optimized machine code and so on, and to discover whether that trace that we're running is still correct or maybe we need to throw it away because we thought this thing that we're running is like adding to integers, right?

47:25But now we're seeing, oh, there's a float that we did not expect. So we need to throw out this optimized code and actually go back to the interpreter again. So all this infrastructure is already in 3.13. And so far, we can proudly say that with all this added, Python is not slower. Right, 0 % faster. Yes, it is 0 % faster because, again, with the release cadence that we're having, 12 months of work is a lot, right, obviously, but it is not enough to just say we started from zero JIT and ended up with a mature JIT like V8, we are done. It is just simply impossible, especially that our team is much smaller than the V8 teams, much smaller than the Java hotspot team, and so on and so on.

48:11So yeah, so so far, there is another experimental feature in Python 3.13, which is the JIT. you have to compile with that, and you have to enable it when you're running Python. So it does not have most of the optimizations that you would expect from the just-in-time compiler, because that entire technology of just-in-time compilation is essentially a framework on top of which you can now make optimizations that make things faster. For example, if within the trace that you're building, you know, connecting all those machine code blocks that are now faster and removing indirection, you have multiple checks whether something is none or something is an integer.

48:54You can move that check of that same piece of data to the front of the trace and just do that check once and you don't have to repeat it in different parts of the code and so on. and to make sure that the traces are as long as possible. Those are different levels of optimization. You want to JIT relatively quickly so that people can see the uplift very quickly, but you don't want to JIT every code that is being executed because that would make the memory usage crazy. So there's things that you need to dial in there and implement. So it is very much an in-progress project. It looks very promising, but so far Python 3.13 is a snapshot of the current stage of that effort as well.

49:40Pablo, anything to add? Well, one of the things that is very exciting with the JIT right now is that the approach itself is quite cool. Like for instance, if you're making a JIT for a dynamic language, one of the biggest challenges is not only adding the infrastructure itself for a particular CPU, but like architecture is that you need to do it for all of them for every single OS because at the end you're writing machine code and machine code depends on your CPU architecture, your platform. So you need one. So the JIT is not the JIT or as we like to call it legit. But you need one for Python, sorry, one for Windows, one for Mac OS, one for Linux, but also one for AMD64, one for AR64, one for ARM v7.

50:24So you can imagine how hard this actually is because you basically are implementing now compiler, right? But this approach that the brand is taking not only has the advantage that it's basically leveraging an existing compiler behind in this particular case, LLVM, but also is doing it at the front. So it doesn't happen at compile time. Like all that assembly code has been generated before and the only thing that we need to do at runtime is just stitch those things together. So we have these templates. So a plan is basically or LLVM is not a runtime dependency. Like it's just a build time dependency.

50:56So you build all this assembly code, and it uses the native compiler for your platform. So if you're in some forsaken platform, like, I don't know, AX or something like that, it will work if you are able to run Clang there. So that's great. But also, which is really cool, and most JITs need to also implement themselves, is that we are able to leverage an optimizing compiler for this particular workload. So not only that assembly code works for every architecture because someone else actually implemented the backend, but also we can leverage all the optimizations that everybody that's using LLVM is using.

51:29It's the same optimizations that Rust is using. Rust is using LLVM these days still. And they are using, I mean, if you program the IR, the intermediate representation correctly, and then you are able to leverage that well, which is not, you know, it's easier to set that down. But the idea is that now you can use this set common of very, like these many years of compiler research just to make your core faster. And you can just leverage that immediately as opposed to having to re-implement all those things and have like SSA ourselves and like, you know, trimming and all that stuff. Like now, you know, you just run Clang, you get some super cool assembly out.

52:03We need to just stitch it together and feel like the symbols and whatnot. But like, it's a much, like you can get results much, much faster than if we need to implement a full-blown JIT for Python. So very exciting. It's like a JIT factory. Yeah, right. Java style. You guys use factories in Python? JIT template factory. What's up, friends? I'm here in the breaks with Dennis Pilarinos, founder and CEO of Unblocked. Check him out at getunblocked.com. It's for all the hows, whys, and WTFs. So Dennis, you know we speak to developers. Who is Unblocked best for? Who needs to use it? Yeah, Unblocked helps large teams with old code bases understand why something has been done in the past.

52:50It helps them understand what happens if they make changes to it. Basically, all the questions that you would typically ask a coworker, you no longer have to interrupt them. You don't have to wait for their response. You don't have to, if you're geographically distributed, you don't have to wait for that response. You don't have to wait for, you know, you don't have to dig through documentation. You don't have to try to find the answer in Confluence and JIRA. What we basically do is give you the answer by you just asking a question. The way that we got to the problem was a consequence of our kind of lived experience.

53:22We're actually going to call the company Bother, which is like, you don't bother me. I don't bother you. Right. Instead of like being tapped on the shoulder or interruptive Slack messages, we could just use Bother and get to get the answers that we wanted. We didn't go with that name because it's a little bit of a negative connotation, but Helping developers get unblocked by answering questions or surfacing data and discussions within the context of their IDE relative to the code that they're looking at is something that thousands of developers love so far. I think our listeners are really familiar with AI tooling, very familiar with code generation, LLMs.

53:57How is unblocked different from what else is out there? A lot of code generation tools help you write the code to solve a problem. We sit upstream of that. Our goal is to help provide the context that you need. If you think about where you spend your time when you're trying to solve a new problem, understanding how that system works, why it was built that way, what are the ramifications of changing something? That's the problem that Unblocked tries to solve for you. We take the data and discussions of all of these, the source code and all those systems to provide that answer for you so that you can get that context and then you can go and write that code.

54:31We have this great example of a company who hires, you know, very competent developers. It took them five days, that developer five days to write 12 lines of code. And his feedback to us was it's not that it takes you five days to write 12 lines of code. It took them five days to get the information that they needed to write those 12 lines of code. And that takes probably about 30 minutes to write those 12 lines of code and rip off that PR. OK, the next step to get unblocked for you and your team is to go to get unblocked dot com. yourself your team can now find the answer they need to get their jobs done and not have to bother anyone else on the team take a meeting or waste any time whatsoever again get unblocked.com that's g-e-t-u-n-b-l-o-c-k-e-d.com and get unblocked

55:26if we can move this conversation up a level for a moment do you guys like this one-year release cadence, it seems like it has its drawbacks. Well, this guy implemented that idea. So what do you think he's going to say? Who did? You did this? Yes. So we can make this good cop, bad cop, or actually devil's advocate. And this guy is trying to buy this. This is good. So I'm also the release manager of some ancient Python versions. In fact, when Python 3.13 comes out, the first version I released, which is Python 3.8, is going end of life. So we're going to say goodbye to that. It was a very, very nice release.

56:06I really enjoyed Python 3.8, but it's time to move on. So if your use case, if your production is still running on 3.8, well, now it's high time to move since it's not going to receive any security updates anymore. Right, so at the time I joined as the release manager, the release process was a little bit hand-wavy in terms of when are we releasing Python. So the theory said we're releasing it every 18 month-ish, but it could be anywhere between 18 and two years. And, you know, you couldn't really tell. And definitely what you could not expect was which time of the year is this going to happen in.

56:45And the consequence of this was that Linux distributions in particular, but also other operating systems and end users, were lagging behind adoption of Python 3 versions a lot. You know, people are like, well, Python 2, no, because Python 2.7, right? But like Python 2.7 was out and then essentially frozen for a long, long time. So people forgot how this thing looked like before in times of Python 2.3, 2.4, 2.5, 2.6, and then 2.7. That unstable release cadence was problematic. And that cost, for example, for Fedora, which later is source for what is happening in Red Hat, was lagging behind with the releases that we had.

57:26And it turns out that the infrastructure that the big Linux distributions have, like Ubuntu and like Red Hat, is extremely useful for us to early find bugs in a new version that we are developing in the betas, in the release candidates, and so on. So it would be worthwhile if we could actually ship the newest version of Python with those Linux distributions as they are also developed and go through their respective betas and so on. That's also great for the end user, right? Because then by your autumn release of Fedora, you are released with the newest version of Python that was just released.

58:07That is a great experience for the end user. So I wrote APEP, a Python enhancement proposal document, arguing those things that I'm just telling you, that it would be amazing to have a fixed release cadence that we can trust and share with the Linux distributions such that they can essentially rely on us having certain milestones in the release, like reaching beta one, like reaching release candidate one, so that they can use the latest version as soon as possible. And we switched to the release cadence in Python 3.9, and we've been using it ever since. So yeah, your honor, that's my position on this issue.

58:50Well, as the other release manager of Unseam version now, so I released 3.10 and 3.11. I mean, one of the problems that this thing has, I mean, it has worked really well, I think, like people in general, and just to be clear, like, you know, so Wookies doesn't kill me after we finish the podcast. I think it's that positive chase, right? So in general, very good. I think it has been some predictability. People actually now have dates and they can put their lives around. One of the interesting side effects of this is that now people have this impression that Python is releasing too often. So every single year there's a new version and then the old version gets out of support or like first backfix support and then security support.

59:35So they have this feeling that, you know, they're going to be, if they don't move fast enough, they are going to be able to support super soon, which was always true. It is that you get the numbers like changing slower. So the impression was that it's happening slower. But right now, this is like moving this impression that, you know, Python is changing too quickly, right? Or whatever it is. The other downside that this has is that, as you can see, for instance, for Python 3.13, it means that because we need to release every single year, we are releasing these features, maybe like big ones, like the NoGil or the JIT in particular are not being super exciting.

1:00:07So for instance, JIT in particular is 0 % faster. Well, it's kind of cool, but where would you use it? Well, if we have a 2G release, maybe it would be a bit better. So you get these weird situations. So planning big projects right now is a bit more challenging. For instance, one of the things that happened in Python 3.9, if I recall, we changed the entire parser of the language. Like Bonk, different parser, which I'd like to describe as changing the wheels of a car that is moving. not the best feeling but that was that will have we could have some extra rest if that was a two year release cadence and we didn't have to finish everything and all the validation in one year.

1:00:47I mean this doesn't mean that it was rushed it's just that we have to put more hours because we wanted to make it to the release but on the other hand it's humans making it even before when the situation was like Lukasz was describing the day of beta freeze, which is the moment when no new features can come in and only backfixes from that point on in the release, everybody was rushing their feature on the last day, you know, like all the new features to do, like, so, you know, humans are humans, so it doesn't really matter. But that's one of the, like, certainly the hit has been raised a bit because, like, right now it's released more often and the numbers are changing faster.

1:01:22That was more the angle I was coming from, was from the, when the train leads the station once a year. You better get your stuff on it, whether it's ready or not. And I appreciate, I didn't know the history of where it came from. So I think this is definitely progress and improvement. Predictability, you know each time of year it's going to go out, et cetera, et cetera. And it makes a lot of sense, especially a large project like this with tons of people using it. You want to be predictable, but also not too frequent. I would probably tend towards more frequent because now you have the train leaving the station more times.

1:01:55And if you don't, if you miss this one, that's all right. We got another train leaving six weeks from now. And so you can hop on when it's ready versus having to hop on once a year. And we've seen success in projects like Chrome, et cetera, with rolling every six weeks, whatever's ready, we're going to ship a new version. I don't know if that works with languages in runtimes. You guys know way better than I do, which is why I asked the question. Right, I guess for compiled languages that might in fact work better, for us the difference is that we're interpreted. So there's a lot of code that is lying next to your interpreter and then you're trying to run it with whatever is available to you.

1:02:31With a lot of the libraries that we're supporting, they're a particular issue. So, you know, like now I'm going to be the devil's advocate because like Pablo like was too apologetic. So like, you know, let me like... Wow, wow. Yeah, like... So that's right. Yeah, let me just say like, hey, this is all a compromise, you know, and there's obviously going to be things that are better and things that are worse. The price to be paid for shorter release cadences is that the matrix of tests of CI that a library maintainer has to now kind of suffer with, has to run, is larger, right? Because the number of versions that is out there that people are using, there's just more versions than there were before, right?

1:03:14You had 2.7 and 3.6 maybe. And then 3.7 appeared some 18 or 19 months later. So the growth of this matrix was sort of spread out in time. Now that matrix is bigger. I would claim that, okay, it's good that we're testing with every particular version to identify where incompatibility arose or something. However, I always like to say that it's still the same sausage. We're still the same sort of team that is developing the thing. We still have the same velocity, only we slice it thinner now. So every release is easier to migrate to compared to the last release because necessarily there were fewer changes.

1:03:59Yeah, psychologically speaking, that's a very good point, actually. I'm glad that you mentioned because one of the things that happen when you release every two years or more, especially for a language that needs to evolve and needs to change things and maybe doesn't have, I mean, we get out of backwards compatibility, don't get me wrong, but other languages like C++ clearly care even more. so for a language that changes this much changing and adapting your code and all the libraries for a two year worth of changes is brutal which means that we saw that before right which means that until new libraries are compatible with the new version there is a non-trivial amount of time so as a user or as a company employee that you want to leverage these new features you probably want to wait a lot because now people talk about scheduling the work and finding even how much it is and library maintainer says well i will fix it later but when the changes are smaller there is like psychologically speaking obviously in reality is more of the same modulus and non-linear interaction but in general uh when the changes are smaller then it's more enticing because like at least you know okay maybe this this this and this thing and sure maybe you need to wait for another library to do it but like normally like and right now what we have seen uh compared with 3a when we started doing this right is that that people jump earlier like before uh for instance one of the biggest libraries in this Numpy, which is numerical computing in C, right?

1:05:19And Fortran and whatnot. So that library used to take a long time to be compatible with new versions, like months, right? Or sometimes even like half a year or something. And right now we are having it almost day one. So this is fantastic because people that want to use the new features in the language, they are not held back by the fact that their own particular dependencies don't have the support for this. So most of these things, I don't think you can approach it from a data-driven kind of perspective because it's very difficult to say what is better and what is worse. But certainly from a psychological point of view, I think you have a very good point.

1:05:57Not too soon, not too often, sorry, but also not to lagging behind. I think earlier is better, maybe not too early. Every three weeks it will be impossible for a compile language. We will ship a JIT compiler and optimizes your code. It will be like 6 % lower or something. But one year, yeah, good. I want to add one thing. So NumPy is not doing as well as Pablo is claiming. They're doing way better. There are already wheels for Python 3.13. Before the release of Python 3.13, those wheels are going to work on the final version of Python 3.13 released October 1st. We're releasing this like mid-September.

1:06:40They're already there. Moreover, they're also already there for the specific version that is this experimental interpreter that does not have the GIL. I think everyone, a non-trivial amount of listeners probably are saying, what is that wheel? What is this thing? Yeah, so it's this binary package that lives on the Python package index on PyPI that people download NumPy from when they say pip install NumPy. So if you say pip install NumPy from Python 3.13, on day one, you're going to be using wheels that are already there before the release of Python 3. Is this another word for a build? Basically it's a binary package.

1:07:21So it's a package that doesn't contain Python code only. Why do we make up another term for it? Well, this is the key, right? Because the Python package index used to, well, I think it still is, it's called a cheese shop because it's all about these Monty Python references and jokes, right? So it's a sketch of the Monty Python that goes to this cheese shop. And a wheel is a wheel of cheese. Oh, gosh. So, you know, ha, ha, ha. I've taken that one a bit too far, maybe. Or like PIP. Do you know PIP? I mean, this is a slightly different joke, but do you know PIP? PIP is the package installer for Python, right?

1:07:51So when you install a Python, you do PIP install. Do you know what PIP means? Python install packages? PIP installs packages. Yes. Okay. So a recursive acronym. Recursive. Yeah, yeah, yeah. Recursive acronyms I'm well aware of, but I'm not a Monty Python guy, so I missed completely this reference. Right. Well, you know, I was missing this reference for years and someone pointed it out and said, well, I suppose it makes sense. You're like, I have to watch it. Is it from Holy Grail or which movie is this? No, I think it's one of the Flying Circus movies. Yes, one of the episodes of the Flying Circus has a skit about the cheese shop.

1:08:26And that's where that comes from. A little obscure reference to build an entire Python ecosystem around, but okay. I think it's not helping for Python programmers to look like they're in a coin or something. But you just need to have some file extension for those things and some name for those things. We used to have a more primitive early format for this sort of thing. And those were called eggs. Eggs were because, you know, snakes lay eggs, right? So that made this other sort of reference. Because Python in the name comes from Monty Python. But a lot of people just see the name Python and they're like, yes, snakes are cool.

1:09:05So snakes. We accept both sort of ways of interpreting this because snakes are indeed cool. Yes, sure. But yeah, like now that we knew that eggs are not enough for what we need for proper binary packages, we were like, where to go now with names? And naming is hard. Let me tell you this much. Like naming is very hard. Someone jumping in the middle of the podcast listening to us talk about eggs and snakes. Wasn't this about tech? Again, this background is spectacular because you do run out of references with snakes. snakes. Like how many things can you actually reference? But with Monty Python there's just countless references you can make.

1:09:41Just people won't get them. But yeah, they're very obscure these days. It's an increasingly old and hard to get. For instance, one of the funny ones one of these attempts at the compiler from back in the day, I think from Google was called Unladen Swallow. From the Monty Python Life of Rangio? Holy Grail. Holy Grail, yeah. The speed of Unladen Swallow. Good stuff. Okay, good meta conversation on release cadence. I think I agree with all your points. You have to find that sweet spot and as frequent as you can be reliable without being too frequent. And that probably varies per project. And I'm happy that you guys are happy with once a year.

1:10:23Certainly better than what you all were doing before. That's very cool. Let's go back to Python 3.13 because Pablo, at the beginning you mentioned it's officially iOS compatible or something. what does this mean? You can just fire up Python on your iPhone? Well, not quite yet, I would say. So CPython has a very old build system. It's AutoTools, which is this computer make. People that have been doing Linux for a long time probably have run into this. And one of the challenges of this is that, and then it's another different for Windows, but both macOS and most Unix platforms use that. So for a long time, that was okay, but there was a lot of platforms that we couldn't support partially because of the build system and partially because the source requires changes.

1:11:10For instance, many of the efforts that we are trying to do right now to make Python available in the browser, I don't know if you hear about this, but right now there's a good effort to ensure that Python can run in the browser and it's a good alternative to JavaScript. This is a project, for instance, called PyScript, which allows people to run Python instead of JavaScript, which is quite cool. so those require changes in these build systems and even in the source of itself and iOS in particular has quite a lot of challenges, actually probably Gukesh can chime in to specify those since this is very close to his heart but one person in particular that has been working for iOS support which is Russell Kimmagee he works right now at Anaconda which is one of these Python companies so he has been working quite hard to ensure that both the build system and the source itself works in iOS, which has a lot of challenges, right?

1:12:05Because iOS has not only an alien way of doing things, but many APIs are not available there, or other stuff are available, but you need these specific things that you need to Apple to bless you with their power just to be able to use. Like the G-Compiler, how does the G-Compiler run in iOS? Who knows? Maybe let's start from the very beginning there. So the goal that we have with this iOS support, for the time being at least, is not to have an application on the App Store called Python. That's not what we're doing. What we are doing right now is to have the Python distribution, the source distribution, ready entirely for being embedded within another application that somebody wants to release on the App Store for iOS.

1:12:52This was theoretically possible in the past with already a ton of hacks that Russell Keith Magee historically made for a project called Beware that is just running Python on mobile on both Android and iOS. But the Python source code, our source distribution was not making this easy. There are plenty of changes that you may have to make because the phone platform that Apple came up with is very limited. It is limited by design such that it's more secure and it doesn't waste your battery too much. And it does a bunch of other things that make it easier for applications to be just swiped up to kill them.

1:13:36And they don't run any runaway processes and whatnot. So there are APIs there that are not allowed. There are others that are just not implemented, so they're not available. So you cannot run threads and so on. So this sort of thing causes embedding Python to be complex. Then there's the new and increasingly strict rules around when you're packaging some sort of binary blob and want to put it on the official app store for Apple, they have this automated and sometimes in person, but mostly automated review process, whether what you're publishing is safe. So there's plenty of things that interpreted language like Python would do that would be flagged by those automated systems as unsafe, that this code is doing something fishy.

1:14:24So there are changes needed for those things to just pass review so that your application that embeds Python can successfully actually appear on the App Store. Like including until very recently, you couldn't really have just dynamic libraries lying around in your bundle that you're publishing on the App Store. Everything had to be statically compiled, which puts, depending on libraries like NumPy, into jeopardy because it is very hard for external C extensions to be built in a way that they are both all part of a single binary. Now this extension is lifted. However, you have to sign everything that you're publishing to the App Store separately.

1:15:05So it has to be notarized by Apple. It has to be signed with your application developer certificate and so on. And so, you know, you know, when you're sending out this code to Apple that, okay, this is the code I built, nobody meddled with it. And then Apple notarizes it that, okay, this one is safe to be run by, you know, users, phones and so on. So all this infrastructure, like it seems like something that shouldn't really be our concern, but all of this was making it hard for Python to just be embedded. And now with Russell's hard work, this is a platform that we support. So if you want to have a text editor that has automation in Python, if you do want to have an editor for Python source code, there's a few of those, like Pythonista, Pyto, and so on.

1:15:51Now it's going to be much easier for the authors of those to upgrade to a newer version of Python. This is why all those apps lagged behind and they were still on Python 3.8 or 3.10, because it was a ton of work to upgrade and redo all those hacks in a newer Python version. So now, unnecessary, we support it. That definitely wasn't what I was thinking, but it makes a ton of sense. I think it's going to make some people very excited. What were you thinking about? Yeah, what were you thinking about? Write some Python code and deploy an app based on it. Like, build an app in Python. Right, so this is kind of step two for the Beware project that Russell Keith Magee is part of, and which is now being essentially sponsored by Anaconda, where, yeah, you're going to be able to build Python applications from scratch and end up with an app on the iPhone that you can put on the App Store and it's fine.

1:16:44But the first step of this is that all this additional tooling that ViewViewer is building on top of Python needs to rely on Python being embeddable. Right. Also, conceptually speaking, it's not different from the first use case because you're not going to have one single Python in the App Store that everybody can use. and you need to have everyone will have a different Python version. So it's a trivial embedding. So you're not embedding this on a text editor. You're embedding it on a binary that is just Python. But you also want to see not only your Python code, you want to see your dependencies as well.

1:17:19So now, okay, this is the same. If a text editor, you want a text editor to have Python inside to do some extension, you may want some extensions that Python can do to interact with something else. So maybe you want also that on the dependencies. so it's technically if you screen hard enough it's really the same case it's just that for the developer that just wants to put some files and maybe to say I require this and that that's a lot of work you're in the business of compiling Python yourself what are you talking about so it's much better if we can offer a specific workflow that says like okay your case is only like this and that and you only have these dependencies then you just click this, put this here and the build process will take that for you.

1:18:04So you don't need to, that's the step two. But, you know, conceptually speaking, it's just a dev tooling, not really a huge undertaking. But really it's like, you know, it's again, incremental steps. So the important thing here is as well that like this is now part of our CI. We are building Python on iOS every time you make a change. So it's going to be much less probable for us to introduce changes without knowing that they break support for running this on the phone, which should be great news for anybody that actually wants to see Python on mobile devices, because now we are going to care to keep this running.

1:18:41Well, now you have me thinking about the future. Let's close on this wish list. Imagine that the three of us are sitting down next September and talking about 3.14. What do you want us to be talking about? What do you hope we're talking about going into 3.14, each of you? Oh, okay. So my dreams are simple. I want everything. I like that. So honestly, I sponsored the, you know, making the optional PEP. So this is very close to my heart. What I would like to see is for this to pay off or the library developers to step in, try it out and say like, hey, actually this is helping us. What is related to this to see that the performance, you know, efforts that we are having separately from the global input lock removal actually converge and kind of do work with this free-threaded version.

1:19:38And also, finally, again, free-threading. We don't expect people to actually start new threads randomly and end up in this crazy world of free-threading where you can just easily just write bad applications by writing on top of your variables and accessing data you shouldn't. And, you know, kind of we don't want to reintroduce models that already we know failed in other languages. We want tooling to be more high level. So I want this more high level tooling to start popping up slowly. We deliberately didn't suggest any so far. We want to see what the community actually wants to use. There's kind of things that happened in other more younger programming languages.

1:20:21You know, you have go routines, right? like you have channels in different programming languages and so on. So I'm very excited to see where this goes. So this free threading evolution is what I would like to be able to tell you next year that, you know, has done great leaps and it's now way more usable and it's literally the future. So, yeah. Well, in my case, I think, apart from what Gukesh just said, I think the key for me is that this pays off, which is non-trivial, right? It may pay off for some users, for sure, that are really hungry about this. But what I really want to know is that people, you know, for a lot of time, removing the guild has been almost like jokeable.

1:21:04Like, you know, this is what is missing. Like, you know, this will be the last thing and everything will be so green and fantastic when we remove the guild. Well, no, no, no. Prove it. So I want to see everybody saying, you know, now that there is no guild, oh, I finally can do this and dip this thing. Which is not going to happen fully, right? because people still will reach to other languages for other reasons and performance. But I just want to see a big impact there. And ideally, we talk again for 3.14, we are saying, okay, this makes a difference. So it was cool, sure, and some people really like it, but wow, now it's a different play field.

1:21:40So it's different. The other thing that I would like to see, and this is a bit more personal, so this is a project that I started, I think, in Python 3.10, which is that a lot of people were complaining that the user experience of the language was not really great in terms of error messages and things like that. So syntax errors and you mistype something and there is some important problem. So that was really bad to the point that there were some situations that was laughable. And we have worked quite a lot with the new parsers and with many other improvements to improve this to the point that right now I'm super happy to see that many, like even papers that are researching error messages are citing Python as like one of the, obviously not in the same category as Rust because like we can all agree that Rust has fantastic error messages.

1:22:25Different kind because those are compiled time error messages that we are talking about runtime ones. If you don't consider the parser bad. But we are right now like quite high in that sense to the point that I think there was some research about like how LLMs can reason about what's wrong about your code from the errors and you run it in 3.11, it was like 13, a 30 % better performance of those languages because it's telling you a bit more, right? So you can fix the code a bit better, which is quite cool. So we continue doing this thing and we are trying to add even bigger changes to the interpreter that help in this regard, which is challenging because those changes sometimes require some non-trivial performance games there.

1:23:07So obviously we don't want to affect performance, but to do that we need to put a lot of engineering there. So in 3.14 we are probably going to try some new ones that are quite big so hopefully when we talk again those have actually been implemented and they have been successful which I would love to see yeah that's awesome, I was reading about your improved error messages and I think that's the kind of thing that has to be a sustained effort over time it's like one of those paper cuts kind of things where it's like we're just going to continue to polish this thing and so a great effort, I'm glad that you struck that up well the podcast, core.py if you are a super python nerd and want to nerd out with these two about internals on a more frequent basis then we'll cover it here on the changelog which sounds like it's going to be about once a year love to have you guys back next September and be talking about 314 hopefully all of our wishes will have come to fruition and we'll have people out there free birding it I mean free threading it cue Leonard Skinner here

1:24:16and enjoying the new Python with no gill and the performance improvements that are possible there. Lots of potential. I'm certainly looking forward to it. Guys, thanks so much for coming on the show this week. Thanks for having us. Yeah, thanks for having us. We have a pleasure.

1:24:34okay so now that python is officially experimentally free birded i mean free threaded are you going to defenestrate the gill let us know in zulip yes the changelog community's conversations are now happening in zulip join today by signing up at changelog.com slash community. If you've already signed up, but you want your Zulip invite, just sign in and click on send me a Zulip invite. You'll see it. Of course, it's totally free and a totally cool place to hang out and discuss. Let's do one more big thanks to our sponsors of this episode. Sentry. We love Sentry. You might too. Use code changelog.

1:25:17Save 100 bucks. Fly.io, the home of changelog.com. Ship something in five minutes or less. Learn how at fly.io and of course to coder.com and unblocked please support our sponsors they support us and thank you of course to the goat breakmaster cylinder is the greatest beat freak of all times finally thank you for listening to our shows we truly appreciate you spending time with us each week that is all for now but on Friday we are talking about developer happiness and unhappiness and productivity or lack thereof with our friend Abinoda from DX. That is a fun one. So stay tuned. That's all for now.

1:26:02But we'll talk to you again on Friday.

1:26:33Game on.