WebAssembly 3.0 with Andreas Rossberg

31:13So, right. So more high level languages, especially GC languages, are much more reasonable to compile them to WebAssembly now. I would think that actually it's reasonable for most applications. The only thing if you're really building something that all what it's doing is DOM interaction and manipulating the DOM, then you probably won't get much value out of it. right? Definitely not in terms of performance. It won't necessarily be slower either, but you might have to jump through extra hoops with all the glue code and so on and so forth, which is much quicker if you can just do it in JavaScript directly.

31:54So that is the main domain where I would say, okay, it's probably still better to use JavaScript. Or potentially, if you're compiling a language that is very closely integrating with JavaScript by design, right, then that might also be better to still target JavaScript itself. So that's kind of actually an interesting thing to discuss. So when I think about building an application in JavaScript, and particularly for the web, you have JavaScript, the language, all the runtime and libraries, but you also have all of these web-specific APIs that are available in JavaScript. Are those exposed directly in WebAssembly as well if you're building for the browser?

32:33Or do you have to hop over to JavaScript to access the API and then hop back? Yeah, that's a famous kind of question and complaint. Give me DOM access, right? The thing is, yeah, right now you don't have that. And that is actually a very important part of the whole WebAssembly security model that it's completely sandboxed, right? You have no ambient capabilities to have any access to your environment whatsoever. Anything you can do or interaction you have with your environment has to come through imports. And that's like core to its very tight sandboxing functionality. So what you have, though, is this very tight integration with JavaScript in the sense that a WebAssembly function you import can just be supplied as a JavaScript function, providing the kind of signatures match up sufficiently well.

33:26So basically, you can just import JavaScript functions and then just call them as if they were WebAssembly functions. The problem, of course, is that all the web APIs are really totally designed for JavaScript, and they don't really make a lot of, or many of them don't make a lot of sense in terms of a more moderate kind of object system language. So you usually end up having to write some, or may have to do some glue code there. But this is more like a tooling problem. So people often make the assumption that it would be way faster if they could access, say, the DOM from WebAssembly directly.

34:05But that's not actually necessarily the case, in particular, since JavaScript engines already go to way more lengths to optimize all sorts of things about accessing objects and JavaScript objects and DOM objects. And that would be a ton and a megaton of stuff that you would have to replicate in the WebAssembly code generator. and I don't think anybody is willing to do that because it's kind of silly, right? You wouldn't even, probably not even get much performance out of that because the overhead of calling from WebAssembly into JavaScript is actually mostly dominated by the actual work done in the DOM itself, right?

34:45With most of these calls. So the assumption that it would be way faster or something is not actually very accurate. That being said, there is some work now. So one thing you would like to have is at least have some kind of interfaces for accessing the DOM, which people sort of agree upon. And that is something that is sort of a separate layer from WebAssembly core itself, right? So people have now started looking into this in the context of this component model work. and so that might be something like maybe similar to these WESI libraries but for the web so we start defining interfaces and they grow kind of organically and maybe at some point when they are mature enough we've iterated enough on them and we might even standardize some of that but until then it might just be a library or tool chain that just does this for you and you don't have to worry about it anymore because that's the other thing right with the web API is a gigantic and be a moving target.

35:53So standardizing anything around that is kind of a challenge, right? Like you, you were more generous than I was. I was going to say a fool's errand. And you have to go through the pain of like trying to find the nice mapping to something that makes sense in WebAssembly and possibly for more than just one language that wants to interface. That was the thing that I was going to ask is, you know, now that you have relatively flexible, a relatively flexible type system, is there actually a substantial difference in terms of the interfaces per language, right? WebAssembly is pretty, leaves a lot of that to the language compiler.

36:33So there's still no like sort of automatic interoperability between languages, And that is very intentional because that in practice never works, because even the smallest impedance mismatches basically usually break everything. So it was very much an axiom in WebAssembly's design that we not try to solve this problem because it's essentially insolvable, at least on that level. And trying to solve this is sort of what the component model tries to do. that defines a somewhat more high-level language agnostic type layer, which is more high-level as well than GC types we have in WebAssembly now because I mentioned that these are very low-level.

37:13They don't know what an object is or anything like that. They just talk about low-level representations, really. So the component model has more high-level types, and they are particularly designed for multiple languages to be able to map to them and talk through these with each other. And that would be the preferred way to also define maybe a web API for WebAssembly. So on this level of type system that everybody knows already, at least if they want to implement against the component model, they already know how to talk to and have mapped to their language. Let's maybe step back a moment and describe the component model.

37:52So this is another WebAssembly spec or what exactly is it? So this is ongoing work still. And at some point, it might be a different spec, but we're not quite there yet. So this is trying to define a separate layer on top of WebAssembly itself, which is like a more advanced module system, if you want. That's one view of it, plus a more high level language agnostic type language to describe interfaces, right? And with that, you can basically specify how you bundle WebAssembly modules. You can define how they instantiate each other probably and things like that. And then there is this closely connected to that, this WESI effort, the WebAssembly systems interface, which tries to define a whole set of kind of standard in air quotes interfaces with respect to this component model.

38:49so that like that are operating system abstractions for example so you can use them and there are different ways to implement these interfaces either a web assembly engine implements natively so you have direct access to your host environment or they could be virtualized in other ways and the component model is pretty rich in allowing various ways of implementing interfaces and plugging things together for virtualization and stuff like that but yeah so that that is ongoing work still. It's a very complicated problem to solve, as you can imagine. And many people tried to solve that problem before.

39:24And there have been various technologies that don't have the best reputation, let's say, historically. I'm not much involved in that other than talking to the people doing most of the work. But we are kind of very careful to not try to repeat the mistakes, have a more constrained system and take advantage of some of the inherent design advantages that WebAssembly has, like the strong sandboxing and all that, right? And the strict portability. So it tries not to solve all the problems in the world at once. It makes some informed opinionated choices, like when modules of different languages talk to each other, it's all like a shared nothing concurrency model.

40:07So you can't really pass pointers across the boundary because that introduces all sorts of extra problems, right, that you want to avoid. That might also mean that it doesn't solve everybody's problem, but at least it solves a certain set of problems in that space. And from there, we can maybe get more experience about other points in the space after that. Got it. So to sort of play back to make sure that I understand, right now in the WebAssembly world, if you had different language implementations of different modules, there's no guarantee at all that they're going to be able to talk to that.

40:42That's basically on the developer to say, here's the defined interface that you have to match and go. And so probably most people are developing these things as a pluggable piece, either all built in one language or they're owning the whole system that's interacting. And what the component model, if I'm understanding correctly, is saying, let's define a higher level interface that is true across languages so that you could build essentially libraries or modules that can interface with each other without having to know the whole system front to end. Yeah, I mean, the situation with bare WebAssembly is kind of like with hardware, right?

41:17So you have your hardware types and that's it. And then if you want to have different compilers be able to talk to each other or to talk to an operating system or some other kind of environment thing, you need an ABI, right? An application binary interface. And that is something you can well define on top of WebAssembly. So far, nobody has done that. Although like a few weeks ago when we had the last face-to-face meeting, there was actually talk about defining a C-ABI for WebAssembly, which is the main one that everybody wants, right? So you at least know how to talk to a CFFI or something, and that is standardized across multiple compilers.

41:54And the component model you can view as an ABI on steroids if you want, because it's not a fixed ABI. You have a glue layer in between. You can define how you interface to that. But yeah, other than that, that's exactly what it is. Well, and this starts to, I think, take you out of the world of WebAssembly is a way to build singular web applications and into a world where you say, OK, WebAssembly might be a generalized operating system or virtual machine that you can target for different reasons. So let's maybe actually divert over here. We've talked a lot about web use cases for WebAssembly. What are some of the non-web use cases that people are wanting to use this for?

42:40Oh, there are many web use cases at this point. Actually, the web is just one among many at this point, which kind of we hoped for from the beginning, and it worked out better than we expected. I mean, we made very sure that the WebAssembly design and also the spec is not at all tied to the web or JavaScript for that matter, right? And the web in the name was more like a marketing thing, if you want. it's really intended to be like this maybe a bit hyperbole but a universal vm if you want right and so what are the use cases so people are using it on for edge computing a lot right like fastly is the biggest company doing that they bet everything on web assembly there lots of use cases in embedded space which kind of confused me for a while because why would they take this overhead when they have small CPUs anyway.

43:34Well, it turns out the big feature for them is portability. So we have people from Siemens, for example, in the working group who explain that, you know, we have all these embedded devices and there are new processors every year or month or whatever. And it's a pain in the ass every time you have to rebuild your entire tool chain. And now they don't want to do that anymore. They want to have a WebAssembly tool chain and then basically just switch out the backend for their CPU directly, right? But they have this very different use case from the web because, first of all, they have much smaller systems, so they need a much leaner runtime, idly, like zero size if you want.

44:17And they also want to do ahead-of-time compilation. They don't want to do JIT compilation like you do on the web, right? They have this whole tool chain, but in the end, they compile to the actual hardware they want. And of course, WebAssembly was designed with AOT in mind as well. We always wanted that to be an option. So we also took a lot of care to not shut the door on that. And there are other kind of optimizers that take benefit of that. But yeah, actually seeing that happening for realists is also interesting. there are more fancy use cases like in ai where you kind of use web assembly to control pruning in some search or whatever so you make some things programmable but in a portable way right with a very universal language like you you script your your pruning algorithm that way i think microsoft did something like that there are blockchains who use it as their execution platform And one of the features they are particularly interested in, besides the sandboxing, is the well-defined deterministic semantics.

45:24Because if that's the code you execute on the blockchain, I mean, blockchain is a whole hype word, right? But what matters there technically is you do replicated computation, and then you rely on consensus between the different replica to deliver the same result. And of course, that won't fly if you have underdefined, underspecified behavior all over the place, because then everybody will disagree all the time, right? And WebAssembly is, to a large extent, fully specified and fully deterministic. And with 3.0, we even have a deterministic profile now, which is kind of specifying if you really want to be fully deterministic.

46:05This is what it means. So everybody has a clear picture of what that means and can depend on that. So you always get the same result. So let me actually dig in a little bit on this, because I think naively I tend to think of most traditional programming environments as being deterministic. As contrasted to now we do all this stuff with machine learning and LLMs, and that's where I think of, oh, that's non-deterministic. I should explain here that deterministic is used here in more like an abstract way. So we say that language semantics is non-deterministic if it is may be specified, but allows multiple different behaviors.

46:44So, and then any particular execution or any particular implementation can pick one of these. But the point being, if you run on different hardware or maybe under different hardware constraints, right, then you could get these different behaviors, right? And that's what you don't want. So the way you model that semantically is usually as non-determinism. So just you get a random pick of the allowed behavior. Got it, got it, got it. So this is places where essentially the language spec does not fully pin everything down. And so different implementations of that on hardware or just different compilers or whatever can make different choices.

47:19And so it may be deterministic within a single environment, but across environments, it's non-deterministic. Yeah, and even within a single one, it sometimes might not be. You never really know. So WebAssembly 1.0 was very neat in that regard. There was only one little source of non-determinism, which was with floating point arithmetics. So if they generate NANs, then unfortunately, the IEEE standard for floats failed to define what the bit patterns are for NANs. And all the hardware vendors completely disagree on what they do and even how they interpret these. So there are funny things like IEEE defining a quiet bit or signaling bit on NANs.

48:00But it doesn't define what way it's to be interpreted, whether one is signaling or zero. and surely hardware disagrees on that choice, right? So silly things like that. And initially we tried to even knock that down and say, okay, if your hardware's behavior diverges from what we specify, you have to normalize all your NANDs. But it turns out that is very expensive. You don't really want to pay the cost because then all float computations get more expensive. Although that's what, for example, blockchain implementations do exactly, right? They do that. It's not that bad, But you pay a certain price for that.

48:38Right. So that is the kind of thing why we have non-determinism on the spec level. Of course, you also have natural non-determinism once you start talking about things like threats, right? Because that's essential in the concurrency model that you have non-determinism. That is more like intentional non-determinism. That's really core to what the feature is about. But all the other are sort of like accidental non-determinisms that are just really under specifications because we can't do better. And the less we have of that, the better. Yeah. Well, and this is sort of a level of care or if you're on the other side, like pedanticism that most developers never have to worry about.

49:18But if you're a language designer, all the way down. Okay. So we talked about some of those use cases. We talked about how hardware embedded folks might be doing this to simplify their toolchain, blockchain. You talked about determinism. What are some of the other drivers for non-web use cases to adopt WebAssembly as their target? I think it's usually two things. One is portability and the other is the sandboxing model. And of course, the fact that it's pretty universal as well. So it's like Turing Complete. You can compile everything into it. It's not constrained in any particular way. So those are the three reasons why it's a good fit for various places where you want to kind of embed some form of compute without tying yourself to whatever, like the specific scripting language or open security holds or whatever.

50:07Right. So that's kind of how you make use of it. Yeah. And it's also fast, which sometimes is a bonus as well. Yeah, so I think the example that comes to mind for the closest to that sort of universal virtual machine model that anyone's come before, I think, is the JVM and all the different JVM targeting languages. What are the factors that might cause somebody to move from a JVM environment to a WebAssembly environment? Are they exactly what we just said? Is there some other nuance we're missing? Yeah, I mean, the JVM is interesting because that is really a VM that was designed for Java, right?

50:43And it's basically an abstract syntax for Java, if you want. So it has all the Java object model built in, which is pretty heavyweight, including all the reflection stuff and so on. And there are other things it doesn't have built in, right? You can't do. So when you compile to JVM, and many languages have done that, usually if you don't happen to be very much exactly like Java, then it's not going to be very fast. There are things like, so for example, with the GC extension now, one thing we have that other VMs don't have is we have tagged pointers. So I don't know if you know what tagged pointers are, but if you have a garbage collection runtime, then usually you have the garbage collector has to know what a pointer is.

51:34And sometimes you want to unbox integers so that you don't have to put everything into a box and allocate. So you usually use pointer tagging, which is you take a bit off every pointer that marks it as an integer. And then you can just use an integer as if it was a pointer and the GC knows about this bit and just follows the real points. And that's something we actually provide directly as a WebAssembly feature, whereas in other VMs you don't have that. But there are many language runtimes that make heavy use of such a feature to avoid boxing all their small types. and do things like polymorphism much more efficiently.

52:13And with the JVM, you pay the price for the heavyweight object model. So if you're compiling a language that, for example, has lots of very small allocations, which is, for example, the characteristics of many functional language, then that would not be ideal for you because your characteristics run against what it's optimized for. And then there are other things like, I don't know, if you want to do closures and stuff, that's also very costly in the JVM because it's not like native to it and you have to encode it in heavyweight ways and all these things, right? Like you don't have access to the low level as much as you have in WebAssembly.

52:55So you have to jump through more hoops in a way. So let's now look forward a little bit. What does the next generation in WebAssembly look like? what can't WebAssembly do well right now? We mentioned a little bit the component model, and that is a big area of exploration and focus. You also sort of had a throwaway comment about continuations. Maybe we can dive into that. But yeah, what's coming down the road? Right. So in terms of bigger features, I mean, there are various small features in the making, but the two big features I would say that are still missing from core WebAssembly, so not the component model, is threads.

53:34is the one which is sort of close to the finishing line, finally, I hope. And the other is what used to be known as stack switching, which turns out we model it as this form of delimited continuations. And the reason this feature is important because this is what allows you to compile at least efficiently all these control abstractions that modern languages have. And control abstractions are things like, I don't know, async await, generators, green threads, you name it, right? All these things where you have non-local control flow. And usually these are implemented or often they are implemented in some way or the other by having multiple stacks that you switch between.

54:20And when you are in a system like WebAssembly where you have to be type safe and where also the actual stacks are kind of encapsulated, you can't really get to them for safety security reasons. Then it turns out that there is a neat abstraction that is already well known in the programming language field for decades, which are continuations or delimited continuations. So that is kind of like how we model them. So you have a way of, and basically a continuation is a new stack if you want. You have a way of creating a new one and then you can resume their computation there, which basically switches over to that stack.

54:57And then that computation can suspend, which switches you back to the point where you resumed, very roughly speaking. And you can have as many of those as you want. So I assume that since those control flow features that you mentioned are in a number of languages, including some that I believe already target WebAssembly, how are they supported today? What will this take you forward to? There are many kind of ways to do that. So the most powerful way where you can do all that is you do a global program transformation called CPS. So continuation passing style where for every function you have an additional parameter which tells it where to continue instead of returning.

55:39That is also a well-known technique, but you can imagine that it's pretty expensive. So some languages actually do that. Others use tricks by, I don't know, trampolining through JavaScript, for example. That is, by the way, also how C++ implemented exceptions before WebAssembly 3.0, right? It had to call into JavaScript to create an exception handler, then call back into WebAssembly to actually run its body and so on and so forth, which is also kind of crazy. And of course, it only works when you have JavaScript. That's one technique, yeah. So once you have exceptions, you can do trampolining, which is you just build up your call stack and eventually you throw back to a scheduler that then somehow has an idea of how to build up the call stack again if it has to.

56:26And there are various techniques like that. They all have in common that they tend to be brittle and costly to some extent and not very modular. So one of the features our proposal has, which is something that came out of academia in the last kind of 15, 20 years, it's based on this idea of effect handlers, which is essentially the simplest way to understand it is a generalization of exception handlers that allow you to resume at the point where you're through. but you can resume at any later time. And the thing, the continuation is essentially your object that allows you to resume. What effect handlers give you, as opposed to these other techniques, is that you can compose these handlers in a nice way and without them interfering with each other.

57:21Whereas if you have to manually do these transformations and you have more than one of these control effects in your language, then you have to do all of them kind of together. With effect handlers, you can do them in modular ways separate from each other. They don't have to know much about each other. You can just suspend over another handler, essentially. And that is very nice, for example, if you want to do both threads, so green threads, virtual threads, and say async await or generators, then one is in a much smaller scope, but you still want to be able to suspend the entire thread that runs the generator directly going to the outer one, right?

58:00Without the generator logic having to know anything about that. No, I think it's really interesting. We've had a number of conversations with different folks about kind of advances in the programming language world. And like we have these newer languages that are building on these capabilities that have really just happened in the last 10, 15 years. If you look forward even further, so that's what's on the horizon now, what else is in the back of your head of we're going to want to get there eventually, we're going to need to get there, what you want to enable with WebAssembly? Well, yeah, I mean, we've already got quite far as far as I'm concerned.

58:42So one thing I would like to see is WebAssembly having more built-in support, like in standard operating systems, for example. So it's in every browser, but why does, I don't know, macOS or Linux or Windows not support it out of the box, right? So that you can actually run just desktop applications using WebAssembly. So it becomes even more of a portable format. And the extreme point of that would be to put it into hardware. So I keep hearing people talking about that and considering the ideas. So far, as far as I'm aware, nobody has actually tried to do that for real. And there are some hurdles you have to get over, but in principle, it should be possible.

59:25And that might also be very interesting. Because it's a virtual CPU, ultimately, and at least some feature subset of it should be possible to implement more or less directly in hardware. So what you're saying is the next ARM spec or something like that is directly targeting WebAssembly? Well, not the ARM spec, but there might be a WESM processor, a WESM CPU of kinds, and maybe a custom chip or whatever, or FPGA or something that people who want to run that in certain environments can take advantage of. I don't know, something like that. Love it. Well, we're coming close to the end of our time here.

1:00:06Is there anything we haven't talked about yet that you think would be good to leave our listeners with? Yeah, I mean, one thing I could briefly mention is because I'm heavily involved with that side is the specification side of things with WebAssembly. So one thing we're particularly proud of is that not just don't we have undefined behavior at all, right? We have everything formally mathematically specified and verified. So we have machine verified proofs that we don't have undefined behavior. And there's a lot of work going into that and extending the kind of infrastructure for that now with various research teams that hook it up to various theory improvers to do even more fancy things of theory improving reasoning about programs and things like that, which I think is a whole new level for an industrial programming language that nobody has done before either.

1:00:59So that's something I'm particularly interested in as well, because that ties back to my academic self, half of myself that likes that stuff, too. But I think it's very valuable. And I mean, the reason that WebAssembly has such a clean and fairly safe design is because we design or one of the reasons, at least I would claim, is that we designed it with formalization hand in hand. We didn't just build this artifact and then try to make sense of it after the fact. We really did it at the same time. And that's when you kind of have to be honest with yourself when you're designing something and you have to formalize it.

1:01:40And the formalization becomes hard. And probably the design isn't quite right. So it's a nice feedback loop in the similar way that implementation is a nice feedback loop. You want to do all these things in parallel and feedback into the design. So both for semantics and performance and these things, which I would love people to explore more, also for other languages. And that would give us also the other nice thing about that is that this gives a way of actually doing verification in depth, right? So because we now have verified hardware, we have a verified engine like WebAssembly, and maybe we can now build more verified compilers, and then you can actually verify the entire software stack at some point.

1:02:24I would love to see that kind of thing because, frankly, the state of the art with our kind of craft is still like these things to be desired. And not to be too much on the bubble area, but if we're using non-deterministic machines to generate code, which is what more and more people are doing with LLM-based programming, having a way to deterministically validate and verify it after the fact is going to be ever more important. Right. Yeah, absolutely. Yeah. The more code we let be generated by dubious sources. I mean, humans are pretty dubious as well. I've read a lot of human written code that feels pretty dubious to me.

1:03:11Fair enough. Yeah. But for some reason, people are less willing to trust humans when writing code than they're willing to trust AI when it's writing code. There was just a study coming out I just read about today that is a bit scary. So when you're pair programming with a human partner, then you usually question everything he's doing. When you do that with an AI, not so much, although it's at least as failable. Formal validation of the outputs. Yeah. Definitely would be nice. But that's very much an open research area as well because it's a very hard problem in general. Awesome. Well, thank you.

1:03:51And let's wrap. Thank you very much.