CodeSandbox with Ives van Hoorne

28:39Yeah, I didn't reflect on that, but you're right. This simple approach, it solved all of it. And it is also the simplest approach. No fancy AI or detection heuristics. Before we move on to the client code runs the entire code sandbox portion of the DevStack, I have one last question on the back end. And at Codecademy, I always wanted to build something where if we did detect someone was crypto mining, we would let them and then steal their crypto coins and give them something fake in response just to really stick it to them. Ever tried something like that that you're willing to talk about? I have fantasized about this as well.

29:17Yeah, I did think about it. Never did it. There was a case where people did not just use it for crypto, but they were also and still are sometimes using it to watch advertisements and get money from it. It's very interesting. They start a browser inside a VM and then they have like a VNC connection to it so you can see the browser. And then they start on like 20 VMs all at once. They start going to different pages where they watch advertisements and then they get money back for watching those advertisements. It's very interesting. And I once, I found this, I found one of those VMs and the VM was just public so I could open it.

29:53And I saw the VNC window open in the preview. So I just started doing a lot of things inside that you can see, like I opened Notepad and I sent a message and I could see them at some point, like looking at that sandbox and getting very confused and closing all the windows because they felt like they were caught. That was funny. Yeah, the backend team for us, I hear they used to do things to mess with people to make it not worth their while to abuse the platform. It's a great way to stop people. it's similarly when we detected crypto miners for a while we just throttled their sandboxes their dev boxes so then they would feel like they would mine crypto and everything would still work everything would still run but they would only be running at five percent speed of what the full vm could run so that's another way to confuse them i guess i love it but okay let's move closer to the front end.

30:48So let's stick with the original code sandboxes. For now, your Postgres database is continuing to scale wonderfully. You've served up the file contents to the user. What happens in the user's browser now? Yeah, when we started code sandbox, we had a very low budget. So using a server was out of the question to run the code. Like we were students, we had a budget of like$100 a month, that was the max that we could have. And that was all from student loans in any case. So I started to look into whether I could run Webpack in the browser. Webpack was by far the most popular bundler at the time. And initially, I got Webpack to run, but the bundle size of Webpack itself was huge.

31:32It was like eight megabytes, which was even more for 2017. That was huge. So then I started to look into more how these different pieces of code are executed. And And I started to try to execute the code myself, like try to build a very simple version of Webpack that would run in the browser to execute the code. And in essence, it all revolves around eval, like the JavaScript function eval, where you can give it a string and it will evaluate that code. And everyone says that you should never use eval, but that's the core functionality of CodeSemble. That's what we built kind of the startup around.

32:10and what happens is we receive all the codes and the first thing we do is we parse all the codes. We try to understand which files are used and which files import which other files. This results in a dependency graph as they call it and with this graph we transpile all the codes. So you could have for example your source code could be TypeScript or it could be JavaScript but not suitable to run in the browser yet. So we would transpile that code to code that would be possible to run in the browser. And then later on, once everything is transpiled, we would execute that code by using eval. And these pieces of code, they could import other files.

32:51And they would do that using a require function call, the common JS way of running code. And what we would do is whenever we would eval the code, we would wrap everything in the function where we provide require as a function. So we would overwrite the require function with our own function. And whenever it would require, call require, we would resolve that code and we would run eval on that code, creating this kind of if you have infinite imports. And that is ultimately how it works. The logic itself is not extremely complicated. I also have given a talk and I think I've done a blog post about how this works.

33:31And I also have a sandbox that has like a mini bundler implemented. that is in a nutshell how it works it became more advanced because there are bigger also big challenges like for example how are you going to support node modules like installing dependencies alone everyone makes the joke that node modules are bigger than the universe so how do you make that efficient which is a challenge in of itself but the core functionality is essentially this loop of creating the dependency graph transpiling all those files in that dependency graph and then call an eval with a required override on the code.

34:07So code sandboxes are known for having beautiful visuals, right? How does that interact with the DOM then or the HTML page? Okay, so code sandbox, if you are looking at the editor page, you're actually looking at two applications. You're looking at the editor itself, that's the Create React App application. But the preview on the right, that's a completely different application that is rendered inside an iframe and that iframe refers to a different entry point so to say that entry point is the bundler so when we the editor downloads all the code from our postgres api server then the editor sends that code to the iframe of the preview iframe it would call post message to send like all the files to it the reason that we have to run everything in an iframe is twofold one is it isolates the user application completely from our editor so they cannot mess with our editor but the other reason is security because we ultimately are running user code so they shouldn't be able to access our cookies or our local storage those kind of things so the editor would call post message on the iframe send the user code to that preview inside the preview we would have the bundler sitting idle and listen for any message that comes in and when it would receive a message it would then do this loop of executing the code.

35:29And that would then ultimately fill up the preview. And whenever the user would change code, we would just send the whole bundle, like everything from the editor back to the preview iframe again. And the preview iframe would then, the bundler in that would make a diff. It would look at the current version of the code and the new version of the code, and it would compare everything. And then for every file that changed, it would reevaluate those files. It would just run eval on those files and its parents. So you're using, let me get this straight, eval, iframes, and window.postmessage. Yeah, yeah, that's the core functionality.

36:12Yeah, it sounds counterintuitive, right? Yes, but then on the inside, you're also building up a full dependency graph and doing dynamic reloads of the graph based on impacted changes from the editor. So you have this incredible mixture of pre-2017 era tech with very lovely computer science concepts for efficient redeploys. Yeah, that's the interesting part because you're building essentially two things. You're building a, I would not say hardcore, but like pure developer tool like Webpack. But you're also building a UI around it, like a developer experience around it. And that opens up some unique opportunities because you have the whole stack.

36:50You control the whole stack. So a simple thing that we did, for example, is whenever we would get an error inside the bundler that it could not resolve a dependency. Let's say you import Lodash, but you have not installed Lodash yet. We would create a specialized error message in the bundler. There would be a button saying, oh, we could not resolve Lodash. And then it would be a suggestion that you could click, which says install Lodash. And when you click that button, the bundler would post message back to the editor saying, we need to install Lodash. And then the editor would create, would show the UI of installing Lodash.

37:26It would install Lodash and call re-evaluate on the bundler. That is just one of those examples where because you control the whole experience, you can create a pretty nice experience. Another one was someone in our Discord was frustrated because they were using CodeSandbox for a job interview and they hadn't capitalized their components, their React components. And the React component was not working. I think it was back in the day when React components had to be capitalized to work. And because of that, they failed their interview. so as a response i created like a small detection heuristic on that too where if we detect that there's an error that's about like a custom component that's not a default html element and react would throw that error then we would show a suggestion like have you capitalized your component and then we would have a button capitalize component and when you would click it we would update the code to capitalize the components to catch those kind of things and that was the most exciting to me because it's a mix of like the core developer tooling building a bundler but there is also a developer experience and user experience that you can refine based on what comes back from that bundler did the person ever respond that they got a job afterwards or did that help them no correspondence after i didn't implement any analytics to know how often that button was pressed maybe no one has seen it afterwards but who knows just one clarification point for technical areas when you say install what is installing in the client yeah installing dependencies it has changed over time but the biggest challenge that was there was installing npm dependencies because npm dependencies tend to be pretty pretty big sometimes a library an npm dependency could be two megabytes and that's not a big problem in of itself but that dependency could also say if i have 20 other dependencies and those dependencies could also be 22 megabytes suddenly you have 80 megabytes of dependencies you have to download so we added support for npm dependencies by creating a separate service an aws lambda service and that Lambda service, we would say, I want Lodash, for example.

39:51And Lambda would install Lodash and then it would look at what files have a high probability of being required to run the dependency. So it would look at the entry point. So for example, Lodash would say, oh, my entry point is in source slash index. And we would then look at that file and we would look at all those imports in that file. And we would again go through the same process of creating a dependency graph. And we would make sure to only include the files that were required to run the main dependency. And we would send that also back as a JSON to the bundler itself. If the user then would require a file that was not included in this bundle, then we would manually download that single file from a service called Unpackage, which hosts the files for NPM dependencies.

40:39So when I say installing a dependency, the only thing the editor does is add, for example, Lodash to the list of dependencies. But ultimately, the bundler is the one that calls a service to download the files required for that dependency. And this is all cached under Redis? This is cached in S3. Yeah, it's something that came later. and it's very interesting how it works because working with dependencies itself is a very interesting challenge because what if, for example, if you install dependency A and it requires Lodash version 2 and then you install also dependency B and it requires Lodash version 3, how can you make that work?

41:27Or if you install dependency A, it requires Lodash version, any version that's in 2 and then you install dependency B and it requires Lodash, specifically version 2.5. How are you going to make sure that both versions get Lodash 2.5 to be most efficient in the download? So there are some very interesting challenges with building up this dynamic NPM install service. And so we don't just cache singular dependencies. We also cache combinations of dependencies, like, for example, React and React DOM. We cache that combined, that merged bundle. But we also cache React, React DOM, Lodash, and React icons.

42:05for example, as combined bundle, so that we don't have to recompute these combinations every time. So yeah, that bucket is huge. It's a couple terabytes, I think. So there's a benefit then to users using common combinations of packages. Like if I'm using React and React DOM the same way everyone else is, that's already pre-cached before I make that installation request. Yeah, for sure. If you, and very often people will use, I think React and React DOM is the most common combination. And all of this, this whole S3 bucket is cached. Cloudflare is in front of it. So because of that, we also have a lot of requests to S3.

42:47So dependency installations, it's very interesting. If you try to install, if you want to run a sandbox with a dependency combination that has been used by another user before, then installation will be within a second. It would just be a matter of downloading the file from the S3 bucket. It would already be pre-bundled. Sometimes we even do pre-transpilation on the source files to make it also faster to run so that the bundler doesn't have to do transpilation on it anymore. We would even store information for every file, like the dependency graph. We would also embed the dependency graph into dependencies to know which file imports which file.

43:26So the bundler would also save work. The bundler wouldn't have to do the parsing of all those files. that to run the code and that is the interesting thing you could see it as a super fast npm because the dependency combinations you are most likely using a dependency combination that has been used by someone else before and in that case you would just have to download that file from an s3 bucket okay so we've covered most of the stack we're nearing the end we've got database and correlated s3 and redis caches around it when i make an installation request you've got a whole bunch of heuristics and strategies there.

44:02That all gets sent to the clients in some kind of transpiled bundled form. And then you have the two apps on the client, the editor and the preview or viewer. Whenever the editor causes the file to be changed, we post message over to the iframe to eval the code as necessary. I think the last bit I want to touch on is the editor itself, because that on its own is an entirely separately complex and interesting application. The code editor that works in the browser, how does that work? I would argue that's the most complicated part of that specific stack. The first version of the editor was very simple.

44:34It was all React components. It was like the File Explorer were React components. And the editor itself, the code editor was CodeMirror, which is a library created by a Dutch person. Later on, we would move our code editor to Monaco. Monaco is a little piece of VS Code. VS Code has an incredible code base. It's one of the best code bases I have ever seen. It has really good composition. It has really good APIs. And you can already see that it's so good because they were able to extract the core editor from VS Code and make it a library and keep it in sync with VS Code relatively easily. So Monaco is that.

45:21It's essentially the VS Code editor, but then exposed as a library. It doesn't have all the fancy things from VS Code, like extension support or a file explorer or all those. It just has the editing part, but it works and it works really well. So we moved to Monaco because it had better, well, it had a more familiar experience for everyone using VS Code. And when Code Samo started, Atom was the most popular editor, but later on VS Code became the most popular editor. But there was always a case where people were wondering if they could do the same thing as VS Code. And code editor is an incredibly complicated piece of technology.

45:56You have things like the command palette. You have key bindings. You have themes because developers, they want themes. You have settings for all the little things like line height, font size, font itself, how the editor should behave when you press alt and move your cursor around. How should multi-cursor work? All of these, there were a ton of settings. And then you have extensions, where extensions alter the editor behavior or add new editor behavior. So Code Editor is one of the most ambitious UIs to ever work on. And it's an incredible thing to work on. You learn a ton. Oh, and performance.

46:35Just making it performant, like when you click on a file, that it can open that file within 10 milliseconds. Incredibly hard. Initially, we were doing everything with Monaco. We had our own UI built around it. And Monaco was the core editor, but we had our own UI. And because of that, we were able to create a lot of custom experiences. But later on, I started to become more and more interested in VS Code. And in 2018, during my Facebook internship, I was looking at a way to make VS Code run in the browser. VS Code didn't run in the browser back then. Initially, VS Code was built to run in the browser, but it was not performance enough.

47:11So they made VS Code an Electron application. So it used web technologies to render the UI of VS Code, but it did not run in the browser because it did use things like Node file system, a lot of Node APIs. So I was looking if we could use VS Code Semos because that would save a lot of work. It would create familiarity for developers, like their own custom key bindings, those kind of things. We would start to allow extensions. We would be able, it would be faster because ultimately VS Code is much faster than React. mostly because VS Code is imperative instead of declarative. It would have a ton of benefits.

47:48So making VS Code run in the browser ultimately came down to emulating Node in the browser because Node ultimately is JavaScript with native Node modules. So for example, File System, Net, HTTP, OS, all of these Node modules. If you create a browser equivalent version of that that would run in the browser, then essentially you're re-implementing nodes and you're tricking VS Code in thinking that it's running on your local computer, but it's running in the browser. And so that's initially what I did to make VS Code run in the browser. Later on, the VS Code team actually ported VS Code back to the browser.

48:26And now with Code Sandbox, we're running the browser version of VS Code. We're still emulating some node components to run extensions sometimes in the browser but if there is for example a vm then we use vs code server so that you have the native vs code experience but we do have a little add-on that we put on vs code that allows us to render react components inside the vs code ui and that allows us to create custom experiences still using react but within the context of vs code so how does that work with type information say i want to hover my mouse over Lodash, import it from Lodash and see what are the methods available under it?

49:08Yeah, that's a very interesting question. So if we are looking at the DevBox version, it's quite simple because VS Code server gets the files from the file system. And well, in fact, the extension, the TypeScript extension runs on the server itself, and it would read the files. That's kind of like a solved problem. But for sandboxes that completely run in the browser, we have a bit of an interesting solution. We run VS Code extensions in the browser. Those VS Code extensions expect to run in a node environment. They expect to have. When they do require FS, they expect to have the FS module available.

49:45They want to be able to run read file, read file, read deer, write file, all those kind of things. So the first thing that we did was implement file system, but then to run in the browser. So we can run VS Code extensions in the browser. They will think that they are in a Node environment, but they're running in the browser. And the second thing is, whenever you install a dependency like Lodash, we do the same thing that we essentially did for the Bundler. We have a service that calls npm install for Lodash. And then it looks at all the types files that are installed. And it will create like, again, a JSON bundle that just only includes all the type files.

50:29So only the.ts files or the.ts files. And then in the browser, we would download this bundle and we would write all of those files into our in-memory file system, as we call it, for this fake file system that we've created in the browser. And then when the TypeScript extension would run, it would, again, think that it's in a Node environment and it would read files from Node modules. And those files, those are all coming from this in-browser file system that was populated from this dependency packager. Is that a different service, just to clarify, from the runtime JS file service? Yeah, yeah. But it shares a lot of code.

51:06I essentially copied the code from the dependency packager and then changed some things to make it more fitting for typefetching. I have to say that's how it works for the first six years. And now recently I've made a change that we actually download the tar file directly from npm. Dowsers have become better and better. And at this point, unpacking a gzip tar file is really fast in the browser. So for a lot of dependencies, we don't go to our own service anymore. We literally download from the npm registry directly the tar file and we unzip it in memory and then we save it. It's a bit less efficient, but we had to do it because half of our bandwidth usage was just from this typefetcher.

51:54And Cloudflare told us that we had to go on a more expensive plan. And I was thinking, well, what if we then just download directly from the NPM registry, then we don't pay for the bandwidth of those bundles. And the funny thing is, because of that, we significantly reduced our Cloudflare bill. But Cloudflare backs the NPM registry. So Cloudflare itself doesn't notice a difference in bandwidth. In fact, the bandwidth has increased because now tar files, they don't create these perfect bundles of typings. But at least we don't get built for that bandwidth. Microsoft owns NPM. If they want to reduce that, they can always add a feature to NPM to help you out.

52:33Yeah, that's true. That's true. It was an interesting thing. And unpacking a gzip tar file definitely would be super inefficient back in 2017 or back in 2018. But nowadays you have, I think it's a browser feature called decompression stream or something, which makes it super fast because ultimately it uses the native decompression of the computer. It is really beautiful how the browsers have made things like this possible. And more and more over time, more and more of your stack is kind of becoming built in. True. I would even say that if I would rebuild CodeSandbox today, I would probably not build the Bundler as it is today.

53:10Because right now everything is based on CommonJS. with this require override. I think if I would build it today, I would use the ES module capabilities of the browser and then perhaps use a service worker that acts similar to a Vite dev server. It would, whenever you try to download a file, a JavaScript file, our service worker would maybe transpile that file and give it back. But then the execution and bundling is done by the browser. It's not done by us anymore. We would not even be using eval. Yeah, it would essentially, it would be fully ESM. It would be like a Vite. It would literally be an in-browser version of Vite.

53:47If I would rewrite everything, I would do it that way because it's much more native to the browser and the world is moving to ESM in any case. Eventually. Right, eventually. It'll take a while. Do you think you'll ever have time to try out, say in a few years, the fully browser bundle and transpiler for CodeSandbox? Yeah, if no one else has done it, that's also the big question because i think that's a big opportunity and i can understand if people would would explore it right now the bundler as it works today works really well i'm a bit worried that if i would rewrite it completely using this that we would introduce bugs it would still be an interesting experiment yeah it is something that i would want to explore i had it in my mind for a while like but i'm not sure when i will explore it what's another long-term exploration that you would love to get to if and only if you had time what i find very interesting about this whole firecracker stack that we now have for development environments is i think the the fact that we can clone a vm within a second that we can resume a vm within a second that's incredibly powerful technology that is not just applicable to development environments, but also CICD systems, even to deployments.

55:09Yeah, I think there's a whole plethora of different use cases for that technology. I would love to generalize this infrastructure so that people can use it for other purposes that go beyond the cloud development environments. Yeah, I think it's incredibly powerful. It could be used for so much more. So that's something that I would find very interesting, like making it so that the infrastructure that we have today, we can open source it, that we can make it generic enough that people can use it for their own use cases. A lot of companies, a lot of projects have CI, CD times where you have whatever, let's say a half dozen tasks.

55:45Each of those tasks spends 20 seconds in NPM or PNPM or Yarn install, and then three seconds in the task itself. It's very frustrating. It's very wasteful. Yeah. And this technology, I was thinking you could, for example, one thing that you could do is you could do all the preparation that needs to be done for CICD and then create a snapshot. And whenever you need to do a CICD run, it will continue exactly from that snapshot. It will just then pull the latest code and then run the tests. But also for parallelization, like if you, for example, want to run a test suite across 12 different workers, what you could do is you could create a snapshot, you could do all the preparation work on one VM.

56:22And then after that, you clone this VM 12 times. And all these 12 VMs, they run a different part of the test suite in parallel. And one of the things that we even support, or we have never deployed it, but it's an experiment that I built, is that you can do cloning of VMs over the network even. So you have a VM running on one machine, and then another machine starts a clone of that VM, and they use a network to sync the memory between the VMs. That's also a capability. Then you could really think, like if you have like a CICD cluster that is like, I don't know, 20 servers big, you could do the preparation on one machine and then clone that machine to all the other 20 machines and they run all 1 20th of the test stack using the result of the preparation that the single server has done.

57:13And it's actually worth it, both time and compute to do it that way. Yeah, I'd say so. It's just an idea. Yeah, it's like you probably will find things that don't work in practice, but that always happens when you try things. I also wonder, for example, there's now a ton of research being done in how to efficiently train AI models using hundreds of servers at the same time, because ultimately that's the biggest challenge. Like how can you scale so much, so much compute and data transfer in a data center? I think those challenges that they have, they are not the same, but they are in a similar vein, which is, it's an interesting, it's an interesting space, at least.

57:59How soon will I be running my CI jobs on CodeSandbox then, given all your caching? I think not soon. Well, it would be good to explore like different use cases and work on the different capabilities for the stack itself. Now, in fact, you could run your CICD today already. It's just that the platform has not been built for it, but we can expose an API or you can even create a dev box that runs CICD. Let's say just next year. Then next year you can ask me, can I run CICD? And then I will say, yes, we have this API. And then you can give it a Docker image and it will run your code. We'll follow up in a year.

58:39Same with the AI models. This has already run longer than our typical CICD interviews because you've got so much fascinating tech to talk about. I want to end on a less technical, perhaps still fascinating note on your personal points. Can you tell us a little bit about volleyball and why it's such a fantastic game to play? Yeah, I mentioned this before. In my free time, I do a lot of volleyball. Volleyball is something that I started doing when I was like, I don't know, nine years old or something and did it a lot. During high school, I did it an extremely amount of lots if that would be worked.

59:18I think I trained like six times in a week. I got a bit burnt out on it during high school. I was a bit too, I guess, a bit too competitive. So I stopped for a while and didn't do much volleyball. But I recently picked it up again, I think a year ago. Yeah, about a year ago. And it's so incredibly good to play volleyball because it's when I, during the training or during the match, I completely forget everything. like everything surrounding like I could be worrying about something but when I play volleyball I don't worry about anything I just worry about the ball so it's very good from a psychological vantage point but also I'm much fitter now and the day after I feel much more energized because of the sports so yeah I'm a big fan of volleyball it's kind of a reminder for me that it's important to not just do coding or meetings the whole day.

1:00:12Ultimately, it's also important to stay healthy and exercise. I think a lot of people maybe did high school volleyball, just hanging around in gym, maybe have done some casual beach volleyball with friends. What is different about your areas and brands of volleyball that we're not just standing around awkwardly waiting for the ball to come near us? Oh, so you mean, what makes it fast? What makes it a good workout? You know, what are you doing when i play volleyball i i'm mostly found on the ground i think there's a lot of diving going on like if you want to get to a ball and you it's just out of reach you just jump and recently i started to also um track my heart rate during volleyball to see like how intense it is i've now done it two times and my heart rate goes up to 195 when uh when i play volleyball it's really especially at the net it's also you jump a ton because for example when we play volleyball if i play mid the thing you do is whenever the setter has the volleyball like whenever the ball goes to the setter you have to jump and fake an attack so you always jump when the setter has the ball because that way you confuse the opponent about where the ball will be attacked so that you confuse the blockers so you can jump like in just one rally you can jump like two or three times and then you have like 40 rallies and that's four times 160 times yeah you you jump like 350 times or something in a single match it's very intense in that sense it's very explosive there's no long running but it's it's it requires a lot of short bursts of energy that sounds very intense it's also very psychological you're not turning your brain off you're still thinking.

1:01:58Yeah, it's true. There's a lot of prediction going on. Like you can confuse your opponents, they can confuse you. But I like what I like the most is the competitive aspect of it, like winning matches or losing matches too. That makes it a lot of fun compared to, for example, running. I also do a lot of running nowadays. But for me, something is only when it comes to sports, I find something mostly exciting when it's a competition that keeps me in it, I guess. Great. Well, this has been an absolutely phenomenal interview. Ivis, thank you so much for talking us through the start of CodeSandbox, the backend, the database, the caching layers, the frontend.

1:02:34This is a lot of stuff. If folks want to find out more about you and our CodeSandbox on the internet, where would you direct them? My Twitter is where I'm most active, I would say. That's Compuives, C-O-M-P-U-I-V-E-S. That's my Twitter. and CodeSandbox can be found on CodeSandbox.io or CodeSandbox.com or CodeSandbox.org. That's a small tidbit. We went with CodeSandbox.io because the domain was only$30 and CodeSandbox.com was$3 ,000. But then later on, a lawyer who learned how to code using CodeSandbox gifted the CodeSandbox.com domain to us. That's how we ultimately got the CodeSandbox.com domain.

1:03:18Well, all that being said, This has been Josh Kulberg and Ivis von Horta with Software Engineering Daily. Thanks, all. Cheers. Thanks.