WEBVTT NOTE This file was generated by Descript 00:00:13.855 --> 00:00:15.565 This is Self-Directed Research. 00:00:15.695 --> 00:00:19.428 Our hosts, James and Amos, get hyped about different topics, and take turns each 00:00:19.428 --> 00:00:21.198 week presenting their ideas to each other. 00:00:21.595 --> 00:00:24.355 You can check out the website, YouTube, or Spotify to watch this 00:00:24.385 --> 00:00:29.225 episode's presentation, and visit sdr-podcast.com/episodes for 00:00:29.225 --> 00:00:33.125 previous episodes, presentations and videos, show notes, and transcripts. 00:00:33.445 --> 00:00:35.415 New episodes are published every Wednesday. 00:00:36.315 --> 00:00:39.905 James is on again this week, this time talking about "Async 00:00:39.906 --> 00:00:41.435 Allocators," but first, a quick note. 00:00:41.632 --> 00:00:43.092 This episode is sponsored by CodeCrafters. 00:00:43.092 --> 00:00:49.222 Visit sdr-podcast.com/codecrafters for more information or to sign up today. 00:00:49.412 --> 00:00:51.792 Or stay tuned to the end of the episode for more information. 00:00:58.094 --> 00:01:00.446 So now that Amos has raised the blast shield, 00:01:01.014 --> 00:01:02.154 Yes, I'm good to go. 00:01:02.584 --> 00:01:03.034 I think. 00:01:03.544 --> 00:01:04.857 Amanda: Good to go. 00:01:05.427 --> 00:01:06.437 Your poor mic arm. 00:01:07.037 --> 00:01:07.767 Yes, exactly. 00:01:07.767 --> 00:01:11.257 That was, I think as well, I can hide now. 00:01:11.257 --> 00:01:11.887 This is great. 00:01:11.957 --> 00:01:13.417 I can act as a dungeon master. 00:01:14.748 --> 00:01:16.608 So I'm going to talk about something that's been... 00:01:16.668 --> 00:01:18.948 I talk about this a lot with a friend of mine, Eliza. 00:01:19.150 --> 00:01:24.020 This is one of those ideas that's so close to being good, but all the like real world 00:01:24.020 --> 00:01:27.270 realities of it make it obnoxious to use. 00:01:27.590 --> 00:01:33.490 But my pitch today is that making allocators async is a bad great idea. 00:01:33.900 --> 00:01:37.350 Not a great bad idea, but a bad great idea in that it is a great 00:01:37.350 --> 00:01:39.540 idea, but it is practically bad. 00:01:40.185 --> 00:01:41.835 Mhm, mhm, I'm listening. 00:01:41.985 --> 00:01:44.865 So at least in Rust, there's like two... 00:01:46.435 --> 00:01:48.715 you know, if you come from C, there's like malloc and free. 00:01:49.075 --> 00:01:53.225 C++ has something that I don't know, but in Rust, there's like the classic 00:01:53.245 --> 00:01:57.455 interface, which is the GlobalAlloc trait, which I'm showing on my screen, 00:01:57.535 --> 00:02:00.505 but it's got two main interfaces. 00:02:00.505 --> 00:02:03.855 Alloc, where you give it a layout, like how big and what's the 00:02:04.135 --> 00:02:06.665 padding or alignment of a type. 00:02:07.115 --> 00:02:12.245 And it infallibly gives you back a pointer to a byte, like a byte 00:02:12.245 --> 00:02:14.255 pointer, sort of like how malloc works. 00:02:14.425 --> 00:02:17.985 And then there's dealloc, which is an unsafe function where you give it that 00:02:17.985 --> 00:02:21.425 pointer and the layout again, because when you dealloc, you need to have the 00:02:21.425 --> 00:02:24.570 same layout when you unallocate something. 00:02:25.030 --> 00:02:26.850 And these are sort of like the core fundamentals. 00:02:26.850 --> 00:02:29.720 There's a lot of like things that build on top of this, but essentially 00:02:29.720 --> 00:02:33.460 like, give me a chunk of memory this big and with this alignment. 00:02:33.780 --> 00:02:36.530 And I am no longer using this chunk of memory that is 00:02:36.540 --> 00:02:38.160 this big and this alignment. 00:02:38.530 --> 00:02:43.460 This is sort of like the foundation of alloc and dealloc in Rust today. 00:02:43.840 --> 00:02:46.170 I just want to mention that the reason `dealloc` asks for 00:02:46.170 --> 00:02:49.600 the layout as well is so that you don't have to store it inline.. 00:02:49.600 --> 00:02:54.490 "in-band", I guess, in the memory itself, unlike C `free`, right? 00:02:54.904 --> 00:02:55.124 Yeah. 00:02:55.124 --> 00:02:57.904 I think most allocators do that. 00:02:57.904 --> 00:02:58.064 Yeah. 00:02:58.064 --> 00:03:02.064 Like they shove the metadata of the allocation in the allocation. 00:03:02.064 --> 00:03:03.654 It's just hidden from the user. 00:03:03.834 --> 00:03:06.384 If you were to take a pointer returned by `malloc` and you'd 00:03:06.384 --> 00:03:10.144 look like a few bytes before, then you would not get a segmentation fault. 00:03:10.154 --> 00:03:11.824 You would just get the metadata for that allocation. 00:03:11.864 --> 00:03:14.484 I'm pretty sure, but that's highly implementation-dependent. 00:03:14.504 --> 00:03:14.804 So... 00:03:14.934 --> 00:03:15.414 so, don't. 00:03:15.584 --> 00:03:16.544 It's an impl detail. 00:03:16.544 --> 00:03:16.864 Yeah. 00:03:16.904 --> 00:03:17.244 Yeah. 00:03:17.424 --> 00:03:21.994 So that's today's implementation of it, but there's some stuff missing about it. 00:03:22.004 --> 00:03:24.994 And I'm going to show you the unstable, but interface that 00:03:24.994 --> 00:03:26.174 I like a little bit better. 00:03:26.464 --> 00:03:28.314 And this is the Allocator trait. 00:03:28.324 --> 00:03:32.084 So not the GlobalAlloc trait, but the unstable Allocator trait. 00:03:32.184 --> 00:03:37.764 Which has two very similar methods: alloc or allocate, instead of just returning 00:03:37.764 --> 00:03:44.831 a pointer, it returns a result NonNull slice of u8 or an allocation error. 00:03:44.831 --> 00:03:50.859 And the deallocation takes a NonNull u8 and a layout. 00:03:51.134 --> 00:03:53.774 So I know that fallible allocation was a requirement for 00:03:53.774 --> 00:03:55.804 the Rust in the Linux kernel people. 00:03:56.034 --> 00:03:58.641 Did the Allocator trait come from there or was it there before? 00:03:59.149 --> 00:04:02.559 I don't know where it came from, but it's definitely to address those 00:04:02.569 --> 00:04:09.034 kinds of things, because at least in like userspace programs, you most of the time 00:04:09.034 --> 00:04:10.944 assume that the allocator will never fail. 00:04:11.184 --> 00:04:14.605 These days it's very rare to be out of memory. 00:04:14.615 --> 00:04:15.735 And this is for two reasons. 00:04:15.745 --> 00:04:17.165 One, we have a lot. 00:04:17.555 --> 00:04:19.585 And two, your operating system lies. 00:04:19.795 --> 00:04:23.835 It will say that you have an allocation and it will do things 00:04:23.835 --> 00:04:25.245 like swap memory in and out. 00:04:25.275 --> 00:04:28.480 It will do things like, tell you that you have allocated it, but 00:04:28.480 --> 00:04:30.210 not actually wired up that memory. 00:04:30.450 --> 00:04:34.960 So if you don't touch it, it lazily allocates basically, like it lies and 00:04:34.960 --> 00:04:38.470 says that you have memory, but you don't until you actually touch it, and then 00:04:38.470 --> 00:04:40.450 it gets wired up, and things like that. 00:04:40.907 --> 00:04:46.907 And it can also lie in time in that when you call alloc, if you allocate and that 00:04:46.907 --> 00:04:51.407 memory isn't available, it might pause the whole world, like basically unschedule 00:04:51.407 --> 00:04:56.177 your program, set up that memory for you, and then come back and give that to you. 00:04:56.237 --> 00:05:01.277 So it can lie in a bunch of different ways that aren't exposed in the old interface. 00:05:01.287 --> 00:05:04.837 It just says like, "You will immediately get back a pointer of memory." 00:05:05.257 --> 00:05:09.227 I think technically the old Allocator trait is allowed to return a null pointer 00:05:09.237 --> 00:05:15.262 if you are out of memory, but it's a weird thing and not everything handles it well. 00:05:15.492 --> 00:05:18.252 But like the Linux kernel itself, because there is no operating system 00:05:18.302 --> 00:05:24.202 underneath Linux to lie to Linux, it has to like expose this reality. 00:05:24.442 --> 00:05:28.592 So this is one of those like abstractions that you pretend don't exist, but at some 00:05:28.592 --> 00:05:31.902 point you have to recognize that they exist once you start working on operating 00:05:31.902 --> 00:05:34.702 systems or things of, of similar caliber. 00:05:35.648 --> 00:05:38.368 So we talked a little bit about what it means to be out of memory. 00:05:38.764 --> 00:05:42.384 On a desktop, usually like when a desktop runs out of memory for real, like when 00:05:42.384 --> 00:05:44.974 Linux says, "Okay, finally, I'm giving up. 00:05:45.344 --> 00:05:48.854 You asked for memory that I could never give you, or there's too much going on 00:05:48.854 --> 00:05:50.344 and I've decided you're not important." 00:05:51.020 --> 00:05:54.760 Usually the reason why you also don't observe out of memory on Linux 00:05:54.770 --> 00:05:58.080 a lot of time is by the time the operating system says 'You are out of 00:05:58.080 --> 00:06:00.260 memory,' it just kills the program. 00:06:00.460 --> 00:06:03.900 Like, the program doesn't get to see 'I'm out of memory,' you get 00:06:04.050 --> 00:06:05.750 killed by the operating system. 00:06:05.872 --> 00:06:09.996 Or you get a signal that most programs don't handle, which kills you, because 00:06:09.996 --> 00:06:11.466 you have an unhandled signal, essentially. 00:06:11.476 --> 00:06:13.256 It's like having a segfault, basically. 00:06:13.696 --> 00:06:17.006 So, this is something that's like, extremely rare. 00:06:17.740 --> 00:06:20.150 The good default is to just kill the program. 00:06:20.380 --> 00:06:22.020 There are some things that don't like that. 00:06:22.040 --> 00:06:25.240 If you're running like a database or something like that, you might decide, 00:06:25.240 --> 00:06:28.130 'Hey, I don't like that,' and so I might handle that signal and there 00:06:28.130 --> 00:06:32.463 are some hooks for that, but it's a 99 percent of programs don't think about 00:06:32.463 --> 00:06:36.903 this ever, because it's challenging to actually handle in a reasonable way. 00:06:37.173 --> 00:06:40.673 So just giving up is actually a pretty good outcome for this, especially 00:06:40.673 --> 00:06:43.093 on desktops where you've got virtual memory and stuff like that. 00:06:43.198 --> 00:06:43.448 Yeah. 00:06:43.448 --> 00:06:46.555 And it's assumed that you're always going to have some breathing room. 00:06:46.645 --> 00:06:49.985 I forget what the technical term is called, but especially on servers 00:06:49.985 --> 00:06:52.545 and whatnot, you never aim for a hundred percent utilization, right? 00:06:52.545 --> 00:06:55.105 You want somewhere to grow up to realize- Oh, you need to scale up. 00:06:55.875 --> 00:06:57.035 So you always want some headroom. 00:06:57.915 --> 00:06:58.625 Yeah, exactly. 00:06:58.635 --> 00:07:00.225 Headroom or overhead, exactly. 00:07:00.785 --> 00:07:03.618 Where this is less true is on embedded systems. 00:07:04.288 --> 00:07:08.189 And that's because embedded systems have a lot less memory, they don't typically 00:07:08.189 --> 00:07:12.719 have an MMU, so they're not like, remapping memory that could get paged 00:07:12.739 --> 00:07:16.989 out or swapped out, because you don't have a disk to swap to a lot of the time. 00:07:17.649 --> 00:07:20.939 You don't have an operating system that can always pause the world and lie. 00:07:20.959 --> 00:07:24.159 And you might not even have threads that can be paused. 00:07:24.169 --> 00:07:27.589 There might just be the thread, which is the entire program. 00:07:28.099 --> 00:07:33.149 But you might still have cases where you only have so much memory or 00:07:33.279 --> 00:07:36.429 the memory is a little fragmented and you might make a request for 00:07:36.429 --> 00:07:39.069 memory that the system can't honor. 00:07:39.506 --> 00:07:43.723 But in my opinion, in embedded, these cases tend to be a lot 00:07:43.723 --> 00:07:47.873 more ephemeral than on a desktop. 00:07:48.233 --> 00:07:51.493 There might be a case where right now you don't have enough 00:07:51.503 --> 00:07:53.103 free contiguous memory for that. 00:07:53.463 --> 00:07:56.803 But if you were to wait like 20 milliseconds until you're done drawing 00:07:56.813 --> 00:08:00.793 the display, all of a sudden the display is going to drop that frame buffer, 00:08:00.813 --> 00:08:02.543 and that memory becomes available. 00:08:03.093 --> 00:08:05.903 Where in the same way that the operating system could just go, "We're gonna 00:08:05.903 --> 00:08:07.933 pause this thread for a little bit. 00:08:08.283 --> 00:08:11.813 Okay, now 50 milliseconds later, memory did become available. 00:08:12.613 --> 00:08:15.053 Free up that memory, give it to this other one, unpause 00:08:15.063 --> 00:08:16.263 the world," you're good to go. 00:08:16.623 --> 00:08:19.505 But that had to be done by the operating system. 00:08:19.525 --> 00:08:24.955 It's not in-band control, it's out-of-band control by the operating system. 00:08:25.785 --> 00:08:25.935 Yeah. 00:08:25.995 --> 00:08:27.655 Plus there's no mechanism for timeouts. 00:08:27.665 --> 00:08:31.065 It just blocks and then the OS decides how long it blocks, if it 00:08:31.065 --> 00:08:33.145 ever unblocks and fails and whatnot. 00:08:33.155 --> 00:08:37.495 But I can imagine, which is surely in the next few slides, that if you have 00:08:37.495 --> 00:08:41.025 an async method in the future, which you can drop, you can combine with a 00:08:41.045 --> 00:08:44.875 timeout to actually decide when you want to give up and say, "Okay, well, I 00:08:44.885 --> 00:08:46.283 guess I'm not using memory after all." 00:08:46.542 --> 00:08:47.092 Exactly. 00:08:47.432 --> 00:08:51.582 So this is an experiment that I did a while back for the operating system that 00:08:51.582 --> 00:08:54.412 I've been hacking on, Mnemos, where: 00:08:54.432 --> 00:08:58.192 What if the signature for allocation and deallocation look like this? 00:08:58.282 --> 00:09:02.742 We have a function called alloc that takes a layout, an async function that 00:09:03.222 --> 00:09:06.342 takes a layout and returns a NonNull u8. 00:09:07.142 --> 00:09:09.322 Notice that, it's, again, there's no error. 00:09:09.562 --> 00:09:12.992 So this is taking a layout, but it's not returning a result. 00:09:13.497 --> 00:09:15.207 It's just returning a NonNull. 00:09:15.526 --> 00:09:22.076 And we have a non-async function called deallocate that takes a pointer. 00:09:22.126 --> 00:09:24.086 It probably should be a NonNull, but takes a pointer. 00:09:24.416 --> 00:09:27.366 And also the layout so it can do deallocation. 00:09:28.011 --> 00:09:34.081 This is for exactly the reasons you said, because in async, you can combine futures. 00:09:34.091 --> 00:09:38.941 So I could have a future for allocation and select on it with a timeout that 00:09:38.941 --> 00:09:40.711 says, "Okay, we'll try and allocate. 00:09:40.731 --> 00:09:45.811 And if no one gets me memory in a hundred milliseconds, then just give up and 00:09:45.811 --> 00:09:50.381 either do something else or send a NAK If you're waiting on like packet memory 00:09:50.381 --> 00:09:55.791 or something like that, you can choose in-band- or in userspace or however you 00:09:55.791 --> 00:09:57.731 want to call it- how to respond to that. 00:09:58.439 --> 00:10:01.209 Now, one thing you'll notice is that the dealloc isn't async. 00:10:01.259 --> 00:10:05.914 And this is because at least in Rust, drop is not async, or at least today and for 00:10:05.924 --> 00:10:10.994 the near future is not async, which means if you were to drop memory, you need to 00:10:11.204 --> 00:10:13.024 deallocate that memory when it's freed. 00:10:13.734 --> 00:10:19.904 Essentially, there's no way to have an async drop today, which is annoying, but 00:10:19.914 --> 00:10:22.014 not necessarily the end of the world. 00:10:22.154 --> 00:10:25.694 So I know there's a lot of controversy around async drop and 00:10:25.754 --> 00:10:29.994 let's not get into it, but I keep forgetting how would we even make it work? 00:10:31.214 --> 00:10:35.224 And I think the answer is we, we wouldn't mostly, but in this specific 00:10:35.224 --> 00:10:39.364 case, when thinking about memory in an embedded context and deallocating, 00:10:39.404 --> 00:10:41.484 would you ever want that to be async? 00:10:41.494 --> 00:10:44.364 What kind of network banner or something operations would you 00:10:44.364 --> 00:10:45.464 do when deallocating memory? 00:10:46.076 --> 00:10:47.986 I can tell you how I handled it. 00:10:48.646 --> 00:10:52.486 And the answer is: When you allocate memory, your allocator can choose that 00:10:52.486 --> 00:10:56.626 there might be some minimum amount of space that an allocation can be, just 00:10:56.636 --> 00:10:59.446 from the implementation of the allocator or for whatever reason it feels like. 00:10:59.636 --> 00:11:02.666 If you ask the allocator for two bytes, it might give you 16. 00:11:03.086 --> 00:11:07.366 It will tell you that they are two, but it really has given you 16 00:11:07.376 --> 00:11:09.916 bytes and just lied a little bit. 00:11:10.956 --> 00:11:16.596 So what I did when you allocate, every allocation is the minimum size of 00:11:16.596 --> 00:11:21.524 what you asked for, and enough room for an intrusive linked list node. 00:11:22.164 --> 00:11:25.354 So essentially a pointer header, and a layout. 00:11:25.384 --> 00:11:31.440 So I actually have a union of whatever allocation you asked for, and a linked 00:11:31.440 --> 00:11:33.750 list node, and a layout in there. 00:11:34.270 --> 00:11:38.630 And so what happens when you call dealloc, I run the destructor on the 00:11:38.630 --> 00:11:43.640 memory, so essentially now that memory is whatever it was in the allocation before. 00:11:44.260 --> 00:11:48.160 And then I replace what was there with a linked list node, and 00:11:48.160 --> 00:11:49.760 the layout of the actual type. 00:11:50.200 --> 00:11:54.830 And then I link the node into essentially a free list to the allocator. 00:11:55.270 --> 00:11:59.280 And because it's an intrusive linked list, it can be infinitely lengthed 00:11:59.430 --> 00:12:01.660 because essentially it's like a linked list of things like that. 00:12:02.000 --> 00:12:06.250 And then the next time that you call alloc or at some periodic garbage collector- not 00:12:06.250 --> 00:12:10.740 garbage collection, but like maintenance window- it just starts draining the link 00:12:10.740 --> 00:12:16.680 list of free items and then repopulating the allocator table with those items. 00:12:16.870 --> 00:12:20.886 Which means you can always free immediately at the cost 00:12:20.886 --> 00:12:24.863 of the minimum allocation size being essentially three words. 00:12:24.873 --> 00:12:30.613 It's like the size, the alignment, and then the pointer to the 00:12:30.623 --> 00:12:31.873 item, or something like that. 00:12:31.873 --> 00:12:36.313 So it ends up being like three words, which might be like 12 or 24 bytes 00:12:36.313 --> 00:12:38.013 depending on the system that you're on. 00:12:38.453 --> 00:12:41.243 That's an implementation detail and just how I worked around it. 00:12:41.243 --> 00:12:44.383 There's nothing like intrinsic of how you have to do it, but that's how I got 00:12:44.383 --> 00:12:48.408 around of: Having to worry about what happens if you drop everything at once, 00:12:48.428 --> 00:12:49.768 how do you have enough room to store that? 00:12:49.958 --> 00:12:52.028 And the answer is, well, you're already allocating chunks of memory, 00:12:52.028 --> 00:12:55.698 so just make sure that they are big enough to recycle themselves. 00:12:55.918 --> 00:12:59.668 Which was a very fun and clever trick, and a lot of very fun, unsafe code. 00:13:00.498 --> 00:13:01.238 I can imagine. 00:13:01.238 --> 00:13:02.088 I like that trick. 00:13:02.088 --> 00:13:03.538 I didn't know about it before. 00:13:03.838 --> 00:13:07.348 I don't think you specifically answered my question though, which is: because 00:13:07.378 --> 00:13:09.346 that's not actually asynchronous, right? 00:13:09.756 --> 00:13:13.510 Putting it in the, the potentially infinite linked list of available 00:13:13.510 --> 00:13:14.550 space, recently free space. 00:13:14.600 --> 00:13:15.640 It's not actually asynchronous. 00:13:15.640 --> 00:13:18.850 So when would you actually need to do something over in the network or 00:13:18.850 --> 00:13:20.980 like some, I don't know, some desk IO. 00:13:21.010 --> 00:13:22.250 I'm still thinking in terms of desktop. 00:13:22.250 --> 00:13:22.420 Okay. 00:13:22.420 --> 00:13:25.150 My brain is desktop wired and your brain is microcontroller wired. 00:13:25.150 --> 00:13:25.420 So... 00:13:26.045 --> 00:13:27.495 Yeah, the answer is I don't know. 00:13:28.155 --> 00:13:31.315 The funny thing about my drop list is it's almost exactly how 00:13:31.790 --> 00:13:35.050 ready lists work for async tasks. 00:13:35.300 --> 00:13:39.560 So the issue with drop right now is just the function itself in Rust is not async. 00:13:39.850 --> 00:13:43.070 There's no way to pass it the context of the executor you're running on. 00:13:43.200 --> 00:13:47.230 It has no way of like spawning a task that does the drop and then awaiting that. 00:13:47.460 --> 00:13:49.380 There's a lot of proposed ways of doing that. 00:13:49.617 --> 00:13:52.707 In Rust, because we have this decoupling of the system and the 00:13:52.707 --> 00:13:58.637 executor you choose, there's no like blessed way that like a general async 00:13:59.277 --> 00:14:03.097 function could be scheduled on that, because something might be dropped. 00:14:03.327 --> 00:14:07.427 Like you could drop a future in a non-async context, because you've 00:14:07.437 --> 00:14:12.537 got to have a function that returns a future and then drops it, but 00:14:12.577 --> 00:14:14.117 outside of an async function. 00:14:14.392 --> 00:14:16.942 I was going to say, I've definitely spawned tokio background 00:14:16.942 --> 00:14:21.122 tasks from drop, but I guess I just got lucky because it just happened to be 00:14:21.122 --> 00:14:22.982 dropped from an async context already. 00:14:22.982 --> 00:14:23.452 So it was... 00:14:24.002 --> 00:14:27.602 One thing that clicked when I actually started looking into async runtimes 00:14:27.622 --> 00:14:32.915 was that the current runtime stores a thread local, at least in tokio. 00:14:33.115 --> 00:14:34.225 It is actually an optimization trick. 00:14:34.225 --> 00:14:36.875 You can actually send objects to a different thread and drop them from 00:14:36.875 --> 00:14:40.305 there, or, you know, if you're short lived program, just don't drop them 00:14:40.305 --> 00:14:43.145 at all because the operating system's job is to reclaim all that memory. 00:14:43.145 --> 00:14:44.335 So you don't actually have to do it. 00:14:44.689 --> 00:14:44.929 Yeah. 00:14:45.029 --> 00:14:47.589 And when I come up with my problems with an async allocator, this is actually 00:14:47.589 --> 00:14:49.489 the first one, is that drop isn't async. 00:14:49.779 --> 00:14:53.159 I told you how I deal with that and at least practically why it's fine. 00:14:53.359 --> 00:14:55.949 I think in the long term you would want to figure out what async drop 00:14:55.949 --> 00:14:59.109 is, but at least in my opinion, from the allocator's perspective, the 00:14:59.109 --> 00:15:04.569 allocator doesn't care about how you free that memory necessarily. 00:15:04.679 --> 00:15:05.519 There's workarounds. 00:15:05.949 --> 00:15:10.399 It'd be nice if we had async drop because then we could make dealloc async as well, 00:15:10.399 --> 00:15:12.379 but that's sort of beside the point. 00:15:13.374 --> 00:15:18.104 The more pressing, really painful thing is that the whole point of 00:15:18.114 --> 00:15:22.334 using an allocator most of the time is to have nice collection types. 00:15:22.604 --> 00:15:27.554 Things like Vecs and HashMaps and Box and Arc, and all of those 00:15:27.594 --> 00:15:29.404 lovely, nice things that you'd like. 00:15:29.756 --> 00:15:34.356 They're part of the standard library, which means you can't mess with them 00:15:34.356 --> 00:15:38.736 usually, and they usually have internal implementation details that from outside, 00:15:38.736 --> 00:15:41.406 like in userspace, you can't mess with. 00:15:41.586 --> 00:15:44.146 And this sucks, because it means that when you're writing something, you either 00:15:44.146 --> 00:15:49.876 need to use this, like, totally separate standard library collection, and I have 00:15:49.876 --> 00:15:54.526 to write all of my own collections, which isn't horrific- I did it, basically. 00:15:54.526 --> 00:15:57.416 Not everything that's in the standard library, but a lot of them, like, 00:15:57.416 --> 00:16:01.436 Box and Arc and Vec and things like that- just so you can have, like, 00:16:01.476 --> 00:16:03.196 async constructors, basically. 00:16:03.416 --> 00:16:08.201 Because when you want to call Vec::new, you need to await that. 00:16:08.661 --> 00:16:12.771 Which actually is really nice in practice, but when you're trying to 00:16:12.781 --> 00:16:16.821 mix it with existing code that doesn't do that, it sucks because you're just 00:16:16.821 --> 00:16:18.311 duplicating it for no good reason. 00:16:18.476 --> 00:16:20.486 And your async constructors, or your kind of 00:16:20.486 --> 00:16:24.186 wrappers around the collections, they're made possible by methods like 00:16:24.186 --> 00:16:26.296 from_raw_parts or something like that? 00:16:27.071 --> 00:16:27.491 Yeah. 00:16:27.491 --> 00:16:31.660 When possible you try and construct something from scratch- like Vec or 00:16:31.670 --> 00:16:37.598 Box, you can create from scratch using from_raw_parts, those kind of things. 00:16:37.778 --> 00:16:41.958 Things like Arc, which have a private struct called ArcInner there's 00:16:41.958 --> 00:16:43.507 no way to make that from scratch. 00:16:43.633 --> 00:16:44.593 Do you transmute? 00:16:44.779 --> 00:16:46.599 No, I just have my own Arc type. 00:16:46.919 --> 00:16:49.779 When I can't use the existing collection types, like I can't create 00:16:49.779 --> 00:16:51.219 them manually and then use them. 00:16:51.539 --> 00:16:55.149 I just have completely separate types that work the same way, 00:16:55.349 --> 00:16:56.569 but aren't the same type. 00:16:56.804 --> 00:16:56.994 Right. 00:16:57.004 --> 00:16:57.814 James, that's boring. 00:16:57.814 --> 00:17:00.824 I thought you were doing ABI crimes and like having runtime assertions 00:17:00.824 --> 00:17:04.595 that the contents of Arc, the layout is exactly what you expect it to be. 00:17:04.834 --> 00:17:06.394 I actually avoid those a lot of the time. 00:17:06.494 --> 00:17:10.742 But yeah, my favorite, at least the thing that makes me happy is when you 00:17:10.752 --> 00:17:17.182 do horrifically cursed unsafe things, but make them totally stable and 00:17:17.182 --> 00:17:18.972 they totally pass things like Miri. 00:17:19.392 --> 00:17:22.722 Like when I was working on that intrusive linked list for the free list and things 00:17:22.722 --> 00:17:25.162 like that, that was like my whole point. 00:17:25.225 --> 00:17:28.955 I use like unions, I use a lot of transmute casting. 00:17:28.955 --> 00:17:29.985 I used a lot of that. 00:17:30.335 --> 00:17:35.509 But it was all stable and it was all passing Miri and testing, so 00:17:35.509 --> 00:17:37.539 that's like what brings me joy. 00:17:37.725 --> 00:17:41.784 But the last- I don't know, this is maybe a separate side of the last challenge 00:17:41.784 --> 00:17:47.950 here is you can't turn off non-async allocations because when someone does 00:17:47.950 --> 00:17:53.894 call Vec::new that calls the regular allocation interfaces, which means if 00:17:53.894 --> 00:17:58.299 you're in one of these out of memory cases, you just segfault or you crash 00:17:58.319 --> 00:18:01.599 because it returns a null pointer and you get a panic or something like that. 00:18:01.599 --> 00:18:06.019 There's no way to force third party libraries to use this interface or even 00:18:06.019 --> 00:18:08.459 to play nice with fallible allocations. 00:18:08.459 --> 00:18:12.639 Maybe in the future when more libraries use a fallible allocations interface, 00:18:12.779 --> 00:18:18.502 you could wrap that, but it's very challenging to use this effectively. 00:18:18.802 --> 00:18:23.192 Like you have to kind of go all in on this split universe other thing, which 00:18:23.362 --> 00:18:27.302 defeats a lot of the purpose which is: I would like to move faster by using nice 00:18:27.312 --> 00:18:31.242 existing collection types, and use off the shelf libraries and things like that. 00:18:31.659 --> 00:18:33.959 This is a much worse split for the record. 00:18:33.969 --> 00:18:38.479 It feels to me at least that this would be a much worse split than the executor 00:18:38.479 --> 00:18:42.399 situation that we currently have and which isn't as bad as people think it is. 00:18:42.770 --> 00:18:45.920 Yeah, it's up there with having fallible allocations really. 00:18:45.920 --> 00:18:50.330 I think- I think once Rust stabilizes fallible allocations, we will have 00:18:50.430 --> 00:18:55.528 a while where it's just awkward because so much of existing code 00:18:55.558 --> 00:18:58.078 assumes that allocations never fail. 00:18:58.158 --> 00:18:58.728 But... 00:18:59.168 --> 00:19:00.148 they can. 00:19:00.308 --> 00:19:03.558 And I think the Linux kernel project is going to be the one that shakes this 00:19:03.558 --> 00:19:08.028 out, of like pushing more of the like the ecosystem of crates to be like, 00:19:08.078 --> 00:19:11.448 "Hey, you should probably have a try new function for everything in the 00:19:11.448 --> 00:19:15.128 same way that we have like fallible constructors in a lot of the cases. 00:19:15.808 --> 00:19:20.088 And for that one, I think that the paradigm isn't so weird, but introducing 00:19:20.198 --> 00:19:24.658 async allocators would be then like a third flavor of weird, where then you have 00:19:24.658 --> 00:19:32.136 like unfallible, fallible and failable async or infallible async constructors. 00:19:32.156 --> 00:19:36.761 So, I don't know, and I don't think this is widespread enough that 00:19:36.771 --> 00:19:39.941 the standard library would take it, which means it's sort of like 00:19:39.951 --> 00:19:44.661 always this weird side research thing that is really, really cool. 00:19:44.928 --> 00:19:48.028 I might start using it for some network stack stuff, but essentially 00:19:48.028 --> 00:19:49.898 I can't make it a public interface. 00:19:49.918 --> 00:19:54.608 It'll just be this like weird internal pattern that I use because it's 00:19:54.608 --> 00:19:58.868 so nice, but it just does not play nice with the rest of the universe. 00:19:58.888 --> 00:20:01.088 But I feel like the only way to make it work, first of 00:20:01.088 --> 00:20:04.738 all, like you said, it is like in your own highly controlled code base. 00:20:04.768 --> 00:20:08.963 And second of all, this reminds me of the 'never panic' crate or whatever it's 00:20:08.963 --> 00:20:12.063 called, where you use it- I think they use a linker trick to make sure that you 00:20:12.063 --> 00:20:15.403 never actually end up- I was going to say calling the panic macro, but there's no 00:20:15.403 --> 00:20:16.933 such thing as calling a macro at runtime. 00:20:16.963 --> 00:20:18.433 So I don't know exactly how it works. 00:20:18.556 --> 00:20:19.136 Do you wanna know? 00:20:20.156 --> 00:20:20.566 Sure. 00:20:20.793 --> 00:20:24.183 Essentially you define an undefined symbol, like an extern undefined 00:20:24.183 --> 00:20:31.073 symbol as your panic handler, which means that if all the panic handlers are 00:20:31.123 --> 00:20:34.863 optimized out because it's never used in the program, the linker just throws 00:20:34.863 --> 00:20:38.703 away that symbol and it never tries to resolve that unresolvable symbol. 00:20:38.868 --> 00:20:40.338 It only works in release mode? 00:20:40.932 --> 00:20:41.038 Basically. 00:20:41.038 --> 00:20:44.348 Because like in debug, you can't rely on it to be sturdy. 00:20:44.738 --> 00:20:48.765 There's false positives because if the optimizer can't guarantee 00:20:48.765 --> 00:20:52.235 that there are no panicking branches, even if there aren't practically any, it 00:20:52.235 --> 00:20:53.665 still fails because it's a linker error. 00:20:54.185 --> 00:20:54.432 Right. 00:20:54.475 --> 00:20:57.475 But if we had a generic way, and it can't be a Clippy lint, right? 00:20:57.485 --> 00:20:58.775 Because it's not just in your own code. 00:20:58.775 --> 00:21:01.185 It has to be in all your transitive dependencies. 00:21:01.195 --> 00:21:03.925 If we have a generic way to ban methods and say, "No, 00:21:03.925 --> 00:21:05.385 nobody gets to call Vec::new. 00:21:05.580 --> 00:21:06.940 If it does, I want to compile time error. 00:21:06.950 --> 00:21:07.850 Don't rely on the linker. 00:21:07.860 --> 00:21:08.630 That's dirty." 00:21:08.980 --> 00:21:10.050 Then we could do something like that. 00:21:10.079 --> 00:21:12.279 I can think of other use cases. 00:21:12.849 --> 00:21:15.839 Not right now, but I'm sure we could think of other use cases. 00:21:15.889 --> 00:21:16.169 Yep. 00:21:16.569 --> 00:21:19.669 So for now it just lives on the shelf as a fun thing, but if you're designing 00:21:19.669 --> 00:21:23.239 the next programming language out there, maybe think about having async 00:21:23.239 --> 00:21:26.029 allocators in the same way that we've finally just gotten people to admit 00:21:26.029 --> 00:21:28.989 that fallible allocators are- are maybe something that you should think about as 00:21:28.989 --> 00:21:30.969 a first class aspect of your language. 00:21:30.969 --> 00:21:34.619 So, my hope is that maybe the next language beyond Rust gets to use this. 00:21:40.104 --> 00:21:42.384 This episode is sponsored by CodeCrafters. 00:21:42.571 --> 00:21:45.841 CodeCrafters is a service for learning programming skills by doing. 00:21:46.511 --> 00:21:49.801 CodeCrafters offers a curated list of exercises for learning programming 00:21:49.801 --> 00:21:52.811 languages like Rust or learning skills like building an interpreter. 00:21:53.286 --> 00:21:56.826 Instead of just following a tutorial, you can instead clone a repo that contains 00:21:56.836 --> 00:22:00.566 all of the boilerplate already, and make progress by running tests and pushing 00:22:00.566 --> 00:22:03.906 commits that are checked by the server, allowing you to move on to the next step. 00:22:04.546 --> 00:22:07.516 If you enjoy learning by doing, sign up today using the link at 00:22:07.516 --> 00:22:11.636 sdr-podcast.com/codecrafters, or use the link in the show 00:22:11.636 --> 00:22:12.866 notes to start your free trial. 00:22:13.286 --> 00:22:15.806 If you decide to upgrade, you'll get a discount and a portion of 00:22:15.806 --> 00:22:17.346 the sale will support this podcast. 00:22:17.791 --> 00:22:21.481 That's sdr-podcast.com/codecrafters. 00:22:22.181 --> 00:22:24.561 Thanks to CodeCrafters for sponsoring this episode.