WEBVTT NOTE This file was generated by Descript 00:00:13.396 --> 00:00:17.616 This is Self Directed Research, where our hosts, Amos and James, give each 00:00:17.616 --> 00:00:21.516 other 20 minute presentations that usually last longer than that about whatever 00:00:21.516 --> 00:00:22.996 they've been obsessing over lately. 00:00:23.406 --> 00:00:28.426 For more episodes, show notes, and transcripts, visit sdr-podcast.com. 00:00:28.848 --> 00:00:30.968 New episodes drop every Wednesday. 00:00:31.310 --> 00:00:33.400 Today's episode is sponsored by Tweede golf. 00:00:33.670 --> 00:00:34.820 More about them later. 00:00:35.310 --> 00:00:39.460 This week, Amos presents, "I Was Wrong About Rust Build Times." 00:00:44.013 --> 00:00:44.403 All right. 00:00:44.403 --> 00:00:48.363 So my topic today is: I was wrong about Rust build times. 00:00:48.633 --> 00:00:51.123 So a while ago, let's check it out. 00:00:51.143 --> 00:00:52.343 I have the article open here. 00:00:52.393 --> 00:00:57.193 I, in December of 2021, I published a 30 minute read, you know, that's a lie. 00:00:57.223 --> 00:01:00.453 My reading time estimators is lying, but I have unit tests. 00:01:00.998 --> 00:01:04.738 with some of my articles, checking that the reading time is what it should be. 00:01:04.738 --> 00:01:07.298 So I can't change it now because then I would have to update the test. 00:01:07.308 --> 00:01:10.028 Anyway, it's about Why is my Rust build so slow and there's a bunch of 00:01:10.218 --> 00:01:12.408 headers here: what is cargo even doing? 00:01:12.408 --> 00:01:13.978 How much time are we spending on these steps? 00:01:14.088 --> 00:01:18.588 This is talking about cargo build --timings, which I have here as 00:01:18.618 --> 00:01:21.868 this still relevant in my slides because I have slides today. 00:01:22.118 --> 00:01:25.383 I read this blog post literally last week, or two weeks ago, when 00:01:25.383 --> 00:01:29.613 I was troubleshooting why certain builds were slow, and the answer was 00:01:29.613 --> 00:01:31.253 just a lot of generated serde code. 00:01:31.253 --> 00:01:34.683 So all that stuff I was talking about last time, of measuring that, I read 00:01:34.683 --> 00:01:37.403 this article of going through and making sure that I had all of my, 00:01:37.403 --> 00:01:40.593 like, line items that I wanted to check when I was digging into it. 00:01:40.918 --> 00:01:43.328 How did you end up finding out that it was serde? 00:01:43.433 --> 00:01:45.923 The way that I ended up figuring it out was I looked at 00:01:45.923 --> 00:01:47.963 the volume of generated code. 00:01:48.013 --> 00:01:53.063 So the step that I ended up doing was running cargo expand, which gives you 00:01:53.063 --> 00:01:57.623 the output of everything after expansion and counting the line counts before 00:01:57.623 --> 00:02:01.943 and after, but also then running that expanded code through a tool called Form. 00:02:02.383 --> 00:02:06.293 Basically it takes a single file and by module splits it up into multiple files... 00:02:06.873 --> 00:02:07.333 Okay. 00:02:07.983 --> 00:02:11.758 Which gives you one output and then run it through Form, you end up almost 00:02:11.758 --> 00:02:15.118 with the same folder structure, which means that when you then run it through 00:02:15.118 --> 00:02:18.738 like Tokei or another tool you can do the before and after and be like, "Ah, 00:02:18.738 --> 00:02:23.668 this was 2,000 lines before expansion and 54,000 lines after expansion." 00:02:23.908 --> 00:02:28.628 So I don't have like a smoking bullet that it was serde, but serde and one 00:02:28.628 --> 00:02:32.908 other proc macro were generating like literally seven or eight times the size 00:02:32.908 --> 00:02:35.763 of an already 50,000 line Rust crate. 00:02:36.363 --> 00:02:37.103 Right, right, right. 00:02:37.103 --> 00:02:40.373 So the figures you just gave, like 2,000 to 54,000. 00:02:40.393 --> 00:02:43.073 Those are not exact, but it was in that ballpark, right? 00:02:43.693 --> 00:02:44.593 For some of them, yeah. 00:02:44.683 --> 00:02:47.743 If they were just data structures in there, then it would blow up like that. 00:02:47.943 --> 00:02:51.433 If there was a lot of code, which wouldn't expand as hard, but there 00:02:51.433 --> 00:02:54.843 was a couple files we had that were just data structures, that 00:02:54.843 --> 00:02:56.903 became very large generated code. 00:02:57.463 --> 00:02:57.773 Okay. 00:02:57.773 --> 00:03:02.363 So what the heck should we do about proc macros and can we get introspection 00:03:02.363 --> 00:03:06.473 going on again in the Rust project are two topics that I think will come up a 00:03:06.473 --> 00:03:09.213 bunch if we keep doing these, but okay. 00:03:09.253 --> 00:03:10.843 I'm going to go back on track on the article. 00:03:11.003 --> 00:03:11.983 I'm happy that it helped you. 00:03:12.013 --> 00:03:14.053 It did help a lot of other people as well. 00:03:14.053 --> 00:03:14.263 I know. 00:03:14.263 --> 00:03:15.933 Cause I got a, got a lot of good feedback. 00:03:15.933 --> 00:03:17.433 I'm generally happy about this article. 00:03:17.783 --> 00:03:19.963 It talks about picking a better linker. 00:03:20.343 --> 00:03:23.753 Since what has happened is that at Worldwide Developer Conference, 00:03:23.783 --> 00:03:26.503 I think that's what their acronym stands for: WWDC, the Apple 00:03:26.513 --> 00:03:28.113 Developers Conference thingy in 2022. 00:03:28.543 --> 00:03:33.253 Uh, they made their linker twice as fast, which makes it less useful 00:03:33.253 --> 00:03:34.683 to switch to a different linker. 00:03:34.913 --> 00:03:39.333 I don't have it here, but the author of Mold tried different schemes. 00:03:39.333 --> 00:03:42.353 Mold was a third party alternative linker that was faster than all the 00:03:42.353 --> 00:03:44.273 other linkers written in C++ I believe. 00:03:44.683 --> 00:03:49.648 And the author tried a bunch of schemes to monetize it and eventually 00:03:49.728 --> 00:03:53.788 kind of gave up, I believe, and just said, "Okay, I guess I'm not gonna," 00:03:53.848 --> 00:03:54.948 because I think he quit his job. 00:03:55.298 --> 00:03:56.838 I should, you know, I should source this better. 00:03:56.903 --> 00:04:00.323 He was the developer that actually developed lld for LLVM, 00:04:00.343 --> 00:04:01.883 and then went on to build Mold. 00:04:02.223 --> 00:04:07.573 And was selling it as sold, but what came around was essentially Apple figured 00:04:07.573 --> 00:04:12.773 out what they were doing with sold, rewrote ld64 for their own thing, and 00:04:12.773 --> 00:04:16.413 then went, "Well, that's kind of, you just ate all of my competitive gain. 00:04:16.863 --> 00:04:20.453 I'm not going to maintain a product anymore because there's 00:04:20.453 --> 00:04:23.423 no reason for me to recommend it for anyone to buy it now that ld64 00:04:23.423 --> 00:04:25.643 is just as fast in the same way." 00:04:25.853 --> 00:04:31.293 Yes, which is interesting because we know Apple does that with apps, right? 00:04:31.643 --> 00:04:32.963 It's called getting Sherlocked. 00:04:33.371 --> 00:04:35.611 Like the app of the year, one year is going to be 00:04:35.841 --> 00:04:37.751 the next Apple app next year. 00:04:38.031 --> 00:04:38.801 So they do that. 00:04:38.801 --> 00:04:42.461 They let developers make things in their walled garden, and then they make their 00:04:42.461 --> 00:04:44.381 own version and then you're irrelevant. 00:04:44.741 --> 00:04:49.721 But for me, I don't, I can't think of another high profile instance of this 00:04:49.721 --> 00:04:51.551 happening in open source, I guess. 00:04:51.991 --> 00:04:53.651 Well they do tiling windowing now. 00:04:53.681 --> 00:04:55.511 So I guess that chewed a little bit. 00:04:55.786 --> 00:04:56.266 Yeah. 00:04:57.131 --> 00:04:57.851 Yeah. 00:04:58.086 --> 00:04:58.896 I'm always surprised. 00:04:58.896 --> 00:05:03.856 Like ever since I went back to macOS full time, I used to have like a Windows 00:05:03.856 --> 00:05:06.406 machine, Windows 11 with Linux VMs. 00:05:06.456 --> 00:05:08.536 And now I just have Macs all around the house. 00:05:08.576 --> 00:05:11.656 It's just, there's no problem that several thousand dollars cannot fix. 00:05:11.870 --> 00:05:14.680 I'm always surprised to see the thriving ecosystem of like little 00:05:14.680 --> 00:05:17.710 Mac apps that just add a little thing that's missing in the ecosystem. 00:05:18.140 --> 00:05:20.860 And I'm always happy to throw them like between 10 and 30 bucks. 00:05:20.860 --> 00:05:22.820 Cause I'm like, it's a lifetime license. 00:05:22.820 --> 00:05:23.740 It's something I need. 00:05:24.050 --> 00:05:25.530 I should make an article about all these. 00:05:25.640 --> 00:05:29.560 So Apple killed mold and now the built in thing is kind of okay. 00:05:29.560 --> 00:05:32.520 So RIP, but also good for everyone else. 00:05:32.580 --> 00:05:32.870 I don't know. 00:05:33.315 --> 00:05:36.645 In terms of linker as well, that slide went missing for some 00:05:36.645 --> 00:05:42.505 reason, but recently rustc nightly started shipping their own... 00:05:43.725 --> 00:05:48.125 not their version of, just like they started shipping lld and they started 00:05:48.145 --> 00:05:50.205 using it by on Linux x86-64 bit. 00:05:50.485 --> 00:05:55.815 I don't know how much is in common between like- Apple's linker is as lld, no? 00:05:55.870 --> 00:05:57.750 I don't think it is, but I'm, I'm guessing here. 00:05:57.780 --> 00:06:00.130 It's not the same, but it might be a fork. 00:06:00.150 --> 00:06:00.700 I don't know. 00:06:00.845 --> 00:06:02.145 Well, someone should check it out. 00:06:02.175 --> 00:06:06.537 If only there were two talking and I'll check it out before next week. 00:06:06.867 --> 00:06:08.077 So what else in the article? 00:06:08.077 --> 00:06:09.567 The article talks about incremental builds. 00:06:09.567 --> 00:06:14.227 I get sad every time I look at this because I, I've talked to 00:06:14.287 --> 00:06:19.497 some people who were involved with rustc around those features. 00:06:19.527 --> 00:06:24.580 And my understanding is that there were plans to do incremental building, but 00:06:24.580 --> 00:06:26.307 better, or for more things, like... 00:06:26.377 --> 00:06:31.317 it's partially done and it works well enough locally, but definitely for CI, 00:06:31.317 --> 00:06:33.317 for example, it's not even an option. 00:06:33.807 --> 00:06:36.487 Because it's just so hard to know what to commit to cache. 00:06:36.487 --> 00:06:41.077 So if you're thinking of a traditional GitHub actions, CI cache workflow, you 00:06:41.087 --> 00:06:43.507 restore the cache at the beginning of the build and you save the cache at the end 00:06:43.507 --> 00:06:46.927 of the build and the first problem you end up having is that suddenly the cache is 55 00:06:46.927 --> 00:06:48.597 gigabytes and that's too many gigabytes. 00:06:48.957 --> 00:06:51.447 And so you don't know what to prune, what to keep. 00:06:51.447 --> 00:06:55.187 You don't know if you can use the main branch cache for the PRs, if you should 00:06:55.197 --> 00:06:56.437 bring back the PR cache to the... 00:06:56.817 --> 00:06:59.267 there's security concerns, what if someone poisoned the cache, 00:06:59.267 --> 00:07:00.227 there's all sorts of things. 00:07:00.697 --> 00:07:04.877 And that's kind of still doable with the non incremental builds in 00:07:04.877 --> 00:07:06.617 Rust, but with incremental builds... 00:07:06.697 --> 00:07:08.287 I don't think anyone's trying to do it. 00:07:08.317 --> 00:07:12.487 I think a lot of tools, like if you build Rust with Nix, or if you build Rust with 00:07:12.497 --> 00:07:18.067 like Bazel or Buck, I don't know, I still need to try those and do a comparison. 00:07:18.977 --> 00:07:19.677 It's just... 00:07:19.687 --> 00:07:22.257 incremental building is nice locally for your machine. 00:07:22.667 --> 00:07:25.577 Actually, there is one person or one company doing incremental 00:07:25.577 --> 00:07:27.017 builds with caches in CI. 00:07:27.367 --> 00:07:30.957 But what they did is just essentially have persistent workers. 00:07:31.097 --> 00:07:33.327 I'm assuming we still cloud-based somewhere, but they have a bunch 00:07:33.327 --> 00:07:36.817 of workers and they just like clone them and they just bring 00:07:36.817 --> 00:07:39.667 the build to them instead of like restoring the cache somewhere else. 00:07:40.312 --> 00:07:43.692 So like old-school Jenkins, not GitHub Actions where everything 00:07:43.692 --> 00:07:47.362 was a persistent machine and there's no isolation or VMing. 00:07:47.717 --> 00:07:49.447 I'm assuming they're careful about it. 00:07:49.487 --> 00:07:52.387 I'm assuming it's still "cloudy" and like they can scale it up. 00:07:52.387 --> 00:07:55.377 Cause it's easy to like clone a VM and do Copy-on-Write. 00:07:55.397 --> 00:07:55.617 I don't know. 00:07:55.617 --> 00:07:57.207 There is some magic happening. 00:07:57.207 --> 00:07:57.697 Probably. 00:07:58.032 --> 00:07:59.292 It's the Earthly people. 00:07:59.792 --> 00:08:03.172 I will, I will add the links because I asked ChatGPT what article I 00:08:03.172 --> 00:08:06.262 was thinking of and it finally, it found it when you let it do 00:08:06.462 --> 00:08:07.852 web searches, it can find things. 00:08:07.852 --> 00:08:10.382 So it gave me all articles and I was like: no, no, no, not this 00:08:10.382 --> 00:08:12.412 one, not sccache, none of that. 00:08:12.422 --> 00:08:16.242 And it found it, the Earthly folks who did it for, I forget 00:08:16.292 --> 00:08:17.692 who, but we will have the link. 00:08:18.362 --> 00:08:21.852 There's a lot of things about like rustc self refining that is still accurate. 00:08:21.862 --> 00:08:25.832 As far as I know, some crates still hit pathological cases in rustc that's, 00:08:25.907 --> 00:08:30.367 it's still a thing, you can get some nice gains if you know where to look 00:08:30.677 --> 00:08:35.507 and there's cargo LLVM lines that people are using with great success. 00:08:36.047 --> 00:08:41.257 But the one thing this article says that is actually, I think in retrospect, a very 00:08:41.257 --> 00:08:43.767 bad idea is splitting it into more crates. 00:08:44.747 --> 00:08:47.787 And I will explain both the original reasoning, why I think 00:08:47.787 --> 00:08:49.207 it's bad and what I'm doing instead. 00:08:50.087 --> 00:08:52.577 I was gonna say, because this was my only suggestion to my client when they 00:08:52.577 --> 00:08:53.757 said, "How do we make this go faster?" 00:08:53.757 --> 00:08:56.927 And I go, "Well, we probably shouldn't have one crate that's 50,000 lines 00:08:56.937 --> 00:09:02.127 before expansion and like 200 or 300,000 lines after expansion," because 00:09:02.277 --> 00:09:04.127 you lose parallelism at that point. 00:09:04.127 --> 00:09:07.447 But I'm interested to hear why you're turned around, because before I was real 00:09:07.447 --> 00:09:11.247 ready to suggest what you suggest in this blog post, because it made sense to me. 00:09:12.407 --> 00:09:16.347 So I ended up having too many crates is one of the things. 00:09:16.457 --> 00:09:22.427 And it ended up- it's not directly related to compile times, but like development 00:09:22.427 --> 00:09:26.097 in general, having to maintain those 14 different crates, even though you have 00:09:26.097 --> 00:09:30.597 like a cargo upgrade by cargo-edit, it's just a lot of, even if you inherit 00:09:30.797 --> 00:09:34.407 dependency versions from the workspace, even if you inherited the crate version 00:09:34.407 --> 00:09:37.602 from our workspace, which are all things you can do, it's I don't know. 00:09:37.822 --> 00:09:40.632 I had to think about a lot of things, uh, all the time. 00:09:40.632 --> 00:09:42.752 It was, it was a barrier to just developing the thing. 00:09:43.262 --> 00:09:44.152 So that's one thing. 00:09:44.582 --> 00:09:49.432 I think where my calculation went wrong- and I, I want to actually go 00:09:49.432 --> 00:09:53.532 and back it with hard data, which I don't have today, but I may have in 00:09:53.572 --> 00:10:00.157 the future for an actual write up- is that, sure, the crates can compile in 00:10:00.157 --> 00:10:02.237 parallel, but define "compile," right? 00:10:02.237 --> 00:10:03.487 They can be parsed in parallel. 00:10:03.497 --> 00:10:04.537 That was already a thing. 00:10:04.537 --> 00:10:08.347 I don't know if cargo/rustc actually does it, but it could, right? 00:10:08.647 --> 00:10:10.737 I'm sure there's opportunities for parsing things in parallel. 00:10:10.747 --> 00:10:11.957 I haven't checked, but that- 00:10:12.077 --> 00:10:14.627 There's the parallel front ends feature, and then I can't 00:10:14.627 --> 00:10:17.817 remember all the stages that are real or nightly only today. 00:10:18.297 --> 00:10:18.677 Right. 00:10:19.087 --> 00:10:19.377 Yeah. 00:10:19.377 --> 00:10:22.377 If you run cargo build --timings, you can see your different crates 00:10:22.417 --> 00:10:23.777 actually building in parallel. 00:10:24.077 --> 00:10:30.502 Uh, but it ends up doing more work because since you're now using 00:10:30.512 --> 00:10:33.542 things acRust crates, you have to declare a bunch of things pub. 00:10:34.062 --> 00:10:38.772 So whereas previously you could do like intra-crate, so inside of the 00:10:38.772 --> 00:10:39.942 same crate, it could do inlining. 00:10:39.942 --> 00:10:42.282 It could do a lot of things because it has all the code. 00:10:42.552 --> 00:10:45.372 It doesn't even need to like export those symbols. 00:10:45.592 --> 00:10:48.172 It's, they're not in the object files and already the archive files, the 00:10:48.172 --> 00:10:49.632 linker doesn't have to know about them. 00:10:50.232 --> 00:10:55.132 Rustc can just do its little, its little cooking on the side, and then generate 00:10:55.142 --> 00:10:56.632 only what LLVM needs to know about. 00:10:56.632 --> 00:10:59.862 Now there's a bunch of things that are pub and so it's all exported. 00:10:59.862 --> 00:11:02.832 It's all in file and the linker now has to worry about all of that, 00:11:03.042 --> 00:11:06.482 which would be fine because as we just said, linkers got faster. 00:11:07.042 --> 00:11:13.762 But the other thing is now we have link time optimization (LTO), which I really 00:11:13.762 --> 00:11:17.192 want to measure because I feel like it's taking so long, even if you have like 00:11:17.192 --> 00:11:21.342 the fast linker on earth that does all the optimizations that Mold did and still 00:11:21.342 --> 00:11:27.217 does, you still have this thing where like compilers, well, linkers will call 00:11:27.237 --> 00:11:32.547 a plugin to figure out what optimizations can they do acRust different compilation 00:11:32.547 --> 00:11:35.367 units and that takes a lot of time. 00:11:35.367 --> 00:11:38.037 And if you're splitting all your crates in different crates, it has to 00:11:38.037 --> 00:11:43.102 do this every time, because you have one binary that depends on all your 00:11:43.112 --> 00:11:47.612 workspace and every time you change any of these, it has to link again. 00:11:48.262 --> 00:11:51.462 If it was like one crate and you have incremental caching, then 00:11:51.472 --> 00:11:52.932 it can actually some of that. 00:11:52.932 --> 00:11:55.362 But I don't think the result of LTO is cached anywhere 00:11:55.397 --> 00:11:57.867 Well, I think ThinLTO does. 00:11:57.877 --> 00:12:00.977 So I think this is the difference between ThinLTO and FatLTO. 00:12:00.997 --> 00:12:06.067 So FatLTO, my working model for it is: it takes all the compilation artifacts, 00:12:06.347 --> 00:12:11.237 vomits them as if they were one giant object file, and then runs optimization on 00:12:11.237 --> 00:12:14.137 them again, without having this separate. 00:12:14.147 --> 00:12:21.182 Whereas ThinLTO keeps the LLVM like, bitcode for each of the stages and can't 00:12:21.192 --> 00:12:24.682 do quite as powerful optimizations, but it doesn't have to, like you said, 00:12:24.682 --> 00:12:27.962 re vomit everything into one pile so that it can optimize it again. 00:12:28.292 --> 00:12:31.352 It can do this all by parts, but it does lose some 00:12:31.992 --> 00:12:34.302 optimization opportunities there. 00:12:35.092 --> 00:12:35.512 Sure. 00:12:35.627 --> 00:12:39.937 But again, I wish I had numbers to back it up, but in my experience, 00:12:40.167 --> 00:12:41.487 it still spends time there. 00:12:41.537 --> 00:12:42.827 It is yeah, for sure. 00:12:43.267 --> 00:12:48.487 FatLTO, which is non ThinLTO, whatever, regular classic LTO, definitely 00:12:48.487 --> 00:12:49.727 takes more time than ThinLTO. 00:12:49.977 --> 00:12:54.027 Also, I think the default for dev builds is something called 00:12:54.537 --> 00:12:56.637 thin local LTO or something? 00:12:56.667 --> 00:12:57.747 Like crate local... 00:12:57.747 --> 00:13:00.287 Yeah, that's, that's, that's like another level. 00:13:00.962 --> 00:13:04.342 The LTO flag in cargo.toml is one of the worst ones because 00:13:04.602 --> 00:13:08.562 it has a value- like you can put a string or a boolean there and if you 00:13:08.562 --> 00:13:13.207 put "false" it does ThinLTO, and if 00:13:13.207 --> 00:13:13.647 yes. 00:13:13.647 --> 00:13:13.732 Yes. 00:13:13.732 --> 00:13:15.347 "true" it does full LTO... 00:13:15.637 --> 00:13:18.862 So "false" does a thin-local LTO, the thing I was just talking about. 00:13:19.012 --> 00:13:20.532 "True" or "fat" is the same thing. 00:13:21.167 --> 00:13:23.007 It trips people up all the time. 00:13:23.027 --> 00:13:26.487 And it's one of those, I always have to go and look at it, because I'm like- it makes 00:13:26.487 --> 00:13:30.907 sense historically why it's like that, but it makes me so upset, because we don't 00:13:30.907 --> 00:13:33.147 usually deal with truthiness in Rust. 00:13:33.147 --> 00:13:35.247 Like, and this is one of the few times we have to, and it 00:13:35.247 --> 00:13:36.637 confuses everyone when I see it. 00:13:36.637 --> 00:13:39.622 This is worse than YAML because yeah, false and off 00:13:39.622 --> 00:13:40.762 are different values here. 00:13:40.812 --> 00:13:42.332 At least in YAML it's consistent. 00:13:42.382 --> 00:13:42.722 Yeah. 00:13:42.947 --> 00:13:43.877 And also Norway. 00:13:44.022 --> 00:13:46.452 I think in retrospect, it was bad advice. 00:13:46.512 --> 00:13:49.722 It didn't end up helping much for me. 00:13:49.752 --> 00:13:51.702 I think the numbers are like wishy washy. 00:13:51.702 --> 00:13:53.642 I didn't, I wasn't very scientific about it. 00:13:53.837 --> 00:13:56.057 The other thing you haven't mentioned is the Orphan Rule. 00:13:56.067 --> 00:14:00.147 This is the other one that I see people complain about a lot when they have 00:14:00.187 --> 00:14:01.857 broken things up into many crates. 00:14:01.877 --> 00:14:06.007 Is, like you said, the other maintenance things of having to span multiple crates. 00:14:06.067 --> 00:14:09.127 The Orphan Rule is a big one where you want to be able to impl your 00:14:09.127 --> 00:14:14.047 trait for a lot of things and that becomes way harder when you leave, 00:14:14.112 --> 00:14:16.112 when you go from modules to crates. 00:14:16.122 --> 00:14:18.262 So do you have something that you would recommend today? 00:14:18.262 --> 00:14:21.462 Or is this where the research- this is like, proposal for more research 00:14:21.462 --> 00:14:23.142 of, I need to figure out which- 00:14:23.257 --> 00:14:23.847 So, okay. 00:14:23.857 --> 00:14:26.787 So the slides end here, but my- my thoughts don't. 00:14:27.042 --> 00:14:27.422 Okay. 00:14:28.467 --> 00:14:31.007 It's not just, "I was wrong and I think it was a bad idea." 00:14:31.007 --> 00:14:37.052 I do have a plan that I'm currently enacting in, in my proprietary code base. 00:14:37.082 --> 00:14:39.592 I don't know how much proprietary code I have, but actually it's 00:14:39.592 --> 00:14:41.922 pretty easy to find out now because I just brought everything together. 00:14:42.242 --> 00:14:43.262 That sounds wrong. 00:14:43.312 --> 00:14:46.682 I don't have a million 800,000 lines of code. 00:14:46.952 --> 00:14:48.152 Where does that come from? 00:14:48.262 --> 00:14:51.782 I have about 18,000 lines of Rust because my website does a lot of things. 00:14:51.782 --> 00:14:53.322 It has this whole asset pipeline. 00:14:53.322 --> 00:14:57.667 It encodes images so like AVIF and WebP and whatnot. 00:14:57.667 --> 00:15:00.917 And I have a whole video pipeline as well, because I have a clone of the 00:15:00.917 --> 00:15:04.697 YouTube player on my website for patrons, so you don't have to sit through ads. 00:15:05.157 --> 00:15:07.077 I have a Markdown pipeline. 00:15:07.077 --> 00:15:08.967 I have KaTeX for equations. 00:15:09.027 --> 00:15:10.097 I have a lot of dependencies. 00:15:10.097 --> 00:15:12.687 I think it's about a thousand crates and it used to be separated into 00:15:12.687 --> 00:15:13.947 a bunch of different code bases. 00:15:14.227 --> 00:15:16.477 And one of the code bases, the blog used to be separated a bunch 00:15:16.477 --> 00:15:18.197 of crates because of that article. 00:15:18.497 --> 00:15:21.537 So what I've spent the last couple of days doing is actually put 00:15:21.537 --> 00:15:23.467 everything back into one single crate. 00:15:23.727 --> 00:15:28.467 And then the rest of the plan is to isolate things by how long it takes 00:15:28.467 --> 00:15:34.647 to build and make those into dynamic libraries, which in most other languages. 00:15:34.797 --> 00:15:38.937 Well, no, in C or C++ would be okay, I guess, because 00:15:39.257 --> 00:15:40.337 people have been doing forever. 00:15:40.577 --> 00:15:42.587 And it's, it's kind of how Linux distributions work. 00:15:42.587 --> 00:15:45.477 You just install dynamic libraries somewhere and then everything like 00:15:45.677 --> 00:15:47.027 shares the same version of things. 00:15:47.297 --> 00:15:50.702 But in Rust specifically, it's complicated because we don't have a 00:15:50.702 --> 00:15:52.512 stable ABI for a good reason, but still. 00:15:52.922 --> 00:15:55.612 The compiler kind of doesn't want you to do that. 00:15:55.612 --> 00:15:58.262 It's not just the switch from like, "Oh, this crate should be dynamic. 00:15:58.302 --> 00:15:58.912 Just do that." 00:15:58.912 --> 00:16:02.412 I think it's not that easy, or maybe it is, but like, then you 00:16:02.412 --> 00:16:05.682 have to make sure that the compiler version used is the exact same. 00:16:05.712 --> 00:16:07.332 So you've run into these, this kind of issues. 00:16:07.902 --> 00:16:11.402 You already have like, extern rust unsafe stuff. 00:16:11.492 --> 00:16:12.422 Which you can do. 00:16:13.237 --> 00:16:16.267 Well, actually tell me, cause I just, yeah, I just skipped over it, 00:16:16.267 --> 00:16:19.147 like knowing it's a headache, but maybe you know the details of the headache. 00:16:19.147 --> 00:16:19.857 So please tell me. 00:16:20.522 --> 00:16:24.052 I was gonna say, if you manually "I solemnly swear that I'm only using 00:16:24.052 --> 00:16:25.512 one version of the compiler with it." 00:16:25.892 --> 00:16:30.522 Then you can extern and like consume extern definitions 00:16:30.522 --> 00:16:31.872 and have them be linked in. 00:16:32.202 --> 00:16:35.162 Like I said, I do a lot of embedded, so we haven't done a lot of dynamic 00:16:35.162 --> 00:16:37.792 linking because most microcontrollers don't have dynamic linkers. 00:16:38.252 --> 00:16:43.762 But I know that's how the compiler generally works with things like compile 00:16:43.772 --> 00:16:45.372 derive macros and things like that. 00:16:45.372 --> 00:16:49.332 Those are essentially compiler plugins and we can do the same thing, but it, 00:16:49.402 --> 00:16:51.842 it ends up being very easy to foot gun. 00:16:51.842 --> 00:16:54.272 But if you promise that you're always doing a clean build of the entire 00:16:54.272 --> 00:16:55.762 system so that you can do a deployment. 00:16:56.277 --> 00:16:58.547 Then that's not necessarily very unreasonable. 00:16:58.597 --> 00:17:02.187 It's just not very portable to give to someone else. 00:17:02.247 --> 00:17:04.217 I was, I was fact checking you in real time. 00:17:04.217 --> 00:17:05.457 Here's the funny thing that happened. 00:17:05.467 --> 00:17:07.857 So I looked at extern Rust because I forgot that was a thing, but then 00:17:07.857 --> 00:17:08.877 I remembered as you brought it up. 00:17:08.877 --> 00:17:11.537 So I looked at the Rust reference, but then I looked at the proc macro 00:17:11.537 --> 00:17:14.267 bridge, because you explained that's how it works, just does extern Rust. 00:17:14.597 --> 00:17:17.487 And the first result is of course, an article by me about proc 00:17:17.487 --> 00:17:19.607 macro support in rust-analyzer. 00:17:19.847 --> 00:17:23.467 I forgot, they do a weird, interesting thing, which might 00:17:23.467 --> 00:17:24.997 be worth discussing another time. 00:17:25.297 --> 00:17:27.727 If you can guarantee that you're using the same version of everything 00:17:27.737 --> 00:17:28.817 everywhere, then that's fine. 00:17:28.827 --> 00:17:30.267 But I don't want to have that constraint. 00:17:30.267 --> 00:17:34.257 So one crate that lets you have a stable ABI, no matter what 00:17:34.257 --> 00:17:36.337 the rustc version is, is Stabby. 00:17:36.537 --> 00:17:40.317 stabby-abi is the ABI crate itself, but stabby is the thing with like 00:17:40.387 --> 00:17:45.717 proc macros and the README does a good job at explaining what it does, but 00:17:45.717 --> 00:17:51.407 yeah, essentially instead of making everything a repr(C), which is what you 00:17:51.407 --> 00:17:54.167 would do if you wouldn't want a stable ABI most of the time, or if you have 00:17:54.167 --> 00:17:59.457 to interface with C and C++ you use this, and then you can have this sort 00:17:59.457 --> 00:18:05.947 of plugin system where you have some traits and they are implemented by shared 00:18:05.947 --> 00:18:07.207 objects that you can load at runtime. 00:18:07.297 --> 00:18:10.167 And for me, for this code base specifically, this makes sense 00:18:10.167 --> 00:18:13.167 for me because I'm not actually going to load and unload them. 00:18:13.237 --> 00:18:16.967 Unloading them is the, the thing that you really should never do with Rust. 00:18:17.477 --> 00:18:21.207 I, I also have an article about loading and unloading things for 00:18:21.207 --> 00:18:22.597 like, I forget what it's called... 00:18:22.677 --> 00:18:22.767 So. 00:18:22.777 --> 00:18:23.907 You're iterating on some things. 00:18:23.917 --> 00:18:25.167 You're just recompiling bits of it. 00:18:25.177 --> 00:18:27.117 So they do that with Bevy for game development. 00:18:27.317 --> 00:18:31.727 For this project, my blog, I just need to load all the plugins at runtime. 00:18:32.067 --> 00:18:35.797 And I think I can get a lot of gains here because for example, KaTeX 00:18:35.817 --> 00:18:37.257 pulls in an entire Javascript engine. 00:18:37.267 --> 00:18:39.777 So if I only have to build that once, that's good. 00:18:40.037 --> 00:18:43.037 I don't know, all of, all of SQLite, I can put SQLite into its 00:18:43.037 --> 00:18:44.647 own dynamic thing and it's fine. 00:18:44.817 --> 00:18:47.887 And so of course you lose out on like, there's no LTO going on anymore 00:18:47.887 --> 00:18:49.097 because it's actually dynamic. 00:18:49.157 --> 00:18:50.207 It's too late to inline. 00:18:50.207 --> 00:18:52.137 There's no JITs there to save you. 00:18:52.497 --> 00:18:56.887 For like clear interfaces, I don't know, S3 storage pulls in a bunch of crates. 00:18:57.097 --> 00:18:59.667 It's all async, async interfaces do work with Stabby. 00:19:00.227 --> 00:19:02.267 Just put them in a, in a shared object and see what happens. 00:19:02.267 --> 00:19:03.387 So this is my new approach. 00:19:03.487 --> 00:19:05.347 I haven't finished it yet. 00:19:05.487 --> 00:19:09.787 I will keep you posted, but I just wanted to say that actually breaking 00:19:09.787 --> 00:19:13.377 things down into crates is not enough, because you'll get caught at LTO 00:19:13.377 --> 00:19:17.917 time, it slows down development in general, and also, I did it too much. 00:19:17.957 --> 00:19:21.267 And I don't think it helped much, because, especially in a workspace, 00:19:21.297 --> 00:19:25.687 maybe that's the last thing I want to cover on this, is that, unless you use 00:19:25.687 --> 00:19:29.942 cargo-hakari, I don't know if you've used it before, which does the workspace hack. 00:19:30.952 --> 00:19:32.392 Can you explain the workspace hack? 00:19:32.392 --> 00:19:35.032 Because I have a customer using it and I don't totally wrap my 00:19:35.032 --> 00:19:37.632 head around it because someone else at the customer set it up. 00:19:37.702 --> 00:19:40.462 But it broke some stuff and then the stuff got fixed. 00:19:40.502 --> 00:19:44.372 What is the workspace hack and how does hakari help you with the workspace hack? 00:19:44.512 --> 00:19:47.622 There's a chance the explanation I will give is completely wrong because 00:19:47.622 --> 00:19:48.872 I want to do it off the top of my head. 00:19:49.062 --> 00:19:51.312 I can tell you the problem it's supposed to solve. 00:19:51.582 --> 00:19:55.512 If in a workspace, you have different members that depend on the same 00:19:55.512 --> 00:19:58.262 dependencies, say you have like three different crates in your workspace. 00:19:58.262 --> 00:20:02.772 They all depend on serde, but they have slightly different feature flags, then 00:20:03.157 --> 00:20:05.007 they don't actually share the artifacts. 00:20:05.007 --> 00:20:06.817 Like you're going to have three different builds of serde for this 00:20:06.837 --> 00:20:10.837 workspace and the workspace hack is just, okay, let's have a single crate 00:20:11.357 --> 00:20:14.737 that has all the dependencies that anything in your workspace depends on. 00:20:15.057 --> 00:20:15.617 And then- 00:20:15.852 --> 00:20:18.572 Force unification on, on all of it. 00:20:18.622 --> 00:20:19.332 Yeah, okay. 00:20:19.437 --> 00:20:23.127 It's going to have all the features you want enabled, like the union of all 00:20:23.127 --> 00:20:24.647 features that all the others were using. 00:20:24.647 --> 00:20:29.667 And then there's a crate: a cargo subcommand called cargo-hakari, I don't 00:20:29.667 --> 00:20:34.187 know how to pronounce it, that sets it up for you and updates it for you. 00:20:34.417 --> 00:20:34.957 And it's really nice. 00:20:34.957 --> 00:20:38.817 And I've used it in the past, but yeah, if you don't do that, if you forget to 00:20:38.817 --> 00:20:43.282 do that, then you might think that, yeah, okay, I had this big dependency tree. 00:20:43.282 --> 00:20:45.482 I put it in a different crate and I don't touch it. 00:20:45.482 --> 00:20:47.162 So every time it's going to be cached. 00:20:47.822 --> 00:20:51.152 Actually, no, the rewards is not as good as I thought it was. 00:20:51.722 --> 00:20:55.212 I'm interested in this because I want to see someone do dynamic 00:20:55.212 --> 00:20:58.412 loading because I haven't seen many people mess with that, so I'm excited. 00:20:58.412 --> 00:20:58.662 Okay. 00:20:58.787 --> 00:21:01.387 Because I know when you write it up, it will be written up. 00:21:01.767 --> 00:21:07.067 The other thing though is: I want to poke you to have a more realistic baseline. 00:21:07.327 --> 00:21:11.067 That if you are saying, "Hey, I'm not going to get LTO because I am 00:21:11.607 --> 00:21:14.587 isolating all these components," make that your baseline. 00:21:14.597 --> 00:21:17.687 Like change your cargo profile so that you have LTO completely turned 00:21:17.687 --> 00:21:20.427 off, you have maximum code gen units. 00:21:20.437 --> 00:21:25.777 You have maybe even nightly, like parallel front end and things like that of: okay, 00:21:25.777 --> 00:21:29.067 well, if you really are saying that you're willing to give these things up because 00:21:29.067 --> 00:21:32.147 you're going to have dynamic plugins, what happens if you go all the way back? 00:21:32.187 --> 00:21:35.387 Because I've seen some A's and B's before we go, "Ah, this is the 00:21:35.387 --> 00:21:37.967 worst that this can get with full LTO and everything like that." 00:21:38.207 --> 00:21:43.147 And then, you compare it with the leanest possible setup ever where 00:21:43.147 --> 00:21:44.867 you've made all of these compromises. 00:21:44.867 --> 00:21:47.457 And I'm not saying you specifically, but this is that- I think we talked 00:21:47.467 --> 00:21:50.867 about this before- the perils of benchmark of like, I've implemented 10 00:21:50.867 --> 00:21:54.757 percent of the problem and "Hey, look, it's 10 times faster" kind of hint. 00:21:54.777 --> 00:21:58.947 But I'd be interested to hear, especially if you have realistic perf numbers 00:21:58.967 --> 00:22:01.257 of how much slower is it without LTO? 00:22:01.267 --> 00:22:05.067 Or how much slower is it with dynamic linking if you have any sort of like 00:22:05.507 --> 00:22:08.867 perf numbers to play with, because not just how fast it compiles... 00:22:08.987 --> 00:22:10.827 what are you losing for that trade off? 00:22:11.422 --> 00:22:11.782 For sure. 00:22:12.062 --> 00:22:12.352 Yeah. 00:22:12.422 --> 00:22:15.192 If I do a write up about this or a video, I haven't decided yet. 00:22:15.452 --> 00:22:18.022 It will be either of these because I think it's a very interesting topic. 00:22:18.472 --> 00:22:22.052 I will definitely start from like, if you don't have any dynamic 00:22:22.052 --> 00:22:26.042 loading, you do like full static linking FatLTO, all the things, like 00:22:26.042 --> 00:22:28.492 you said, code gen units to one. 00:22:28.562 --> 00:22:30.232 Is that it? 00:22:30.232 --> 00:22:30.392 Yeah. 00:22:30.802 --> 00:22:33.602 Well, one would make it more optimized, but slower. 00:22:33.852 --> 00:22:34.532 Yeah, yeah. 00:22:34.582 --> 00:22:38.012 So we go from like the most optimizations with the slowest build to like the other 00:22:38.012 --> 00:22:41.852 end of the spectrum without changing the linking style and then to dynamic linking. 00:22:42.052 --> 00:22:46.122 But one thing I'm specifically excited about with dynamic linking is that even if 00:22:46.162 --> 00:22:51.372 like, cause the C dependencies I've talked about get recompiled, like SQLite, like 00:22:51.452 --> 00:22:53.682 the JavaScript engine: those get rebuilt. 00:22:53.842 --> 00:22:57.142 I don't know when I upgrade to a new major Rust version or like I cargo 00:22:57.142 --> 00:22:59.802 clean accidentally or something, but most of the time when you just 00:22:59.832 --> 00:23:02.382 run cargo check, what happens when you save the file in a text editor? 00:23:02.737 --> 00:23:06.897 They don't actually get rebuilt, but cargo check is still slow. 00:23:07.457 --> 00:23:10.537 It's still freaking slow because there's like so many, 00:23:10.557 --> 00:23:12.317 so many crates in the dep tree. 00:23:12.497 --> 00:23:16.097 One thing that didn't go all the way, talking about incremental compilation 00:23:16.097 --> 00:23:19.292 is that: I really wish we just had a compiler server, instead of a 00:23:19.292 --> 00:23:24.762 compiler CLI because you can design the cache file format the best you can. 00:23:24.942 --> 00:23:26.742 It still has all this work to do. 00:23:26.752 --> 00:23:27.852 Like it wakes up. 00:23:27.852 --> 00:23:30.952 It's like, "Okay, do we have a Rust tool chain file around here? 00:23:31.172 --> 00:23:31.542 Okay. 00:23:31.542 --> 00:23:32.652 We found the right rustc. 00:23:32.672 --> 00:23:33.452 Let's boot it up. 00:23:33.482 --> 00:23:33.792 Okay." 00:23:34.402 --> 00:23:37.742 And it has like, "Oh boy, we have, we have a lot of cache files on 00:23:37.742 --> 00:23:39.012 this, better read all of those!" 00:23:39.242 --> 00:23:41.502 And even if it's like, obviously it's going to be memory cache, 00:23:41.502 --> 00:23:43.732 the operating system, it's not actually going to read from disk. 00:23:44.062 --> 00:23:47.392 That's still a lot of work to then realize, "Oh, darn it. 00:23:47.412 --> 00:23:48.152 Everything's up to date. 00:23:48.152 --> 00:23:49.302 We don't have anything to do here!" 00:23:49.302 --> 00:23:50.622 And it just shuts down. 00:23:50.712 --> 00:23:50.992 Sure. 00:23:50.992 --> 00:23:51.812 It's not Node.js. 00:23:51.812 --> 00:23:52.492 It's not Ruby. 00:23:52.492 --> 00:23:55.532 It does that in like a couple hundred milliseconds, maybe, but 00:23:55.532 --> 00:23:57.952 it's still work that a server could just say, "Yeah, I know it's up to 00:23:57.952 --> 00:24:00.122 date, nothing changed," instantly. 00:24:00.302 --> 00:24:05.502 If you haven't read Aleksey matklad's writings on different approaches 00:24:05.512 --> 00:24:09.932 for different compilers, he makes sort of an A and B explanation of at least 00:24:09.932 --> 00:24:14.102 rust-analyzer versus rustc where rustc is a compiler, where rust-analyzer is 00:24:14.102 --> 00:24:16.482 sort of a live incremental compiler. 00:24:16.492 --> 00:24:20.322 There's one post that he has that's a really good proposal that's like, "Hey, 00:24:20.332 --> 00:24:25.612 if you're designing a new language now, start with an incremental server first, 00:24:25.872 --> 00:24:29.762 rather than a batch compiler, because most users end up wanting those features." 00:24:29.822 --> 00:24:34.092 Basically he makes a proposal that says, "Everyone starts with a batch compiler 00:24:34.092 --> 00:24:38.602 because it's easy, and then generally works towards an incremental compiler. 00:24:38.902 --> 00:24:42.102 And no one started with an incremental compiler first." 00:24:42.442 --> 00:24:43.102 Unless... 00:24:43.687 --> 00:24:47.827 well, I've talked to some folks about like Smalltalk more dynamic environments 00:24:47.837 --> 00:24:50.907 where it's meant to be sort of a live system that you are playing with and 00:24:50.907 --> 00:24:55.357 introspecting and poking, where you have a more interactive view of your compiler. 00:24:55.357 --> 00:24:57.817 So if you haven't read those articles, you should. 00:24:57.817 --> 00:25:01.607 And if anyone's interested in what exactly you are talking about right 00:25:01.607 --> 00:25:04.837 now, they should go look at matklad's blog, which is also tremendous. 00:25:05.562 --> 00:25:06.332 That is true. 00:25:06.762 --> 00:25:11.632 However, I should mention that I have read, I think, everything matklad has 00:25:11.632 --> 00:25:14.822 written on the topic and I think everyone listening to this should read about them. 00:25:15.102 --> 00:25:18.862 I also know that Alex has experimented with other languages. 00:25:18.882 --> 00:25:19.302 I don't know. 00:25:19.362 --> 00:25:22.612 Don't know if he's even active in Rust nowadays anymore. 00:25:22.882 --> 00:25:25.672 There was a point where I was thinking, "Oh, cool, rust-analyzer 00:25:25.692 --> 00:25:28.882 is the future, is going to take over rustc because it's incremental. 00:25:28.882 --> 00:25:30.612 It's already re implemented a lot of the same stuff." 00:25:30.612 --> 00:25:34.952 So like I was thinking of a future with like rust-analyzer, polonius for 00:25:34.952 --> 00:25:39.662 borrow checking, the new trait engine chalk, I think, but then all those 00:25:39.662 --> 00:25:44.014 illusions have kind of, been removed from me because polonius turns out may 00:25:44.014 --> 00:25:46.324 not be the actual, the right way to go. 00:25:46.344 --> 00:25:48.554 Maybe there's another way to do all of this. 00:25:48.784 --> 00:25:53.644 Chalk, I think is being abandoned in favor of rewriting it another time. 00:25:53.834 --> 00:25:59.694 And rust-analyzer will never replace rustc for a lot of reasons and does not even 00:25:59.694 --> 00:26:01.224 support all the things rustc supports. 00:26:01.224 --> 00:26:04.349 And I think we've already added things to the language that makes it 00:26:04.349 --> 00:26:05.809 impractical to have that approach. 00:26:05.809 --> 00:26:08.609 So rust-analyzer does like the best it can, but it can 00:26:08.609 --> 00:26:09.849 never actually replace this. 00:26:09.899 --> 00:26:15.059 But I do believe that if you gave someone a bit of money, which has given them 00:26:15.059 --> 00:26:19.059 a bit of time, they could like hack it and just have the server and instead of 00:26:19.059 --> 00:26:20.849 reading everything from disk all the time. 00:26:20.974 --> 00:26:25.754 Just that, just like have the interface be parsing cargo args, whatever, but 00:26:25.754 --> 00:26:28.264 then keeping things in memory instead of reading them from disk all the time. 00:26:28.594 --> 00:26:31.444 Cause I know the Rust compiler does cache things in memory, right? 00:26:31.444 --> 00:26:35.074 It has this query system by, by something like Salsa, which rust-analyzer 00:26:35.094 --> 00:26:37.674 also has, and if you query the same thing a bunch of times, which 00:26:37.674 --> 00:26:40.964 happens a lot during compilation, it doesn't actually recompute all the 00:26:40.974 --> 00:26:42.534 things it does, memoization heavily. 00:26:42.854 --> 00:26:45.534 So just keep that in memory, right? 00:26:45.564 --> 00:26:46.414 How hard can it be? 00:26:46.649 --> 00:26:49.739 I think the people that complain about how much memory rust-analyzer use 00:26:49.849 --> 00:26:53.729 can talk exactly about how hard that is, but I'm interested to see more people 00:26:53.739 --> 00:26:55.869 hacking on it, like, a hundred percent 00:26:56.040 --> 00:26:56.630 Me too. 00:27:03.118 --> 00:27:04.768 This episode is sponsored by Tweede golf. 00:27:05.178 --> 00:27:08.298 Tweede golf is a Rust consultancy from the Netherlands that you may know from 00:27:08.298 --> 00:27:13.198 their open source work like `ntpd-rs` and `statime`, or from their work organizing 00:27:13.228 --> 00:27:15.838 RustNL, the Dutch yearly Rust conference. 00:27:16.348 --> 00:27:18.718 I've worked with them for a couple years, and I would personally recommend 00:27:18.718 --> 00:27:21.728 reaching out to them if you need help building software in Rust, embedded 00:27:21.728 --> 00:27:24.688 or otherwise, or to book a training to get your teams up to speed on topics 00:27:24.688 --> 00:27:26.498 like using async on bare metal systems. 00:27:27.328 --> 00:27:29.888 Links to contact them are available in our show notes, and don't forget 00:27:29.888 --> 00:27:30.798 to let them know we sent you! 00:27:31.148 --> 00:27:33.228 Thank you again to Tweede golf for sponsoring this episode!