Transcript
[00:00:00] Welcome
Tim: Sometimes people stop everything else and decide to focus on that one thing that must change.
David Lattimore is one of those people. He’s currently on sabbatical working on a linker that will speed up builds for every rust project that compiles to Linux. It’s ambitious, it’s audacious, and it is fascinating to listen to.
Oh, and by the way, David is going to be talking at the Rust Forge Conference in New Zealand during the last week of August.
He’s gonna be talking about Wild, his new linker. And if you would like to attend with a 15% discount, please visit rustforgeconf.com/compose.
David: It’s exciting to be here and I love talking about linkers I’m sure we can go into some interesting technical details.
Tim: Yes, please. No, honestly, there are questions that have popped into my mind, ever since I discovered that there is more to a compiler than just being quote, a compiler. Like there are other steps involved, and this word linker popped up and I see in Rust, I’ve got this in my cargo file, I’ve got this option to, add, LTO link time optimization I can enable some flags and I’m really, I’ve been really curious about what’s going on there.
[00:01:26] Introducing wild
David: Sure. So, my linker is called wild, which, the naming comes because linkers traditionally, at least on, on Linux, traditionally end in ld. The original linker was called ld. And so subsequently, ’cause I’ve ended in LD like gold and mold and LLD and so I started with LD and then I—my plan is to eventually make it incremental—although it’s not incremental yet.
And so I added an I and then I just did a dictionary search for words ending in ILD, and Wild was the one that popped out at me.
Tim: Yeah it works, doesn’t it? I think like it’s clear what I mean, I think it’s clear and interesting.
David: It’s an
Tim: it all, yeah, it’s good. I also like that the idea, there’s a kind of a connotation of a project that’s attempting to potentially be, I guess a challenger to orthodoxy or potentially, or trying things new.
Is that how you see wild, I guess with the focus on incrementalism? Is that an indication that you are looking to build a tool that works differently than the others that you mentioned?
David: I’m certainly looking for opportunities where I can do things differently whilst being as compatible as possible. So I started the linker, I guess about a year and a half ago, so maybe September, 2023. I’ve been working on it more or less full time since then. And yeah it’s it’s still got, it’s still got a way to go, it is pretty usable already.
So I’ve been using it as my default linker for six months or so now for all my rust work. And
Tim: Is it one of those projects which you think, okay, maybe this is like a six week kind of thing, and then it’s actually there’s everything else to implement. Is that kind of how it happens? Or did you know that this was this big audacious project right from the start?
David: No, I knew it was going to be a big project. So I had actually attempted to write a linker once before, back maybe around 2017. And I lost access to the source code to that when I left my previous employer. But so I knew that running a link was a lot of work and yeah.
Tim: So you thought, okay, but yeah, I, okay, I appreciate that, and so when, actually, let’s start with the question of,
[00:03:41] How to opt-in to a non-default linker with Cargo
Tim: let’s say that I wanted to use something that’s different than the default. And there, as you mentioned, gold mold. Now wild. What do I do inside of cargo to enable that?
David: So the first thing you need to do is switch to using clang to link rather than, I guess it probably uses GCC by default, but if you switch to using clang, clang, then you can pass the appropriate link as to clang to tell it to use. While as the linker I think it is possible to use GCC, but it’s a little bit trickier because GCC, the way that you select the linker, it doesn’t let you pass a path to the linker. It just has a predetermined set of linkers that it supports and while isn’t current one of those linkers, so you’ve got trick it into using. And thinking that it’s using LD or something.
Tim: So if, if someone was to create their hobby linker, they would need to somehow patch or, get a patch accepted upstream into G CCC potentially to get that main line so that downstream users would be able to invoke it through the compiler.
David: I mean it the easiest way is to trick it. So
Tim: yeah, sure.
David: a directory with a file called LD in it, and it’s actually your linker. Yeah.
Tim: Oh I expected that you’d say you’re just sim link ld, but that sounds fine too. The and so that kind of goes to something which is interesting, that the compiler is actually invoking other tools. GCC, the GNU Compiler collection, or clang, the C Lang compiler as part of the LLVM project, they’re actually, in some sense modular, they’ve, they outsource some of their work to produce an executable.
And one of those is the linker.
David: Yeah, that’s right. And the other interesting thing there is that even when you’re compiling a rust program, it’s still invoking the C compiler in order to actually link your program. So the rust compiler, rust C doesn’t invoke the linker directly. In general, it can, but if you’re linking against, if you need to link against the C runtime, which you do for most programs, then generally you need to invoke the C compiler.
’cause the C compiler is the one that knows how to link against the C runtime.
Tim: Actually, so yeah. And that c runtime is bolted into your program. But if you had a bare metal target, so if you were genuinely writing a rust code for some compiler target with no operating system, then you’re completely on your own. But if you have essentially any operating system you’re going to bring all along.
The c runtime for the ride, which in kind of by des by design and partially, the way that, that, the way that’s created is you essentially require a c compiler to compile rust.
David: That’s right. Yeah. Certainly if you’re running, if you’re writing a desktop application for Rust, something that uses the Rust Standard Library on Linux, for example, then you need the C runtime. And so various object files that get added in, stuff that runs at startup to initialize that runtime and figuring out all those flags is tricky.
Tim: I, all right, so this is, so we as programmers, we write a main function, but that isn’t actually the what? That is not the function that’s initial, like initially jumped to by the operating system. When when the program is loaded, that’s actually looking for a different symbol, which is underscore start, isn’t it?
David: Yeah, that’s right. Yep. Yeah. On Linux, the default is under sourced underscore start. So there can
Tim: Which.
David: of initialization that happens. So if your program is dynamically initialized, then the first thing it will run will be underscore start from your dynamic loader. Will then run and do a bunch of initialization of loading all the other parts of your program. And then it will call underscore start in your actual binary, which will then do further initialization, and then it’ll eventually run main. So there can be quite a lot that happens before you actually get to main.
Tim: I feel as though we’ve just created quite a few like items on a stack of things we need to explain. Let’s see if we can we, because I think this is all fascinating, right? So it turns out that starting a program itself is quite hard, or at least is more complicated, and it looks I, I’m just gotta write a couple of things down.
We’ve got dynamic linking or dynamic library loading and so forth. I was curious though about this. Notion of that the compiler itself is tool like is it? We think of the compilation step as a we, we push a button, we do car cargo build or rusty if we invoking it directly. But there’s lots of things happening.
What are some of the other tools in the list as well as the linker?
Do we need to have an assembler, for example?
David: I don’t think it usually invokes a separate assembler. It might in some cases, but for the most part, like if we were compiling rust code, the rust compiler embeds, LLVM and ll VM knows how to convert, take L-L-V-M-I-R and reduce it to machine code, which then puts in object files and,
Tim: right?
[00:08:31] What is a linker?
Tim: And so what is the linker actually doing?
David: Okay. So the linker takes a bunch of object files and combines them together into an executable, and the linker needs to decide where to put the different sections of those object files. So each object file we compose of a whole bunch of sections. there’ll be sections that contain executable code, sections that contain read-only data, sections that contain read, write data. might be debug info. There’s lots of different sections in each of these object files. And the linker takes all of those and merges them together into an executable.
Tim: I have heard the like ELF files and DWARF files. Is that’s related to this process, isn’t it? Yeah.
David: On Linux, the executables are ELF files, which is, it’s a container format. It’s also used
Other unixes as well. It’s not used on Windows and Mac. They have their own formats that are different. dwarf is the debug info format. So dwarf is contained within the alpha usually.
Tim: So if I had a rust project comprised of let’s say 10 dependencies, which will probably end up mushrooming into 50 or 500, but let’s just say we’ve got 10 dependencies are will the rust compiler create? So if, will it create one file for each of those dependencies? And the linker job is to wire them together and produce something else.
So when you say an object file, is that the result of compiling one crate?
David: Sort of. So by default, rust will, order to speed up compilation rust, will use multiple code gen units so that it can have multiple threads doing work at one time. And each of those code gen units goes into a separate object file rust does then, however, take all of those object files and combine them together into an archive file. Which it calls, which it then renames as a lib. And so that lib is an archive file that contains multiple object files. from each code gen unit.
Tim: Huh. And this is one of the reasons why increasing or, changing that code gen u unit’s number changes like the behavior of the compiler, it speeds things up, it decreases things. So if the trade off there would be, if I have multiple code, if so, under the debug mode, by default, I think it’s 256 code gen units.
But you can reduce that down and release to, let’s say, one. You’d have a slower build that is somehow magically more performant. Is that correct? Or
David: Yeah. It gives the compiler more opportunity to optimize because it’s got visibility over more of your code at once. That said though, if you’re using something like link time optimization, which I guess you’d like to get into at some point, then maybe mitigates that because it, yeah.
Tim: ah, okay. So actually, yes. I actually hope that we’re gonna spend lots of time on LinkedIn optimization because I’m very curious. I’ve but I. I just wanted to lay out the land of, okay, so my compile, like when I hit, when I run Cargo build, it’s doing a lot cargo, is invoking rust sea, which itself is doing a bunch of work for along all of my dependencies, producing object files, which are probably going to be wrapped in a files, which are an archive with one or more object files at some point.
We now have a bunch of archives, bunch of libs, and we need to fuse them. I said wired together before, but I think that a better analogy would be like fusing them or like welding them together to produce one binary. ’cause in rust things are generally statically linked except for a lip C, but we can modular that out potentially.
Now. When, and the linker comes in at that welding step, or essentially taking all of those archives and fusing them together, is that,
David: Yeah, so the linker opens all of those object open, opens all the archives, finds all the object files and checks to see what symbols they all have. And then it makes various decisions about which archive entries to load. Because particularly for things like libey, we don’t always load all archive entries.
We will only load archive entries in certain circumstances if they define symbols that are referenced. once it’s decided which archive entries to load, then it. It might do, say, a garbage collection sweep where it traverses the graph of symbols in order to remove dead code. then it works out how much space it needs in all the sections.
Lays everything out and writes the file.
Tim: what is a symbol? Is this just a, I assume just a short string?
David: It’s a name, so it’s a string that contains a name of something. So that name could be a function, it could be a bit of data. Yeah.
Tim: And is there some sort of, how do, is there name spacing involved? How does a, is this actually part of the design of the compiler or the programming language to make sure that symbols that are generated in multiple places don’t somehow collide? I.
David: Yeah. So unless he put no mangle on your symbol, on your functional variable. Then it’ll get mangled by the compiler, which means it’ll take your name and it’ll take the name of the modules you’re in. It’ll take the name of the crate you’re in, and it’ll also add of a unique suffix.
So if you have multiple versions of the same crate, then you need to make them not collide. So it gets all of these different bits and pieces and mans them together in something that’s semi human readable. But we’ll have bits in it that look a bit funny.
Tim: But if I wanted to somehow destroy my own compilation process and I did accidentally, if I put no man on five functions from different and I tried to if I tried to push them all together it’s presumably almost non-deterministic, which function Act actually gets called or which variable is actually referenced.
Is that correct?
David: So you’ll probably get a link error. You’ll probably get
Tim: Okay.
David: ddu duplicate symbols ah,
Tim: Ah, okay. So the link is actually gonna protect me and be nice and say, Hey look, this is ridiculous. You need to figure this out.
David: Yeah, there are circumstances in which it’ll will allow multiple definitions of the same symbol. So if the symbol is declared as weak, then it will just take one of the definitions. But if
Tim: of the,
David: yeah,
Tim: one of the things that you mentioned earlier was the L-L-V-M-I-R, this intermediate representation, and would that be the place where this wink, this weak annotation on, let’s say a would, is that where something would be placed or, I, is that available in the rust language to be able to, I don’t actually know.
David: I like it might be, but I’m not sure. So most of the testing of these things, like when I’m testing the linker, I write little programs to test out different aspects of the linker. But most of my test programs I write in see, just because it gives you that lower level access to a lot of weird compiler features.
And I’m much more familiar with doing those things in C than I am in Rust. But..
Tim: Because essentially C does not provide, Rust semantics, right? It is essentially it’s some sort of higher level assembler to use a cliche. And if you wanted to, yeah, I guess really test out the linker. Yeah, of course you would use the thing that’s like easiest to write in. It makes me curious.
So the intent, and you actually, you mentioned this earlier, the intent is to have a, this is not a unique to rust problem like any while it should be usable by any compiler backend for
David: Yeah. For any compiled language, natively compiled language. A Linux ideally, yes.
Tim: But, oh, okay. So here was the an interesting, potentially not interesting question of if I wanted to support cross compilation, I would need a linker that also understands the calling conventions of the target as well. Is that right?
David: So calling conventions, the linker
Tim: Different.
David: anything with calling conventions.
Tim: Okay. Sure.
David: it does need to do stuff with the machine code sometimes. So there’ll be bits of, machine code that the linker needs to transform, and so it needs to understand the instruction set. There’s also a lot of of the main things the linker deals with are things called relocations.
So a relocation is an instruction that’s emitted by the compiler to tell a linker to put a particular value at a particular location. For example, when the compiler emits your fun, your compiler function it generates the machine code for your function. But wherever your function, say, calls another function or references some data. compiler doesn’t know where that other function is. It doesn’t know where the data is going to be. ’cause the linker is the one that decides those things. And so the compiler will emit a relocation to say, please put the address of function foo at this particular offset in my function. And so the linker is applying those relocations and some of those relocations maybe won’t actually get processed until runtime. a lot are comp, a lot are processed at link time. Even the ones that are processed at runtime, the linker still needs to deal with them ’cause it needs to transform them into a different platform and put them in different tables and stuff like that. and so when the linker is processing those relocations, sometimes it will transform the machine code into a more optimal form, depending on what it’s linking. So the link needs to understand those things. You mentioned cross compilation, so you need to if you wanted to cross compile to a different architecture, your linker would need to support that architecture. Yep.
Tim: In order to be able to facilitate these relocations because it needs to be able to basically generate the correct binary,
David: Yes. Yes.
[00:18:11] Link-time optimizations
Tim: The, so ag, yeah. Let’s start to chat about link time optimization. Now I have some naive questions to ask you, so I apologize in advance, but I I’ll start there. The so the first thing is.
Does the linker decide itself? Could the linker decide to inline code itself? Or do things like loop unrolling if it wanted to, or?
David: Generally not. So I guess there’s two kinds of that can happen in linkers. So there’s the transformations that I mentioned where the compiler has emitted one kind of relocation, and the linker transforms that into a different kind. And these are micro optimizations. They’re much templates, very localized.
They’re not crossing function boundaries. They’re just that little bit of machine code that relocation gets transformed. So they’re a kind of optimization but usually in linkers, we’ll call those relaxations. I think technically relaxations are maybe where they make the code shorter, but
I just refer to them all as relaxations, whether they make the code shorter or not. so there’s those, then there’s full on link time optimization, which is where, and that’s actually very involve, very much involving the compiler. And so you’re effectively running the compiler at link time. And the, it’s not actually the linker that’s doing those optimizations. It’s the compiler. And so usually the way that will work is you’re compiling your code, instead of having the compiler emit machine code, it will just store the, its own intermediate representation in the object file. And then when you come to linking. The compiler will take, will get invoked somehow. There’s a few different ways that it can be invoked and it will take that intermediate representation that was stored in the object files do the actual optimizations then. So it’s really the compiler that’s doing the optimizations, not the linker.
Tim: Wow. So I had this idea of, okay, let’s say have a constant, which is used in five places. The linker has somehow given information, and that constant might be the same in multiple dependencies. I just think of this, for some reason I’ve defined, I don’t know, pie in multiple places, and the linker can see that constant is used. And I wondered if the let’s say a static variable has the same value in multiple different crates. It would somehow delete all of the redundant copies and refer everyone back to a single canonical place where this thing is defined.
David: So there, there can be a little bit of that sort of stuff happening in linkers. So you can have merge sections in particular string merge sections where you have a section containing multiple strings. compiler, sorry. The linker job there in that case is to take all those strings and de-duplicate them. Similarly, you can have merge sections where the entire section is a single and the linker will look for other objects with the same section name with identical content and deduplicate them. So there’s a little bit of that happening there, but it’s very much the compiler needs to tell the linker to do that.
So the compiler needs to set the merge flag on the section, and it’s telling the
Please deduplicate this.
Tim: So the linker is really thinking about sections of memory.
David: Yep.
Tim: Time in some sense because I was going to think, the follow on to the constant or statics was whole functions of essentially if the signature matched, or let’s say even the resulting code were to match, could you essentially deduplicate across crates?
Although, I dunno how much completely redundant code that any given project might be, but potentially if I have multiple versions of a single crate and there might be just say a patch release, but for some reason I need two versions of it, there might be quite a lot of overlap between between that and maybe the linker could do a lot of janitorial work to clean up the code paths.
[00:22:12] Identical Code Folding
David: So, there is another feature that linkers can support called Identical Code Folding, where it does look for bits of code that are identical and deduplicate those. The caveat there is that kind of doing that kind of work in the linker is expensive. It can be an expensive operation and whilst priority is very much to be as fast as possible, particularly for development use. And so that kind of feature goes against
[00:22:35] Why is linking slow?
Tim: Yeah, because the, I have noticed that if I, again, we say we’ve got a hundred dependencies and I’m running cargo build. I get to 99 out of a hundred steps, and that 100th, the N minus one step, which presumably is the link step, takes a long time. It’s frustrating because you’re like, I just wanna get to 100% done.
And the. Related question is why is it that linkers have this insatiable amount of appetite for RAM and like compute, they just seem to take as much as you’ve got. Why do they need so much so much Ram and for example if you were compiling, let’s say the Linux kernel, it’s like hard.
David: Linkers do use different amounts of memory. So wild does use less memory generally when I’ve profiled it than the other linkers. That said, it does need to read all of your input files and it reads your input files by mem-mapping them.
The amount of ram will at least be as much as the sum of all of your, all of the object files that are fed into the linker. It won’t necessarily actually page all of those in. So if there’s large parts of those object files that are never referenced, then they effectively don’t count towards the memory usage, even though they’re mapped in. If you never actually touch those pages of memory, then they won’t count towards the processor’s memory usage.
But for every page of the input files that we need to read, at some point that will count towards the memory usage of the linker.
So my main interest is in making the linker as fast as possible, uh, making, making, I’m very interested in making a fast development cycle. And so I, I want the linker to be as fast as possible and some of the, of the designs of the way object files are set up in some cases are not ideal towards making the linker as fast as possible. So you’ll have bits of information that you don’t have at the right time, since you’ve got a structure your code in a weird way in order to figure something out and then go back and fix it up once you know the information later on. So there’s a lot of sort of gymnastics you’ve gotta do in order to get things because you don’t have the information you need necessarily upfront. There’s also, so one of the slowest things in linking is string merging. So d that deduplication of strings. that’s particularly a problem if you’re linking large c plus programs with debug info because they have just so much duplicated debug info
Tim: Huh.
David: all the header files duplicate the same, in same debug info over and over again the linker really has a lot of work to do. If I’m linking something like clang with debug info, then there might be six gigabytes of input to the linker. And most of that is just these string sections. And the linker just has to go through them and hash them and deduplicate them. And it’s just a lot of accessing at memory. And that’s, it’s
Tim: It is not free.
David: yeah, it’s not ideal from a making a fast linker perspective.
Tim: No. Do you think that, so there’s almost two kind of spinoff questions. One is, do you think that it would ever be feasible? We finish my sentence. Do you think it would ever be finished, feasible to have, in some sense a developer, like a debug development time linker and a release linker, something that is like heavily, that is optimized for producing very good.
Machine code or like almost a petition in the projects that would use.
David: To an extent you can get that just by switching flags. I think caveat there is that I think a lot of developers don’t necessarily there’s plenty of developers that don’t fine tune their builds. Like I, I hear of lots of people who are still using the default Linux linker on Linux, like using NU ld and it’s really slow.
And and yet. If people complain about compile times, then that’s like the first thing you should change. But certainly there’s lots of flags you can change to adjust that. Turning off debug info, if you’re not using a debugger, turning off debug info is helpful. Or if you are using a debugger using split debug info so that the debug info goes and, stays in the original object files and doesn’t need to be processed by the linker, that kind of thing.
Tim: Okay. And that. That’s actually how things work over in Windows land, isn’t it? Where the debug information is in a separate file.
David: Puts the debug info in a PDB file or something like that. I haven’t really worked on Windows, but that’s what I’ve heard that but you still need to generate that PDB file. So the other option is split debug info ’cause unpacked, which leaves the debug info in the original object files and it’s up to the debugger to go and find it when it needs it, which I
Tim: yeah.
David: A nice solution.
Tim: Yeah. That does seem like a relatively practical middle ground especially if you are, using a debugger during development and testing, which presumably that’s why when you have debug symbols involved at all, ’cause you probably strip them out at, for the release. Or the final build anyway.
Yeah, that that’s okay, but. The
the Cargo.toml file just provides me with, I’ve got LT O zero thin and fat slash one, but those are, that those would provide the sufficient specificity of what you are saying. So I need to go into this case. It would be clang or which would be cargo config to be able to really start to, adjust the a knob, the knobs correctly.
Is that right?
David: No, you can tell in your cargo, Tom, you can tell it that you don’t want debug info or you can
Tim: Oh yeah. Sorry. Yeah, we can turn that off. Yeah, sure.
David: or
Tim: Of course.
David: yeah, strip the debug info. And so actually, in terms of linking if I have input files that have debug info in them, but you tell the linker strip equals debug, like that’s a flag strip, which the rust compiler can pass to the linker. then linking is pretty much as fast as if you didn’t have the debug info there. So having the debug info present in the object file is not a problem as long as you tell the linker to strip it. At least in the case of wild and I think mold, I think LLD possibly still processes the debug info and then just gets rid of it after it’s processed it.
But yeah.
Tim: Yeah. Which is an interest. Yeah. I mean there was a, an side note there, which around architecture and one of the yeah, the. One of the points that you made earlier was the files don’t actually l don’t actually lend themselves to having the right information at the right place or the right time to really provide like optimal performance or optimal throughput.
And presumably that means that there is a bit of a co-evolution of file formats versus tools and I wonder when you’re going to call Mutiny on the ELF file format, for example, and say that its time is over or something and propose some radical new solution there.
David: I think to an extent it’s not necessarily even Elf, it’s the problem. It’s all the stuff that’s built on top of Elf. So it’s the way ELF is used, that’s often problematic. But yeah, I mean there’s probably things about Elf as well. Yeah.
Tim: Do you, I know that you, so your mo, your primary motivation is speed.
David: Yes. Yeah. I want so I spent about six years as a small talk developer, and I got very used to having instantaneous feedback from the compiler. Being able to edit stuff, edit code while it’s running, and just see the change immediately. And I guess I want to regain that feeling of spontaneity and instantaneous in a compiled language like rust.
Tim: So I’m going to have a bit of a meta discussion. So for people that have not used small talk when actually no. David, I’m sure that you can explain this yourself. When I am interacting with a small talk program, the small talk virtual machine it feels live. Why does it feel live? It’s this kind of, it feels very immediate, but what’s going on there?
David: I guess it because it is a virtual machine and because you, the, that virtual machine allows you to do things like new things whilst it’s running. It was designed that way from scratch to be a machine that can be updated while it’s running and, yeah.
Tim: Can you disprove? And one my memory would, I haven’t really used say the any of, I used squeak actually for which is like a educational vision, but the but you’re also interacting with binary files like the, you’re not really, there’s no source code really. It’s like you are actually in the environment.
It’s immersive.
David: Yeah, I guess that comes from having development environment is running as part of the same program as your actual application, which is a bit weird. And I’m not saying that’s a good thing but it does lead to some interesting outcomes. It means that you can have, your objects can override the way that they’re displayed in the development environment.
And very extensible and flexible and you can make changes to your development environment while it’s running, which can also break, if you break things that can be bad, but yeah. It’s fun.
[00:31:48] Are there areas where the Rust compiler and the Cargo build system could improve?
Tim: And the let’s say that you have infinite time and infinite resources. Are there other parts of rust or things that where you think, oh gosh, there is an opportunity to fix that and make it more, let’s say snappier.
David: Yeah, I think there’s heaps. I don’t think the linker is necessarily even the most important thing to work on. But I guess I gravitate towards independent projects. I can just work away on unencumbered and on by myself even though I should probably be more social and go and, contribute more to the actual rust compiler.
But I think there’s heaps that can be changed in the rust compiler and in cargo. And, to give an example, so in cargo at the moment if you tell cargo that you wanna strip your binary, so you wanna strip debug info from your binary, then it will go and rebuild everything. though it really only needs to change the flags that’s passing to the linker that’s an example of a change that, I should probably go and contribute, but
Tim: Principle should be trivial ish. But presumably because it was built that way and it works. Yeah, no, it would be fascinating. Yeah. Maybe it would be an a two hour job to go and find the right part of the code and then go and make the change and then push the PR up and see where it lands.
Yeah, no, that’s interesting because I have also had a difficulty with contributing to large projects, which is that I. My time of the available time that I have as quite patchy in the sense of I don’t have a lot of free time and I find it really difficult to commit to providing regular contributions to an established project.
And doing something by yourself essentially in isolation is one way to guard against needing to let anyone down or it gives you a little bit more freedom to experiment ’cause you’re not bound by code conventions and so forth and gaining agreement from lots of the rest of the community.
So I can see why working on something off to the side is like a, it’s quite a sound strategy for many people.
David: Yeah.
[00:34:04] On being an open source maintainer
David: By the same token I really like working with other people and collaborating with people. And it’s been great actually having, I’ve got a few very active contributors to Wild that have been contributing a lot of great new features which is, and
Tim: all right. This is it’s inverted now you’re in charge. The, do you find the process of, and so that’s really nice to hear actually that the process of becoming or being a maintainer of an established project is a really positive one, or at least it has been in the main the, are there any other sort of reflections on doing or behaving or I guess open source, being an open source maintainer or, is there anything that has been different than what you expected about that process?
David: No, I think it’s been, I’ve had other open source projects in the, in Rust that I’ve
On previously, and they’ve had contributions, but I guess they haven’t had the same sort of long term, people were there actually to work on the project. My other projects have more been they’ve maybe had a few repeat contributors, but for the most part it’s felt that they’re implementing a feature that they particularly want for their use. ’cause they’re a user and they wanna just implement the features that they want. Whereas this very much feels contributors who are there as part of the project who are there to help drive it forward and wanna see it succeed.
And yeah, that’s it. It feels like we’re a team working on that.
Tim: That’s neat.
David: that’s a different feeling and I quite, I really like that.
Tim: Oh, cool. I. Keep thinking about something that you mentioned about the structure of of, we were talking about our files and so forth and things not being quite in the right place. But I was thinking about this more broadly in terms of the way that you’ve structured your code and with the time. I’m curious about whether or not you invested a lot of time early thinking about the codes architecture and how you wanted it to grow over the time, or did you start with something small and then just radically hack and change to make, and then did you adjust the codes a structure or architecture according to new needs that popped up?
David: Yeah, it’s, it is definitely gone through a lot of evolution. And that evolution has to an extent evolved as I’ve gained. understanding of things I’m trying to solve. Early on I might have done something a particular way because I had a particular model in my mind as to how that thing worked. And then I discovered an exception to that case. I discovered some inputs to the linker that meant, oh, actually mental model of this was actually wrong. I need to actually, fit a different model. And so then I have to, make, restructure my code in order to fit that new model of how I now understand this problem actually needs to be solved. So it’s evolved quite a lot as time has gone by. But I think Rust makes it, rust, makes it a lot easier to make those big refactorings, like you can really let the compiler drive the change. And often, once you’re done making the change and everything compiles, then it, it works.
And, a
Tim: Yeah it’s almost a shame that’s a cliche because it’s quite true.
David: Yeah.
Tim: The because I wonder about that from two points of view. One is you’re now, you’re a team, and so you can’t really impose massive radical restructures on everybody anymore, presumably.
David: Even when I’m doing the code base is big enough now that even when I’m doing a big restructuring, it won’t be touching all the files. Like it might be a massive restructuring and say the way layout is done or the way writing is done, or, the way sim the symbol database is being built, or that, that kind of thing.
Then and as long as I am aware of what other people are working on, then I can schedule at work to reduce merge conflicts. There’s some changes at the moment that I wanna make, but I’m waiting for some other big pull requests to get in before I look to do those changes because I know that if I don’t, then I’ll get probable merge conflicts or someone will.
Tim: Do you, and you also need to have, think about things upstream in terms of you need to be able to be compliant with. I guess command line flags that are specified for other linkers, those must be very stable now. It’s not as though, or are people still introducing new experimental flags or features that that CC compiles are using?
David: Yeah. Yeah. There’s certainly, new flags are added all the time and there’s, and they’ve been being added for decades, so there’s a massive long tail of often very obscure flags. If you look through the Goo LD man page there’s so many flags and we don’t support a lot of them.
You’ll look at them and you’ll be just like, okay is anyone using that? I don’t know. Maybe someone was using it at the time they implemented it, but now they’ve stopped using it. And so I’ve let implementation of flags be driven by us finding real world real world projects that are actually using the flag we need to make it work without project. but new flags are being added and. Yeah.
Tim: Yeah, and there’s another thing of large projects collapse on their own weight over time. You could imagine being a longstanding project like the GNU linker and probably started quite simple. But now you need to support every architecture and every command line flag from, the early seventies until now into the future.
And that weight of expectation and stability means that. The sort of the code OSS in some way which limits your ability to really experiment, or at least that’s my presumption, which is where the opportunity for projects like wild come in.
David: Yeah, to an extent. A lot of the flags don’t have, like a lot of the flags are relatively simple. There’s plenty of flags in there that, you’ve got the think the code to pass the flag, and then you’ve got a little bit of code to process the flag and it’s pretty isolated and you don’t have to worry about it too much.
There are other cases where it’s a lot more cross-cutting and you, the flag really does change a lot. You can, as long as you’ve got tests of the flag, you can restructure the way things work internally and then verify that you haven’t broken that flag. So
Tim: yeah, and it sounds like, you write your little c program that exploits some little kind of corner case and then see if it behaves the way that you expect. And then hello fed magic, the, the next kind of, I was really curious about whether or not it’s easier to implement new optimizations or new ideas for optimizations in a newer code base. Are there any things in terms of, even though you’re focusing on speed, has a consequence of trying this project meant that you are now looking at new ways to get better output as well.
Anyway.
David: Mostly I’m focused on link, how fast I can link, and I guess I’ve written the linker from the start with the idea that it’s very multi-threaded and I do everything multi-threaded, and that’s certainly where you get on of speed from. But also I think my approach to multi-threading I don’t actually know how the other linkers do that, so I haven’t really looked particularly at the code of any of the other linkers because I just wanted to figure out things myself do things my own way.
And so I’ll observe the other linkers mostly as a black box. I’ll run GNU LD and see how it behaves with particular inputs and see what outputs it produces. And that’s how I’ve mimicked their behavior. Sorry, I don’t, I feel like there’s more to your question than that. I’m not actually sure
Tim: No, I was wondering whether or not one of the, there was a, when you mentioned that you were getting new contributions, I was wondering whether or not you had interesting optimization passes that you had not really thought of just emerge, or, and the hidden question is it easier to write system software in rust than it is to write system software in C?
David: I think it is. Yeah, I think it definitely is. Certainly, having a very multi-threaded program like Rust is great for multi-threading. It, it really if you make mistakes with the threading, then you tend to get compile errors. And scope threads are awesome. I’ve made so much use of effectively scope threads.
I’m using rayon for parallelism, but the same sort of idea. And so that lets me have. Data that’s just shared across the threads without any kind of locking or there’s no mut texts, there’s no if it’s read only access, then I don’t even need atomics. I can just access the data just as is. And I think that’s really great from a performance perspective. And Rust makes it easier to, or a lot easier to get that stuff right.
[00:42:46] Rust’s suitability for large projects
Tim: You already were an experienced Rust programmer before you started, but, um, the. Uh, but it seems like it’s reinforced your belief that this is a language for doing quite big things.
David: Yeah, I very much fell in love with Rust from the moment I first learned about it. Back around about when 1.0 was released. I was, when I first heard of Rust and watched a few videos and I. I could see, because I’ve been programming in CC plus plus, or c and c plus plus since the, since the mid nineties.
So I’ve been using C++ for a long time, and Russ just solved so many of the problems that, that I’ve encountered over the years in those languages.
[00:43:27] Changes in Rust style over time – indentation
David: And my, my learning of rust is certainly it’s always evolving. Like even in the time I’ve been writing the linker, I’ve seen my, the way that I write rust evolve and change.
As a small example the way that I’m using more vertical white space now than I used to, so I’m putting blank lines in. I had a comment early on in, when I’d started the linker someone who was working with someone who a screen reader, was having trouble with reading the code and they said, they commented on the lack of blank lines, and I’d previously always put blank lines in to have meaning, but then I was never quite sure where I. Where they needed to have meaning. And so I didn’t put them
Tim: Yeah.
David: Whereas now I’m pretty much just putting blank lines in. If there’s multiple, if I’ve got a big if else, then I’ll put a blank line before it and after it just to makes it a bit easier to separate those things out and see them visually.
And that’s, I’m still updating the code as I go, so it’s not like I went through and updated all the code, but, whenever I’m editing a function anyway, I’m adding in those blank lines and things out.
Tim: That’s fascinating actually. Could you want do you, did you investigate why it’s easier for non-sighted, people that are using screen reader technologies to pause code with blank lines, presumably because there are pauses added when it’s spoken or.
David: maybe, I’m not sure, and in fact I, I spoke to to someone at Rust Week who uses a. A screen reader and they didn’t think it would affect them. But now that I’m used to it, I prefer it myself. Like even as a sighted person, I just prefer, like, when I look at my old code from my old style, it looks very squished.
Tim: Yeah no. I actually quite like a relaxed style of code as well. I am continuously fighting one part of rust, which is its tendency to have this right wood drift. Especially when you start to match things. And then if you have results and objects and, sorry, result wrapping an option of something which might be an eno.
And now I’ve got three layers of match potentially and I’m still trying to figure out exactly the optimal way to do that. I tended to go with if else and sorry if let and, treat, but actually match on the exceptional case early and early return. And but yeah, no the it is interesting how much style or so presumably like the compiler doesn’t care where your white space, the compiler doesn’t care the way that things look, and yet it seems to have this really, we have this intimate connection with with code.
[00:46:01] Changes in style over time – error handling
David: And another place where my style has changed a little bit is in regards to panics. I’d previously very much followed the same school of Thoughters. I dunno if you’ve read Burnt Sushi’s writeup on panics, and basically the position is. Panics are fine for bugs, and if it’s not, if it’s not a bug, you should return an error.
And and I very much subscribe to that, I’m perfectly happy to call unwrap. If I think that this thing should not be none, and if it’s none, then that’s a bug, then I’ll call unwrap on it because, I’m happy if it’s, if it is none, then that’s a bug. And panicking when there’s a bug is fine in my per, from my perspective. However, where I’ve changed slightly is I’ve found there’s often benefits to returning a result, even in cases of a bug, especially if the bug is non-local and fairly cross. I’ve got cases in the linker where, for example, you I need to allocate space in one section of the code and then allocate address in another section of the code, and then actually write those things out and yet another section of the code, and those three things need to be sync.
If I allocate too much or too little space, then I won’t be able to allocate addresses correctly, or I won’t be able to write it out correctly. And so if I discover there’s too much space or too little space, a bug in the linker. and I could panic, but I found it’s often better to, in those cases, to return an error I can attach more semantic information. I don’t really care about the back. Basically. I know where I am in the code. There’s only really one path to get there. The back trace is irrelevant. What I care about is which symbol was I processing, which file was I processing, which section was I processing? That’s the sort of the semantic information that I really care about.
And I get that much more easily if I return an error rather than a result. So that’s how I’ve shifted a little bit in that
Tim: So you’re packing in, making use of the fact that you can result as a value and you can pack in whatever information that you actually need in order to handle that value. And yeah, in this case it sounds like your collecting up information from higher up in the call stack and then propagating the result back up to that.
So you can gather all of that information and then spit it out there is smarter, or at least it reduces the debugging time than doing it just panicking down the bottom.
David: Yep.
Tim: There is actually a. I think that error handling or idiomatic error handling is one of these areas where, I don’t know if the Russ community really has a unified story.
Or at least I think there’s kind of multiple levels of maturity in every project. And one of the things that people will lean on are trait objects quite a lot, especially when they start to intermix or intermingle different era types. And then they have this problem of, type ratio.
And I struggle a little bit to trying to give people generalized advice on how to handle errors because the tools that Russ provide you, I. Give you lots of options for being able to handle your specific case, but it’s not as general purpose as say, like something in the JVM where you just get the stack trace or what have you, and, or it’s just an exception which you then have to handle.
But actually and it strikes me as quite hard to provide like one canonical answer that will fit all cases.
David: Yeah, I think it’s very much dependent upon what the program you’re writing is or what the library you’re writing is. And in the case of a library, it’s tricky because you might have multiple different users that have different competing needs. Certainly I’ve worked, when I’ve worked in embedded before we had to pretty much avoid all code that even possibly could panic because. Not so much because we didn’t wanna panic as much as if we had code that could panic, then that would pull in the panic machinery, which would pull in all the formatting machinery, which was very large, and it would increase our binary size. And so from that
Tim: So just, the print line macro is not free.
David: no. Yep.
Tim: if you’ve got format_args! and format_args! needs to know how to basically format everything and convert it to a string. Turns out that’s pretty tricky for things like floating point numbers and other, basically everything else.
So yeah, I can imagine why if you only have a couple of kilobytes of ram or presumably tens of kilobytes but that’s still not free at all. That’s funny because I always thought that non panicking, we need panic free code. Partially be mostly because we just don’t wanna panic. Like we don’t want it to stop, but it’s also, we don’t want to include code that can handle a panic.
Is the way out there just abort or, and just, or. Essentially panicking. Your panic handler is a no op. You just jump to either an in bus, busy loop or you just shut down the device or something. Or
Can you,
David: case we returned results. Like even though
Tim: okay.
David: panicking is reasonable, like returning an error result is also. think you can do. I think it comes back to there’s lots of different contexts in which, and there’s no one way of doing things in rust because it really much depends on what you’re building, as to what the right approach is. And so diff different approaches work better and worse in different contexts.
Tim: And David, I need to let you go
David: Ah,
Tim: because I’ve dominated your time so far. I, I. But was there anything else that you would like to add to the discussion?
David: I guess
Tim: I.
David: wants to help build a linker, then, I’m always looking for more contributors. Come along to and file an issue or discussion or book some time in my calendar and we can have a chat about what you wanna do. And it’d be great to have more people come along and join in.
Tim: Superb. Now are you a social media person?
David: I have discord. That’s about it
Tim: Okay I will make sure that there is, they will try and make sure that it’s very easy to find wild and very easy to find how to take you up on the offer because I strongly recommend to anyone that wants to learn more and maybe contribute to do it’s a getting this over the line, getting this sort of stabilized or, it is stable, but bringing more prominence to this area.
It could revolutionize the experience of developing with the language a really strong, just hearing that this qualitative effect of like near, instantaneous builds would just be, would it, would, it would be glorious. And no. So thank you so much for dedicating so much time to it and and for being so generous with your time now and also supporting and nurturing other people who wish to contribute to the project.
David: Yeah, no worries. Thanks. It’s been a fun conversation.
Tim: Oh, cool. Okay. Thanks David.