Systems Software for Linux with Rust – Interview with Zeeshan Ali Khan

​

[00:00:00] Tim: When we started recording this episode, I thought we’d be talking about inter process communication and parallelism. What ended up happening was a huge discussion about Linux and Rust and C and the future of software. It’s a long episode, but I think you’re going to enjoy it.

[00:00:23] Welcome

[00:00:23] Tim: Welcome to Compose! It is our third episode and I have the author of a crate, in fact a couple of crates, who has the best logo that I think I’ve ever seen.

[00:00:38] Zeeshan Ali Khan: Thank

[00:00:39] The Story Behind the zbus bus

[00:00:39] Tim: Do you to describe the zbus and the animated gif or gif that greets people on the readme? There must be a story behind that.

[00:00:50] Zeeshan Ali Khan: Sort of, yeah. So there’s a logo and then there is the how do I call it? It’s an illustration, actually. Yeah, that’s what you call it. And yeah, this is done by Jakob Steiner. A great artist. He works on GNOME. He’s a designer. And the logo is done by Tobias Bernhardt, who is another GNOME designer. So as you can see, I’m very well connected in the GNOME world.

[00:01:16] What is the zbus crate?

[00:01:16] Zeeshan Ali Khan: But the library itself, I would describe it as just, crate that makes you helps you talk dbus, right? It’s an inter process communication protocol. and a lot of system system services on Linux. They use that. , so if you want to talk to, let’s say systemd or network manager and modem manager, all of those.

That’s where you need Dbus.

And when you, and that’s how most people come to this library. They, all of a sudden, for any reason, they need that. And they look around and they find a few libraries and Zbus. I would say myself, it’s the best one that you can find because a lot of people are porting away from the old one that already existed.

That we will talk about later as well. But yeah, that’s the description.

[00:02:01] Tim: So even let’s say I’m, is it possible with ZBus, which is your library, to, is there a Py03 wrapper around ZBus to talk the Dbus protocol from, let’s say Python or has that been, is, are you, is the library that you

[00:02:22] Zeeshan Ali Khan: I don’t think

[00:02:23] zbus – ease of use

[00:02:23] Tim: easier to use than, okay.

[00:02:26] Zeeshan Ali Khan: It is, yeah. For me one of the greatest compliments was when one of my friends he was not interested in Rust, actually, for a long time. And then when he did do Rust, he was doing some dbus, and he used zbus, and he said, oh, this is the reason I would love Rust now, and this is the reason I want to do all my dbus stuff in Rust because of zbus.

So that I really felt so good about that when that happened. So yeah it’s not there, I don’t know of any wrappers for Python or anything, but I’m pretty sure that they can be made and I think they would be much better off than the C versions the wrapper for C APIs.

Yeah.

[00:03:00] Is zbus a pure Rust implementation of the dbus protocol?

[00:03:00] Tim: Correct me if I’m wrong or if I’m mistaken, you have created a pure Rust implementation of the Dbus protocol. You have not wrapped, let’s say, I

[00:03:14] Zeeshan Ali Khan: Yeah.

[00:03:15] Tim: don’t know, is there a lib Dbus that you then created a library over? Yeah. So that,

[00:03:21] Zeeshan Ali Khan: I can tell you the story if you want to here , should I tell you the story?

[00:03:25] Tim: please do, please. Yeah. Love. I’d

[00:03:26] Zeeshan Ali Khan: Alright let’s do the story. , the origin story. My, my background is in systems programming, right? Because of that, I actually maintained a systems a component called Geo Clue, which runs on Linux.

A modern Linux machine has geo clue most of them have which the task of that service is simple. Find your location by any means. You can, and I wrote the re-implementation of it and the main, feature of the re implementation was privacy in mind. So that users have a choice to, to decide how much, which accuracy and if they want a relocation services at all in the first place, even.

And that was in C. So when I got into Rust in around 2016, I I wanted to rewrite that in Rust and and that’s a Diva service. The first thing I need is a DBUS library.

And there was an existing library called DBUS. Full name is DBUS RS on the repo. But and the create name is DBUS.

[00:04:17] Name squatting in crates.io

[00:04:17] Zeeshan Ali Khan: And so of course that’s what you find, right?

You search for it, you search for DBUS, you find DBUS. And it’s still the first hit like zbus cannot be the first hit because it’s the exact match They have squatted the name. And I tried to try to use it And this is the time when I was really new to Rust right?

[00:04:34] Starting the project on a whim

[00:04:34] Zeeshan Ali Khan: So I have a lot of trouble with all those types that were being mapped to C types, and they were getting really complex, actually, and I was having a lot of trouble, and I was at this hackfest, and I said to my friend, Sebastian hey how about I rewrite this in Rust and make a pure implementation.

I was really joking at first.

And he said, yeah, why not? And I was like I’m here. I’m at a hackfest. Let’s try it.

And I started writing, I’m a systems guy, so I started from the very scratch.

From the bottom I first worked on this. thing called zvariant, which is the encoding part of it, with no transport or anything.

And I did something in three days that kind of worked.

I could create a specific method that could be called. And I did some very basic socket, right? Just to test it out. And I got a response from the from the other side, which was correct, and I was able to decode it.

That was like, actually, this is not that hard.

And then that’s how it started. This whole project. So it is a pure [Rust project].

[00:05:29] GVariant

[00:05:29] Tim: Wow. Yeah. So the variant it’s GVariant isn’t it? And that is part of the GNOME project.

[00:05:36] Zeeshan Ali Khan: Yeah. So the, about, about GVariant

Dbus the wire protocol, the encoding of it that had some limitations, right? It’s, it started in 2003. So it’s it’s been more than two decades of this whole thing existing. And this one person Allison Lortie she decided that she wanted something a bit better.

So she wrote a new encoding called GVariant, which is super duper, like Dbus was already pretty efficient protocol, but she made it even more efficient efficient and added some of the missing features. And the idea was that it will become Dbus 2. 0. It wasn’t supposed to be something different.

But somehow, historically, all those things, moving parts, happening. She was a glib contributor, so she added this support in glib. Dbus to this day, when it, when you need to do any dbus stuff with glib glib api You have to talk this GVariant that the API is designed for this new, different specification slightly different than dbus and it’s not compatible with it, right?

And it internally translates that to dbus when it’s, when it actually sends us off and also the vice versa. So it’s a bit slightly inefficient. It’s not as inefficient as it sounds, but it, there is a bit of, Conversion going on there. So that thing became GVariant and that is now separate from the Dbus, specification.

And I was, when I started this whole thing, I was not just new at D Rust. I was also new at. Dbus, the protocol. I had used Dbus a lot. And I’ve implemented services. I had done a lot of things with Dbus before, but I had no idea about the protocol itself. So I had, I didn’t know that GVariant is completely incompatible.

So when I started, that’s why I named it zVariant. And I said, Oh, I’ll just support both. And that would be easy.

[00:07:23] Tim: If we didn’t say to ourselves that it was going to be easy, we wouldn’t start, right? The if we knew as much as possible, about how difficult things would be that we, the knowledge that we gained after being part of a project for a few years, then you probably would never have started.

And so this literally

[00:07:45] Zeeshan Ali Khan: That’s

[00:07:45] Tim: At a three day hack fest. And now eight years later, you are the maintainer probably of the most performant, most easy to use IPC system in the Linux ecosystem.

[00:08:03] Zeeshan Ali Khan: Sort of. The g the there, the other bindings, the C bindings, they, I would say they’re more efficient at the moment. I’m actually working on that. So I found the hotspot the bottleneck of my performance. All the CPU usage, basically. And I have a pretty, I’m pretty confident that I can make it more efficient than the C implementations out there.

So yeah, I’m currently working on that actually. So hopefully it will happen. But at the moment, it’s not the case, I would say. But they are I would say almost as efficient and they are that’s one thing is for sure, they are a lot more easier to use, like Zbus is a lot more easier to use than the C libraries or the bindings to the C libraries.

And that was the idea, so I started with making it possible that it’s as easy as possible. That was the whole idea. Yeah. And about starting things like when you start things because they’re easy that’s a really cool thing that it happens that we think it’s easy but there’s also the other way around like sometimes there’s a thing and you think it’s this is going to be super difficult And you don’t start it you’re procrastinating for years or months or whatever and then you start it and then after a few days You’re like, oh, I’m almost done So it wasn’t that hard.

So there are like two sides of it.

[00:09:19] Tim: Yeah, it’s, and actually I am curious about your impressions about the phrase systems programmer and the Rust programming language, because I don’t have a systems background. I was, I have a data science background actually. And so that was, that means Python and the. And I think that over the last couple of years, has outgrown the phrase systems programming language.

Like it’s become a much more general purpose. But I actually think that there are a lot of people who are intimidated, I don’t know, by like low level systems details unnecessarily, but actually. If you spent a couple of hours, maybe you would find that actually it’s much more approachable or accessible than you originally expected that you don’t need a PhD to start.

[00:10:22] Zeeshan Ali Khan: Yeah, that’s that’s certainly true. It’s it’s actually funny to me because what I have noticed is that there’s a lot more, there are a lot more people from web and from cloud and all those backgrounds than they are from systems program programming background in Rust, which is very surprising to me because for me the main thing about Rust was that it’s, It’s a systems programming language.

That’s what it’s designed for. So where are all the systems programmers? So I rarely meet actually systems programmers. I mostly meet people from other backgrounds. And and I was very surprised also, like it’s, it went into crypto world. It’s like most of the jobs that you see are from crypto world.

And you’re like, this is not what it was designed for. So

[00:11:04] Tim: Yeah. And actually probably C is a lot easier to write a lot of code. cryptographic primitives. You’ve got more access to constant time operations, you have at least easier access to intrinsics,

[00:11:17] Zeeshan Ali Khan: yeah,

[00:11:19] Tim: gets out of your way a little bit, whereas the Rust compiler is much more opinionated.

[00:11:25] Defining Systems Programming

[00:11:25] Zeeshan Ali Khan: I actually wanted to discuss that a bit later about coming from C background, but the first thing about what you asked about systems programming, um, Defining systems programming is very difficult. I have learned that a bit the hard way because I’ve been interviewing people to, for a systems programming job.

And a lot of time they come and they say, yeah, I have systems programming background. Then I ask, and it’s completely different from what I was expecting. And they’re not lying there. They do have systems programming from what they understand it. The main thing is for me is that the system, right?

The operating system, you have some idea of how the components on your systems are talking to each other. What are they doing, basically? And A lot of times these people come for interviews, they are when I ask them like, Okay, so system D what IPC they talk, right?

And they’re like, I don’t know. I’m like, but that’s how can yeah, there’s very different definitions and I can’t, Put my finger on one specific one, but for me from my background, it’s like knowing the systems services how they work essentially and all the essential features of them, like what do they do for example, if there’s a service, for example, and it’s, you don’t want them to run, be running all the time, right?

If they’re not being used, you want them, the process not to be running, but how would it work? What Starts the process when there is a call for them. And what is the process for that? And the answer is, of course Socket activation, which systemd’s, the systemd’s feature but Most people wouldn’t know about that.

So for me that’s systems programming and also Using those libraries and those services like to have some idea of how to use them when you need to. That’s also systems programming skill, right? So that’s how I would very vaguely describe what I mean by system programming.

But of course, definitions vary.

[00:13:16] Tim: They do. And see, for example, my definition of systems programming would be something, would

be something closer to middleware. So systems programming, involves writing software for other programs or And at this point, there are two questions that have been shouting at me, in the back of my mind.

[00:13:45] Introducing Zeeshan (Finally)

[00:13:45] Tim: One is we need to introduce you. Who are you? And why are you talking to us?

It’s now nearly, we’re over a quarter of an hour into the interview. And I haven’t given you the courtesy of allowing you to introduce yourself. And the other thing that I think we need to discuss is what is IPC or what is inter process communication? Yeah who are you and why are you right?

Why do you write dbus? Inrust.

[00:14:18] Zeeshan Ali Khan: I’m ZK . The correct pronunciation is Khan, but you can call it Khan as well. It’s okay. Yeah.

[00:14:25] Tim: Oh, sorry, I shouldn’t have interrupted, but

[00:14:28] Zeeshan Ali Khan: no. It’s okay.

[00:14:29] Tim: so is that the anglicization of the script? And,

[00:14:36] Zeeshan Ali Khan: Yeah, it is. Yeah. It’s I think in Pakistan, we call it Latin, like you write it in Latin, Latinization or something. But yeah, it’s it’s that. Actually, no, it’s called Romanization or something. That’s weird. But yeah. So

[00:14:50] Tim: Anglicization from English Latinization about the script Romanization because of the Latin. Anyway, sorry from Romans languages, but yeah, that’s yeah. So yeah. Thank you. Thank you for that. Sorry about the division. Carry on

[00:15:03] Zeeshan Ali Khan: no worries. So yeah my, my nation is called Pashtuns and we are everyone you meet from that nation is called their last name would be Khan. So it’s it’s quite common because of that.

[00:15:13] Guest’s Background and Journey

And yeah, I’m from this country called Pakistan originally. And I moved to Finland and the, in 2005, it’s been almost 20 years of that.

[00:15:24] Zeeshan Ali Khan: And I I went there to work on something called MIMO or MIGO project. People know it from different names because I changed names over the time. And the idea was to make a device by Nokia that is Linux based. And it was in the beginning, it was just internet tablets, but then we started also doing these phones at some point because they wanted to become mainstream, right? Things didn’t go in the end so well. And Yeah people know about the story, what happened to Nokia, so I wouldn’t go into that. But for five years, I was working there, and I was doing what I would call systems programming, but also more embedded systems programming. And that’s, that was a bit different in the sense that we were mostly using like GNOME software, the stack but we are putting our own UI on it, and underneath there were our own drivers.

A lot of them were women proprietary, but there was a lot of stuff that was open source and even Nokia was like funding creation and development of a lot of open source projects. They were part of GNOME basically for a long time. And and we had a really good relationship with with GNOME and then later it was Qt became very important there.

So I was interacting a bit with that too, but I was mostly a GNOME guy. So basically doing desktop stuff. But some do it in a way that it works performantly on and then a better device as well with resource constraints. So that was a, it was a very nice challenge. And I got because of that background, I got so used to having things very optimized and having things very like very small.

And it’s just in now in my, I would say DNA that I need things to be super efficient. And that’s why I got into Dbus as well, because of that. And and we can define IPC at the same time as we it’s just one process talking to another, right? On, on Linux, on I, I I guess all pro all operating systems, you call them processes, right?

Like, when you launch a program and it’s running, it’s a process then. So that living instance of a program is called a process. And when that needs to talk to another one this is called an IPC. And this can be a very low level thing. Like a lot of IPCs and RPCs I’ve seen, they’re actually pretty low level.

They think in terms of messages. So you have a message and you have defined an encoding and. When you’re both sides agree to what the encoding is how it will be encoded and decoded then they can talk, basically talk to each other. And one side sends the message, the other one decodes it and figures out what message it is.

In Rust world, we will have an enum, right? Usually oh, the other side sends me a command and what command it is and you match it on it and stuff like that. So on a basic,

[00:17:56] Tim: this might, so even more low level, I, so something like zero MQ, for example,

[00:18:04] Zeeshan Ali Khan: yeah,

[00:18:05] Tim: yeah there are two programs essentially, two processes and I’m sending messages between one and the other. And at the, on the receiving side, there is essentially no knowledge yet about what the message will contain.

Is this what you mean by, you need to define both a transport layer, a way to send bytes between two passages and an encoding so that both sides can actually make sense of the bytes, the bits that are being sent between one

[00:18:39] Zeeshan Ali Khan: yeah. And they’re like self describing formats, right? Like JSON, where you don’t need to any more information from the other side, except that it’s JSON. And then they can, Sorry, decoded. But in, then there are non self describing like DBUS, which is a binary protocol. Everything, as I said needed to be super efficient.

They created a binary protocol so that it’s not for human eyes, right? So it’s only for programs for code for programmers to decipher. It, it doesn’t need to be self describing. All you need is how to. how to decode it. And the how to is is the important bit.

And it’s they also define like alignments. This, it is defined so that you can read directly from the encoded message. So you don’t need to put it in, into another thing, right? Like you, when you’re decoding it, you don’t need to deserialize to another thing.

[00:19:32] Tim: So this is known as zero copy.

[00:19:36] Zeeshan Ali Khan: Yeah. Yeah, so it was very much designed for zero copy reading or deserialization or whatever you call it. And it’s the case for the DBS. That’s the important bit. I have this condition that I forget what I was talking about. I

[00:19:54] Tim: to remind you of this. It

[00:19:55] Zeeshan Ali Khan: was introducing myself.

[00:19:57] Tim: Yeah, that’s right. And also, by the way, you described the internals of IPC. That’s okay. So you are at Is it MeMo? And the reason why I say that is because I wanted to interrupt you at that point and say, I think I read every blog post about MeMo and there were a few other names or It seemed like there was a lot of A lot or about between 20, 2003, 2005, it seemed like Linux on mobile might actually happen.

And in fact, I think Nokia did actually produce one really good phone just, and then got acquired by Microsoft. And then needed

[00:20:43] Zeeshan Ali Khan: just, ​

[00:20:48] Tim: Is it

[00:20:49] Zeeshan Ali Khan: So they first set up, yeah, it was N9. And I think N9.

[00:20:56] Tim: N95 is the mask.

[00:20:58] Zeeshan Ali Khan: It’s there was a developer version of it with a keyboard. with a slide, sliding keyboard. But the main one was called N9 at the end, but they had already signed an agreement with Microsoft of partnership. So they were going with Windows or whatever it was for in the future in any case before N9 was produced.

We knew that the ship was going down. We left so at that point I left and I joined Red Hat. And then I was doing. Purely desktop stuff. So I was not doing like I didn’t need to care about super efficiency anymore. And I was there for five years and I was doing mostly C in all this time, but there was also another language called Vala.

[00:21:36] The Evolution of Programming Languages

[00:21:36] Zeeshan Ali Khan: Because I was also tired a bit of C because it was just too low level, right? So I needed something a bit higher level that can be as efficient as

[00:21:45] Tim: Isn’t that a scheme?

[00:21:46] Zeeshan Ali Khan: Vala sorry Vala.

[00:21:47] Tim: Vala a scheme? Or, oh no, I thought it was a Lisp dialect. I,

[00:21:52] Zeeshan Ali Khan: No, it’s not at all. Actually, it’s it’s based on C sharp. The syntax is C sharp based, but it compiles, transpiles, as people call it, to C. So it was basically writing C in a higher level language like C sharp. That was very convenient. And I wrote my, one of my project called Rigel, it’s a I don’t know if people still remember this thing called DLNA or UPnP, which is about talking from one device to another, it was more like an RPC, not IPC, but RPC and it, most, a lot of media servers, media renders, TVs still do that actually and that, that’s what I was working on, and it’s an open source project, Rigel, and it was done in Vala And my library was in C.

So I was doing this thing called this thing, that I was doing libraries in C and the application I would write in Walla. And but then like when GNOME like when Nokia, that was like going in the other direction and I left Nokia and Walla was The interest was going down very fast.

And then Rust came along and that interest in Vala, basically a flat line. It’s not it’s still like people really like it in GNOME still, and they still do it some people, but the number of people doing Vala is like really low and there’s a lot more people interested in Rust now. So I think Rust basically killed Vala at some point.

But I was still. Interested in it. So even at Red Hat I was doing Vala until I got into Rust and then I was like, okay I don’t need Vala at all anymore. It’s Rust has,

[00:23:19] Tim: Yeah, I think that there are a few languages that are essentially C With different syntax, and for example, there’s an extension language for Python called Cython which allows you to write native extensions, and, But the thing that gets me with these languages is they still have memory safety issues and you can still have null pointers.

[00:23:48] Zeeshan Ali Khan: yeah. Null pointers is very common. I think most of my time was spent on that. And that’s actually, that brings me to why I wanted to rewrite GeoClue, right? Because all the bugs that I was getting, I was not that interested in maintaining it anymore. And I was the maintainer. Nobody wanted to take it. And all the bugs I was getting were crash reports.

Basically, and because of some pointer getting in a different way, and the funny, not so funny actually, in that time, was that I would solve a crash, and it would solve that specific crash, but And that fix would introduce other crashes that would come out later. And I was like, Oh, I have to do this again.

So I was getting really tired of that. And I was like, if there is a way to, to just completely remove them, why not? So I started rewriting in in Rust and all these years later, I think it’s been four years since I started Zebus or maybe even five. I think at least four years now.

And I still haven’t read it in GeoClue.

[00:24:49] Maintaining Open Source Projects

[00:24:49] Zeeshan Ali Khan: I actually, in the between, gave up like that project because Zebus itself became such a big project and it, I have to do so much for that now that I just don’t have the time. So yeah I’m mostly just doing Zebus now, not GeoClue.

Are you on mute?

[00:25:06] Tim: Sorry, the, so your GeoClue project is currently, it works, but as far as, but essentially you have the problem that you probably, you are the quote, the maintainer or the owner of the project. However, you don’t have enough capacity to actually,

[00:25:30] Zeeshan Ali Khan: Yeah, so I have given it up like to other folks who were interested in contributing and I was like, yeah, you are contributing a lot. Please, just maintain it from now on. And they also, at least one of them needed for their work. So I was like, yeah, it makes sense that for you to take it over because I for me, it wasn’t work project anymore.

It was I wasn’t getting paid to work on it at all. And I just didn’t have the capacity. And as I have a kid now, so it’s been two years that I have a little one as long, as

[00:25:56] Tim: I

[00:25:57] Zeeshan Ali Khan: it’s,

[00:25:58] Tim: is it’s challenging. The interesting thing about this that I think is a more broad or more general discussion that so much work gets pushed onto so few people. In open source, the, and the, and it even to hear, Oh, look, now ZBus is doing very well,

that doesn’t reduce the amount of work, although it probably get different bug reports, or at least different feature reports but it seems As things become more successful,

[00:26:32] Zeeshan Ali Khan: yeah,

[00:26:33] Tim: the job almost becomes harder because you don’t want to push out a release that is broken or

[00:26:41] Zeeshan Ali Khan: Of course.

[00:26:42] Tim: And things like this,

[00:26:43] Zeeshan Ali Khan: Yeah it’s yeah, it’s certainly true. And Zebus is used quite a lot nowadays, like in actual production. People actually, even I met once a person who was using a beta version, like I had specifically marked it as beta because I I did not want anyone to use it in production, but he was using it in production.

I was like you have to immediately change to a supported version because i’m not gonna support you on this so yeah, it’s used, quite a lot. In by so many companies big and small Since it’s insane, but the contribution that are coming they are not They’re not in proportion. They’re people just assume yeah, this is the project.

It’s cool. And we will use it. It works. If they have a problem, they usually work around it. I joined a few companies where they were already using it. And then I found out that they were working around some problems. I’m like, why are you working around? Why didn’t you tell me? I could have fixed this.

No they just work around it. One of them actually had a huge document explaining how to work around a very small problem in Zebas that I could have in five minutes fixed it. But no, he had to write a whole document to work around it.

[00:27:52] Challenges in Open Source Communication

[00:27:52] Tim: I wonder, yeah, it’s very hard for commercial users of open source to speak to maintainers

[00:28:01] Zeeshan Ali Khan: Yeah.

[00:28:02] Tim: in a sensible way, I think,

[00:28:04] Zeeshan Ali Khan: Yeah. I think they just don’t have this sorry to interrupt, but I just that I think they just don’t have this thing that these are people you can talk to. I think they think of open source maintainers, developers as some sort of guards or some out of reach people who are really awesome, but they, Yeah, it’s yeah, that’s a bit of a problem. You can fix it. It’s you just tell them like, yeah, this is a guy like me. I’m working with you now, right? Like I’m not a God. I’m just like you. Yeah, it is a bit of a problem. Sorry. I interrupted you. Yeah.

[00:28:38] Developing Software in Rust

[00:28:38] Tim: But I am curious around how the process of developing software in Rust comes across. So there’s two aspects. One is the friction between communities. Essentially, you’ve already talked about how Rust in at least the GNOME, the context of GNOME project kind of, destroy, destroyed the momentum for Vala, let’s say.

And then the, but I think. With Dbus, if the Rust implementation becomes, if you remove these bottlenecks, you discover that. You could potentially go faster and you are easier to use that. Does that not cause issues with the people who developed the original code in a different language? And the other related question is what does it feel like to write rust for something that is spiritually in a different language?

You cannot force Dbus to change.

[00:29:52] Rust’s Compatibility Challenges

[00:29:52] Tim: You need to comply with It’s rules and Rust has its own opinions. And I assume that there is conflict at a technical level between what Rust wants or Rust, Rust compiler wants and the, what the specification requires.

[00:30:12] Zeeshan Ali Khan: If you don’t mind, I’ll answer your, Last question. No, second question first, because I understood that question and I know the answer. And the first question, I have to clarify,

[00:30:24] Tim: Okay. Sure.

[00:30:25] Working with (or perhaps around?) Serde in Rust

[00:30:25] Zeeshan Ali Khan: So yeah, so the second question about compatibility, right? So that’s something I learned fairly early on because I first provided an API like for zVariant.

It wasn’t zBus yet. zVariant was already there. And the first issue that was filed against it after the first release was, why are you not providing a serde API? And that’s like the standard in Rust. The answer that I probably didn’t give to that person, but the answer was really that I was new to Rust.

I was already struggling with Rust itself and and to go for serde, which I looked at and I got scared, especially to implement it, right? Serde, it’s so easy, so interesting, so good. But when you implement it That’s where things get a bit hairy, especially if you’re new to Rust. You see the lifetimes, you’re like, what the hell is going on?

So I got scared basically. And I was like, okay, I can’t deal with multiple things at the same time. I’m learning Dbus protocol. I’m learning Rust. I can’t be learning. Serde as well. So I didn’t do serde, but the, after that, I was like, okay, let’s create a serde based API because that’s what people would be interested in Rust world.

[00:31:37] The Complexity of Serde with Binary Protocols

[00:31:37] Zeeshan Ali Khan: So when I did that, I realized soon that there was a huge, there was a fundamental problem between Dbus and serde. And the reason is that, When you’re doing serde, implementing serde deserializer or serializer when you’re,

[00:31:53] Tim: I should pause there just, sorry. So for people who have not implemented a file format for serde, serde, so you will be used to the serialize and deserialize traits that you derive. They’re very common, we just, again, we just bolt them on to our struct and serde does the rest of the work.

Now, when you want to provide a file format, On the serialize or deserialize side, you implement different traits these are called serialize er and deserialize er, which have this slightly odd visitor pattern API, which itself seems very mystical and it’s difficult and complex. So carry on. I just thought I would just explain for people who

[00:32:47] Zeeshan Ali Khan: No, no worries. I should have done that.

[00:32:49] Tim: that’s okay.

It’s okay.

[00:32:51] Zeeshan Ali Khan: I should have done that. Yeah, as as Tim just pointed out it’s also for example, like when you use a specific format, right? JSON, SerdeJSON and TOML and all that. So they are implementing behind the scene for you, the serializer and deserializer.

And for those self describing formats that I talked of before ser json and toml and stuff it’s much easier and serde, it seems to me from looking at it from internal perspective, it seems to be very much defined very much created for these formats, so it’s much easier to implement it for those.

But it’s much, much harder to implement it for binary protocols that are not self describing in general. And also, as Tim pointed out, there’s the visitor pattern for de serializer. So de serializer is a bit harder to implement than serializer, but it’s similar. The good thing, the good news was actually that all the basic types that DBAs has are the same as in rust and survey world.

So they’re they match. So that was a good thing. But the ba the part that. That where the things fall apart is when you want to implement a container like Array, for example, dbus calls it Array, but in in Serde world, it’s called a sequence, which is like a whack, or or a slice, for example that when you encode or decode that’s where the problems comes because a Serde, when when you’re serialize, when you’re implementing this serializer or deserializer trait it does not tell you the type, because Rust doesn’t have what do you call that?

Reflections, right? It doesn’t tell you exactly what the type is. It’s very defined in a very generic way. You get a generic type that, oh, this type implements serialize and deserialize, and that’s it. And you don’t have no other information. So in, in Dbus, unfortunately, there is a weird. Requirement that when you are encoding an empty array, empty sequence, basically, you have to still pad for the first first element, even if you don’t have the first element.

If

[00:34:48] Tim: this is because of the alignment. Yeah,

[00:34:54] Zeeshan Ali Khan: because of this alignment requirement, you have to add some zeros in the encoding based on what type it is. And since you don’t know what type it is, what does it contain inside you can’t do it. It’s just impossible doing it purely with serde itself, with the serde API and Rust.

[00:35:10] Addressing Serde Incompatibility

[00:35:10] Zeeshan Ali Khan: I had to implement a requirement in there, so that your type doesn’t just have to implement serialize or deserialize depending on what you’re doing. But also, another create, which I called type which gives me a specific string, which is in Dbus, defines what is on, in, inside, right? Because it’s not a self describing format, as I said.

So to decode a series of bytes that you receive from the other side, you need something to tell you what’s in there. And that thing is called a signature. And the signature is just a string that tells about each. Type that is encoded in sequence in there. Okay, this is an array of strings, and then now a string so basically a struct.

Everything is a struct of different sizes, right? And it describes that. So I had to introduce, I was so annoyed by this, because I really did not want the, people to have to do this because you have a random type, right? Like from a different crate and you have this orphan rule in Rust.

And now all of a sudden you need to, for Dbus, how do you encode it or decode it from Dbus, right? You need. The signature, all of a sudden, and it’s more work serde, deserialize, and serialize, they’re so common, all those traits crates they provide serde features usually, and they provide this implementation for you, but they will not do that for Zebus, because as famous as it could be, it doesn’t matter because it cannot come to the same, Fame as it says it is and they will not go that far for it.

So it was really annoying, but that’s where the incompatibility happened between the two worlds and I had to do that. But I did provide a macro. So that you don’t have to do it manually. It’s really hard to prove, to always implement that. But it’s I have, I’ve done that.

So it’s not that hard. So because in the end, I wanted to make it as easy as possible. And this is as easy as it could be because of this fundamental incompatibility between these two worlds. Now we can talk about your first question, if I have answered your second question.

[00:37:16] Tim: I think that no I appreciate that there is I only have a half thought in response, which would be something along the lines that there is a, I want to shake serde and say, look, I wonder if we, so the thought is, shouldn’t we have some way to tell serde something extra? Can we provide some sort of context about the type that we, the information that we expect, and then we could go back to, I wonder if we have just created the same the page of documentation about workarounds in the file format.

Maybe we should talk to David Tolman. However, the, I actually think in the case of serde, that it’s design means that adjusting it now is going to be so hard that effectively you need the page of documentation or in your case, the macro. To work around it. I think I have this personal grumpiness about how prominent serde is for serialization and deserialization, because I think that it forces itself.

Anyway, I wish that serialized and deserialized were part of the standard library and that they were slightly more neutral or

[00:38:45] Zeeshan Ali Khan: Yeah the thing is that this specific issue was filed already by the previous creator, the dbus RS, which was a library, which is a wrapper around a C library. And I think they wanted to also move away from C library and create a new API based on serde. So they filed this issue already and they discussed with David Tolney and in the end they themselves closed, the issue.

The issue saying that, okay, I think this is going to require breaking serde API. And that’s not something you want but to make it actually even go even further one step, because I don’t want people to do having to do this. So all the famous crates, let’s say time and What is the other one?

Chrono, and all those kind uuid, everything, all those crits that are the most famous one that provides types that are used all over. I in Zebus have provided these features that are that when you enable it, it provides you serialization, deserialization for, not deserialization, but the type.

implementation for you. So you don’t, in most cases, you don’t actually have to do this, but mostly for your custom types, for which you have to implement serialize, deserialize anyway. So you might as well derive one more thing. And that’s, it’s not so bad. So I think in the end, what I have ended up with is not so bad.

And I can always add more types in in ZBus if Someone comes around anything so they don’t have to work around actually

[00:40:06] Tim: you very much for doing the work to,

[00:40:08] Zeeshan Ali Khan: No worries, and there’s other people also contributing recently actually someone contributed more types and stuff. I think you there was another question from

[00:40:16] Social Dynamics in the D-Bus Community

[00:40:16] Tim: there was, and the question was more along the lines of the social aspect of, do you think that people in the dbus community are, you mentioned, clearly, I should finish my sentence, they, is the hostility to you as someone coming in from the outside who didn’t know dbus when they started? I suppose now you have established yourself as being, Part of the Dbus community at large, but the Dbus is very important in the Linux world.

It even though it’s weird, you’ve got these strange paths and okay, fine. But and the yeah, so I’m, I am curious about whether or not you have had any pushback from people who have said it would have been better if you had just made the C thing better.

[00:41:21] Zeeshan Ali Khan: I, I actually never had that because for C we already had a library like Gile provides GIO, which already does this in a, and it provides as nice an API as it possibly could for C, it’s like there’s a limitation that you cannot make it much any better than that, I think.

And actually, that was my inspiration. So I was used to that. So I wanted something similar in Rust world for yeah, for Rust people. And then it grew out and I realized that I could actually provide much better than that. I think C and C they have reached their limitation and there’s already an API and nobody has asked for it.

Except for one person who was trying it. to do this, but they were not looking for a better API in C. What they were trying to look for was trying to introduce Zebus as a dependency into libraries like GTK a system library in C. And they were hoping that they can snug it in so that Rust is a dependency of core GNOME and Linux libraries.

And they were, that’s what their main agenda. Which I agree with, but I was telling them like, there’s, it’s going to be hard to get it and make it a C, good C API and then being able to sell it that, okay this is a better C API than the other one. And that one cannot be improved. In the same way.

I think that would be a harder sell. So I think that this would, this could be a wasted effort. It would be a big effort. And if it’s wasted, then I really don’t want to do it. There is one.

[00:42:56] Tim: do you want to talk a little bit about your thoughts on that?

[00:43:00] Rust in Linux Systems Programming

[00:43:00] Tim: About Rust and Linux, not in the kernel, but I suppose in user space or maybe in kernel space as well, but in all of the utilities and things.

[00:43:12] Zeeshan Ali Khan: yeah, so I think it, it also ties into what you said earlier about you coming from a web background and I was talking about a lot of people actually come from web background and other backgrounds, not from this Linux systems programming background, and that creates a bit of friction. And the main project where this friction arises is cargo for us from Rust world.

Like I’m now part of the Rust world. I’m mostly I really understand the Attraction of cargo for people from other worlds. They see cargo as some kind of like amazing thing that solves all problems. And actually I don’t know if you would be surprised to know this, but it’s not the case for syst Linux systems programmers.

That’s actually the project that they point to that’s a very good excuse for C programmers to say, Why should I learn Rust? It has bad tooling.

[00:44:06] Challenges with Cargo and Linux Systems

[00:44:06] Zeeshan Ali Khan: It doesn’t have good tooling and in a way they’re right, because they’re expecting that a very similar tooling to exist for Rust as well, that they’re used to foresee, because when you’re a system programmer especially an open source system programmer, you have to worry a lot about you provide a package that is very easy for for distros, for Linux distributions to, to package and what and give them something they’re used to and also to be able to cross compile in the same way that they’re used to and not what how cargo or there’s this thing called I forgot the name.

Cross. Yeah. Cross tool. They’re not used to that. They don’t know what that is about. And where is the my binary? And especially one aspect of it as installation of because in the rest of the world, we are more used to of like single binary, like it compiles to always the same binary as the name of the cargo.

And that’s it. And then you run it. or you install it and you call it with the same name, that’s the end of it. But in, in system, Linux system programming, you always have auxiliary data and that you also have to install. You don’t have just the binary. And also, the binary, where do you install it?

So all that stuff especially with installation, that is not there in Cargo. There is a there are issues. And they are, they’re there for ages. Nobody really does anything about it because of this friction. So the cargo people, the people who actually work on cargo and contribute to it and are into it, they’re Rust developers, right?

So they’re, they’ve already bought into Rust and they’re very passionate about it. And

[00:45:37] Tim: So I think what you’re saying next is that, they don’t. They don’t want to teach, they don’t want to change, they

[00:45:48] Zeeshan Ali Khan: no, it’s not about, Yeah, I think it’s not about being able wanting to teach. It’s more about they want to care about their own use cases, which is natural. Like everyone cares about their use cases. I went Canadian for a bit for some reason. I’m not a Canadian. I have family in Canada, but I’ve never looked there myself.

Anyway they don’t naturally they care about their use cases, which are not Linux systems programming. But on the other hand, the people who you are teaching. trying to sell it to Linux system people they wouldn’t get into it. So it’s a bit of a chicken and egg problem. They wouldn’t get into it because cargo, the tooling doesn’t offer what they need.

And if they don’t get into it, they wouldn’t solve it themselves because they’re not interested. So it’s a bit of a chicken and egg problem.

[00:46:32] Efforts to Integrate Rust into System Programming

And I’ve been trying to Actually fighting this war, actually, trying to get Rust into Linux systems programming. And that’s why one of the reasons I started this project, right?

[00:46:41] Zeeshan Ali Khan: Now like one year ago, I started this other project calledbusd, which is the daemon. I think I should have mentioned this that dbus usually typically involves a broker. So you don’t talk to other processes directly because it’s a bit of a higher level. Protocol, because you don’t have just messages passing between.

You usually have an API and that API means that you have a service that provides a specific API and and the clients call to that. And. Having this broker in the middle takes care of a lot of things for you and including the fact that if you are in you have any problem with your service or your or your client you want to debug it what’s going on?

Why am I not getting these messages? You monitor this broker, and you get your answers. So this broker solves a lot of problem problems. And all these brokers currently are written in C. And they’re old. And I want to change that, and I want to do it in Rust. And the good thing is that with ZBus, making it Making dbus so easy getting the first version done was a matter of two weekends.

So I, like all the basics all the essentials were already done. You could talk through it and it would work. But of course it’s a lot more, there’s a lot more to it than that. So that project is part of my effort to get Rust into systems programming. World system. World, linux world.

But it’s a bit of a problem because of this cargo and and the thing that is used in the sea world right now is called Mason, which is quite sort of new. It’s it’s Python based. It’s in the sense that it it’s implemented in Python and it’s, it’s the syntax is very Python ish.

It’s like a configuration thing. I guess viewers might have heard of something called CMake or AutoTools, which are famous for being notoriously bad. But nowadays

[00:48:30] Tim: I, I actually went to a talk at LinuxConfAU in January, 2020. Like just. as the gates were closing.

[00:48:45] Zeeshan Ali Khan: right?

[00:48:47] Tim: and he was there presenting Mason. Mason?

[00:48:51] Zeeshan Ali Khan: mean, Yussi.

[00:48:52] Tim: Yeah. He’s, yeah, I was about to say

[00:48:54] Zeeshan Ali Khan: He’s a good friend of mine, yeah.

[00:48:55] Tim: Oh good.

[00:48:56] Zeeshan Ali Khan: Yeah, he’s a friend of mine.

[00:48:58] Tim: and he also, he gave a wonderful talk, very jet lagged about, about font kerning and and shaping the Euler curve, which is this sort of.

Perfect spiral. Anyway, sorry, the, so the, they, so in Linux. There’s a Linux distro. So the build system is now preferred is Mason, which is very different in its approach than Cargo, sorry, Rust project. Mason actually does understand how to build Rust. It can invoke Rust C, but then you give up on the rest of Cargo.

You don’t have the whole of crates

[00:49:48] Zeeshan Ali Khan: dependency downloads. Yeah that’s very true. And the main thing is that what the idea is that he doesn’t want to, talk to Cargo. He wants to talk to Rust C. Because that’s his approach for all the projects. So I see his his problem with Cargo. And and the problem is that Rust people are like, no, but Cargo is the tool to use.

Like, why don’t you use Cargo? No, I want to talk to the compiler because my tool is on the same level as Cargo. So why should I be talking to Cargo? There is now there’s now some momentum in there. There’s one person, Javier, he has done already some work. You already can have cargo like Crystal.

io dependencies in your Rust projects. It’s already possible, but there is some Some work you need to do for that. So you need to write these files I think they’re called RAP files if I’m not mistaken I could be wrong about the name But you have to for each of your dependency you have to write that and tell Mason with that what is my dependency?

Where do you get it and all that stuff? But he’s now working on making that obsolete so they can be automatically generated for all your dependencies And then after that is done, you there is, this excuse is gone for not using Rust in systems programming world. For application, it’s already not the case.

There is projects in GNOME world a lot of actually new apps in, in GNOME. They are in done in Rust, so because the new blood, they want to use some, a new modern programming language that is out there that they don’t want to use, see that is going away, hopefully. So yeah it’s there.

It’s coming. And I’m talking to people, right? I’m, friends with Leonard the system D guy, and friends with UC. And so I’m trying my best, but it’s a bit hard.

[00:51:32] The Future of Rust in System Programming

[00:51:32] Zeeshan Ali Khan: It’s especially if you have developed a huge project, very successful, huge project, it has a huge momentum like system D moving towards Rust.

It’s not an easy task. So you want everything to be in place before you start that project. That whole project. So

[00:51:48] Tim: it here, folks. They are rewriting system D and Rust. Linut.

[00:51:52] Zeeshan Ali Khan: are many folks who have tried that, and I think the only time it will happen is that when Lennart says, okay, enough is enough let’s do it. And the thing is, Lennart is very, is an extremely pragmatic guy. He he has his points, and it will take time, but I think eventually I’m hopeful that it will happen.

He hasn’t said no ever. Whenever I talk to him about Rust, coding in Rust, he has never, ever said no. He has always talked about problems, why he’s not doing it. So there is, that’s a very good sign. Eventually it will happen.

[00:52:24] Tim: I look, I think that the things that are so fundamental like system D should no longer be written in languages that are memory unsafe.

[00:52:37] Zeeshan Ali Khan: Yeah, for sure.

[00:52:40] Tim: It’s too dangerous. We,

[00:52:44] Zeeshan Ali Khan: Lennart agrees. He agrees to that. Like it’s on theoretical level. He does agree. So yeah, you’re right.

[00:52:51] Tim: Or you probably had a few questions or that you would like to talk about and I don’t think I’ve given you the opportunity to ask or to add to the conversation.

[00:53:00] Zeeshan Ali Khan: No worries. What I could share it’s interesting because, as like I came from the C world, right? So for me, Everything was less unsafe, right? I was just not used to this way of always thinking in terms of safety, but when I started doing Rust more and more, I stopped thinking in those terms, right?

I was more thinking in terms of how do you do it? And I would never, I was never really thinking how to do it in unsafe Rust. Even if in some cases it would be fine. So I feel like I lost a skill because in, in Rust, you have that opportunity to still do like possibly unsafe things, but it doesn’t have to be unsafe, right?

Like just because you’re saying unsafe doesn’t really mean that. And there’s a lot of times when I had later thought about my code and I was like, why did I do it this way? So inefficient. So this was not needed. All I had to do was use some unsafe API or just even myself do some pointer Stuff and then I could do it.

And also because of that when I have to inter interface each time with any c which I rarely do nowadays, but each time I have to do it with the FFII I’m bit lost. I’m trying to understand how the C call is working, even for example, even with libc recently, or windows sys, I was doing something with it, and it took me half a day to understand how to make that call.

It was just one function call with four parameters, and I just couldn’t figure it out, like, how do I do it safely? So that was a bit of a skill. I feel like I lost it. And I regret that, and I want to Get that back a bit. Because it, it has its advantages, those skills. So that’s something I would say to anyone who comes from C background.

If if you’re to start writing Rust, just be careful. Don’t lose that completely. Don’t be afraid of unsafe. I know I read it in the book in the beginning, but I feel like I am a bit scared of unsafe still because of how Rust works, right? Like you become scared of it. Just don’t be scared of it.

Sometimes it’s okay. Just do unsafe. It will be okay.

[00:55:17] Tim: I think of the word the keyword unsafe as meaning I am taking responsibility for the safety. Not that what you’re doing is inherently unsafe, although yeah, and maybe it comes back to the other point that you made earlier that sometimes you think things that are very difficult actually turn out to be quite easy or at least easier than you expected.

And on the other side, you have, things that look easy, like doing everything safely or using safe APIs. It turns out to be much, much harder. And so if you do have the experience of manipulating pointers or doing lots of pointer arithmetic, and you understand represent internal representations and Indian ness and lots of other you have lots of knowledge about systems programming, then.

You have tools available inside of Rust too, and so you should feel free to use them.

[00:56:18] Zeeshan Ali Khan: Yeah. And I actually, I just got, I just remembered that you are the author of the book that talks about it. I think you mentioned right in your book. It was a long time ago. I read it. I don’t remember. But I think you talked about how to see unsafe and how to use unsafe. And

[00:56:34] Tim: Yeah I did. So I I wrote Rust in Action the book with the with the boxer

[00:56:40] Zeeshan Ali Khan: yeah, I know that you have written that book, but I just remembered that you talk about unsafe bits in there.

[00:56:46] Tim: Yeah I actually I received a lot of criticism.

[00:56:49] Zeeshan Ali Khan: Oh, really?

[00:56:51] Tim: From anonymous reviewers. So these are people, when you write a technical book, the publisher will send your book around to people to provide anonymous feedback. And I got some comments saying, Oh, you shouldn’t introduce unsafe for a book and for beginners. I’m like no, it’s part of the language and don’t be scared of, it’s just software.

[00:57:17] Zeeshan Ali Khan: Guess maybe they came from a different background where they didn’t need this ever and they thought like they will never need it and they probably are right.

[00:57:24] Tim: yeah. And I, and the other thing is I thought it was fun to say let’s try mem transmute and let’s treat a floating point. It’s like an unsigned integer and it’s just bits and we can, yeah but if the time has come where we really need to wrap up the call,

[00:57:42] Zeeshan Ali Khan: yeah, it’s good. It’s so interesting. I don’t want to wrap it up, but

[00:57:48] Tim: We just got to the good bit. Maybe we will schedule another call to talk about

[00:57:54] Zeeshan Ali Khan: sure, I’m here. Yeah,

[00:57:57] Tim: In a few months time, I’ll give you a break. Enjoy your time off. I hope everything is well with you and your family. It’s been a real pleasure to talk. Thank you very much for supporting me on on this new podcast idea.

It’s been a pleasure. Zeeshan and I met in Berlin. When did we first meet?

[00:58:18] Zeeshan Ali Khan: That was no, that was second time. I think we met in the rest. EuroRust, I think.

[00:58:23] Tim: Oh, EuroRust. Okay.

[00:58:24] Zeeshan Ali Khan: Yeah. Yeah, we were hanging out in the I don’t even remember which city it was, but Brussels, I think.

[00:58:29] Tim: It was Brussels. Yeah.

[00:58:30] Zeeshan Ali Khan: it? Yeah.

[00:58:31] Tim: Oh yeah, Brussels, we had lunch.

[00:58:33] Zeeshan Ali Khan: We had breakfast but I think we had lunch too. Anyway, yeah,

[00:58:38] Tim: Yeah, okay. Okay. You have a nice day. I am going to go to bed. I will see you online. Take care. Bye.

[00:58:43] Zeeshan Ali Khan: Yeah. Thanks so much, Tim.