246
Productivity of Rust teams at Google
(programming.dev)
Slide with text: “Rust teams at Google are as productive as ones using Go, and more than twice as productive as teams using C++.”
In small print it says the data is collected over 2022 and 2023.
That’s pretty cool. I’m not advanced enough to really understand all the ways rust is better but I read nothing but good things about it. It seems pretty universally loved.
Basically modern language with modern tooling. It's what C++ would look like if it had been designed today. The big thing though is the abstraction of ownership and lifetimes which took C++ ideas of scopes, smart pointers, and destructors and polished them into something much more powerful. Simply put it's possible to design APIs in Rust that are literally impossible to express in any other language, and that's a big deal.
Added on top of that is a modern dependency management system that is severely needed in languages like C and C++, and a very powerful meta programming system that enables compile time code generation and feature selection that's much safer and powerful than C and C++ fairly primitive pre-processor (although C++ STL does come close).
This sort of violates what I've always heard about computer science. I've always heard logic is logic.
Hmm, yes and no. You can express a program that does anything in any language, but API design is as much about what can't be expressed (with that API) as what can. A well designed API lets you do the things that are desirable while making it impossible to do things that aren't. You can of course bypass APIs to do anything the language allows, even in Rust if you break out the unsafe blocks and functions there's pretty much nothing you can't bypass with enough effort, but you very much have to set out to not use the API to do that.
I think your quantifier of "any other language" is the issue. There are certainly languages with far more powerful type systems than Rust, such as Coq or Lean.
Maybe, although I'm not aware of any other language that has the same abstraction around ownership and lifetimes. Most other languages I'm aware of that have more (or equivalently) powerful type systems are also GCed languages that don't let you directly control whether something gets stack or heap allocated. Nor due they allow you to guarantee that a variable is entirely consumed by some operation and no dangling references remain. While at a high level you can write something that accomplishes a similar result in other higher level languages, you can not express exactly the same thing due to not having direct access to the lower level memory management details.
See this example for Scala:
https://blog.tmorris.net/posts/scala-exercise-with-types-and-abstraction/index.html
Now go further and say you can't compile a call that leaks memory, or things like that.
Disclaimer: I don't know Rust so can't verify the claim. All I can say is it sounds somewhat plausible.
You can leak memory in Rust if you want to (and it's occasionally desirable). What the type system prevents is mainly accessing memory that has been deallocated. The prevention of memory leaks uses RAII just like C++. The main difference related to allocation is that there's no "new" operator; you can pretty much only allocate memory through the initialization of a smart pointer type.
I'd argue it also prevents you from accidentally leaking memory. You have to be pretty explicit about what you're doing. That's true for pretty much anything in Rust, every bad thing from C/C++ is possible in Rust, you just have to tell the compiler "yes, I REALLY want to do this". The fact that most of the really dangerous things are locked behind unsafe blocks and you can set the feature flag to disable unsafe from being used in your code goes a long way towards preventing more accidents though.
I agree Rust makes it virtually impossible to leak memory by accident. The difference I wanted to point out is that leaking memory is explicitly not considered unsafe, and types like Box have a safe "leak" method. Most "naughty" things you can do in Rust require using the "unsafe" keyword, but leaking memory does not.
Cyclic reference-counted pointers are the most probable way to accidentally leak memory. But it's a pretty well known anti-pattern that is not hard to avoid.
Yeah, I didn't think of that case, because any time I use ref counting, cyclic references are at the from of my mind.
So, it's C#?
/s
No, that's what Java would look like today if designed by a giant evil megacorp... or was that J++. Eh, same difference. /s
This did make me laugh though. Anyone else remember that brief period in the mid-90s when MS released Visual J++ aka Alpha C#? Of course then Sun sued them into the ground and they ended up abandoning that for a little while until they were ready to release the rebranded version in 2000.
I won't disagree, but what Rust did is not the correct answer. Better than C++ perhaps, but not good enough. In the real world my code is more than Rust. I'm having trouble using rust because all my existing code is C++ and the dependency management does not work well with my existing build system and dependency management. If you want a dependency manager it needs to cover all languages and be easy to plug in whatever I'm doing currently. This is NOT an easy problem (it might not even be possible to solve!), but if you fail you are useless for all the times where dependency management is hard.
It's hard to say that what Rust did was not correct when it's better than what C++ has (nothing, or rather it punts to the OS and user to handle it). I agree it's far from perfect but it's as good as pretty much any other languages dependency management system that I'm aware of. If you know a better one I'd love to hear about it because yes, there are still gaps and pain points. I'd argue many of those gaps and pain points are a legacy of C and C++ though. The fact that C/C++ never had an actual dependency management system meant that the OS had to provide one, and every OS more or less went about it in an entirely different fashion (and even worse in the case of Linux, every distro went about it in a different fashion). This has massively complicated things because there is a HUGE body of C/C++ libraries that are very desirable to use with absolutely no consistent way to do so. It's not as simple as just adding the ability to pull from the C/C++ repo for any of those dependencies, because there is no such thing.
OCaml's OPAM. They actually took into account that it could be desirable to use software written in other languages in your OCaml project. It even has a bunch of stuff packaged that's written in Rust. Imagine that the other way around. It only has stub packages for compilers like gcc but I assume that's likely because they don't want to have people spend hours building the whole thing themselves when there's a perfectly good one on their system, rather than it not being possible to do.
I love Rust but I will die on this hill that combining package manager and build system like Cargo does and then only making it work for a single language is a lot worse than what C++ does, because if it doesn't work for your project you're screwed. Everything expects you to use Cargo, especially if you intend to publish a library, with C++ you can at least pretty much always get the build setup to do what you need, and you can import whatever as long as it comes with a pkg-config file.
You're looking for Nix (unless you're a Windows developer, work on getting that to work is ongoing). There's very likely other good ones too, but this is the one I like and am familiar with. The difference is that it's not a package manager for C++, but a package manager that also packages C++ packages. Which makes it so much more versatile than something like Cargo, because you can accurately represent dependency chains regardless of what language each package is written in. My Nix + CMake projects will build consistently on every Linux or Mac computer (you can't say the same for Rust crates because they will look for stuff in system directories because Cargo can't package anything that isn't Rust), and you can depend on them similarly to how you would a Rust crate, with the difference that you can depend on them not only in another C++ project, but also in a Python package, a Go package, or whatever else that can be packaged with Nix. And if you can't use Nix, then you can always build the CMake project directly, package it somewhere else maybe, because the two parts are not coupled together at all.
I'll look into OPAM, it sounds interesting.
I disagree that combining build and package management is a mistake, although I also agree that it would be ideal for a build/package management system to be able to manage other dependencies.
A big chunk of the problem is how libraries are handled, particularly shared libraries. Nix sidesteps the problem by using a complex system of symlinks to avoid DLL hell, but I'm sure a big part of why the Windows work is still ongoing is because Windows doesn't resemble a Linux/Unix system in the way that OS X and (obviously) Linux do. Its approach to library management is entirely different because once again there was no standard for how to handle that in C/C++ and so each OS came up with their own solution.
On Unix (and by extension Linux, and then later OS X), it was via special system include and lib folders in canonical locations. On Windows it was via dumping everything into C:\Windows (and a lovely mess that has made [made somehow even worse by mingw/Cygwin then layering in Linux style conventions that are only followed by mingw/Cygwin built binaries]). Into this mix you have the various compilers and linkers that all either expect the given OSes conventions to be followed, or else define their own OS independent conventions. The problem is of course now we have a second layer of divergence with languages that follow different conventions struggling to work together. This isn't even a purely Rust problem, other languages also struggle with this. Generally most languages that interop with C/C++ in any fashion do so by just expecting C/C++ libraries to be installed in the canonical locations for that OS, as that's the closest thing to an agreed upon convention in the C/C++ world, and this is in fact what Rust does as well.
In an ideal world, there would be an actual agreed upon C/C++ repository that all the C/C++ devs used and uploaded their various libraries to, with an API that build tools could use to download those libraries like Rust does with crates.io. If that was the case it would be fairly trivial to add support to cargo or any other build tool to fetch C/C++ dependencies and link them into projects. Because that doesn't exist, instead there are various ad-hoc repositories where mostly users and occasionally project members upload their libraries, but it's a crap-shoot as to whether any given library will exist on any given repository. Even Nix only has a tiny subset of all the C/C++ libraries on it.
Dependency management has to deal with the real world where what we didn't know in 1970 hurts us.
I'm having trouble understanding the point you're trying to make here. You seem to be angry at the Rust dependency manager for not being perfect, but also admit that it's better than C++. Is there some dependency manager you like more than what Rust provides? Do you have any suggestions for how Rust could improve its dependency management?
I said this is a hard problem and might not even be solvable.
rust is not better than C++ if you are in any of those cases where rust doesn't work. Not really worse, but not better either. If it works for you great, but it is worse for me as rust fight our homegrown system (which has a lot of warts )
So you're point is that your custom home grown workaround to a failure of C++ doesn't play well with Rusts official solution to the same problem? And therefore Rusts solution isn't better than C++ lack of a solution?
While that is unfortunate for you and you certainly seem to have tech-debted yourself into a particularly nasty corner, I'm not sure that logic follows.
Rust's package manager doesn't manage c++, python, or anything else. Since the real world should not be one language to rule them all any package system must handle all possible languages. Otherwist they are lacking.
Compared to C++, Rust does a lot of things for you to prevent common mistakes. This reduces a lot of the mental overhead that comes with writing C++ programs. Go does this as well, but at the expense of slower programs. Rust programs are still as fast as C++ programs.
I think focusing on speed of programs is looking at the wrong thing. It's true that at the moment Rust programs are usually faster than equivalent Go programs, but it's already possible to very carefully tune a Go program to achieve similar levels of speed. The real difference is in productivity.
Go makes the tradeoff that it's much more verbose, it takes many times the lines of code to achieve in Go what's possible in either Rust or C++. This is because of their dogmatic obsession with keeping the language "simple". Yes that makes it easy to learn, but it means if you have something complex to express you need to use more of those simple building blocks to do so rather than just a handful of the more complicated ones you're provided in Rust or C++. More lines of code is more opportunity for mistakes to be made and more code to maintain. In a more powerful language you can offload much of the work to libraries that are maintained by other people and that expose powerful APIs that are safe to use. In Go because it's "simple" it's hard to write powerful APIs that aren't filled with footguns and are more complicated to use.
The problem with C++ wasn't that it was a complicated language, it's that it was an inconsistent language. There were many competing ways of accomplishing things and many of them were mutually exclusive with each other and introduced footguns. It was far far too easy to write code in C++ that looked correct but was utterly broken in very subtle and hard to detect ways. The Go guys looked at that situation and decided the problem was that the language was too complex and had too many features (arguably true), but decided to make the exact opposite mistake by designing a language that was too simple and had too few features.
It is much, much more difficult to make Go run as fast as Rust compared to writing that faster in the first place Rust program.
It’s a fair point that the speed of a language is not everything, but that’s not my point. My point is that with C++, the programmer must often play a puzzle of avoiding common pitfalls with e.g. memory allocation - on top of the actual problem the programmer is intending to solve. This hurts productivity, because there’s so much more to be mindful about.
Both Rust and Go are more free from this kind of extra mental overhead. The programmer can focus more attention on the actual problem. This is probably why Google has observed that both Rust and Go developers are twice as productive than C++ developers.
Go makes the trade off of using garbage collectors, which is easier for programmers to work with but comes with extra performance cost.
Having a simple and verbose language is not necessarily a downside. I’d rather take a simple language over all the syntactic sugar that comes with Perl.
This isn't entirely correct. Rust you do still need to worry about those same problems, it just gives you much better abstractions for modeling and thinking about them, and the tooling of the language "checks your homework" so to speak to make sure you didn't make any mistakes in how you were thinking about it. The upside is that you can be very specific about how you handle certain tasks allowing you to be super efficient with resources. The downside is that you do still need to worry about those resources at least a little bit. If you wanted to you could write Rust like Go by just making every variable a
Box<Arc<...>>and using.clone()like mad, but your performance is going to take a hit.Go (and other GCed) languages on the other hand, do entirely free you from having to worry about memory utilization in a general sense as the language runtime takes care of that for you. The downside is that it often does a poor job of doing so, and if you do run into one of the edge cases it's not so great at your tools for dealing with that are severely limited. Further it's very easy to accidentally screw up your memory usage and use far more than is necessary leading to excessive heap churn and serious performance degradation. Because the language makes it easy, even if what you're doing is wrong, and it lacks the tools to alert you to that problem before you trip over it at runtime.
As programmers, our bread and butter is abstractions. We use abstractions because in a very real sense what we do day to day if we removed all the abstractions would be a herculean effort that not even the best of us could manage for any period of time. Go's idea of "simple" is limiting the number of abstractions that the language provides, and so it's up to the programmer to use that small handful to construct more powerful ones. Every code base becomes a snowflake where each team rolled their own solution, or everyone just uses the same sets of libraries that provide the solution. You haven't removed the complexity, you've just shifted it out of the language and onto a 3rd party. It's better to have a consistent set of abstractions provided and maintained by the language and centrally developed by everyone, rather than a hodge-podge of abstractions by random 3rd parties.
I disagree about comparing languages by speed. Just because you can make Go programs as fast as Rust programs, it's not going to be as straightforward as doing it in Rust. I'd much rather spend slightly more effort up front to write idiomatic Rust code that's fast by construction than try to make Go code faster by applying a bunch of arcane tweaks to it.
It is fair to compare speeds, I just think it's probably the wrong argument to focus on if you're trying to convince people of the value of a language. It's definitely a supporting point, but at the end of the day, most programs don't actually need to be blazingly fast, they just need to not be dog slow. Ease of writing (correct) code and even more importantly maintaining and debugging code are generally far more important factors in a languages success, and those are all areas that Rust excels in.
The problem with a purely speed focused argument is that it's always possible to cherry pick examples in both directions. You can find some piece of idiomatic Go code that just happens to be faster than the equivalent idiomatic Rust code and vice versa. The fact that it's undoubtedly much easier to find idiomatic Rust code that out performs most Go code (idiomatic or not) is a much harder argument to use to convince people. The Go proponents will just argue that the ease of understanding the Go code outweighs whatever speed gains Rust has. That's why I think it's important to also point out that Go might be easier to write small snippets of, but for any realistic program it's going to be harder to write and maintain, and it will be more prone to bugs.
A lot of it is about moving problems from runtime to compile time. JS, for example, has most problems live in runtime.
Imagine you're hiring an event planner for your wedding. It's an important day, and you want it to go well and focus on the things that matter to you. Would you rather hire an even planner that barely interacts with you up until the wedding because they're so "easy to work with"? Or one that gets a ton of info and meets with you to make sure they can get everything they need as early as possible?
Rust is like the latter. JS is like an even planner who is just lazy and says "we'll cross that bridge when we come to it" all the time.
C++ is like a meth addict.