Blog Archive About

Psychopath Renderer

a slightly psychotic path tracer

2017 - 06 - 13

Switching from C++ to Rust

I'll get right to it: I've rewritten Psychopath in Rust. And I feel like this requires an explanation.

The short version is that Psychopath is a personal project that I work on for fun, and I had fun rewriting it in Rust. The rest of this post is the long version, as seen through the lens of my experiences developing in Rust.

But first I want to be really, really clear that I do not advocate rewriting things in Rust. It's apparently a meme to say that things should be rewritten in Rust (or so I hear... I haven't actually witnessed it much). If you have a working code-base that's actually used for real things, then rewriting it in Rust (or any language) requires strong justification. In my case, Psychopath isn't used for real things, so it doesn't take much justification aside from "I felt like it".

With that out of the way, let's dive in!

The good

I'm going to first talk about the things I like most in Rust. But don't worry, we'll get to the bad stuff too!

Cargo

The thing that seems to get talked about most with Rust is its memory safety. But for me, that's actually pretty far down the list of things I like about Rust. What I've enjoyed most, by far, about Rust is the tooling around it.

In C++, if I want to build a project that has a bunch of dependencies it's... kind of a pain. Unless your OS ships with the needed dev dependencies, you're going to need to download and build them all, make sure whatever build system the code-base uses can find them, and probably wrestle with a bunch of other fiddly things. It's a pain. And it's also recursive, because the dependencies may have dependencies. And it's also a pain to help other people when it's your code-base that they're trying to build.

In Rust, you download the code and run:

cargo build --release

That's it.

Cargo is Rust's official build tool, and it automatically downloads, caches, and builds the needed dependencies (both direct and indirect) and links them into your build. The only time I've seen it get messy is when there are indirect C/C++ dependencies (e.g. via wrappers for C/C++ libraries). But even then, it's no messier than a standard C or C++ code base, and only for those dependencies.

The extent to which that makes my life easier is difficult to over-state. It feels like I can breath again. Want to just try using a library in your code-base quick? Super fast and easy. Decide you don't like it? Get rid of it just as quickly.

Other tooling

I could go on and on about the tooling, but I'll summarize by pointing out two other things that Rust comes with:

  • Documentation generation
  • Unit testing

Having a standard way to generate documentation from your code-base is great, and it has a trickle-down effect on the ecosystem where more things seems to be documented than not. (Admittedly, the quality of the documentation varies.)

And adding unit tests is as simple as adding the test attribute to a function:

#[test]
fn my_test() {
    assert!(some_condition);
}

Having these kinds of basic things included just makes life easier.

In short, Rust comes with all the basics of modern tooling for a programming language. And it's done well.

No headers

Another thing that I love about Rust is the lack of header files. I hate header files. They're awful. I don't care what anyone says.

Instead of header files, Rust has a really nice module system. Some people seem to get confused by it, but it made perfect sense to me from the beginning. But even if it's a little confusing at first, it's still far better than headers. I don't have to write function signatures twice, keep them in sync, etc.

Strict typing

C and C++ have strict-ish typing, which is nice. But Rust has super-duper strict typing, which is even nicer. Combined with some other aspects of Rust, it makes refactoring a much more confident exercise.

The super strict typing is also the foundation of Rust's type-inference. Within a function, Rust is able to infer most of your types. So there's almost a contradiction in that Rust's super strict typing actually lets you worry about and wrangle with types less. Both more correct and easier.

(As an aside: Rust does not do inter-function type inference: you have to specify the types in function signatures. Essentially, Rust believes in API boundaries being explicit. This keeps things easy to reason about for the programmer and prevents APIs from accidentally changing.)

Memory safety

Okay, you knew it was coming. I do, in fact, like the memory-safe-by-default approach that Rust takes. I spent days pulling my hair out tracking down segfaults and other weird behaviors in the C++ version of Psychopath, and it's just not fun.

In Rust, this rarely happens. And when it does happen it's only because I dropped into unsafe code somewhere, so I know where to start looking. Unsafe code aside, the worst that happens is a run-time panic with a stack trace showing exactly where things went wrong. And usually it doesn't even make it that far. Usually it's a compile error.

But I don't want to over-state this. Memory safety is nice, but I would use Rust even without it. Rust has a lot more going for it than just memory safety. It's a really well-rounded language... assuming you want to do low-level coding.

The bad

So, Rust isn't all sunshine and rainbows, unfortunately. And I would be remiss if I didn't talk about the things I find frustrating.

No unions

(UPDATE: Rust now has unions, as of release 1.19. Yay!)

This is actually on its way, so this will likely be outdated quickly. But up to this point Rust doesn't have unions.

That's not quite as bad as it sounds, because Rust does have sum types (using the enum keyword) which are essentially tagged unions and really nice. But when working on e.g. the BVH in Psychopath it would have been really nice to use the lower-level version of unions to really squeeze those extra bytes out of the node structs.

No SIMD intrinsics

(UPDATE: Rust now has stable SIMD intrinsics, as of release 1.27. Yay!)

Stable Rust has no support for SIMD intrinsics. You can use them on the unstable 'nightly' compiler, but there's no current track for stabilizing them as far as I'm aware.

What this means for Psychopath is that I have a flag you can pass to cargo that tells it to build alternate SIMD versions of some code, but you can only pass that flag (and have it successfully build) if you're using nightly.

EDIT: on the reddit thread stusmall points out that SIMD is planned to be stablized at some point and is considered "on the horizon". What I meant to say is that there isn't a clear timeline for it. It is considered important and will be stablized eventually, but it's anyone's best guess when that might be.

Slow build times

This isn't so much of a hit against Rust compared to C++ (which also has slow build times), but hearing about e.g. Go and D's build times makes me really jealous. And the language that Jonathan Blow is developing seems to just blow things out of the water.

As a new language without all the historical baggage, I would have hoped for Rust to be much faster at building. The Rust team is definitely working on improving the situation, but it is so far off from where I feel like it ought to be that the incremental improvements they're making seem like drops in a bucket.

Mind you, it's entirely possible that I'm being overly pessimistic here. So take it with a grain of salt. But that's how I feel about it at the moment.

Fortunately, Psychopath isn't that large of a code-base, and I've also split it into multiple sub-crates which definitely helps. Building Psychopath itself (assuming external dependencies are already built) takes around 17 seconds on my machine. So it's not terrible, and is about what I would expect from C++ as well.

Immature ecosystem

There aren't a lot of stable libraries out there for Rust. Fortunately, this is being actively worked on. But even so, rendering and vfx is a pretty niche space. And there are just so many things written in C++ for rendering and VFX that are unlikely to get robust Rust equivalents any time soon. It makes writing a renderer in Rust a little daunting.

Of course this isn't Rust's fault by any means. But it is a reality.

The saving grace (perhaps) is that Rust's C FFI story is excellent. Me and another fellow have been working on a Rust wrapper for OpenEXR and it has been a largely pleasant experience. There are some fiddly bits to get right since OpenEXR is C++ rather than C. And it does take time. But it takes a lot less time than writing a full-featured OpenEXR library from scratch.

So I suspect that writing wrappers is what I'll be doing when I need non-trivial functionality from C++ libraries. But that's certainly not as straightforward as just using a library directly. And it negates a little the very first positive point I mentioned about easy builds.

On the bright side, if I'm successful at this, then many of those libraries will already be wrapped for other intrepid developers. Eventually.

Fighting the borrow checker

The borrow checker is the part of Rust's compiler that makes sure ownership rules are properly followed. It is the basis of Rust's memory safety guarantees, which is awesome. But it also takes some getting used to.

By and large, when the borrow checker has refused to compile something I wrote, it's because it was right and I was wrong. So my complaint isn't actually that the borrow checker exists, but rather it's that the error messages are sometimes... confusing. And it's also not always clear how to change things to satisfy it.

Mind you, the error messages are nowhere near as bad as template meta-programming errors in C++. And it's certainly better than tracking down mysterious memory safety bugs. And the borrow checker errors have made great strides in getting better. And actually, most of the time they're pretty clear these days...

...but even so, it's a bit of a hurdle that you have to get over. And every once in a while the borrow checker throws you a curve ball that's just really confusing, and that can be really frustrating.

Having said all that, I left the borrow checker until last for the same reason I put memory safety last in the list of good things: I think its importance is overblown. A lot of people seem to talk a lot about it, but I think it's a minor annoyance.

With most new languages there are some new concepts or ways of thinking about things that you have to wrap your head around, and Rust is no different. But if you code in C++, I can't imagine it being a significant hurdle. It might piss you off for the first week, and then you'll get over it.

Overall

I'm certainly not ready to recommend Rust to anyone doing serious rendering or vfx work yet, mostly because of the lack of unions and SIMD in the stable compiler. And I think it absolutely makes sense for people to be cautious and see how things develop over the next few years.

However, I absolutely do recommend giving Rust a spin for a non-trivial fun project. The immature ecosystem might bite you depending on what you're trying to do, but I think the experience of trying to create something in Rust is kind of... eye opening. You'll likely be a bit frustrated by the borrow checker at first, but once you get over that I think you're equally likely to find programming in Rust really fun. And Rust as a language is clearly extremely capable over-all.