Disabling the borrow checker (DRAFT)

Why?

LogLog Games is an indie game studio that moved from unity to rust 3 years ago. They wrote their own game engine in rust and developed and published a game using it as well. Recently they decided to abandon rust and move back to unity. They wrote an article sharing their reasons for this decision. A lot of the reasons could be boiled down to the statement that rust borrow checker is too strict. The compiler will reject valid programs because it cannot prove that they are safe in the framework defined by the creators of the rust language. This is one of the example they used (in the section "Abstraction isn't a choice"):

Here they are taking a reference to the field selected_duck and in the next line they are passing self to a function. The compilation fails because the function takes a mutable reference to self, and rustc won't allow that because there is already an immutable reference to a field of self in scope. This code won't compile even if character_select_duck_detail isn't mutating the field of self that has been borrowed. The compiler/optimizer simply isn't smart enough to take that into account. So you need to refactor your code to satisfy the compiler. Their argument is that sometimes that refactoring results in suboptimal code. Even if the refactoring doesn't result in suboptimal code, a lot of times it's completely unnecessary and just adds useless burden on the programmer. The model you have of your code in your head might not be the same as the model borrow checker has of your code and in those situations it's just a distraction.

Disabling lifetime and mutability checks.

However, there is a way you can turn off the borrow checker if you are fairly sure that the errors it's giving you are false positives. I have a couple utility functions that do this:

These functions takes a pointer as input, and gives another pointer as output. The output is the same pointer, the same 8 bytes representing a location in memory, but with a different type.

Rust veterans will disagree with me here. A reference is not the same thing as a pointer. But it looks like a pointer and quacks like a pointer, so I'm using it like a pointer.

ms stands for make static. It takes a regular pointer and gives back a static pointer. Static pointers have infinite lifetime, so if the compiler is complaining about a lifetime error which you know is a false positive, you can wrap that pointer with this function and the compiler will stop complaining. For example, this is some graphics code from a game I'm writing:

The engine takes a screenshot every 0.5s and uses it to show the color under the cursor and save the screenshot. If you remove lines 2 and 3 the compiler will start complaining about lifetimes. It will say that the launched thread might outlive these objects and then I will get a use after free bug. What the compiler doesn't know is that these objects live as long as the game. So I use ms to silence the compiler.

I could wrap these objects with Arc<Mutex<>> but it feels unnecessary in this case. I did put the screenshot buffer inside a mutex for correctness' sake but even that isn't necessary. There are millions of pixels on the screen and a write happens only once every 0.5s. The probability that a pixel will be written to while we are reading it is very low. Even then the pixel color we read will be a blend of the RGBA values of the previous frame and the next frame and will only look like that for one frame, assuming the buffer is not overwritten in multiples of 4 bytes, in which case even that won't happen. Rust gives you guaranteed memory and thread safety, but you might not even need that in many cases.

mm stands for make mutable. It takes a constant pointer and returns a mutable pointer. This function can be used to deal with the example they used. self can be wrapped with this function to convert it into a mutable pointer. From the borrow checkers point of view the code is creating two immutable references to self which is allowed:

Of course this is about as far away from conventional rust as you could go. If you use these utilities, rustc cannot guarantee memory safety in your program. Warranty void if seal is broken. You will get segmentation faults, use after free bugs, data races, UB, the whole bunch, if you aren't careful. But then some programmers believe the memory bugs are just another class of bugs that you are supposed to weed out before shipping. I personally haven't seen a memory bug that I wasn't able to fix within 5 minutes.

C like late initialization

I use this wrapper type to do C like late initialization in rust. You can wrap a struct field with it like this:

The default value of this type is zero initialized. Its contents will not follow the rust ownership/RAII model. If you want to update some data inside this field, you'll have to use ci! macro

The data in there will not be dropped automatically, you'll have to drop it yourself when you're done with it.

Should you write rust like this?

You shouldn't program a TLS library the same way you would program a video game. Vice verse, you shouldn't have to program a video game the same way you would program a TLS library. Video game programming is distinct due to its significant emphasis on extensive prototyping. You need to prototype a lot to figure out what looks good and feels fun. A new feature added years down the road could require a big change in the low level implementation of the engine to implement it efficiently. As evident by the article written by LogLog games, rust is very hostile to such programming patterns. And if you're building a single player game, you need to decide if fearless concurrency and memory safety is worth the loss of ergonomics which rust forces on you for your particular use case. Not to mention that a player will not gain anything by hacking the vulnerabilities in his single player game.

For many command-line utilities, it doesn't really matter if your program has vulnerabilities or concurrency problems either, except for how they affect the UX.

When you're writing rust like this, you'll have to maintain a list of unchecked invariants in your mind about your program. This is where the memory and concurrency bugs will originate from. It would be a good idea to write them down as well, or encode them in the structure of your program. And the larger your team size is, the more likely it will be that you'll either forget to document these invariants properly, or someone will not understand them properly and cause problems. And the chances of seeing memory bugs will increase.

It would definitely be a bad idea to write a library like this and publish it for others to use. The borrow checker is the reason it's so easy to import and use an external dependency in your program. The ownership model of rust is also a contract between the library authors and the users. The users are just going to assume that your library is going to follow it, and if it doesn't, they are going to introduce bugs in their library/program. But for moderately sized applications where security/reliability isn't one of the main concerns, or you want/need to be able to prototype very quickly, there are little to no reasons for not using these techniques.

Why even use rust if you're going to bypass the borrow checker?

If you're bypassing Rust's borrow checker, it might seem like you're throwing away its main selling point—memory safety. But Rust offers much more than just the borrow checker. For one, it provides checked array indexing, which prevents common out-of-bounds bugs at runtime, giving you safety even without a manual bounds check. Rust’s straightforward cross-compilation process also makes it easy to build and deploy for multiple platforms, a far cry from the headaches often faced in languages like C++. Speaking of C++, Rust boasts a sane feature set; it's far less cluttered and avoids the excessive complexity C++ has accumulated over the years. Rust also has a rich and growing community, ensuring that help, libraries, and knowledge are readily available. Plus, it has a rich and stable standard library, which means you can accomplish a lot without resorting to third party libraries. And even when you have to reach for third party libraries, using them is about as easy as it could be. Amazing tooling and IDE support, often missing in newer languages, makes Rust a joy to work with on both small and large projects. Even if you bypass some of its strictest features like the borrow checker, these other advantages still make Rust an incredibly valuable tool for building reliable software.