Cross-compilation is an imposing term for a common kind of desire:
-
You want to build an app for Android, or iOS, or your router using your laptop.
-
You want to write, test and build code on your Mac, but deploy it to your Linux server.
-
You want your Linux-based build servers to produce binaries for all the platforms you ship on.
-
You want to build an ultraportable binary you can ship to any Linux platform.
-
You want to target the browser with Emscripten or WebAssembly.
In other words, you want to develop/build on one "host" platform, but get a final binary that runs on a different "target" platform.
Thanks to the LLVM backend, it's always been possible in principle to cross-compile Rust code: just tell the backend to use a different target! And indeed, intrepid hackers have put Rust on embedded systems like the Raspberry Pi 3, bare metal ARM, MIPS routers running OpenWRT, and many others.
But in practice, there are a lot of ducks you have to get in a row to make it work: the appropriate Rust standard library, a cross-compiling C toolchain including linker, headers and binaries for C libraries, and so on. This typically involves pouring over various blog posts and package installers to get everything "just so". And the exact set of tools can be different for every pair of host and target platforms.
The Rust community has been hard at work toward the goal of "push-button cross-compilation". We want to provide a complete setup for a given host/target pair with the run of a single command. Today we're happy to announce that a major portion of this work is reaching beta status: we're building binaries of the Rust standard library for a wide range of targets, and shipping them to you via a new tool called rustup.
Introducing rustup
At its heart, rustup is a toolchain manager for Rust. It can download and switch between copies of the Rust compiler and standard library for all supported platforms, and track Rust's nightly, beta, and release channels, as well as specific versions. In this way rustup is similar to the rvm, rbenv and pyenv tools for Ruby and Python. I'll walk through all of this functionality, and the situations where it's useful, in the rest of the post.
Today rustup is a command line application, and I'm going to show you some examples of what it can do, but it's also a Rust library, and eventually these features are expected to be presented through a graphical interface where appropriate — particularly on Windows. Getting cross-compilation set up should eventually be a matter of checking a box in the Rust installer.
Our ambitions go beyond managing just the Rust toolchain: to have a true push-button experience for cross-compilation, it needs to set up the C toolchain as well. That functionality is not shipping today, but it's something we hope to incorporate over the next few months.
Basic toolchain management
Let's start with something simple: installing multiple Rust toolchains. In this example I create a new library, 'hello', then test it using rustc 1.8, then use rustup to install and test that same crate on the 1.9 beta.
That's an easy way to verify your code works on the next Rust release. That's good Rust citizenship!
We can use rustup show
to show us the installed toolchains, and rustup update
to keep them up to date with Rust's releases.
Finally, rustup can also change the default toolchain with rustup default
:
$ rustc --version
rustc 1.8.0 (db2939409 2016-04-11)
$ rustup default 1.7.0
info: syncing channel updates for '1.7.0-x86_64-unknown-linux-gnu'
info: downloading component 'rust'
info: installing component 'rust'
info: default toolchain set to '1.7.0-x86_64-unknown-linux-gnu'
1.7.0-x86_64-unknown-linux-gnu installed - rustc 1.7.0 (a5d1e7a59 2016-02-29)
$ rustc --version
rustc 1.7.0 (a5d1e7a59 2016-02-29)
On Windows, where Rust supports both the GNU and MSVC ABI, you might want to switch from the default stable toolchain on Windows, which targets the 32-bit x86 architecture and the GNU ABI, to a stable toolchain that targets the 64-bit, MSVC ABI.
rustup default stable-x86_64-pc-windows-msvc
info: syncing channel updates for 'stable-x86_64-pc-windows-msvc'
info: downloading component 'rustc'
info: downloading component 'rust-std'
...
stable-x86_64-pc-windows-msvc installed - rustc 1.8.0-stable (db2939409 2016-04-11)
Here the "stable" toolchain name is appended with an extra identifier indicating
the compiler's architecture, in this case x86_64-pc-windows-msvc
. This
identifier is called a "target triple": "target" because it specifies a platform
for which the compiler generates (targets) machine code; and "triple" for
historical reasons (in many cases "triples" are actually quads these
days). Target triples are the basic way we refer to particular common platforms;
rustc
by default knows about 56 of them, and rustup
today can obtain
compilers for 14, and standard libraries for 30.
Example: Building static binaries on Linux
Now that we've got the basic pieces in place, let's apply them to a simple cross-compilation task: building an ultraportable static binary for Linux.
One of the unique features of Linux that has become increasingly appreciated is its stable syscall interface. Because the Linux kernel puts exceptional effort into maintaining a backward-compatible kernel interface, it's possible to distribute ELF binaries with no dynamic library dependencies that will run on any version of Linux. Besides being one of the features that make Docker possible, it also allows developers to build self-contained applications and deploy them to any machine running Linux, regardless of whether it's Ubuntu or Fedora or any other distribution, and regardless of exact mix of software libraries they have installed.
Today's Rust depends on libc, and on most Linuxes that means glibc. It's technically challenging to fully statically link glibc, which presents difficulties when using it to produce a truly standalone binary. Fortunately, an alternative exists: musl, a small, modern implementation of libc that can be easily statically linked. Rust has been compatible with musl since version 1.1, but until recently developers have needed to build their own compiler to benefit from it.
With that background, let's walk through compiling a statically-linked Linux executable. For this example you'll want to be running Linux — that is, your host platform will be Linux, and your target platform will also be Linux, just a different flavor: musl. (Yes, this is technically cross-compilation even though both targets are Linux).
I'm going to be running on Ubuntu 16.04 (using this Docker image). We'll be building the basic hello world:
rust:~$ cargo new --bin hello && cd hello
rust:~/hello$ cargo run
Compiling hello v0.1.0 (file:///home/rust/hello)
Running `target/debug/hello`
Hello, world!
That's with the default x86_64-unknown-linux-gnu
target. And you can
see it has many dynamic dependencies:
rust:~/hello$ ldd target/debug/hello
linux-vdso.so.1 => (0x00007ffe5e979000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fca26d03000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fca26ae6000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fca268cf000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fca26506000)
/lib64/ld-linux-x86-64.so.2 (0x000056104c935000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fca261fd000)
To compile for musl instead call cargo
with the argument
--target=x86_64-unknown-linux-musl
. If we just go ahead and try that
we'll get an error:
rust:~/hello$ cargo run --target=x86_64-unknown-linux-musl
Compiling hello v0.1.0 (file:///home/rust/hello)
error: can't find crate for `std` [E0463]
error: aborting due to previous error
Could not compile `hello`.
...
The error tells us that the compiler can't find std
. That is of
course because we haven't installed it.
To start cross-compiling, you need to acquire a standard library for the target platform. Previously, this was an error-prone, manual process — cue those blog posts I mentioned earlier. But with rustup, it's just part of the usual workflow:
rust:~/hello$ rustup target add x86_64-unknown-linux-musl
info: downloading component 'rust-std' for 'x86_64-unknown-linux-musl'
info: installing component 'rust-std' for 'x86_64-unknown-linux-musl'
rust:~/hello$ rustup show
installed targets for active toolchain
--------------------------------------
x86_64-unknown-linux-gnu
x86_64-unknown-linux-musl
active toolchain
----------------
stable-x86_64-unknown-linux-gnu (default)
rustc 1.8.0 (db2939409 2016-04-11)
So I'm running the 1.8 toolchain for Linux on 64-bit x86, as indicated by the
x86_64-unknown-linux-gnu
target triple, and now I can also target
x86_64-unknown-linux-musl
. Neat. Surely we are ready to build a slick
statically-linked binary we can release into the cloud. Let's try:
rust:~/hello$ cargo run --target=x86_64-unknown-linux-musl
Compiling hello v0.1.0 (file:///hello)
Running `target/x86_64-unknown-linux-musl/debug/hello`
Hello, world!
And that... just worked! Run ldd
on it for proof that it's the real
deal:
rust:~/hello$ ldd target/x86_64-unknown-linux-musl/debug/hello
not a dynamic executable
Now take that hello
binary and copy it to any x86_64 machine running Linux and
it'll run just fine.
For more advanced use of musl consider rust-musl-builder, a Docker image set up for musl development, which helpfully includes common C libraries compiled for musl.
Example: Running Rust on Android
One more example. This time building for Android, from Linux, i.e.,
arm-linux-androideabi
from x86_64-unknown-linux-gnu
. This can also be done
from OS X or Windows, though on Windows the setup is slightly different.
To build for Android we need to add the Android target, so let's set up another 'hello, world' project and install it.
rust:~$ cargo new --bin hello && cd hello
rust:~/hello$ rustup target add arm-linux-androideabi
info: downloading component 'rust-std' for 'arm-linux-androideabi'
info: installing component 'rust-std' for 'arm-linux-androideabi'
rust:~/hello$ rustup show
installed targets for active toolchain
--------------------------------------
arm-linux-androideabi
x86_64-unknown-linux-gnu
active toolchain
----------------
stable-x86_64-unknown-linux-gnu (default)
rustc 1.8.0 (db2939409 2016-04-11)
So let's see what happens if we try to just build our 'hello' project without installing anything further:
rust:~/hello$ cargo build --target=arm-linux-androideabi
Compiling hello v0.1.0 (file:///home/rust/hello)
error: linking with `cc` failed: exit code: 1
... (lots of noise elided)
error: aborting due to previous error
Could not compile `hello`.
The problem is that we don't have a linker that supports Android yet,
so let's take a moment's digression to talk about building for
Android. To develop for Android we need the Android NDK. It contains
the linker rustc
needs to create Android binaries. To just build
Rust code that targets Android the only thing we need is the NDK, but
for practical development we'll want the Android SDK too.
On Linux, download and unpack them with the following commands (the output of which is not included here):
rust:~/home$ cd
rust:~$ curl -O https://dl.google.com/android/android-sdk_r24.4.1-linux.tgz
rust:~$ tar xzf android-sdk_r24.4.1-linux.tgz
rust:~$ curl -O https://dl.google.com/android/repository/android-ndk-r10e-linux-x86_64.zip
rust:~$ unzip android-ndk-r10e-linux-x86_64.zip
We further need to create what the NDK calls a "standalone
toolchain". We're going to put ours in a directory called
android-18-toolchain
:
rust:~$ android-ndk-r10e/build/tools/make-standalone-toolchain.sh \
--platform=android-18 --toolchain=arm-linux-androideabi-clang3.6 \
--install-dir=android-18-toolchain --ndk-dir=android-ndk-r10e/ --arch=arm
Auto-config: --toolchain=arm-linux-androideabi-4.8, --llvm-version=3.6
Copying prebuilt binaries...
Copying sysroot headers and libraries...
Copying c++ runtime headers and libraries...
Copying files to: android-18-toolchain
Cleaning up...
Done.
Let's notice a few things about these commands. First, the NDK we
downloaded, android-ndk-r10e-linux-x86_64.zip
is not the most recent
release (which at the time of this writing is 'r11c'). Rust's std
is
built against r10e
and links to symbols that are no longer included
in the NDK. So for now we have to use the older NDK. Second, in
building the standalone toolchain we passed --platform=android-18
to
make-standalone-toolchain.sh
. The "18" here is the Android API
level.
Today, Rust's arm-linux-androideabi
target is built against Android
API level 18, and should theoretically be forwards-compatible with
subsequent Android API levels. So we're picking level 18 to get the
greatest Android compatibility that Rust presently allows.
The final thing for us to do is tell Cargo where to find the android
linker, which is in the standalone NDK toolchain we just created. To
do that we configure the arm-linux-androideabi
target in
.cargo/config
with the 'linker' value. And while we're doing that
we'll go ahead and set the default target for this project to Android
so we don't have to keep calling cargo with the --target
option.
[build]
target = "arm-linux-androideabi"
[target.arm-linux-androideabi]
linker = "/home/rust/android-18-toolchain/bin/arm-linux-androideabi-clang"
Now let's change back to the 'hello' project directory and try to build again:
rust:~$ cd hello
rust:~/hello$ cargo build
Compiling hello v0.1.0 (file:///home/rust/hello)
Success! Of course just getting something to build is not the end of the story. You've also got to package your code up as an Android APK. For that you can use cargo-apk.
Rust everywhere else
Rust is a software platform with the potential to run on anything with a CPU. In this post I showed you a little bit of what Rust can already do, with the rustup tool. Today Rust runs on most of the platforms you use daily. Tomorrow it will run everywhere.
So what should you expect next?
In the coming months we're going to continue removing barriers to Rust cross-compilation. Today rustup provides access to the standard library, but as we've seen in this post, there's more to cross-compilation than rustc + std. It's acquiring and configuring the linker and C toolchain that is the most vexing — each combination of host and target platform requires something slightly different. We want to make this easier, and will be adding "NDK support" to rustup. What this means will again depend on the exact scenario, but we're going to start working from the most demanded, like Android, and try to automate as much of the detection, installation and configuration of the non-Rust toolchain components as we can. On Android for instance, the hope is to automate everything for a basic initial setup except for accepting the licenses.
In addition to that there are multiple efforts to improve Rust cross-compilation tooling, including xargo, which can be used to build the standard library for targets unsupported by rustup, and cargo-apk, which builds Android packages from Cargo packages.
Finally, the most exciting platform on the horizon for Rust is not a traditional target for systems languages: the web. With Emscripten today it's quite easy to run C++ code on the web by converting LLVM IR to JavaScript (or the asm.js subset of JavaScript). And the upcoming WebAssembly (wasm) standard will cement the web platform as a first-class target for programming languages.
Rust is uniquely-positioned to be the most powerful and usable wasm-targetting language for the immediate future. The same properties that make Rust so portable to real hardware makes it nearly trivial to port Rust to wasm. The same can't be said for languages with complex runtimes that include garbage collectors.
Rust has already been ported to Emscripten (at least twice), but the code has not yet fully landed. This summer it's happening though: Rust + Emscripten. Rust on the Web. Rust everywhere.
Epilogue
While many people are reporting success with rustup, it remains in beta, with some key outstanding bugs, and is not yet the officially recommended installation method for Rust (though you should try it). We're going to keep soliciting feedback, applying polish, and fixing bugs. Then we're going to improve the rustup installation experience on Windows by embedding it into a GUI that behaves like a proper Windows installer.
At that point we'll likely update the download instructions on www.rust-lang.org to recommend rustup. I expect all the existing installation methods to remain available, including the non-rustup Windows installers, but at that point our focus will be on improving the installation experience through rustup. It's also plausible that rustup itself will be packaged for package managers like Homebrew and apt.
If you want to try rustup for yourself, visit www.rustup.rs and follow the instructions. Then leave feedback on the dedicated forum thread, or file bugs on the issue tracker. More information about rustup is available in the README.
Thanks
Rust would not be the powerful system it is without the help of many individuals. Thanks to Diggory Blake for creating rustup, to Jorge Aparicio for fixing lots of cross-compilation bugs and documenting the process, Tomaka for pioneering Rust on Android, and Alex Crichton for creating the release infrastructure for Rust's many platforms.
And thanks to all the rustup contributors: Alex Crichton, Brian Anderson, Corey Farwell, David Salter, Diggory Blake, Jacob Shaffer, Jeremiah Peschka, Joe Wilm, Jorge Aparicio, Kai Noda, Kamal Marhubi, Kevin K, llogiq, Mika Attila, NODA, Kai, Paul Padier, Severen Redwood, Taylor Cramer, Tim Neumann, trolleyman, Vadim Petrochenkov, V Jackson, Vladimir, Wayne Warren, Yasushi Abe, Y. T. Chung