rustc/debian/README.Debian

Architecture-specific notes
===========================

This section talks about the rustc compiler on your host architecture. For
cross-compiling to a foreign target architecture, see the next section.

armhf armel mips mipsel powerpc powerpcspe
------------------------------------------

We only ship debuginfo for libstd and not the compiler itself, otherwise builds
run out of memory on the Debian buildds, with non-obvious and random errors.

See https://github.com/rust-lang/rust/issues/45854 for details.

If all your armhf build machines have ~8GB memory or more, you can experiment
with disabling this work-around (i.e. revert to normal) in d/rules.


Cross-compiling
===============

Rust supports cross-compiling to many different architectures, and we expose
this feature as fully as feasible in Debian, including to wasm and windows.

Introduction and terminology
----------------------------

Rust uses LLVM, so cross-compiling works a bit differently from the GNU
toolchain. The most important difference is that there are no "cross"
compilers, every compiler is already a cross compiler. For cross-compiling, all
you need to do (on the rustc / LLVM side) is to install the standard libraries
for each target architecture you want to compile to, i.e. libstd-rust-dev.

Before we go further, we must clarify some terminology. The rust ecosystem
generally uses the term "host" for the native architecture running the
compiler, equivalent to DEB_BUILD_RUST_TYPE or "build" in GNU terminology, and
"target" for the foreign architecture that the build products run on,
equivalent to DEB_HOST_RUST_TYPE or "host" in GNU terminology. For example,
rustc --version --verbose will output something like:

    rustc 1.16.0
    [..]
    host: x86_64-unknown-linux-gnu

And both rustc and cargo have --target flags:

    $ rustc --help | grep '\-\-target'
        --target TARGET     Target triple for which the code is compiled
    $ cargo build --help | grep '\-\-target'
        --target TRIPLE              Build for the target triple

One major exception to this naming scheme is in CERTAIN PARTS OF the build
scripts of cargo and rustc themselves, such as the `./configure` scripts and
SOME PARTS of the `config.toml` files. Here, "build", "host" and "target" mean
the same things they do in GNU toolchain terminology. However, IN OTHER PARTS
OF the build scripts of cargo and rustc, as well as cargo and rustc's own
output and logging messages, the term "host" and "target" mean as they do in
the previous paragraph. Yes, it's a total mind fuck. :( Table for clarity:

======================================= =============== ========================
                                        Rust ecosystem, Some parts of the rustc
GNU term / Debian envvar                rustc and cargo and cargo build scripts
======================================= =============== ========================
build   DEB_BUILD_{ARCH,RUST_TYPE}      host            build
  the machine running the build
--------------------------------------- --------------- ------------------------
host    DEB_HOST_{ARCH,RUST_TYPE}       target          host(s)
  the machine the build products run on
--------------------------------------- --------------- ------------------------
only relevant when building a compiler
target  DEB_TARGET_{ARCH,RUST_TYPE}     N/A             target(s)
  the one architecture that the built                     extra architectures
  cross-compiler itself builds for                        to build "std" for
--------------------------------------- --------------- ------------------------

General case for other Debian platforms
---------------------------------------

To manually use the Debian rustc binary for cross-compiling:

0. If you haven't done so previously, run:

   dpkg --add-architecture ${DEB_TARGET_ARCH}
   apt-get update

   (This is something that you need to do for all Debian crossbuilding or
   multi-architecture installing.)

1. Install crossbuild-essential-${DEB_TARGET_ARCH} e.g. arm64.

   (This is something that you need to do for all Debian crossbuilding.)

   For certain (HOST, TARGET) pairs you can instead install gcc-multilib, e.g.
   when compiling from amd64 to i386.

2. Install libstd-rust-dev:${DEB_TARGET_ARCH}.

3. Add the following flags to your rustc invocation:

   -C linker=${DEB_TARGET_GNU_TYPE}-gcc # e.g. aarch64-linux-gnu
   --target ${DEB_TARGET_RUST_TYPE} # e.g. aarch64-unknown-linux-gnu

   For certain (HOST, TARGET) pairs, namely the same ones as above that are
   supported by gcc-multilib, you can omit the linker flag since the default
   ``gcc`` linker (with multilib support) will work.

You can find the right TARGET vars to use in dpkg-architecture(1) and/or
/usr/share/rustc/architecture.mk and/or possibly on the Debian wiki.

These steps are different when cross-building a Debian package, or preparing
one for cross-compiling. (1) is performed automatically by cross-building tools
such as sbuild, and (3) is performed automatically by our cargo wrapper script.
The details of how to do (2) correctly are explained in the section below
called "Using rustc in a Debian package".

Foreign non-Debian platforms
----------------------------

Targeting a non-Debian platform is not a common Debian crossbuilding pattern,
so we do something ad-hoc for our Debian rust packages.

Instead of libstd-rust-dev:$arch (for an $arch that is not in Debian), we
provide a libstd-rust-dev-$platform:$arch package. For example,
libstd-rust-dev-windows:i386. For VM platforms such as WASM, $arch is omitted.

Instead of implicitly relying on crossbuild-essential-$arch (for an $arch that
is not in Debian), we have the libstd-rust-dev-$platform:$arch package
Recommend the appropriate linker. For example, Clang or MinGW.

To use these for manual crossbuilding:

1. Install the appropriate library package, as well as the corresponding linker
   package from its Recommends if it isn't pulled in automatically.

2. Pass in the appropriate ``-C linker`` and ``--target`` flags to ``rustc``.

WASM
~~~~

We ship three different wasm32 targets - wasm32-unknown-unknown, wasm32-wasip1
(previously known as wasm32-wasi) and wasm32-p2 - in the libstd-rust-dev-wasm32
package.

wasm32-unknown-unknown is suitable for web stuff, where you typically will need
to depending on the rust-wasm-bindgen, js-sys, and web-sys crates. Here, calls
to libstd stuff (such as println!()) will silently do nothing, as defined in
``library/std/src/sys/wasm/mod.rs`` and explained in upstream #48564.

wasm32-wasip1/wasm32-wasip2 are suitable for non-web stuff, and are closer to a
"normal" target where you expect libstd to be available, and for println!() to
actually print to stdout. If you just want to cross-compile a regular non-wasm
library or program to wasm for whatever reason, and only want to run it
natively and not inside a web browser, use one of these targets.

To run the generated wasm, you can either:

1. Use /usr/share/rustc/bin/wasi-node, which depends on nodejs.

   Pending #986616, this will be added to the nodejs package directly.

2. Compile and use one of the following runtimes:

   - https://github.com/bytecodealliance/wasmtime
   - https://github.com/bytecodealliance/lucet
   - https://github.com/wasmerio/wasmer

Windows
~~~~~~~

We ship the following targets:

- x86_64-pc-windows-gnu in the libstd-rust-dev-windows:amd64 package
-   i686-pc-windows-gnu in the libstd-rust-dev-windows:i386  package

To run the compiled binaries, you can use wine. You will need to set one of:

- WINEPATH="/usr/lib/gcc/x86_64-w64-mingw32/10-posix;/usr/lib/rustlib/x86_64-pc-windows-gnu/lib"
- WINEPATH="/usr/lib/gcc/i686-w64-mingw32/10-posix;/usr/lib/rustlib/i686-pc-windows-gnu/lib"

If you get "import_dll ... not found" errors, check that these paths are mapped
to some windows drive path - run "winecfg $path" for each path in the component
of WINEPATH; if any begin with "\\?\unix\" then you'll need to map them to a
drive in "winecfg" -> Drives. If all begin with some windows drive letter, then
your error is something unrelated and we sadly can't help you here.


Using rustc in a Debian package
===============================

You are encouraged to support cross-compiling. See the above section for more
details; in summary you need to install rustc for the host architecture and
libstd-rust-dev for the target architecture, so your debian/control would look
something like this:

    Build-Depends:
     [..]
     rustc:native    (>= $version),
     libstd-rust-dev (>= $version),
     [..]

You need both, this is important. When Debian build toolchains satisfy the
build-depends of a cross-build, (1) a "rustc:native" Build-Depends selects
rustc for the native architecture, which is possible because it's "Multi-Arch:
allowed", and this will implicitly pull in libstd-rust-dev also for the native
architecture; and (2) a "libstd-rust-dev" Build-Depends implies libstd-rust-dev
for the foreign architecture, since it's "Multi-Arch: same".

You'll probably also want to add

    include /usr/share/rustc/architecture.mk

to your debian/rules. This sets some useful variables like DEB_HOST_RUST_TYPE.
See the cargo package for an example.

If your build uses cargo, you'll want to add:

    Build-Depends:
     [..]
     cargo:native,
     [..]

and use our cargo wrapper script instead of /usr/bin/cargo directly. See
/usr/share/cargo/bin/cargo for details on how to use it.


Porting to new architectures (on the same distro)
=================================================

As mentioned above, to cross-compile rust packages you need to install the rust
standard library for each relevant foreign architecture. However, this is not
needed when cross-compiling rustc itself; its build system will build any
relevant foreign-architecture standard libraries automatically.

Cross-build, in a schroot using sbuild
--------------------------------------

0. Set up an schroot for your native architecture, for sbuild:

    sudo apt-get install sbuild
    sudo sbuild-adduser $LOGNAME
    newgrp sbuild # or log out and log back in
    sudo sbuild-createchroot --include=eatmydata,ccache,gnupg unstable \
      /srv/chroot/unstable-$(dpkg-architecture -qDEB_BUILD_ARCH)-sbuild \
      http://deb.debian.org/debian

   See https://wiki.debian.org/sbuild for more details.

1. Build it:

    sudo apt-get source --download-only rustc
    sbuild --host=$new_arch rustc_*.dsc

Cross-build, directly on your own system
----------------------------------------

0. Install the build-dependencies of rustc (including cargo and itself):

    sudo dpkg --add-architecture $new_arch
    sudo apt-get --no-install-recommends build-dep --host-architecture=$new_arch rustc

1. Build it:

    apt-get source --compile --host-architecture=$new_arch rustc

Native-build using bundled upstream binary blobs
------------------------------------------------

Use the same instructions as given in "Bootstrapping" in debian/README.source
in the source package, making sure to set the relevant architectures.

Responsible distribution of cross-built binaries
------------------------------------------------

By nature, cross-builds do not run tests. These are important for rustc and
many tests often fail on newly-supported architectures even if builds and
cross-builds work fine. You should find some appropriate way to test your
cross-built packages rather than blindly shipping them to users.

For example, Debian experimental is an appropriate place to upload them, so
that they can be installed and tested on Debian porter boxes, before being
uploaded to unstable and distributed to users.


Test failures
=============

Starting from version 1.20.0+dfsg1-1 the Debian packages of rustc no longer
fail the overall build if > 0 tests fail. Instead, we allow up to around 5
tests to fail. In other words, if you're reading this in a binary package,
between 0 and 5 tests might have failed when building this.

This is due to lack of maintainer time to investigate all failures. Many
previous test failures were reported to upstream and did not receive a timely
response, suggesting the failures were not important. I was then forced to
patch out the test to make the build proceed, so several tests were being
ignored in practise anyway.

This brings the Debian package in line with the Fedora package which also
ignores all test failures. (Many other distributions don't run tests at all.)

If you think that the Debian rustc package is miscompiling your program in a
way that the upstream distributed compiler doesn't, you may check the test
failures here:

https://buildd.debian.org/status/package.php?p=rustc

If you can identify a relevant test failure, as well as the patches needed to
fix it (either to rustc or LLVM), this will speed up the processing of any bug
reports on the Debian side.

We will also examine these failures ourselves on a best-effort basis and
attempt to fix the more serious-looking ones.

Uncommon architectures
----------------------

Debian release architectures armel and s390x currently have more test failures,
being tracked by upstream here:

- https://github.com/rust-lang/rust/issues/52493 armel
- https://github.com/rust-lang/rust/issues/52491 s390x

Ports architectures
-------------------

The number of allowed test failures on certain Debian ports architectures
(currently powerpc, powerpcspe, sparc64, x32) is raised greatly to help unblock
progress for porters. Of course, as a user this means you may run into more
bugs than usual; as mentioned above bugs reports and patches are welcome.


Shared libraries
================

For now, the shared libraries of Rust are private.
The rational is the following:
 * Upstream prefers static linking for now
   - https://github.com/rust-lang/rust/issues/10209
 * rust is still under heavy development. As far as we know, there is
   no commitement from upstream to provide a stable ABI for now.
   Until we know more, we cannot take the chance to have Rust-built packages
   failing at each release of the compiler.
 * Static builds are working out of the box just fine
 * However, LD_LIBRARY_PATH has to be updated when -C prefer-dynamic is used

 -- Sylvestre Ledru <sylvestre@debian.org>, Fri, 13 Feb 2015 15:08:43 +0100