New upstream version 1.20.0+dfsg1

CONTRIBUTING.md

@@ -232,6 +232,7 @@ Some common invocations of `x.py` are:
   guidelines as of yet, but basic rules like 4 spaces for indentation and no
   more than 99 characters in a single line should be kept in mind when writing
   code.
+- `rustup toolchain link <name> build/<host-triple>/<stage>` - Use the custom compiler build via [rustup](https://github.com/rust-lang-nursery/rustup.rs#working-with-custom-toolchains-and-local-builds).
 
 ## Pull Requests
 

COPYRIGHT

@@ -22,12 +22,6 @@ The Rust Project includes packages written by third parties.
 The following third party packages are included, and carry
 their own copyright notices and license terms:
 
-* The src/rt/miniz.c file, carrying an implementation of
-  RFC1950/RFC1951 DEFLATE, by Rich Geldreich
-  <richgel99@gmail.com>. All uses of this file are
-  permitted by the embedded "unlicense" notice
-  (effectively: public domain with warranty disclaimer).
-
 * LLVM. Code for this package is found in src/llvm.
 
     Copyright (c) 2003-2013 University of Illinois at

README.md

@@ -7,9 +7,9 @@ standard library, and documentation.
 
 ## Quick Start
 
-Read ["Installing Rust"] from [The Book].
+Read ["Installation"] from [The Book].
 
-["Installing Rust"]: https://doc.rust-lang.org/book/getting-started.html#installing-rust
+["Installation"]: https://doc.rust-lang.org/book/second-edition/ch01-01-installation.html
 [The Book]: https://doc.rust-lang.org/book/index.html
 
 ## Building from Source
@@ -40,8 +40,9 @@ Read ["Installing Rust"] from [The Book].
 
     > ***Note:*** Install locations can be adjusted by copying the config file
     > from `./src/bootstrap/config.toml.example` to `./config.toml`, and
-    > adjusting the `prefix` option under `[install]`. Various other options are
+    > adjusting the `prefix` option under `[install]`. Various other options, such
-    > also supported, and are documented in the config file.
+    > as enabling debug information, are also supported, and are documented in
+    > the config file.
 
     When complete, `sudo ./x.py install` will place several programs into
     `/usr/local/bin`: `rustc`, the Rust compiler, and `rustdoc`, the

configure

@@ -452,6 +452,7 @@ opt vendor 0 "enable usage of vendored Rust crates"
 opt sanitizers 0 "build the sanitizer runtimes (asan, lsan, msan, tsan)"
 opt dist-src 1 "when building tarballs enables building a source tarball"
 opt cargo-openssl-static 0 "static openssl in cargo"
+opt profiler 0 "build the profiler runtime"
 
 # Optimization and debugging options. These may be overridden by the release channel, etc.
 opt_nosave optimize 1 "build optimized rust code"
@@ -489,6 +490,7 @@ valopt musl-root-armhf "" "arm-unknown-linux-musleabihf install directory"
 valopt musl-root-armv7 "" "armv7-unknown-linux-musleabihf install directory"
 valopt extra-filename "" "Additional data that is hashed and passed to the -C extra-filename flag"
 valopt qemu-armhf-rootfs "" "rootfs in qemu testing, you probably don't want to use this"
+valopt experimental-targets "" "experimental LLVM targets to build"
 
 if [ -e ${CFG_SRC_DIR}.git ]
 then

src/Cargo.toml

@@ -9,6 +9,7 @@ members = [
   "tools/error_index_generator",
   "tools/linkchecker",
   "tools/rustbook",
+  "tools/unstable-book-gen",
   "tools/tidy",
   "tools/build-manifest",
   "tools/remote-test-client",

src/bootstrap/bin/rustc.rs

@@ -75,16 +75,11 @@ fn main() {
         Err(_) => 0,
     };
 
-    // Build scripts always use the snapshot compiler which is guaranteed to be
+    // Use a different compiler for build scripts, since there may not yet be a
-    // able to produce an executable, whereas intermediate compilers may not
+    // libstd for the real compiler to use. However, if Cargo is attempting to
-    // have the standard library built yet and may not be able to produce an
+    // determine the version of the compiler, the real compiler needs to be
-    // executable. Otherwise we just use the standard compiler we're
+    // used. Currently, these two states are differentiated based on whether
-    // bootstrapping with.
+    // --target and -vV is/isn't passed.
-    //
-    // Also note that cargo will detect the version of the compiler to trigger
-    // a rebuild when the compiler changes. If this happens, we want to make
-    // sure to use the actual compiler instead of the snapshot compiler becase
-    // that's the one that's actually changing.
     let (rustc, libdir) = if target.is_none() && version.is_none() {
         ("RUSTC_SNAPSHOT", "RUSTC_SNAPSHOT_LIBDIR")
     } else {
@@ -118,13 +113,6 @@ fn main() {
             cmd.arg("-Cprefer-dynamic");
         }
 
-        // Pass the `rustbuild` feature flag to crates which rustbuild is
-        // building. See the comment in bootstrap/lib.rs where this env var is
-        // set for more details.
-        if env::var_os("RUSTBUILD_UNSTABLE").is_some() {
-            cmd.arg("--cfg").arg("rustbuild");
-        }
-
         // Help the libc crate compile by assisting it in finding the MUSL
         // native libraries.
         if let Some(s) = env::var_os("MUSL_ROOT") {
@@ -149,6 +137,11 @@ fn main() {
             }
         }
 
+        let crate_name = args.windows(2)
+            .find(|a| &*a[0] == "--crate-name")
+            .unwrap();
+        let crate_name = &*crate_name[1];
+
         // If we're compiling specifically the `panic_abort` crate then we pass
         // the `-C panic=abort` option. Note that we do not do this for any
         // other crate intentionally as this is the only crate for now that we
@@ -157,9 +150,12 @@ fn main() {
         // This... is a bit of a hack how we detect this. Ideally this
         // information should be encoded in the crate I guess? Would likely
         // require an RFC amendment to RFC 1513, however.
-        let is_panic_abort = args.windows(2)
+        //
-            .any(|a| &*a[0] == "--crate-name" && &*a[1] == "panic_abort");
+        // `compiler_builtins` are unconditionally compiled with panic=abort to
-        if is_panic_abort {
+        // workaround undefined references to `rust_eh_unwind_resume` generated
+        // otherwise, see issue https://github.com/rust-lang/rust/issues/43095.
+        if crate_name == "panic_abort" ||
+           crate_name == "compiler_builtins" && stage != "0" {
             cmd.arg("-C").arg("panic=abort");
         }
 
@@ -174,7 +170,15 @@ fn main() {
             Ok(s) => if s == "true" { "y" } else { "n" },
             Err(..) => "n",
         };
-        cmd.arg("-C").arg(format!("debug-assertions={}", debug_assertions));
+
+        // The compiler builtins are pretty sensitive to symbols referenced in
+        // libcore and such, so we never compile them with debug assertions.
+        if crate_name == "compiler_builtins" {
+            cmd.arg("-C").arg("debug-assertions=no");
+        } else {
+            cmd.arg("-C").arg(format!("debug-assertions={}", debug_assertions));
+        }
+
         if let Ok(s) = env::var("RUSTC_CODEGEN_UNITS") {
             cmd.arg("-C").arg(format!("codegen-units={}", s));
         }
@@ -218,11 +222,7 @@ fn main() {
                 // do that we pass a weird flag to the compiler to get it to do
                 // so. Note that this is definitely a hack, and we should likely
                 // flesh out rpath support more fully in the future.
-                //
+                cmd.arg("-Z").arg("osx-rpath-install-name");
-                // FIXME: remove condition after next stage0
-                if stage != "0" {
-                    cmd.arg("-Z").arg("osx-rpath-install-name");
-                }
                 Some("-Wl,-rpath,@loader_path/../lib")
             } else if !target.contains("windows") {
                 Some("-Wl,-rpath,$ORIGIN/../lib")
@@ -242,15 +242,20 @@ fn main() {
         // Force all crates compiled by this compiler to (a) be unstable and (b)
         // allow the `rustc_private` feature to link to other unstable crates
         // also in the sysroot.
-        //
-        // FIXME: remove condition after next stage0
         if env::var_os("RUSTC_FORCE_UNSTABLE").is_some() {
-            if stage != "0" {
+            cmd.arg("-Z").arg("force-unstable-if-unmarked");
-                cmd.arg("-Z").arg("force-unstable-if-unmarked");
-            }
         }
     }
 
+    let color = match env::var("RUSTC_COLOR") {
+        Ok(s) => usize::from_str(&s).expect("RUSTC_COLOR should be an integer"),
+        Err(_) => 0,
+    };
+
+    if color != 0 {
+        cmd.arg("--color=always");
+    }
+
     if verbose > 1 {
         writeln!(&mut io::stderr(), "rustc command: {:?}", cmd).unwrap();
     }

src/bootstrap/bin/rustdoc.rs

@@ -41,11 +41,11 @@ fn main() {
         .env(bootstrap::util::dylib_path_var(),
              env::join_paths(&dylib_path).unwrap());
 
-    // Pass the `rustbuild` feature flag to crates which rustbuild is
+    // Force all crates compiled by this compiler to (a) be unstable and (b)
-    // building. See the comment in bootstrap/lib.rs where this env var is
+    // allow the `rustc_private` feature to link to other unstable crates
-    // set for more details.
+    // also in the sysroot.
-    if env::var_os("RUSTBUILD_UNSTABLE").is_some() {
+    if env::var_os("RUSTC_FORCE_UNSTABLE").is_some() {
-        cmd.arg("--cfg").arg("rustbuild");
+        cmd.arg("-Z").arg("force-unstable-if-unmarked");
     }
 
     std::process::exit(match cmd.status() {

src/bootstrap/bootstrap.py

@@ -25,10 +25,11 @@ from time import time
 
 
 def get(url, path, verbose=False):
-    sha_url = url + ".sha256"
+    suffix = '.sha256'
+    sha_url = url + suffix
     with tempfile.NamedTemporaryFile(delete=False) as temp_file:
         temp_path = temp_file.name
-    with tempfile.NamedTemporaryFile(suffix=".sha256", delete=False) as sha_file:
+    with tempfile.NamedTemporaryFile(suffix=suffix, delete=False) as sha_file:
         sha_path = sha_file.name
 
     try:
@@ -55,6 +56,7 @@ def get(url, path, verbose=False):
 
 
 def delete_if_present(path, verbose):
+    """Remove the given file if present"""
     if os.path.isfile(path):
         if verbose:
             print("removing " + path)
@@ -92,12 +94,13 @@ def _download(path, url, probably_big, verbose, exception):
 
 
 def verify(path, sha_path, verbose):
+    """Check if the sha256 sum of the given path is valid"""
     if verbose:
         print("verifying " + path)
-    with open(path, "rb") as f:
+    with open(path, "rb") as source:
-        found = hashlib.sha256(f.read()).hexdigest()
+        found = hashlib.sha256(source.read()).hexdigest()
-    with open(sha_path, "r") as f:
+    with open(sha_path, "r") as sha256sum:
-        expected = f.readline().split()[0]
+        expected = sha256sum.readline().split()[0]
     verified = found == expected
     if not verified:
         print("invalid checksum:\n"
@@ -107,6 +110,7 @@ def verify(path, sha_path, verbose):
 
 
 def unpack(tarball, dst, verbose=False, match=None):
+    """Unpack the given tarball file"""
     print("extracting " + tarball)
     fname = os.path.basename(tarball).replace(".tar.gz", "")
     with contextlib.closing(tarfile.open(tarball)) as tar:
@@ -128,6 +132,7 @@ def unpack(tarball, dst, verbose=False, match=None):
             shutil.move(tp, fp)
     shutil.rmtree(os.path.join(dst, fname))
 
+
 def run(args, verbose=False, exception=False, **kwargs):
     if verbose:
         print("running: " + ' '.join(args))
@@ -245,7 +250,8 @@ class RustBuild(object):
             return
 
         # At this point we're pretty sure the user is running NixOS
-        print("info: you seem to be running NixOS. Attempting to patch " + fname)
+        nix_os_msg = "info: you seem to be running NixOS. Attempting to patch"
+        print(nix_os_msg, fname)
 
         try:
             interpreter = subprocess.check_output(
@@ -293,18 +299,22 @@ class RustBuild(object):
         return self._cargo_channel
 
     def rustc_stamp(self):
+        """Return the path for .rustc-stamp"""
         return os.path.join(self.bin_root(), '.rustc-stamp')
 
     def cargo_stamp(self):
+        """Return the path for .cargo-stamp"""
         return os.path.join(self.bin_root(), '.cargo-stamp')
 
     def rustc_out_of_date(self):
+        """Check if rustc is out of date"""
         if not os.path.exists(self.rustc_stamp()) or self.clean:
             return True
         with open(self.rustc_stamp(), 'r') as f:
             return self.stage0_date() != f.read()
 
     def cargo_out_of_date(self):
+        """Check if cargo is out of date"""
         if not os.path.exists(self.cargo_stamp()) or self.clean:
             return True
         with open(self.cargo_stamp(), 'r') as f:
@@ -357,8 +367,7 @@ class RustBuild(object):
     def exe_suffix(self):
         if sys.platform == 'win32':
             return '.exe'
-        else:
+        return ''
-            return ''
 
     def print_what_it_means_to_bootstrap(self):
         if hasattr(self, 'printed'):
@@ -366,7 +375,7 @@ class RustBuild(object):
         self.printed = True
         if os.path.exists(self.bootstrap_binary()):
             return
-        if not '--help' in sys.argv or len(sys.argv) == 1:
+        if '--help' not in sys.argv or len(sys.argv) == 1:
             return
 
         print('info: the build system for Rust is written in Rust, so this')
@@ -385,6 +394,7 @@ class RustBuild(object):
         if self.clean and os.path.exists(build_dir):
             shutil.rmtree(build_dir)
         env = os.environ.copy()
+        env["RUSTC_BOOTSTRAP"] = '1'
         env["CARGO_TARGET_DIR"] = build_dir
         env["RUSTC"] = self.rustc()
         env["LD_LIBRARY_PATH"] = os.path.join(self.bin_root(), "lib") + \
@@ -460,8 +470,8 @@ class RustBuild(object):
             # always emit 'i386' on x86/amd64 systems).  As such, isainfo -k
             # must be used instead.
             try:
-                cputype = subprocess.check_output(['isainfo',
+                cputype = subprocess.check_output(
-                                                   '-k']).strip().decode(default_encoding)
+                    ['isainfo', '-k']).strip().decode(default_encoding)
             except (subprocess.CalledProcessError, OSError):
                 err = "isainfo not found"
                 if self.verbose:
@@ -561,21 +571,26 @@ class RustBuild(object):
         default_encoding = sys.getdefaultencoding()
         run(["git", "submodule", "-q", "sync"], cwd=self.rust_root)
         submodules = [s.split(' ', 1)[1] for s in subprocess.check_output(
-            ["git", "config", "--file", os.path.join(self.rust_root, ".gitmodules"),
+            ["git", "config", "--file",
+             os.path.join(self.rust_root, ".gitmodules"),
              "--get-regexp", "path"]
         ).decode(default_encoding).splitlines()]
         submodules = [module for module in submodules
                       if not ((module.endswith("llvm") and
-                               (self.get_toml('llvm-config') or self.get_mk('CFG_LLVM_ROOT'))) or
+                               (self.get_toml('llvm-config') or
+                                self.get_mk('CFG_LLVM_ROOT'))) or
                               (module.endswith("jemalloc") and
-                               (self.get_toml('jemalloc') or self.get_mk('CFG_JEMALLOC_ROOT'))))
+                               (self.get_toml('jemalloc') or
-                     ]
+                                self.get_mk('CFG_JEMALLOC_ROOT'))))]
         run(["git", "submodule", "update",
-                  "--init"] + submodules, cwd=self.rust_root, verbose=self.verbose)
+             "--init", "--recursive"] + submodules,
+            cwd=self.rust_root, verbose=self.verbose)
         run(["git", "submodule", "-q", "foreach", "git",
-                  "reset", "-q", "--hard"], cwd=self.rust_root, verbose=self.verbose)
+             "reset", "-q", "--hard"],
+            cwd=self.rust_root, verbose=self.verbose)
         run(["git", "submodule", "-q", "foreach", "git",
-                  "clean", "-qdfx"], cwd=self.rust_root, verbose=self.verbose)
+             "clean", "-qdfx"],
+            cwd=self.rust_root, verbose=self.verbose)
 
 
 def bootstrap():
@@ -667,6 +682,7 @@ def bootstrap():
     env["BUILD"] = rb.build
     env["SRC"] = rb.rust_root
     env["BOOTSTRAP_PARENT_ID"] = str(os.getpid())
+    env["BOOTSTRAP_PYTHON"] = sys.executable
     run(args, env=env, verbose=rb.verbose)
 
 
@@ -690,5 +706,6 @@ def main():
                   format_build_time(time() - start_time))
         sys.exit(exit_code)
 
+
 if __name__ == '__main__':
     main()

src/bootstrap/cc.rs

@@ -42,10 +42,13 @@ use config::Target;
 pub fn find(build: &mut Build) {
     // For all targets we're going to need a C compiler for building some shims
     // and such as well as for being a linker for Rust code.
-    for target in build.config.target.iter() {
+    //
+    // This includes targets that aren't necessarily passed on the commandline
+    // (FIXME: Perhaps it shouldn't?)
+    for target in &build.config.target {
         let mut cfg = gcc::Config::new();
         cfg.cargo_metadata(false).opt_level(0).debug(false)
-           .target(target).host(&build.config.build);
+           .target(target).host(&build.build);
 
         let config = build.config.target_config.get(target);
         if let Some(cc) = config.and_then(|c| c.cc.as_ref()) {
@@ -64,10 +67,13 @@ pub fn find(build: &mut Build) {
     }
 
     // For all host triples we need to find a C++ compiler as well
-    for host in build.config.host.iter() {
+    //
+    // This includes hosts that aren't necessarily passed on the commandline
+    // (FIXME: Perhaps it shouldn't?)
+    for host in &build.config.host {
         let mut cfg = gcc::Config::new();
         cfg.cargo_metadata(false).opt_level(0).debug(false).cpp(true)
-           .target(host).host(&build.config.build);
+           .target(host).host(&build.build);
         let config = build.config.target_config.get(host);
         if let Some(cxx) = config.and_then(|c| c.cxx.as_ref()) {
             cfg.compiler(cxx);

src/bootstrap/channel.rs

@@ -23,12 +23,12 @@ use build_helper::output;
 use Build;
 
 // The version number
-pub const CFG_RELEASE_NUM: &'static str = "1.19.0";
+pub const CFG_RELEASE_NUM: &str = "1.20.0";
 
 // An optional number to put after the label, e.g. '.2' -> '-beta.2'
 // Be sure to make this starts with a dot to conform to semver pre-release
 // versions (section 9)
-pub const CFG_PRERELEASE_VERSION: &'static str = ".4";
+pub const CFG_PRERELEASE_VERSION: &str = ".3";
 
 pub struct GitInfo {
     inner: Option<Info>,
@@ -99,6 +99,10 @@ impl GitInfo {
             version.push_str(&inner.commit_date);
             version.push_str(")");
         }
-        return version
+        version
+    }
+
+    pub fn is_git(&self) -> bool {
+        self.inner.is_some()
     }
 }

src/bootstrap/check.rs

@@ -13,22 +13,23 @@
 //! This file implements the various regression test suites that we execute on
 //! our CI.
 
-extern crate build_helper;
-
 use std::collections::HashSet;
 use std::env;
+use std::ffi::OsString;
+use std::iter;
 use std::fmt;
-use std::fs;
+use std::fs::{self, File};
 use std::path::{PathBuf, Path};
 use std::process::Command;
+use std::io::Read;
 
-use build_helper::output;
+use build_helper::{self, output};
 
 use {Build, Compiler, Mode};
 use dist;
 use util::{self, dylib_path, dylib_path_var, exe};
 
-const ADB_TEST_DIR: &'static str = "/data/tmp/work";
+const ADB_TEST_DIR: &str = "/data/tmp/work";
 
 /// The two modes of the test runner; tests or benchmarks.
 #[derive(Copy, Clone)]
@@ -59,7 +60,7 @@ impl fmt::Display for TestKind {
 }
 
 fn try_run(build: &Build, cmd: &mut Command) {
-    if build.flags.cmd.no_fail_fast() {
+    if !build.fail_fast {
         if !build.try_run(cmd) {
             let failures = build.delayed_failures.get();
             build.delayed_failures.set(failures + 1);
@@ -70,7 +71,7 @@ fn try_run(build: &Build, cmd: &mut Command) {
 }
 
 fn try_run_quiet(build: &Build, cmd: &mut Command) {
-    if build.flags.cmd.no_fail_fast() {
+    if !build.fail_fast {
         if !build.try_run_quiet(cmd) {
             let failures = build.delayed_failures.get();
             build.delayed_failures.set(failures + 1);
@@ -98,7 +99,7 @@ pub fn linkcheck(build: &Build, host: &str) {
 /// This tool in `src/tools` will check out a few Rust projects and run `cargo
 /// test` to ensure that we don't regress the test suites there.
 pub fn cargotest(build: &Build, stage: u32, host: &str) {
-    let ref compiler = Compiler::new(stage, host);
+    let compiler = Compiler::new(stage, host);
 
     // Note that this is a short, cryptic, and not scoped directory name. This
     // is currently to minimize the length of path on Windows where we otherwise
@@ -108,28 +109,20 @@ pub fn cargotest(build: &Build, stage: u32, host: &str) {
 
     let _time = util::timeit();
     let mut cmd = Command::new(build.tool(&Compiler::new(0, host), "cargotest"));
-    build.prepare_tool_cmd(compiler, &mut cmd);
+    build.prepare_tool_cmd(&compiler, &mut cmd);
-    try_run(build, cmd.arg(&build.cargo)
+    try_run(build, cmd.arg(&build.initial_cargo)
                       .arg(&out_dir)
-                      .env("RUSTC", build.compiler_path(compiler))
+                      .env("RUSTC", build.compiler_path(&compiler))
-                      .env("RUSTDOC", build.rustdoc(compiler)));
+                      .env("RUSTDOC", build.rustdoc(&compiler)));
 }
 
 /// Runs `cargo test` for `cargo` packaged with Rust.
 pub fn cargo(build: &Build, stage: u32, host: &str) {
-    let ref compiler = Compiler::new(stage, host);
+    let compiler = &Compiler::new(stage, host);
-
-    // Configure PATH to find the right rustc. NB. we have to use PATH
-    // and not RUSTC because the Cargo test suite has tests that will
-    // fail if rustc is not spelled `rustc`.
-    let path = build.sysroot(compiler).join("bin");
-    let old_path = ::std::env::var("PATH").expect("");
-    let sep = if cfg!(windows) { ";" } else {":" };
-    let ref newpath = format!("{}{}{}", path.display(), sep, old_path);
 
     let mut cargo = build.cargo(compiler, Mode::Tool, host, "test");
     cargo.arg("--manifest-path").arg(build.src.join("src/tools/cargo/Cargo.toml"));
-    if build.flags.cmd.no_fail_fast() {
+    if !build.fail_fast {
         cargo.arg("--no-fail-fast");
     }
 
@@ -140,7 +133,31 @@ pub fn cargo(build: &Build, stage: u32, host: &str) {
     // available.
     cargo.env("CFG_DISABLE_CROSS_TESTS", "1");
 
-    try_run(build, cargo.env("PATH", newpath));
+    try_run(build, cargo.env("PATH", &path_for_cargo(build, compiler)));
+}
+
+/// Runs `cargo test` for the rls.
+pub fn rls(build: &Build, stage: u32, host: &str) {
+    let compiler = &Compiler::new(stage, host);
+
+    let mut cargo = build.cargo(compiler, Mode::Tool, host, "test");
+    cargo.arg("--manifest-path").arg(build.src.join("src/tools/rls/Cargo.toml"));
+
+    // Don't build tests dynamically, just a pain to work with
+    cargo.env("RUSTC_NO_PREFER_DYNAMIC", "1");
+
+    build.add_rustc_lib_path(compiler, &mut cargo);
+
+    try_run(build, &mut cargo);
+}
+
+fn path_for_cargo(build: &Build, compiler: &Compiler) -> OsString {
+    // Configure PATH to find the right rustc. NB. we have to use PATH
+    // and not RUSTC because the Cargo test suite has tests that will
+    // fail if rustc is not spelled `rustc`.
+    let path = build.sysroot(compiler).join("bin");
+    let old_path = env::var_os("PATH").unwrap_or_default();
+    env::join_paths(iter::once(path).chain(env::split_paths(&old_path))).expect("")
 }
 
 /// Runs the `tidy` tool as compiled in `stage` by the `host` compiler.
@@ -197,9 +214,9 @@ pub fn compiletest(build: &Build,
     cmd.arg("--mode").arg(mode);
     cmd.arg("--target").arg(target);
     cmd.arg("--host").arg(compiler.host);
-    cmd.arg("--llvm-filecheck").arg(build.llvm_filecheck(&build.config.build));
+    cmd.arg("--llvm-filecheck").arg(build.llvm_filecheck(&build.build));
 
-    if let Some(nodejs) = build.config.nodejs.as_ref() {
+    if let Some(ref nodejs) = build.config.nodejs {
         cmd.arg("--nodejs").arg(nodejs);
     }
 
@@ -223,7 +240,7 @@ pub fn compiletest(build: &Build,
 
     cmd.arg("--docck-python").arg(build.python());
 
-    if build.config.build.ends_with("apple-darwin") {
+    if build.build.ends_with("apple-darwin") {
         // Force /usr/bin/python on macOS for LLDB tests because we're loading the
         // LLDB plugin's compiled module which only works with the system python
         // (namely not Homebrew-installed python)
@@ -244,10 +261,13 @@ pub fn compiletest(build: &Build,
     let llvm_config = build.llvm_config(target);
     let llvm_version = output(Command::new(&llvm_config).arg("--version"));
     cmd.arg("--llvm-version").arg(llvm_version);
+    if !build.is_rust_llvm(target) {
+        cmd.arg("--system-llvm");
+    }
 
     cmd.args(&build.flags.cmd.test_args());
 
-    if build.config.verbose() || build.flags.verbose() {
+    if build.is_verbose() {
         cmd.arg("--verbose");
     }
 
@@ -261,7 +281,7 @@ pub fn compiletest(build: &Build,
         let llvm_components = output(Command::new(&llvm_config).arg("--components"));
         let llvm_cxxflags = output(Command::new(&llvm_config).arg("--cxxflags"));
         cmd.arg("--cc").arg(build.cc(target))
-           .arg("--cxx").arg(build.cxx(target))
+           .arg("--cxx").arg(build.cxx(target).unwrap())
            .arg("--cflags").arg(build.cflags(target).join(" "))
            .arg("--llvm-components").arg(llvm_components.trim())
            .arg("--llvm-cxxflags").arg(llvm_cxxflags.trim());
@@ -275,7 +295,7 @@ pub fn compiletest(build: &Build,
 
     if build.remote_tested(target) {
         cmd.arg("--remote-test-client")
-           .arg(build.tool(&Compiler::new(0, &build.config.build),
+           .arg(build.tool(&Compiler::new(0, &build.build),
                            "remote-test-client"));
     }
 
@@ -298,6 +318,10 @@ pub fn compiletest(build: &Build,
         cmd.env("SANITIZER_SUPPORT", "1");
     }
 
+    if build.config.profiler {
+        cmd.env("PROFILER_SUPPORT", "1");
+    }
+
     cmd.arg("--adb-path").arg("adb");
     cmd.arg("--adb-test-dir").arg(ADB_TEST_DIR);
     if target.contains("android") {
@@ -333,20 +357,15 @@ pub fn docs(build: &Build, compiler: &Compiler) {
         }
 
         if p.extension().and_then(|s| s.to_str()) != Some("md") {
-            continue
+            continue;
         }
 
-        // The nostarch directory in the book is for no starch, and so isn't guaranteed to build.
+        // The nostarch directory in the book is for no starch, and so isn't
-        // we don't care if it doesn't build, so skip it.
+        // guaranteed to build. We don't care if it doesn't build, so skip it.
-        use std::ffi::OsStr;
+        if p.to_str().map_or(false, |p| p.contains("nostarch")) {
-        let path: &OsStr = p.as_ref();
+            continue;
-        if let Some(path) = path.to_str() {
-            if path.contains("nostarch") {
-                continue;
-            }
         }
 
-        println!("doc tests for: {}", p.display());
         markdown_test(build, compiler, &p);
     }
 }
@@ -370,12 +389,20 @@ pub fn error_index(build: &Build, compiler: &Compiler) {
                              "error_index_generator")
                    .arg("markdown")
                    .arg(&output)
-                   .env("CFG_BUILD", &build.config.build));
+                   .env("CFG_BUILD", &build.build));
 
     markdown_test(build, compiler, &output);
 }
 
 fn markdown_test(build: &Build, compiler: &Compiler, markdown: &Path) {
+    let mut file = t!(File::open(markdown));
+    let mut contents = String::new();
+    t!(file.read_to_string(&mut contents));
+    if !contents.contains("```") {
+        return;
+    }
+
+    println!("doc tests for: {}", markdown.display());
     let mut cmd = Command::new(build.rustdoc(compiler));
     build.add_rustc_lib_path(compiler, &mut cmd);
     build.add_rust_test_threads(&mut cmd);
@@ -444,7 +471,7 @@ pub fn krate(build: &Build,
     cargo.arg("--manifest-path")
          .arg(build.src.join(path).join("Cargo.toml"))
          .arg("--features").arg(features);
-    if test_kind.subcommand() == "test" && build.flags.cmd.no_fail_fast() {
+    if test_kind.subcommand() == "test" && !build.fail_fast {
         cargo.arg("--no-fail-fast");
     }
 
@@ -514,16 +541,14 @@ fn krate_emscripten(build: &Build,
                     compiler: &Compiler,
                     target: &str,
                     mode: Mode) {
-    let mut tests = Vec::new();
     let out_dir = build.cargo_out(compiler, mode, target);
-    find_tests(&out_dir.join("deps"), target, &mut tests);
+    let tests = find_tests(&out_dir.join("deps"), target);
 
+    let nodejs = build.config.nodejs.as_ref().expect("nodejs not configured");
     for test in tests {
-        let test_file_name = test.to_string_lossy().into_owned();
+        println!("running {}", test.display());
-        println!("running {}", test_file_name);
-        let nodejs = build.config.nodejs.as_ref().expect("nodejs not configured");
         let mut cmd = Command::new(nodejs);
-        cmd.arg(&test_file_name);
+        cmd.arg(&test);
         if build.config.quiet_tests {
             cmd.arg("--quiet");
         }
@@ -535,11 +560,10 @@ fn krate_remote(build: &Build,
                 compiler: &Compiler,
                 target: &str,
                 mode: Mode) {
-    let mut tests = Vec::new();
     let out_dir = build.cargo_out(compiler, mode, target);
-    find_tests(&out_dir.join("deps"), target, &mut tests);
+    let tests = find_tests(&out_dir.join("deps"), target);
 
-    let tool = build.tool(&Compiler::new(0, &build.config.build),
+    let tool = build.tool(&Compiler::new(0, &build.build),
                           "remote-test-client");
     for test in tests {
         let mut cmd = Command::new(&tool);
@@ -553,9 +577,8 @@ fn krate_remote(build: &Build,
     }
 }
 
-fn find_tests(dir: &Path,
+fn find_tests(dir: &Path, target: &str) -> Vec<PathBuf> {
-              target: &str,
+    let mut dst = Vec::new();
-              dst: &mut Vec<PathBuf>) {
     for e in t!(dir.read_dir()).map(|e| t!(e)) {
         let file_type = t!(e.file_type());
         if !file_type.is_file() {
@@ -564,10 +587,13 @@ fn find_tests(dir: &Path,
         let filename = e.file_name().into_string().unwrap();
         if (target.contains("windows") && filename.ends_with(".exe")) ||
            (!target.contains("windows") && !filename.contains(".")) ||
-           (target.contains("emscripten") && filename.ends_with(".js")) {
+           (target.contains("emscripten") &&
+            filename.ends_with(".js") &&
+            !filename.ends_with(".asm.js")) {
             dst.push(e.path());
         }
     }
+    dst
 }
 
 pub fn remote_copy_libs(build: &Build, compiler: &Compiler, target: &str) {
@@ -582,7 +608,7 @@ pub fn remote_copy_libs(build: &Build, compiler: &Compiler, target: &str) {
                       .join(exe("remote-test-server", target));
 
     // Spawn the emulator and wait for it to come online
-    let tool = build.tool(&Compiler::new(0, &build.config.build),
+    let tool = build.tool(&Compiler::new(0, &build.build),
                           "remote-test-client");
     let mut cmd = Command::new(&tool);
     cmd.arg("spawn-emulator")
@@ -608,7 +634,7 @@ pub fn remote_copy_libs(build: &Build, compiler: &Compiler, target: &str) {
 
 /// Run "distcheck", a 'make check' from a tarball
 pub fn distcheck(build: &Build) {
-    if build.config.build != "x86_64-unknown-linux-gnu" {
+    if build.build != "x86_64-unknown-linux-gnu" {
         return
     }
     if !build.config.host.iter().any(|s| s == "x86_64-unknown-linux-gnu") {
@@ -633,7 +659,7 @@ pub fn distcheck(build: &Build) {
                      .args(&build.config.configure_args)
                      .arg("--enable-vendor")
                      .current_dir(&dir));
-    build.run(Command::new(build_helper::make(&build.config.build))
+    build.run(Command::new(build_helper::make(&build.build))
                      .arg("check")
                      .current_dir(&dir));
 
@@ -651,7 +677,7 @@ pub fn distcheck(build: &Build) {
     build.run(&mut cmd);
 
     let toml = dir.join("rust-src/lib/rustlib/src/rust/src/libstd/Cargo.toml");
-    build.run(Command::new(&build.cargo)
+    build.run(Command::new(&build.initial_cargo)
                      .arg("generate-lockfile")
                      .arg("--manifest-path")
                      .arg(&toml)
@@ -660,12 +686,13 @@ pub fn distcheck(build: &Build) {
 
 /// Test the build system itself
 pub fn bootstrap(build: &Build) {
-    let mut cmd = Command::new(&build.cargo);
+    let mut cmd = Command::new(&build.initial_cargo);
     cmd.arg("test")
        .current_dir(build.src.join("src/bootstrap"))
        .env("CARGO_TARGET_DIR", build.out.join("bootstrap"))
-       .env("RUSTC", &build.rustc);
+       .env("RUSTC_BOOTSTRAP", "1")
-    if build.flags.cmd.no_fail_fast() {
+       .env("RUSTC", &build.initial_rustc);
+    if !build.fail_fast {
         cmd.arg("--no-fail-fast");
     }
     cmd.arg("--").args(&build.flags.cmd.test_args());

src/bootstrap/compile.rs

@@ -50,7 +50,7 @@ pub fn std(build: &Build, target: &str, compiler: &Compiler) {
     let mut cargo = build.cargo(compiler, Mode::Libstd, target, "build");
     let mut features = build.std_features();
 
-    if let Ok(target) = env::var("MACOSX_STD_DEPLOYMENT_TARGET") {
+    if let Some(target) = env::var_os("MACOSX_STD_DEPLOYMENT_TARGET") {
         cargo.env("MACOSX_DEPLOYMENT_TARGET", target);
     }
 
@@ -158,7 +158,7 @@ pub fn build_startup_objects(build: &Build, for_compiler: &Compiler, target: &st
         return
     }
 
-    let compiler = Compiler::new(0, &build.config.build);
+    let compiler = Compiler::new(0, &build.build);
     let compiler_path = build.compiler_path(&compiler);
     let src_dir = &build.src.join("src/rtstartup");
     let dst_dir = &build.native_dir(target).join("rtstartup");
@@ -199,7 +199,7 @@ pub fn test(build: &Build, target: &str, compiler: &Compiler) {
     let out_dir = build.cargo_out(compiler, Mode::Libtest, target);
     build.clear_if_dirty(&out_dir, &libstd_stamp(build, compiler, target));
     let mut cargo = build.cargo(compiler, Mode::Libtest, target, "build");
-    if let Ok(target) = env::var("MACOSX_STD_DEPLOYMENT_TARGET") {
+    if let Some(target) = env::var_os("MACOSX_STD_DEPLOYMENT_TARGET") {
         cargo.env("MACOSX_DEPLOYMENT_TARGET", target);
     }
     cargo.arg("--manifest-path")
@@ -247,7 +247,7 @@ pub fn rustc(build: &Build, target: &str, compiler: &Compiler) {
     cargo.env("CFG_RELEASE", build.rust_release())
          .env("CFG_RELEASE_CHANNEL", &build.config.channel)
          .env("CFG_VERSION", build.rust_version())
-         .env("CFG_PREFIX", build.config.prefix.clone().unwrap_or(PathBuf::new()));
+         .env("CFG_PREFIX", build.config.prefix.clone().unwrap_or_default());
 
     if compiler.stage == 0 {
         cargo.env("CFG_LIBDIR_RELATIVE", "lib");
@@ -287,7 +287,7 @@ pub fn rustc(build: &Build, target: &str, compiler: &Compiler) {
        !target.contains("windows") &&
        !target.contains("apple") {
         cargo.env("LLVM_STATIC_STDCPP",
-                  compiler_file(build.cxx(target), "libstdc++.a"));
+                  compiler_file(build.cxx(target).unwrap(), "libstdc++.a"));
     }
     if build.config.llvm_link_shared {
         cargo.env("LLVM_LINK_SHARED", "1");
@@ -351,7 +351,7 @@ pub fn create_sysroot(build: &Build, compiler: &Compiler) {
 /// Prepare a new compiler from the artifacts in `stage`
 ///
 /// This will assemble a compiler in `build/$host/stage$stage`. The compiler
-/// must have been previously produced by the `stage - 1` build.config.build
+/// must have been previously produced by the `stage - 1` build.build
 /// compiler.
 pub fn assemble_rustc(build: &Build, stage: u32, host: &str) {
     // nothing to do in stage0
@@ -365,7 +365,7 @@ pub fn assemble_rustc(build: &Build, stage: u32, host: &str) {
     let target_compiler = Compiler::new(stage, host);
 
     // The compiler that compiled the compiler we're assembling
-    let build_compiler = Compiler::new(stage - 1, &build.config.build);
+    let build_compiler = Compiler::new(stage - 1, &build.build);
 
     // Link in all dylibs to the libdir
     let sysroot = build.sysroot(&target_compiler);
@@ -385,7 +385,7 @@ pub fn assemble_rustc(build: &Build, stage: u32, host: &str) {
     let rustc = out_dir.join(exe("rustc", host));
     let bindir = sysroot.join("bin");
     t!(fs::create_dir_all(&bindir));
-    let compiler = build.compiler_path(&Compiler::new(stage, host));
+    let compiler = build.compiler_path(&target_compiler);
     let _ = fs::remove_file(&compiler);
     copy(&rustc, &compiler);
 
@@ -407,6 +407,8 @@ fn add_to_sysroot(sysroot_dst: &Path, stamp: &Path) {
     t!(fs::create_dir_all(&sysroot_dst));
     let mut contents = Vec::new();
     t!(t!(File::open(stamp)).read_to_end(&mut contents));
+    // This is the method we use for extracting paths from the stamp file passed to us. See
+    // run_cargo for more information (in this file).
     for part in contents.split(|b| *b == 0) {
         if part.is_empty() {
             continue
@@ -421,7 +423,7 @@ fn add_to_sysroot(sysroot_dst: &Path, stamp: &Path) {
 /// This will build the specified tool with the specified `host` compiler in
 /// `stage` into the normal cargo output directory.
 pub fn maybe_clean_tools(build: &Build, stage: u32, target: &str, mode: Mode) {
-    let compiler = Compiler::new(stage, &build.config.build);
+    let compiler = Compiler::new(stage, &build.build);
 
     let stamp = match mode {
         Mode::Libstd => libstd_stamp(build, &compiler, target),
@@ -441,7 +443,7 @@ pub fn tool(build: &Build, stage: u32, target: &str, tool: &str) {
     let _folder = build.fold_output(|| format!("stage{}-{}", stage, tool));
     println!("Building stage{} tool {} ({})", stage, tool, target);
 
-    let compiler = Compiler::new(stage, &build.config.build);
+    let compiler = Compiler::new(stage, &build.build);
 
     let mut cargo = build.cargo(&compiler, Mode::Tool, target, "build");
     let dir = build.src.join("src/tools").join(tool);
@@ -473,11 +475,43 @@ pub fn tool(build: &Build, stage: u32, target: &str, tool: &str) {
     build.run(&mut cargo);
 }
 
+
+// Avoiding a dependency on winapi to keep compile times down
+#[cfg(unix)]
+fn stderr_isatty() -> bool {
+    use libc;
+    unsafe { libc::isatty(libc::STDERR_FILENO) != 0 }
+}
+#[cfg(windows)]
+fn stderr_isatty() -> bool {
+    type DWORD = u32;
+    type BOOL = i32;
+    type HANDLE = *mut u8;
+    const STD_ERROR_HANDLE: DWORD = -12i32 as DWORD;
+    extern "system" {
+        fn GetStdHandle(which: DWORD) -> HANDLE;
+        fn GetConsoleMode(hConsoleHandle: HANDLE, lpMode: *mut DWORD) -> BOOL;
+    }
+    unsafe {
+        let handle = GetStdHandle(STD_ERROR_HANDLE);
+        let mut out = 0;
+        GetConsoleMode(handle, &mut out) != 0
+    }
+}
+
 fn run_cargo(build: &Build, cargo: &mut Command, stamp: &Path) {
     // Instruct Cargo to give us json messages on stdout, critically leaving
     // stderr as piped so we can get those pretty colors.
     cargo.arg("--message-format").arg("json")
          .stdout(Stdio::piped());
+
+    if stderr_isatty() {
+        // since we pass message-format=json to cargo, we need to tell the rustc
+        // wrapper to give us colored output if necessary. This is because we
+        // only want Cargo's JSON output, not rustcs.
+        cargo.env("RUSTC_COLOR", "1");
+    }
+
     build.verbose(&format!("running: {:?}", cargo));
     let mut child = match cargo.spawn() {
         Ok(child) => child,
@@ -525,23 +559,24 @@ fn run_cargo(build: &Build, cargo: &mut Command, stamp: &Path) {
             // If this was an output file in the "host dir" we don't actually
             // worry about it, it's not relevant for us.
             if filename.starts_with(&host_root_dir) {
-                continue
+                continue;
+            }
 
             // If this was output in the `deps` dir then this is a precise file
             // name (hash included) so we start tracking it.
-            } else if filename.starts_with(&target_deps_dir) {
+            if filename.starts_with(&target_deps_dir) {
                 deps.push(filename.to_path_buf());
+                continue;
+            }
 
             // Otherwise this was a "top level artifact" which right now doesn't
             // have a hash in the name, but there's a version of this file in
             // the `deps` folder which *does* have a hash in the name. That's
             // the one we'll want to we'll probe for it later.
-            } else {
+            toplevel.push((filename.file_stem().unwrap()
-                toplevel.push((filename.file_stem().unwrap()
+                                    .to_str().unwrap().to_string(),
-                                       .to_str().unwrap().to_string(),
+                            filename.extension().unwrap().to_owned()
-                               filename.extension().unwrap().to_owned()
+                                    .to_str().unwrap().to_string()));
-                                       .to_str().unwrap().to_string()));
-            }
         }
     }
 

src/bootstrap/config.rs

@@ -50,6 +50,7 @@ pub struct Config {
     pub full_bootstrap: bool,
     pub extended: bool,
     pub sanitizers: bool,
+    pub profiler: bool,
 
     // llvm codegen options
     pub llvm_assertions: bool,
@@ -59,6 +60,7 @@ pub struct Config {
     pub llvm_static_stdcpp: bool,
     pub llvm_link_shared: bool,
     pub llvm_targets: Option<String>,
+    pub llvm_experimental_targets: Option<String>,
     pub llvm_link_jobs: Option<u32>,
     pub llvm_clean_rebuild: bool,
 
@@ -79,8 +81,6 @@ pub struct Config {
     pub build: String,
     pub host: Vec<String>,
     pub target: Vec<String>,
-    pub rustc: Option<PathBuf>,
-    pub cargo: Option<PathBuf>,
     pub local_rebuild: bool,
 
     // dist misc
@@ -112,11 +112,18 @@ pub struct Config {
     pub python: Option<PathBuf>,
     pub configure_args: Vec<String>,
     pub openssl_static: bool,
+
+
+    // These are either the stage0 downloaded binaries or the locally installed ones.
+    pub initial_cargo: PathBuf,
+    pub initial_rustc: PathBuf,
+
 }
 
 /// Per-target configuration stored in the global configuration structure.
 #[derive(Default)]
 pub struct Target {
+    /// Some(path to llvm-config) if using an external LLVM.
     pub llvm_config: Option<PathBuf>,
     pub jemalloc: Option<PathBuf>,
     pub cc: Option<PathBuf>,
@@ -162,6 +169,7 @@ struct Build {
     extended: Option<bool>,
     verbose: Option<usize>,
     sanitizers: Option<bool>,
+    profiler: Option<bool>,
     openssl_static: Option<bool>,
 }
 
@@ -187,6 +195,7 @@ struct Llvm {
     version_check: Option<bool>,
     static_libstdcpp: Option<bool>,
     targets: Option<String>,
+    experimental_targets: Option<String>,
     link_jobs: Option<u32>,
     clean_rebuild: Option<bool>,
 }
@@ -303,8 +312,6 @@ impl Config {
                 config.target.push(target.clone());
             }
         }
-        config.rustc = build.rustc.map(PathBuf::from);
-        config.cargo = build.cargo.map(PathBuf::from);
         config.nodejs = build.nodejs.map(PathBuf::from);
         config.gdb = build.gdb.map(PathBuf::from);
         config.python = build.python.map(PathBuf::from);
@@ -318,6 +325,7 @@ impl Config {
         set(&mut config.extended, build.extended);
         set(&mut config.verbose, build.verbose);
         set(&mut config.sanitizers, build.sanitizers);
+        set(&mut config.profiler, build.profiler);
         set(&mut config.openssl_static, build.openssl_static);
 
         if let Some(ref install) = toml.install {
@@ -347,6 +355,7 @@ impl Config {
             set(&mut config.llvm_static_stdcpp, llvm.static_libstdcpp);
             set(&mut config.llvm_clean_rebuild, llvm.clean_rebuild);
             config.llvm_targets = llvm.targets.clone();
+            config.llvm_experimental_targets = llvm.experimental_targets.clone();
             config.llvm_link_jobs = llvm.link_jobs;
         }
 
@@ -404,13 +413,25 @@ impl Config {
             set(&mut config.rust_dist_src, t.src_tarball);
         }
 
+        let cwd = t!(env::current_dir());
+        let out = cwd.join("build");
+
+        let stage0_root = out.join(&config.build).join("stage0/bin");
+        config.initial_rustc = match build.rustc {
+            Some(s) => PathBuf::from(s),
+            None => stage0_root.join(exe("rustc", &config.build)),
+        };
+        config.initial_cargo = match build.cargo {
+            Some(s) => PathBuf::from(s),
+            None => stage0_root.join(exe("cargo", &config.build)),
+        };
 
         // compat with `./configure` while we're still using that
         if fs::metadata("config.mk").is_ok() {
             config.update_with_config_mk();
         }
 
-        return config
+        config
     }
 
     /// "Temporary" routine to parse `config.mk` into this configuration.
@@ -477,6 +498,7 @@ impl Config {
                 ("FULL_BOOTSTRAP", self.full_bootstrap),
                 ("EXTENDED", self.extended),
                 ("SANITIZERS", self.sanitizers),
+                ("PROFILER", self.profiler),
                 ("DIST_SRC", self.rust_dist_src),
                 ("CARGO_OPENSSL_STATIC", self.openssl_static),
             }
@@ -490,6 +512,9 @@ impl Config {
                 "CFG_TARGET" if value.len() > 0 => {
                     self.target.extend(value.split(" ").map(|s| s.to_string()));
                 }
+                "CFG_EXPERIMENTAL_TARGETS" if value.len() > 0 => {
+                    self.llvm_experimental_targets = Some(value.to_string());
+                }
                 "CFG_MUSL_ROOT" if value.len() > 0 => {
                     self.musl_root = Some(parse_configure_path(value));
                 }
@@ -599,8 +624,8 @@ impl Config {
                 }
                 "CFG_LOCAL_RUST_ROOT" if value.len() > 0 => {
                     let path = parse_configure_path(value);
-                    self.rustc = Some(push_exe_path(path.clone(), &["bin", "rustc"]));
+                    self.initial_rustc = push_exe_path(path.clone(), &["bin", "rustc"]);
-                    self.cargo = Some(push_exe_path(path, &["bin", "cargo"]));
+                    self.initial_cargo = push_exe_path(path, &["bin", "cargo"]);
                 }
                 "CFG_PYTHON" if value.len() > 0 => {
                     let path = parse_configure_path(value);

src/bootstrap/config.toml.example

@@ -53,6 +53,14 @@
 # Rust team and file an issue if you need assistance in porting!
 #targets = "X86;ARM;AArch64;Mips;PowerPC;SystemZ;JSBackend;MSP430;Sparc;NVPTX;Hexagon"
 
+# LLVM experimental targets to build support for. These targets are specified in
+# the same format as above, but since these targets are experimental, they are
+# not built by default and the experimental Rust compilation targets that depend
+# on them will not work unless the user opts in to building them. Possible
+# experimental LLVM targets include WebAssembly for the
+# wasm32-experimental-emscripten Rust target.
+#experimental-targets = ""
+
 # Cap the number of parallel linker invocations when compiling LLVM.
 # This can be useful when building LLVM with debug info, which significantly
 # increases the size of binaries and consequently the memory required by
@@ -147,6 +155,9 @@
 # Build the sanitizer runtimes
 #sanitizers = false
 
+# Build the profiler runtime
+#profiler = false
+
 # Indicates whether the OpenSSL linked into Cargo will be statically linked or
 # not. If static linkage is specified then the build system will download a
 # known-good version of OpenSSL, compile it, and link it to Cargo.

src/bootstrap/dist.rs

@@ -50,7 +50,7 @@ pub fn tmpdir(build: &Build) -> PathBuf {
 }
 
 fn rust_installer(build: &Build) -> Command {
-    build.tool_cmd(&Compiler::new(0, &build.config.build), "rust-installer")
+    build.tool_cmd(&Compiler::new(0, &build.build), "rust-installer")
 }
 
 /// Builds the `rust-docs` installer component.
@@ -89,7 +89,7 @@ pub fn docs(build: &Build, stage: u32, host: &str) {
 
     // As part of this step, *also* copy the docs directory to a directory which
     // buildbot typically uploads.
-    if host == build.config.build {
+    if host == build.build {
         let dst = distdir(build).join("doc").join(build.rust_package_vers());
         t!(fs::create_dir_all(&dst));
         cp_r(&src, &dst);
@@ -97,7 +97,7 @@ pub fn docs(build: &Build, stage: u32, host: &str) {
 }
 
 fn find_files(files: &[&str], path: &[PathBuf]) -> Vec<PathBuf> {
-    let mut found = Vec::new();
+    let mut found = Vec::with_capacity(files.len());
 
     for file in files {
         let file_path =
@@ -119,17 +119,9 @@ fn make_win_dist(rust_root: &Path, plat_root: &Path, target_triple: &str, build:
     //Ask gcc where it keeps its stuff
     let mut cmd = Command::new(build.cc(target_triple));
     cmd.arg("-print-search-dirs");
-    build.run_quiet(&mut cmd);
+    let gcc_out = output(&mut cmd);
-    let gcc_out =
-        String::from_utf8(
-                cmd
-                .output()
-                .expect("failed to execute gcc")
-                .stdout).expect("gcc.exe output was not utf8");
 
-    let mut bin_path: Vec<_> =
+    let mut bin_path: Vec<_> = env::split_paths(&env::var_os("PATH").unwrap_or_default()).collect();
-        env::split_paths(&env::var_os("PATH").unwrap_or_default())
-        .collect();
     let mut lib_path = Vec::new();
 
     for line in gcc_out.lines() {
@@ -140,7 +132,7 @@ fn make_win_dist(rust_root: &Path, plat_root: &Path, target_triple: &str, build:
             line[(idx + 1)..]
                 .trim_left_matches(trim_chars)
                 .split(';')
-                .map(|s| PathBuf::from(s));
+                .map(PathBuf::from);
 
         if key == "programs" {
             bin_path.extend(value);
@@ -149,7 +141,7 @@ fn make_win_dist(rust_root: &Path, plat_root: &Path, target_triple: &str, build:
         }
     }
 
-    let target_tools = vec!["gcc.exe", "ld.exe", "ar.exe", "dlltool.exe", "libwinpthread-1.dll"];
+    let target_tools = ["gcc.exe", "ld.exe", "ar.exe", "dlltool.exe", "libwinpthread-1.dll"];
     let mut rustc_dlls = vec!["libstdc++-6.dll", "libwinpthread-1.dll"];
     if target_triple.starts_with("i686-") {
         rustc_dlls.push("libgcc_s_dw2-1.dll");
@@ -157,7 +149,7 @@ fn make_win_dist(rust_root: &Path, plat_root: &Path, target_triple: &str, build:
         rustc_dlls.push("libgcc_s_seh-1.dll");
     }
 
-    let target_libs = vec![ //MinGW libs
+    let target_libs = [ //MinGW libs
         "libgcc.a",
         "libgcc_eh.a",
         "libgcc_s.a",
@@ -203,7 +195,7 @@ fn make_win_dist(rust_root: &Path, plat_root: &Path, target_triple: &str, build:
     let target_libs = find_files(&target_libs, &lib_path);
 
     fn copy_to_folder(src: &Path, dest_folder: &Path) {
-        let file_name = src.file_name().unwrap().to_os_string();
+        let file_name = src.file_name().unwrap();
         let dest = dest_folder.join(file_name);
         copy(src, &dest);
     }
@@ -234,8 +226,6 @@ fn make_win_dist(rust_root: &Path, plat_root: &Path, target_triple: &str, build:
 ///
 /// This contains all the bits and pieces to run the MinGW Windows targets
 /// without any extra installed software (e.g. we bundle gcc, libraries, etc).
-/// Currently just shells out to a python script, but that should be rewritten
-/// in Rust.
 pub fn mingw(build: &Build, host: &str) {
     println!("Dist mingw ({})", host);
     let name = pkgname(build, "rust-mingw");
@@ -366,9 +356,9 @@ pub fn rustc(build: &Build, stage: u32, host: &str) {
 pub fn debugger_scripts(build: &Build,
                         sysroot: &Path,
                         host: &str) {
+    let dst = sysroot.join("lib/rustlib/etc");
+    t!(fs::create_dir_all(&dst));
     let cp_debugger_script = |file: &str| {
-        let dst = sysroot.join("lib/rustlib/etc");
-        t!(fs::create_dir_all(&dst));
         install(&build.src.join("src/etc/").join(file), &dst, 0o644);
     };
     if host.contains("windows-msvc") {
@@ -376,8 +366,8 @@ pub fn debugger_scripts(build: &Build,
         install(&build.src.join("src/etc/rust-windbg.cmd"), &sysroot.join("bin"),
             0o755);
 
+        cp_debugger_script("natvis/liballoc.natvis");
         cp_debugger_script("natvis/libcore.natvis");
-        cp_debugger_script("natvis/libcollections.natvis");
     } else {
         cp_debugger_script("debugger_pretty_printers_common.py");
 
@@ -404,7 +394,7 @@ pub fn std(build: &Build, compiler: &Compiler, target: &str) {
 
     // The only true set of target libraries came from the build triple, so
     // let's reduce redundant work by only producing archives from that host.
-    if compiler.host != build.config.build {
+    if compiler.host != build.build {
         println!("\tskipping, not a build host");
         return
     }
@@ -415,8 +405,9 @@ pub fn std(build: &Build, compiler: &Compiler, target: &str) {
 
     let dst = image.join("lib/rustlib").join(target);
     t!(fs::create_dir_all(&dst));
-    let src = build.sysroot(compiler).join("lib/rustlib");
+    let mut src = build.sysroot_libdir(compiler, target);
-    cp_r(&src.join(target), &dst);
+    src.pop(); // Remove the trailing /lib folder from the sysroot_libdir
+    cp_r(&src, &dst);
 
     let mut cmd = rust_installer(build);
     cmd.arg("generate")
@@ -450,7 +441,7 @@ pub fn analysis(build: &Build, compiler: &Compiler, target: &str) {
     assert!(build.config.extended);
     println!("Dist analysis");
 
-    if compiler.host != build.config.build {
+    if compiler.host != build.build {
         println!("\tskipping, not a build host");
         return;
     }
@@ -498,12 +489,11 @@ fn copy_src_dirs(build: &Build, src_dirs: &[&str], exclude_dirs: &[&str], dst_di
         if spath.ends_with("~") || spath.ends_with(".pyc") {
             return false
         }
-        if spath.contains("llvm/test") || spath.contains("llvm\\test") {
+        if (spath.contains("llvm/test") || spath.contains("llvm\\test")) &&
-            if spath.ends_with(".ll") ||
+            (spath.ends_with(".ll") ||
-               spath.ends_with(".td") ||
+             spath.ends_with(".td") ||
-               spath.ends_with(".s") {
+             spath.ends_with(".s")) {
-                return false
+            return false
-            }
         }
 
         let full_path = Path::new(dir).join(path);
@@ -564,12 +554,12 @@ pub fn rust_src(build: &Build) {
         "src/libstd",
         "src/libstd_unicode",
         "src/libunwind",
+        "src/rustc/compiler_builtins_shim",
         "src/rustc/libc_shim",
         "src/libtest",
         "src/libterm",
-        "src/libgetopts",
-        "src/compiler-rt",
         "src/jemalloc",
+        "src/libprofiler_builtins",
     ];
     let std_src_dirs_exclude = [
         "src/compiler-rt/test",
@@ -595,7 +585,7 @@ pub fn rust_src(build: &Build) {
     t!(fs::remove_dir_all(&image));
 }
 
-const CARGO_VENDOR_VERSION: &'static str = "0.1.4";
+const CARGO_VENDOR_VERSION: &str = "0.1.4";
 
 /// Creates the plain source tarball
 pub fn plain_source_tarball(build: &Build) {
@@ -634,26 +624,26 @@ pub fn plain_source_tarball(build: &Build) {
     write_file(&plain_dst_src.join("version"), build.rust_version().as_bytes());
 
     // If we're building from git sources, we need to vendor a complete distribution.
-    if build.src_is_git {
+    if build.rust_info.is_git() {
         // Get cargo-vendor installed, if it isn't already.
         let mut has_cargo_vendor = false;
-        let mut cmd = Command::new(&build.cargo);
+        let mut cmd = Command::new(&build.initial_cargo);
         for line in output(cmd.arg("install").arg("--list")).lines() {
             has_cargo_vendor |= line.starts_with("cargo-vendor ");
         }
         if !has_cargo_vendor {
-            let mut cmd = Command::new(&build.cargo);
+            let mut cmd = Command::new(&build.initial_cargo);
             cmd.arg("install")
                .arg("--force")
                .arg("--debug")
                .arg("--vers").arg(CARGO_VENDOR_VERSION)
                .arg("cargo-vendor")
-               .env("RUSTC", &build.rustc);
+               .env("RUSTC", &build.initial_rustc);
             build.run(&mut cmd);
         }
 
         // Vendor all Cargo dependencies
-        let mut cmd = Command::new(&build.cargo);
+        let mut cmd = Command::new(&build.initial_cargo);
         cmd.arg("vendor")
            .current_dir(&plain_dst_src.join("src"));
         build.run(&mut cmd);
@@ -716,7 +706,7 @@ fn write_file(path: &Path, data: &[u8]) {
 
 pub fn cargo(build: &Build, stage: u32, target: &str) {
     println!("Dist cargo stage{} ({})", stage, target);
-    let compiler = Compiler::new(stage, &build.config.build);
+    let compiler = Compiler::new(stage, &build.build);
 
     let src = build.src.join("src/tools/cargo");
     let etc = src.join("src/etc");
@@ -777,7 +767,7 @@ pub fn cargo(build: &Build, stage: u32, target: &str) {
 pub fn rls(build: &Build, stage: u32, target: &str) {
     assert!(build.config.extended);
     println!("Dist RLS stage{} ({})", stage, target);
-    let compiler = Compiler::new(stage, &build.config.build);
+    let compiler = Compiler::new(stage, &build.build);
 
     let src = build.src.join("src/tools/rls");
     let release_num = build.release_num("rls");
@@ -1160,7 +1150,7 @@ fn add_env(build: &Build, cmd: &mut Command, target: &str) {
 }
 
 pub fn hash_and_sign(build: &Build) {
-    let compiler = Compiler::new(0, &build.config.build);
+    let compiler = Compiler::new(0, &build.build);
     let mut cmd = build.tool_cmd(&compiler, "build-manifest");
     let sign = build.config.dist_sign_folder.as_ref().unwrap_or_else(|| {
         panic!("\n\nfailed to specify `dist.sign-folder` in `config.toml`\n\n")

src/bootstrap/doc.rs

@@ -27,18 +27,26 @@ use {Build, Compiler, Mode};
 use util::{cp_r, symlink_dir};
 use build_helper::up_to_date;
 
-/// Invoke `rustbook` as compiled in `stage` for `target` for the doc book
+/// Invoke `rustbook` for `target` for the doc book `name`.
-/// `name` into the `out` path.
 ///
 /// This will not actually generate any documentation if the documentation has
 /// already been generated.
 pub fn rustbook(build: &Build, target: &str, name: &str) {
+    let src = build.src.join("src/doc");
+    rustbook_src(build, target, name, &src);
+}
+
+/// Invoke `rustbook` for `target` for the doc book `name` from the `src` path.
+///
+/// This will not actually generate any documentation if the documentation has
+/// already been generated.
+pub fn rustbook_src(build: &Build, target: &str, name: &str, src: &Path) {
     let out = build.doc_out(target);
     t!(fs::create_dir_all(&out));
 
     let out = out.join(name);
-    let compiler = Compiler::new(0, &build.config.build);
+    let compiler = Compiler::new(0, &build.build);
-    let src = build.src.join("src/doc").join(name);
+    let src = src.join(name);
     let index = out.join("index.html");
     let rustbook = build.tool(&compiler, "rustbook");
     if up_to_date(&src, &index) && up_to_date(&rustbook, &index) {
@@ -87,7 +95,7 @@ pub fn book(build: &Build, target: &str, name: &str) {
 fn invoke_rustdoc(build: &Build, target: &str, markdown: &str) {
     let out = build.doc_out(target);
 
-    let compiler = Compiler::new(0, &build.config.build);
+    let compiler = Compiler::new(0, &build.build);
 
     let path = build.src.join("src/doc").join(markdown);
 
@@ -142,7 +150,7 @@ pub fn standalone(build: &Build, target: &str) {
     let out = build.doc_out(target);
     t!(fs::create_dir_all(&out));
 
-    let compiler = Compiler::new(0, &build.config.build);
+    let compiler = Compiler::new(0, &build.build);
 
     let favicon = build.src.join("src/doc/favicon.inc");
     let footer = build.src.join("src/doc/footer.inc");
@@ -209,7 +217,7 @@ pub fn std(build: &Build, stage: u32, target: &str) {
     println!("Documenting stage{} std ({})", stage, target);
     let out = build.doc_out(target);
     t!(fs::create_dir_all(&out));
-    let compiler = Compiler::new(stage, &build.config.build);
+    let compiler = Compiler::new(stage, &build.build);
     let compiler = if build.force_use_stage1(&compiler, target) {
         Compiler::new(1, compiler.host)
     } else {
@@ -268,7 +276,7 @@ pub fn test(build: &Build, stage: u32, target: &str) {
     println!("Documenting stage{} test ({})", stage, target);
     let out = build.doc_out(target);
     t!(fs::create_dir_all(&out));
-    let compiler = Compiler::new(stage, &build.config.build);
+    let compiler = Compiler::new(stage, &build.build);
     let compiler = if build.force_use_stage1(&compiler, target) {
         Compiler::new(1, compiler.host)
     } else {
@@ -298,7 +306,7 @@ pub fn rustc(build: &Build, stage: u32, target: &str) {
     println!("Documenting stage{} compiler ({})", stage, target);
     let out = build.doc_out(target);
     t!(fs::create_dir_all(&out));
-    let compiler = Compiler::new(stage, &build.config.build);
+    let compiler = Compiler::new(stage, &build.build);
     let compiler = if build.force_use_stage1(&compiler, target) {
         Compiler::new(1, compiler.host)
     } else {
@@ -343,17 +351,30 @@ pub fn error_index(build: &Build, target: &str) {
     println!("Documenting error index ({})", target);
     let out = build.doc_out(target);
     t!(fs::create_dir_all(&out));
-    let compiler = Compiler::new(0, &build.config.build);
+    let compiler = Compiler::new(0, &build.build);
     let mut index = build.tool_cmd(&compiler, "error_index_generator");
     index.arg("html");
     index.arg(out.join("error-index.html"));
 
     // FIXME: shouldn't have to pass this env var
-    index.env("CFG_BUILD", &build.config.build);
+    index.env("CFG_BUILD", &build.build);
 
     build.run(&mut index);
 }
 
+pub fn unstable_book_gen(build: &Build, target: &str) {
+    println!("Generating unstable book md files ({})", target);
+    let out = build.md_doc_out(target).join("unstable-book");
+    t!(fs::create_dir_all(&out));
+    t!(fs::remove_dir_all(&out));
+    let compiler = Compiler::new(0, &build.build);
+    let mut cmd = build.tool_cmd(&compiler, "unstable-book-gen");
+    cmd.arg(build.src.join("src"));
+    cmd.arg(out);
+
+    build.run(&mut cmd);
+}
+
 fn symlink_dir_force(src: &Path, dst: &Path) -> io::Result<()> {
     if let Ok(m) = fs::symlink_metadata(dst) {
         if m.file_type().is_dir() {

src/bootstrap/flags.rs

@@ -35,22 +35,12 @@ pub struct Flags {
     pub host: Vec<String>,
     pub target: Vec<String>,
     pub config: Option<PathBuf>,
-    pub src: Option<PathBuf>,
+    pub src: PathBuf,
     pub jobs: Option<u32>,
     pub cmd: Subcommand,
     pub incremental: bool,
 }
 
-impl Flags {
-    pub fn verbose(&self) -> bool {
-        self.verbose > 0
-    }
-
-    pub fn very_verbose(&self) -> bool {
-        self.verbose > 1
-    }
-}
-
 pub enum Subcommand {
     Build {
         paths: Vec<PathBuf>,
@@ -61,7 +51,7 @@ pub enum Subcommand {
     Test {
         paths: Vec<PathBuf>,
         test_args: Vec<String>,
-        no_fail_fast: bool,
+        fail_fast: bool,
     },
     Bench {
         paths: Vec<PathBuf>,
@@ -122,16 +112,15 @@ To learn more about a subcommand, run `./x.py <subcommand> -h`");
         // the subcommand. Therefore we must manually identify the subcommand first, so that we can
         // complete the definition of the options.  Then we can use the getopt::Matches object from
         // there on out.
-        let mut possible_subcommands = args.iter().collect::<Vec<_>>();
+        let subcommand = args.iter().find(|&s|
-        possible_subcommands.retain(|&s|
+            (s == "build")
-                                           (s == "build")
+            || (s == "test")
-                                        || (s == "test")
+            || (s == "bench")
-                                        || (s == "bench")
+            || (s == "doc")
-                                        || (s == "doc")
+            || (s == "clean")
-                                        || (s == "clean")
+            || (s == "dist")
-                                        || (s == "dist")
+            || (s == "install"));
-                                        || (s == "install"));
+        let subcommand = match subcommand {
-        let subcommand = match possible_subcommands.first() {
             Some(s) => s,
             None => {
                 // No subcommand -- show the general usage and subcommand help
@@ -164,7 +153,7 @@ To learn more about a subcommand, run `./x.py <subcommand> -h`");
         let mut pass_sanity_check = true;
         match matches.free.get(0) {
             Some(check_subcommand) => {
-                if &check_subcommand != subcommand {
+                if check_subcommand != subcommand {
                     pass_sanity_check = false;
                 }
             },
@@ -196,9 +185,14 @@ Arguments:
         ./x.py build
         ./x.py build --stage 1
 
-    For a quick build with a usable compile, you can pass:
+    For a quick build of a usable compiler, you can pass:
 
-        ./x.py build --stage 1 src/libtest");
+        ./x.py build --stage 1 src/libtest
+
+    This will first build everything once (like --stage 0 without further
+    arguments would), and then use the compiler built in stage 0 to build
+    src/libtest and its dependencies.
+    Once this is done, build/$ARCH/stage1 contains a usable compiler.");
             }
             "test" => {
                 subcommand_help.push_str("\n
@@ -274,7 +268,7 @@ Arguments:
                 Subcommand::Test {
                     paths: paths,
                     test_args: matches.opt_strs("test-args"),
-                    no_fail_fast: matches.opt_present("no-fail-fast"),
+                    fail_fast: !matches.opt_present("no-fail-fast"),
                 }
             }
             "bench" => {
@@ -311,12 +305,15 @@ Arguments:
 
         let mut stage = matches.opt_str("stage").map(|j| j.parse().unwrap());
 
-        if matches.opt_present("incremental") {
+        if matches.opt_present("incremental") && stage.is_none() {
-            if stage.is_none() {
+            stage = Some(1);
-                stage = Some(1);
-            }
         }
 
+        let cwd = t!(env::current_dir());
+        let src = matches.opt_str("src").map(PathBuf::from)
+            .or_else(|| env::var_os("SRC").map(PathBuf::from))
+            .unwrap_or(cwd);
+
         Flags {
             verbose: matches.opt_count("verbose"),
             stage: stage,
@@ -328,7 +325,7 @@ Arguments:
             host: split(matches.opt_strs("host")),
             target: split(matches.opt_strs("target")),
             config: cfg_file,
-            src: matches.opt_str("src").map(PathBuf::from),
+            src: src,
             jobs: matches.opt_str("jobs").map(|j| j.parse().unwrap()),
             cmd: cmd,
             incremental: matches.opt_present("incremental"),
@@ -347,9 +344,9 @@ impl Subcommand {
         }
     }
 
-    pub fn no_fail_fast(&self) -> bool {
+    pub fn fail_fast(&self) -> bool {
         match *self {
-            Subcommand::Test { no_fail_fast, .. } => no_fail_fast,
+            Subcommand::Test { fail_fast, .. } => fail_fast,
             _ => false,
         }
     }

src/bootstrap/install.rs

@@ -146,5 +146,5 @@ fn add_destdir(path: &Path, destdir: &Option<PathBuf>) -> PathBuf {
             _ => {}
         }
     }
-    return ret
+    ret
 }

src/bootstrap/lib.rs

@@ -161,25 +161,35 @@ pub struct Build {
     flags: Flags,
 
     // Derived properties from the above two configurations
-    cargo: PathBuf,
-    rustc: PathBuf,
     src: PathBuf,
     out: PathBuf,
     rust_info: channel::GitInfo,
     cargo_info: channel::GitInfo,
     rls_info: channel::GitInfo,
     local_rebuild: bool,
+    fail_fast: bool,
+    verbosity: usize,
+
+    // Targets for which to build.
+    build: String,
+    hosts: Vec<String>,
+    targets: Vec<String>,
+
+    // Stage 0 (downloaded) compiler and cargo or their local rust equivalents.
+    initial_rustc: PathBuf,
+    initial_cargo: PathBuf,
 
     // Probed tools at runtime
     lldb_version: Option<String>,
     lldb_python_dir: Option<String>,
 
     // Runtime state filled in later on
+    // target -> (cc, ar)
     cc: HashMap<String, (gcc::Tool, Option<PathBuf>)>,
+    // host -> (cc, ar)
     cxx: HashMap<String, gcc::Tool>,
     crates: HashMap<String, Crate>,
     is_sudo: bool,
-    src_is_git: bool,
     ci_env: CiEnv,
     delayed_failures: Cell<usize>,
 }
@@ -202,20 +212,16 @@ struct Crate {
 /// build system, with each mod generating output in a different directory.
 #[derive(Clone, Copy, PartialEq, Eq)]
 pub enum Mode {
-    /// This cargo is going to build the standard library, placing output in the
+    /// Build the standard library, placing output in the "stageN-std" directory.
-    /// "stageN-std" directory.
     Libstd,
 
-    /// This cargo is going to build libtest, placing output in the
+    /// Build libtest, placing output in the "stageN-test" directory.
-    /// "stageN-test" directory.
     Libtest,
 
-    /// This cargo is going to build librustc and compiler libraries, placing
+    /// Build librustc and compiler libraries, placing output in the "stageN-rustc" directory.
-    /// output in the "stageN-rustc" directory.
     Librustc,
 
-    /// This cargo is going to build some tool, placing output in the
+    /// Build some tool, placing output in the "stageN-tools" directory.
-    /// "stageN-tools" directory.
     Tool,
 }
 
@@ -226,22 +232,9 @@ impl Build {
     /// By default all build output will be placed in the current directory.
     pub fn new(flags: Flags, config: Config) -> Build {
         let cwd = t!(env::current_dir());
-        let src = flags.src.clone().or_else(|| {
+        let src = flags.src.clone();
-            env::var_os("SRC").map(|x| x.into())
-        }).unwrap_or(cwd.clone());
         let out = cwd.join("build");
 
-        let stage0_root = out.join(&config.build).join("stage0/bin");
-        let rustc = match config.rustc {
-            Some(ref s) => PathBuf::from(s),
-            None => stage0_root.join(exe("rustc", &config.build)),
-        };
-        let cargo = match config.cargo {
-            Some(ref s) => PathBuf::from(s),
-            None => stage0_root.join(exe("cargo", &config.build)),
-        };
-        let local_rebuild = config.local_rebuild;
-
         let is_sudo = match env::var_os("SUDO_USER") {
             Some(sudo_user) => {
                 match env::var_os("USER") {
@@ -254,32 +247,64 @@ impl Build {
         let rust_info = channel::GitInfo::new(&src);
         let cargo_info = channel::GitInfo::new(&src.join("src/tools/cargo"));
         let rls_info = channel::GitInfo::new(&src.join("src/tools/rls"));
-        let src_is_git = src.join(".git").exists();
+
+        let hosts = if !flags.host.is_empty() {
+            for host in flags.host.iter() {
+                if !config.host.contains(host) {
+                    panic!("specified host `{}` is not in configuration", host);
+                }
+            }
+            flags.host.clone()
+        } else {
+            config.host.clone()
+        };
+        let targets = if !flags.target.is_empty() {
+            for target in flags.target.iter() {
+                if !config.target.contains(target) {
+                    panic!("specified target `{}` is not in configuration", target);
+                }
+            }
+            flags.target.clone()
+        } else {
+            config.target.clone()
+        };
 
         Build {
+            initial_rustc: config.initial_rustc.clone(),
+            initial_cargo: config.initial_cargo.clone(),
+            local_rebuild: config.local_rebuild,
+            fail_fast: flags.cmd.fail_fast(),
+            verbosity: cmp::max(flags.verbose, config.verbose),
+
+            build: config.host[0].clone(),
+            hosts: hosts,
+            targets: targets,
+
             flags: flags,
             config: config,
-            cargo: cargo,
-            rustc: rustc,
             src: src,
             out: out,
 
             rust_info: rust_info,
             cargo_info: cargo_info,
             rls_info: rls_info,
-            local_rebuild: local_rebuild,
             cc: HashMap::new(),
             cxx: HashMap::new(),
             crates: HashMap::new(),
             lldb_version: None,
             lldb_python_dir: None,
             is_sudo: is_sudo,
-            src_is_git: src_is_git,
             ci_env: CiEnv::current(),
             delayed_failures: Cell::new(0),
         }
     }
 
+    fn build_slice(&self) -> &[String] {
+        unsafe {
+            std::slice::from_raw_parts(&self.build, 1)
+        }
+    }
+
     /// Executes the entire build, as configured by the flags and configuration.
     pub fn build(&mut self) {
         unsafe {
@@ -296,7 +321,7 @@ impl Build {
         sanity::check(self);
         // If local-rust is the same major.minor as the current version, then force a local-rebuild
         let local_version_verbose = output(
-            Command::new(&self.rustc).arg("--version").arg("--verbose"));
+            Command::new(&self.initial_rustc).arg("--version").arg("--verbose"));
         let local_release = local_version_verbose
             .lines().filter(|x| x.starts_with("release:"))
             .next().unwrap().trim_left_matches("release:").trim();
@@ -338,7 +363,7 @@ impl Build {
              mode: Mode,
              target: &str,
              cmd: &str) -> Command {
-        let mut cargo = Command::new(&self.cargo);
+        let mut cargo = Command::new(&self.initial_cargo);
         let out_dir = self.stage_out(compiler, mode);
         cargo.env("CARGO_TARGET_DIR", out_dir)
              .arg(cmd)
@@ -422,36 +447,13 @@ impl Build {
         // library up and running, so we can use the normal compiler to compile
         // build scripts in that situation.
         if mode == Mode::Libstd {
-            cargo.env("RUSTC_SNAPSHOT", &self.rustc)
+            cargo.env("RUSTC_SNAPSHOT", &self.initial_rustc)
                  .env("RUSTC_SNAPSHOT_LIBDIR", self.rustc_snapshot_libdir());
         } else {
             cargo.env("RUSTC_SNAPSHOT", self.compiler_path(compiler))
                  .env("RUSTC_SNAPSHOT_LIBDIR", self.rustc_libdir(compiler));
         }
 
-        // There are two invariants we must maintain:
-        // * stable crates cannot depend on unstable crates (general Rust rule),
-        // * crates that end up in the sysroot must be unstable (rustbuild rule).
-        //
-        // In order to do enforce the latter, we pass the env var
-        // `RUSTBUILD_UNSTABLE` down the line for any crates which will end up
-        // in the sysroot. We read this in bootstrap/bin/rustc.rs and if it is
-        // set, then we pass the `rustbuild` feature to rustc when building the
-        // the crate.
-        //
-        // In turn, crates that can be used here should recognise the `rustbuild`
-        // feature and opt-in to `rustc_private`.
-        //
-        // We can't always pass `rustbuild` because crates which are outside of
-        // the compiler, libs, and tests are stable and we don't want to make
-        // their deps unstable (since this would break the first invariant
-        // above).
-        //
-        // FIXME: remove this after next stage0
-        if mode != Mode::Tool && stage == 0 {
-            cargo.env("RUSTBUILD_UNSTABLE", "1");
-        }
-
         // Ignore incremental modes except for stage0, since we're
         // not guaranteeing correctness across builds if the compiler
         // is changing under your feet.`
@@ -464,8 +466,7 @@ impl Build {
             cargo.env("RUSTC_ON_FAIL", on_fail);
         }
 
-        let verbose = cmp::max(self.config.verbose, self.flags.verbose);
+        cargo.env("RUSTC_VERBOSE", format!("{}", self.verbosity));
-        cargo.env("RUSTC_VERBOSE", format!("{}", verbose));
 
         // Specify some various options for build scripts used throughout
         // the build.
@@ -475,9 +476,15 @@ impl Build {
             cargo.env(format!("CC_{}", target), self.cc(target))
                  .env(format!("AR_{}", target), self.ar(target).unwrap()) // only msvc is None
                  .env(format!("CFLAGS_{}", target), self.cflags(target).join(" "));
+
+            if let Ok(cxx) = self.cxx(target) {
+                 cargo.env(format!("CXX_{}", target), cxx);
+            }
         }
 
-        if self.config.extended && compiler.is_final_stage(self) {
+        if mode == Mode::Libstd &&
+           self.config.extended &&
+           compiler.is_final_stage(self) {
             cargo.env("RUSTC_SAVE_ANALYSIS", "api".to_string());
         }
 
@@ -499,7 +506,7 @@ impl Build {
         // FIXME: should update code to not require this env var
         cargo.env("CFG_COMPILER_HOST_TRIPLE", target);
 
-        if self.config.verbose() || self.flags.verbose() {
+        if self.is_verbose() {
             cargo.arg("-v");
         }
         // FIXME: cargo bench does not accept `--release`
@@ -515,13 +522,13 @@ impl Build {
 
         self.ci_env.force_coloring_in_ci(&mut cargo);
 
-        return cargo
+        cargo
     }
 
     /// Get a path to the compiler specified.
     fn compiler_path(&self, compiler: &Compiler) -> PathBuf {
         if compiler.is_snapshot(self) {
-            self.rustc.clone()
+            self.initial_rustc.clone()
         } else {
             self.sysroot(compiler).join("bin").join(exe("rustc", compiler.host))
         }
@@ -538,7 +545,7 @@ impl Build {
         let mut rustdoc = self.compiler_path(compiler);
         rustdoc.pop();
         rustdoc.push(exe("rustdoc", compiler.host));
-        return rustdoc
+        rustdoc
     }
 
     /// Get a `Command` which is ready to run `tool` in `stage` built for
@@ -546,7 +553,7 @@ impl Build {
     fn tool_cmd(&self, compiler: &Compiler, tool: &str) -> Command {
         let mut cmd = Command::new(self.tool(&compiler, tool));
         self.prepare_tool_cmd(compiler, &mut cmd);
-        return cmd
+        cmd
     }
 
     /// Prepares the `cmd` provided to be able to run the `compiler` provided.
@@ -594,7 +601,10 @@ impl Build {
         if self.config.backtrace {
             features.push_str(" backtrace");
         }
-        return features
+        if self.config.profiler {
+            features.push_str(" profiler");
+        }
+        features
     }
 
     /// Get the space-separated set of activated features for the compiler.
@@ -603,7 +613,7 @@ impl Build {
         if self.config.use_jemalloc {
             features.push_str(" jemalloc");
         }
-        return features
+        features
     }
 
     /// Component directory that Cargo will produce output into (e.g.
@@ -635,8 +645,14 @@ impl Build {
     /// Returns the libdir where the standard library and other artifacts are
     /// found for a compiler's sysroot.
     fn sysroot_libdir(&self, compiler: &Compiler, target: &str) -> PathBuf {
-        self.sysroot(compiler).join("lib").join("rustlib")
+        if compiler.stage >= 2 {
-            .join(target).join("lib")
+            if let Some(ref libdir_relative) = self.config.libdir_relative {
+                return self.sysroot(compiler).join(libdir_relative)
+                    .join("rustlib").join(target).join("lib")
+            }
+        }
+       self.sysroot(compiler).join("lib").join("rustlib")
+           .join(target).join("lib")
     }
 
     /// Returns the root directory for all output generated in a particular
@@ -677,6 +693,11 @@ impl Build {
         self.out.join(target).join("doc")
     }
 
+    /// Output directory for some generated md crate documentation for a target (temporary)
+    fn md_doc_out(&self, target: &str) -> PathBuf {
+        self.out.join(target).join("md-doc")
+    }
+
     /// Output directory for all crate documentation for a target (temporary)
     ///
     /// The artifacts here are then copied into `doc_out` above.
@@ -771,7 +792,7 @@ impl Build {
 
     /// Returns the libdir of the snapshot compiler.
     fn rustc_snapshot_libdir(&self) -> PathBuf {
-        self.rustc.parent().unwrap().parent().unwrap()
+        self.initial_rustc.parent().unwrap().parent().unwrap()
             .join(libdir(&self.config.build))
     }
 
@@ -803,9 +824,17 @@ impl Build {
         try_run_suppressed(cmd)
     }
 
+    pub fn is_verbose(&self) -> bool {
+        self.verbosity > 0
+    }
+
+    pub fn is_very_verbose(&self) -> bool {
+        self.verbosity > 1
+    }
+
     /// Prints a message if this build is configured in verbose mode.
     fn verbose(&self, msg: &str) {
-        if self.flags.verbose() || self.config.verbose() {
+        if self.is_verbose() {
             println!("{}", msg);
         }
     }
@@ -813,7 +842,7 @@ impl Build {
     /// Returns the number of parallel jobs that have been configured for this
     /// build.
     fn jobs(&self) -> u32 {
-        self.flags.jobs.unwrap_or(num_cpus::get() as u32)
+        self.flags.jobs.unwrap_or_else(|| num_cpus::get() as u32)
     }
 
     /// Returns the path to the C compiler for the target specified.
@@ -845,7 +874,7 @@ impl Build {
         if target == "i686-pc-windows-gnu" {
             base.push("-fno-omit-frame-pointer".into());
         }
-        return base
+        base
     }
 
     /// Returns the path to the `ar` archive utility for the target specified.
@@ -853,13 +882,13 @@ impl Build {
         self.cc[target].1.as_ref().map(|p| &**p)
     }
 
-    /// Returns the path to the C++ compiler for the target specified, may panic
+    /// Returns the path to the C++ compiler for the target specified.
-    /// if no C++ compiler was configured for the target.
+    fn cxx(&self, target: &str) -> Result<&Path, String> {
-    fn cxx(&self, target: &str) -> &Path {
         match self.cxx.get(target) {
-            Some(p) => p.path(),
+            Some(p) => Ok(p.path()),
-            None => panic!("\n\ntarget `{}` is not configured as a host,
+            None => Err(format!(
-                            only as a target\n\n", target),
+                    "target `{}` is not configured as a host, only as a target",
+                    target))
         }
     }
 
@@ -877,7 +906,7 @@ impl Build {
             !target.contains("emscripten") {
             base.push(format!("-Clinker={}", self.cc(target).display()));
         }
-        return base
+        base
     }
 
     /// Returns the "musl root" for this `target`, if defined
@@ -1058,7 +1087,7 @@ impl<'a> Compiler<'a> {
 
     /// Returns whether this is a snapshot compiler for `build`'s configuration
     fn is_snapshot(&self, build: &Build) -> bool {
-        self.stage == 0 && self.host == build.config.build
+        self.stage == 0 && self.host == build.build
     }
 
     /// Returns if this compiler should be treated as a final stage one in the

src/bootstrap/metadata.rs

@@ -56,7 +56,7 @@ fn build_krate(build: &mut Build, krate: &str) {
     // of packages we're going to have to know what `-p` arguments to pass it
     // to know what crates to test. Here we run `cargo metadata` to learn about
     // the dependency graph and what `-p` arguments there are.
-    let mut cargo = Command::new(&build.cargo);
+    let mut cargo = Command::new(&build.initial_cargo);
     cargo.arg("metadata")
          .arg("--format-version").arg("1")
          .arg("--manifest-path").arg(build.src.join(krate).join("Cargo.toml"));

src/bootstrap/mk/Makefile.in

@@ -55,7 +55,7 @@ check:
 check-aux:
 	$(Q)$(BOOTSTRAP) test \
 		src/tools/cargotest \
-		cargo \
+		src/tools/cargo \
 		src/test/pretty \
 		src/test/run-pass/pretty \
 		src/test/run-fail/pretty \

src/bootstrap/native.rs

@@ -86,14 +86,20 @@ pub fn llvm(build: &Build, target: &str) {
         None => "X86;ARM;AArch64;Mips;PowerPC;SystemZ;JSBackend;MSP430;Sparc;NVPTX;Hexagon",
     };
 
+    let llvm_exp_targets = match build.config.llvm_experimental_targets {
+        Some(ref s) => s,
+        None => "",
+    };
+
     let assertions = if build.config.llvm_assertions {"ON"} else {"OFF"};
 
     cfg.target(target)
-       .host(&build.config.build)
+       .host(&build.build)
        .out_dir(&out_dir)
        .profile(profile)
        .define("LLVM_ENABLE_ASSERTIONS", assertions)
        .define("LLVM_TARGETS_TO_BUILD", llvm_targets)
+       .define("LLVM_EXPERIMENTAL_TARGETS_TO_BUILD", llvm_exp_targets)
        .define("LLVM_INCLUDE_EXAMPLES", "OFF")
        .define("LLVM_INCLUDE_TESTS", "OFF")
        .define("LLVM_INCLUDE_DOCS", "OFF")
@@ -123,11 +129,11 @@ pub fn llvm(build: &Build, target: &str) {
     }
 
     // http://llvm.org/docs/HowToCrossCompileLLVM.html
-    if target != build.config.build {
+    if target != build.build {
         // FIXME: if the llvm root for the build triple is overridden then we
         //        should use llvm-tblgen from there, also should verify that it
         //        actually exists most of the time in normal installs of LLVM.
-        let host = build.llvm_out(&build.config.build).join("bin/llvm-tblgen");
+        let host = build.llvm_out(&build.build).join("bin/llvm-tblgen");
         cfg.define("CMAKE_CROSSCOMPILING", "True")
            .define("LLVM_TABLEGEN", &host);
     }
@@ -149,7 +155,7 @@ pub fn llvm(build: &Build, target: &str) {
         }
 
         let cc = build.cc(target);
-        let cxx = build.cxx(target);
+        let cxx = build.cxx(target).unwrap();
 
         // Handle msvc + ninja + ccache specially (this is what the bots use)
         if target.contains("msvc") &&
@@ -237,7 +243,7 @@ pub fn test_helpers(build: &Build, target: &str) {
     cfg.cargo_metadata(false)
        .out_dir(&dst)
        .target(target)
-       .host(&build.config.build)
+       .host(&build.build)
        .opt_level(0)
        .debug(false)
        .file(build.src.join("src/rt/rust_test_helpers.c"))

src/bootstrap/sanity.rs

@@ -18,55 +18,70 @@
 //! In theory if we get past this phase it's a bug if a build fails, but in
 //! practice that's likely not true!
 
-use std::collections::HashSet;
+use std::collections::HashMap;
 use std::env;
-use std::ffi::{OsStr, OsString};
+use std::ffi::{OsString, OsStr};
 use std::fs;
 use std::process::Command;
+use std::path::PathBuf;
 
 use build_helper::output;
 
 use Build;
 
+struct Finder {
+    cache: HashMap<OsString, Option<PathBuf>>,
+    path: OsString,
+}
+
+impl Finder {
+    fn new() -> Self {
+        Self {
+            cache: HashMap::new(),
+            path: env::var_os("PATH").unwrap_or_default()
+        }
+    }
+
+    fn maybe_have<S: AsRef<OsStr>>(&mut self, cmd: S) -> Option<PathBuf> {
+        let cmd: OsString = cmd.as_ref().into();
+        let path = self.path.clone();
+        self.cache.entry(cmd.clone()).or_insert_with(|| {
+            for path in env::split_paths(&path) {
+                let target = path.join(&cmd);
+                let mut cmd_alt = cmd.clone();
+                cmd_alt.push(".exe");
+                if target.is_file() || // some/path/git
+                target.with_extension("exe").exists() || // some/path/git.exe
+                target.join(&cmd_alt).exists() { // some/path/git/git.exe
+                    return Some(target);
+                }
+            }
+            None
+        }).clone()
+    }
+
+    fn must_have<S: AsRef<OsStr>>(&mut self, cmd: S) -> PathBuf {
+        self.maybe_have(&cmd).unwrap_or_else(|| {
+            panic!("\n\ncouldn't find required command: {:?}\n\n", cmd.as_ref());
+        })
+    }
+}
+
 pub fn check(build: &mut Build) {
-    let mut checked = HashSet::new();
+    let path = env::var_os("PATH").unwrap_or_default();
-    let path = env::var_os("PATH").unwrap_or(OsString::new());
     // On Windows, quotes are invalid characters for filename paths, and if
     // one is present as part of the PATH then that can lead to the system
     // being unable to identify the files properly. See
     // https://github.com/rust-lang/rust/issues/34959 for more details.
-    if cfg!(windows) {
+    if cfg!(windows) && path.to_string_lossy().contains("\"") {
-        if path.to_string_lossy().contains("\"") {
+        panic!("PATH contains invalid character '\"'");
-            panic!("PATH contains invalid character '\"'");
-        }
     }
-    let have_cmd = |cmd: &OsStr| {
-        for path in env::split_paths(&path) {
-            let target = path.join(cmd);
-            let mut cmd_alt = cmd.to_os_string();
-            cmd_alt.push(".exe");
-            if target.is_file() ||
-               target.with_extension("exe").exists() ||
-               target.join(cmd_alt).exists() {
-                return Some(target);
-            }
-        }
-        return None;
-    };
-
-    let mut need_cmd = |cmd: &OsStr| {
-        if !checked.insert(cmd.to_owned()) {
-            return
-        }
-        if have_cmd(cmd).is_none() {
-            panic!("\n\ncouldn't find required command: {:?}\n\n", cmd);
-        }
-    };
 
+    let mut cmd_finder = Finder::new();
     // If we've got a git directory we're gona need git to update
     // submodules and learn about various other aspects.
-    if build.src_is_git {
+    if build.rust_info.is_git() {
-        need_cmd("git".as_ref());
+        cmd_finder.must_have("git");
     }
 
     // We need cmake, but only if we're actually building LLVM or sanitizers.
@@ -74,51 +89,32 @@ pub fn check(build: &mut Build) {
         .filter_map(|host| build.config.target_config.get(host))
         .any(|config| config.llvm_config.is_none());
     if building_llvm || build.config.sanitizers {
-        need_cmd("cmake".as_ref());
+        cmd_finder.must_have("cmake");
     }
 
     // Ninja is currently only used for LLVM itself.
-    if building_llvm && build.config.ninja {
+    // Some Linux distros rename `ninja` to `ninja-build`.
-        // Some Linux distros rename `ninja` to `ninja-build`.
+    // CMake can work with either binary name.
-        // CMake can work with either binary name.
+    if building_llvm && build.config.ninja && cmd_finder.maybe_have("ninja-build").is_none() {
-        if have_cmd("ninja-build".as_ref()).is_none() {
+        cmd_finder.must_have("ninja");
-            need_cmd("ninja".as_ref());
-        }
     }
 
-    if build.config.python.is_none() {
+    build.config.python = build.config.python.take().map(|p| cmd_finder.must_have(p))
-        build.config.python = have_cmd("python2.7".as_ref());
+        .or_else(|| env::var_os("BOOTSTRAP_PYTHON").map(PathBuf::from)) // set by bootstrap.py
-    }
+        .or_else(|| cmd_finder.maybe_have("python2.7"))
-    if build.config.python.is_none() {
+        .or_else(|| cmd_finder.maybe_have("python2"))
-        build.config.python = have_cmd("python2".as_ref());
+        .or_else(|| Some(cmd_finder.must_have("python")));
-    }
-    if build.config.python.is_none() {
-        need_cmd("python".as_ref());
-        build.config.python = Some("python".into());
-    }
-    need_cmd(build.config.python.as_ref().unwrap().as_ref());
 
+    build.config.nodejs = build.config.nodejs.take().map(|p| cmd_finder.must_have(p))
+        .or_else(|| cmd_finder.maybe_have("node"))
+        .or_else(|| cmd_finder.maybe_have("nodejs"));
 
-    if let Some(ref s) = build.config.nodejs {
+    build.config.gdb = build.config.gdb.take().map(|p| cmd_finder.must_have(p))
-        need_cmd(s.as_ref());
+        .or_else(|| cmd_finder.maybe_have("gdb"));
-    } else {
-        // Look for the nodejs command, needed for emscripten testing
-        if let Some(node) = have_cmd("node".as_ref()) {
-            build.config.nodejs = Some(node);
-        } else if let Some(node) = have_cmd("nodejs".as_ref()) {
-            build.config.nodejs = Some(node);
-        }
-    }
-
-    if let Some(ref gdb) = build.config.gdb {
-        need_cmd(gdb.as_ref());
-    } else {
-        build.config.gdb = have_cmd("gdb".as_ref());
-    }
 
     // We're gonna build some custom C code here and there, host triples
     // also build some C++ shims for LLVM so we need a C++ compiler.
-    for target in build.config.target.iter() {
+    for target in &build.config.target {
         // On emscripten we don't actually need the C compiler to just
         // build the target artifacts, only for testing. For the sake
         // of easier bot configuration, just skip detection.
@@ -126,33 +122,32 @@ pub fn check(build: &mut Build) {
             continue;
         }
 
-        need_cmd(build.cc(target).as_ref());
+        cmd_finder.must_have(build.cc(target));
         if let Some(ar) = build.ar(target) {
-            need_cmd(ar.as_ref());
+            cmd_finder.must_have(ar);
         }
     }
-    for host in build.config.host.iter() {
-        need_cmd(build.cxx(host).as_ref());
-    }
 
-    // The msvc hosts don't use jemalloc, turn it off globally to
-    // avoid packaging the dummy liballoc_jemalloc on that platform.
     for host in build.config.host.iter() {
+        cmd_finder.must_have(build.cxx(host).unwrap());
+
+        // The msvc hosts don't use jemalloc, turn it off globally to
+        // avoid packaging the dummy liballoc_jemalloc on that platform.
         if host.contains("msvc") {
             build.config.use_jemalloc = false;
         }
     }
 
     // Externally configured LLVM requires FileCheck to exist
-    let filecheck = build.llvm_filecheck(&build.config.build);
+    let filecheck = build.llvm_filecheck(&build.build);
     if !filecheck.starts_with(&build.out) && !filecheck.exists() && build.config.codegen_tests {
         panic!("FileCheck executable {:?} does not exist", filecheck);
     }
 
-    for target in build.config.target.iter() {
+    for target in &build.config.target {
         // Can't compile for iOS unless we're on macOS
         if target.contains("apple-ios") &&
-           !build.config.build.contains("apple-darwin") {
+           !build.build.contains("apple-darwin") {
             panic!("the iOS target is only supported on macOS");
         }
 
@@ -199,18 +194,6 @@ $ pacman -R cmake && pacman -S mingw-w64-x86_64-cmake
         }
     }
 
-    for host in build.flags.host.iter() {
-        if !build.config.host.contains(host) {
-            panic!("specified host `{}` is not in the ./configure list", host);
-        }
-    }
-    for target in build.flags.target.iter() {
-        if !build.config.target.contains(target) {
-            panic!("specified target `{}` is not in the ./configure list",
-                   target);
-        }
-    }
-
     let run = |cmd: &mut Command| {
         cmd.output().map(|output| {
             String::from_utf8_lossy(&output.stdout)
@@ -224,6 +207,6 @@ $ pacman -R cmake && pacman -S mingw-w64-x86_64-cmake
     }
 
     if let Some(ref s) = build.config.ccache {
-        need_cmd(s.as_ref());
+        cmd_finder.must_have(s);
     }
 }

src/bootstrap/step.rs

@@ -28,6 +28,7 @@
 
 use std::collections::{BTreeMap, HashSet, HashMap};
 use std::mem;
+use std::path::PathBuf;
 use std::process;
 
 use check::{self, TestKind};
@@ -104,10 +105,10 @@ pub fn build_rules<'a>(build: &'a Build) -> Rules {
     rules.build("llvm", "src/llvm")
          .host(true)
          .dep(move |s| {
-             if s.target == build.config.build {
+             if s.target == build.build {
                  Step::noop()
              } else {
-                 s.target(&build.config.build)
+                 s.target(&build.build)
              }
          })
          .run(move |s| native::llvm(build, s.target));
@@ -124,7 +125,7 @@ pub fn build_rules<'a>(build: &'a Build) -> Rules {
                  Step::noop()
              } else {
                  s.name("librustc")
-                  .host(&build.config.build)
+                  .host(&build.build)
                   .stage(s.stage - 1)
              }
          })
@@ -148,7 +149,7 @@ pub fn build_rules<'a>(build: &'a Build) -> Rules {
                 }
             }
         }
-        return ret
+        ret
     };
 
     // ========================================================================
@@ -215,29 +216,29 @@ pub fn build_rules<'a>(build: &'a Build) -> Rules {
         let mut rule = rules.build(&krate, "path/to/nowhere");
         rule.dep(move |s| {
                 if build.force_use_stage1(&s.compiler(), s.target) {
-                    s.host(&build.config.build).stage(1)
+                    s.host(&build.build).stage(1)
-                } else if s.host == build.config.build {
+                } else if s.host == build.build {
                     s.name(dep)
                 } else {
-                    s.host(&build.config.build)
+                    s.host(&build.build)
                 }
             })
             .run(move |s| {
                 if build.force_use_stage1(&s.compiler(), s.target) {
                     link(build,
-                         &s.stage(1).host(&build.config.build).compiler(),
+                         &s.stage(1).host(&build.build).compiler(),
                          &s.compiler(),
                          s.target)
-                } else if s.host == build.config.build {
+                } else if s.host == build.build {
                     link(build, &s.compiler(), &s.compiler(), s.target)
                 } else {
                     link(build,
-                         &s.host(&build.config.build).compiler(),
+                         &s.host(&build.build).compiler(),
                          &s.compiler(),
                          s.target)
                 }
             });
-            return rule
+            rule
     }
 
     // Similar to the `libstd`, `libtest`, and `librustc` rules above, except
@@ -269,7 +270,7 @@ pub fn build_rules<'a>(build: &'a Build) -> Rules {
     for (krate, path, _default) in krates("std") {
         rules.build(&krate.build_step, path)
              .dep(|s| s.name("startup-objects"))
-             .dep(move |s| s.name("rustc").host(&build.config.build).target(s.host))
+             .dep(move |s| s.name("rustc").host(&build.build).target(s.host))
              .run(move |s| compile::std(build, s.target, &s.compiler()));
     }
     for (krate, path, _default) in krates("test") {
@@ -280,7 +281,7 @@ pub fn build_rules<'a>(build: &'a Build) -> Rules {
     for (krate, path, _default) in krates("rustc-main") {
         rules.build(&krate.build_step, path)
              .dep(|s| s.name("libtest-link"))
-             .dep(move |s| s.name("llvm").host(&build.config.build).stage(0))
+             .dep(move |s| s.name("llvm").host(&build.build).stage(0))
              .dep(|s| s.name("may-run-build-script"))
              .run(move |s| compile::rustc(build, s.target, &s.compiler()));
     }
@@ -291,8 +292,8 @@ pub fn build_rules<'a>(build: &'a Build) -> Rules {
     rules.build("may-run-build-script", "path/to/nowhere")
          .dep(move |s| {
              s.name("libstd-link")
-              .host(&build.config.build)
+              .host(&build.build)
-              .target(&build.config.build)
+              .target(&build.build)
          });
     rules.build("startup-objects", "src/rtstartup")
          .dep(|s| s.name("create-sysroot").target(s.host))
@@ -332,7 +333,7 @@ pub fn build_rules<'a>(build: &'a Build) -> Rules {
               "incremental");
     }
 
-    if build.config.build.contains("msvc") {
+    if build.build.contains("msvc") {
         // nothing to do for debuginfo tests
     } else {
         rules.test("check-debuginfo-lldb", "src/test/debuginfo-lldb")
@@ -352,7 +353,7 @@ pub fn build_rules<'a>(build: &'a Build) -> Rules {
                                          "debuginfo-gdb", "debuginfo"));
         let mut rule = rules.test("check-debuginfo", "src/test/debuginfo");
         rule.default(true);
-        if build.config.build.contains("apple") {
+        if build.build.contains("apple") {
             rule.dep(|s| s.name("check-debuginfo-lldb"));
         } else {
             rule.dep(|s| s.name("check-debuginfo-gdb"));
@@ -463,7 +464,7 @@ pub fn build_rules<'a>(build: &'a Build) -> Rules {
     rules.test("check-linkchecker", "src/tools/linkchecker")
          .dep(|s| s.name("tool-linkchecker").stage(0))
          .dep(|s| s.name("default:doc"))
-         .default(true)
+         .default(build.config.docs)
          .host(true)
          .run(move |s| check::linkcheck(build, s.target));
     rules.test("check-cargotest", "src/tools/cargotest")
@@ -471,10 +472,14 @@ pub fn build_rules<'a>(build: &'a Build) -> Rules {
          .dep(|s| s.name("librustc"))
          .host(true)
          .run(move |s| check::cargotest(build, s.stage, s.target));
-    rules.test("check-cargo", "cargo")
+    rules.test("check-cargo", "src/tools/cargo")
          .dep(|s| s.name("tool-cargo"))
          .host(true)
          .run(move |s| check::cargo(build, s.stage, s.target));
+    rules.test("check-rls", "src/tools/rls")
+         .dep(|s| s.name("tool-rls"))
+         .host(true)
+         .run(move |s| check::rls(build, s.stage, s.target));
     rules.test("check-tidy", "src/tools/tidy")
          .dep(|s| s.name("tool-tidy").stage(0))
          .default(true)
@@ -548,6 +553,10 @@ pub fn build_rules<'a>(build: &'a Build) -> Rules {
          .dep(|s| s.name("maybe-clean-tools"))
          .dep(|s| s.name("librustc-tool"))
          .run(move |s| compile::tool(build, s.stage, s.target, "error_index_generator"));
+    rules.build("tool-unstable-book-gen", "src/tools/unstable-book-gen")
+         .dep(|s| s.name("maybe-clean-tools"))
+         .dep(|s| s.name("libstd-tool"))
+         .run(move |s| compile::tool(build, s.stage, s.target, "unstable-book-gen"));
     rules.build("tool-tidy", "src/tools/tidy")
          .dep(|s| s.name("maybe-clean-tools"))
          .dep(|s| s.name("libstd-tool"))
@@ -590,8 +599,8 @@ pub fn build_rules<'a>(build: &'a Build) -> Rules {
              // Cargo depends on procedural macros, which requires a full host
              // compiler to be available, so we need to depend on that.
              s.name("librustc-link")
-              .target(&build.config.build)
+              .target(&build.build)
-              .host(&build.config.build)
+              .host(&build.build)
          })
          .run(move |s| compile::tool(build, s.stage, s.target, "cargo"));
     rules.build("tool-rls", "src/tools/rls")
@@ -601,8 +610,8 @@ pub fn build_rules<'a>(build: &'a Build) -> Rules {
          .dep(move |s| {
              // rls, like cargo, uses procedural macros
              s.name("librustc-link")
-              .target(&build.config.build)
+              .target(&build.build)
-              .host(&build.config.build)
+              .host(&build.build)
          })
          .run(move |s| compile::tool(build, s.stage, s.target, "rls"));
 
@@ -630,8 +639,8 @@ pub fn build_rules<'a>(build: &'a Build) -> Rules {
     rules.doc("doc-book", "src/doc/book")
          .dep(move |s| {
              s.name("tool-rustbook")
-              .host(&build.config.build)
+              .host(&build.build)
-              .target(&build.config.build)
+              .target(&build.build)
               .stage(0)
          })
          .default(build.config.docs)
@@ -639,8 +648,8 @@ pub fn build_rules<'a>(build: &'a Build) -> Rules {
     rules.doc("doc-nomicon", "src/doc/nomicon")
          .dep(move |s| {
              s.name("tool-rustbook")
-              .host(&build.config.build)
+              .host(&build.build)
-              .target(&build.config.build)
+              .target(&build.build)
               .stage(0)
          })
          .default(build.config.docs)
@@ -648,8 +657,8 @@ pub fn build_rules<'a>(build: &'a Build) -> Rules {
     rules.doc("doc-reference", "src/doc/reference")
          .dep(move |s| {
              s.name("tool-rustbook")
-              .host(&build.config.build)
+              .host(&build.build)
-              .target(&build.config.build)
+              .target(&build.build)
               .stage(0)
          })
          .default(build.config.docs)
@@ -657,27 +666,42 @@ pub fn build_rules<'a>(build: &'a Build) -> Rules {
     rules.doc("doc-unstable-book", "src/doc/unstable-book")
          .dep(move |s| {
              s.name("tool-rustbook")
-              .host(&build.config.build)
+              .host(&build.build)
-              .target(&build.config.build)
+              .target(&build.build)
               .stage(0)
          })
+         .dep(move |s| s.name("doc-unstable-book-gen"))
          .default(build.config.docs)
-         .run(move |s| doc::rustbook(build, s.target, "unstable-book"));
+         .run(move |s| doc::rustbook_src(build,
+                                         s.target,
+                                         "unstable-book",
+                                         &build.md_doc_out(s.target)));
     rules.doc("doc-standalone", "src/doc")
          .dep(move |s| {
              s.name("rustc")
-              .host(&build.config.build)
+              .host(&build.build)
-              .target(&build.config.build)
+              .target(&build.build)
               .stage(0)
          })
          .default(build.config.docs)
          .run(move |s| doc::standalone(build, s.target));
     rules.doc("doc-error-index", "src/tools/error_index_generator")
-         .dep(move |s| s.name("tool-error-index").target(&build.config.build).stage(0))
+         .dep(move |s| s.name("tool-error-index").target(&build.build).stage(0))
          .dep(move |s| s.name("librustc-link"))
          .default(build.config.docs)
          .host(true)
          .run(move |s| doc::error_index(build, s.target));
+    rules.doc("doc-unstable-book-gen", "src/tools/unstable-book-gen")
+         .dep(move |s| {
+             s.name("tool-unstable-book-gen")
+              .host(&build.build)
+              .target(&build.build)
+              .stage(0)
+         })
+         .dep(move |s| s.name("libstd-link"))
+         .default(build.config.docs)
+         .host(true)
+         .run(move |s| doc::unstable_book_gen(build, s.target));
     for (krate, path, default) in krates("std") {
         rules.doc(&krate.doc_step, path)
              .dep(|s| s.name("libstd-link"))
@@ -705,7 +729,7 @@ pub fn build_rules<'a>(build: &'a Build) -> Rules {
     // ========================================================================
     // Distribution targets
     rules.dist("dist-rustc", "src/librustc")
-         .dep(move |s| s.name("rustc").host(&build.config.build))
+         .dep(move |s| s.name("rustc").host(&build.build))
          .host(true)
          .only_host_build(true)
          .default(true)
@@ -790,7 +814,7 @@ pub fn build_rules<'a>(build: &'a Build) -> Rules {
          .host(true)
          .only_build(true)
          .only_host_build(true)
-         .dep(move |s| s.name("tool-build-manifest").target(&build.config.build).stage(0))
+         .dep(move |s| s.name("tool-build-manifest").target(&build.build).stage(0))
          .run(move |_| dist::hash_and_sign(build));
 
     rules.install("install-docs", "src/doc")
@@ -839,8 +863,8 @@ pub fn build_rules<'a>(build: &'a Build) -> Rules {
     /// Helper to depend on a stage0 build-only rust-installer tool.
     fn tool_rust_installer<'a>(build: &'a Build, step: &Step<'a>) -> Step<'a> {
         step.name("tool-rust-installer")
-            .host(&build.config.build)
+            .host(&build.build)
-            .target(&build.config.build)
+            .target(&build.build)
             .stage(0)
     }
 }
@@ -1036,8 +1060,8 @@ impl<'a> Rules<'a> {
             build: build,
             sbuild: Step {
                 stage: build.flags.stage.unwrap_or(2),
-                target: &build.config.build,
+                target: &build.build,
-                host: &build.config.build,
+                host: &build.build,
                 name: "",
             },
             rules: BTreeMap::new(),
@@ -1187,25 +1211,26 @@ invalid rule dependency graph detected, was a rule added and maybe typo'd?
             if paths.len() == 0 && rule.default {
                 Some((rule, 0))
             } else {
-                paths.iter().position(|path| path.ends_with(rule.path))
+                paths.iter()
+                     .position(|path| path.ends_with(rule.path))
                      .map(|priority| (rule, priority))
             }
         }).collect();
 
+        if rules.is_empty() &&
+           !paths.get(0).unwrap_or(&PathBuf::new())
+                 .ends_with("nonexistent/path/to/trigger/cargo/metadata") {
+            println!("\nNothing to run...\n");
+            process::exit(1);
+        }
+
         rules.sort_by_key(|&(_, priority)| priority);
 
         rules.into_iter().flat_map(|(rule, _)| {
             let hosts = if rule.only_host_build || rule.only_build {
-                &self.build.config.host[..1]
+                self.build.build_slice()
-            } else if self.build.flags.host.len() > 0 {
-                &self.build.flags.host
             } else {
-                &self.build.config.host
+                &self.build.hosts
-            };
-            let targets = if self.build.flags.target.len() > 0 {
-                &self.build.flags.target
-            } else {
-                &self.build.config.target
             };
             // Determine the actual targets participating in this rule.
             // NOTE: We should keep the full projection from build triple to
@@ -1214,19 +1239,18 @@ invalid rule dependency graph detected, was a rule added and maybe typo'd?
             // the original non-shadowed hosts array is used below.
             let arr = if rule.host {
                 // If --target was specified but --host wasn't specified,
-                // don't run any host-only tests. Also, respect any `--host`
+                // don't run any host-only tests.
-                // overrides as done for `hosts`.
                 if self.build.flags.host.len() > 0 {
-                    &self.build.flags.host[..]
+                    &self.build.hosts
                 } else if self.build.flags.target.len() > 0 {
                     &[]
                 } else if rule.only_build {
-                    &self.build.config.host[..1]
+                    self.build.build_slice()
                 } else {
-                    &self.build.config.host[..]
+                    &self.build.hosts
                 }
             } else {
-                targets
+                &self.build.targets
             };
 
             hosts.iter().flat_map(move |host| {
@@ -1304,7 +1328,7 @@ invalid rule dependency graph detected, was a rule added and maybe typo'd?
         for idx in 0..nodes.len() {
             self.topo_sort(idx, &idx_to_node, &edges, &mut visited, &mut order);
         }
-        return order
+        order
     }
 
     /// Builds the dependency graph rooted at `step`.
@@ -1343,7 +1367,7 @@ invalid rule dependency graph detected, was a rule added and maybe typo'd?
         }
 
         edges.entry(idx).or_insert(HashSet::new()).extend(deps);
-        return idx
+        idx
     }
 
     /// Given a dependency graph with a finished list of `nodes`, fill out more
@@ -1404,13 +1428,20 @@ mod tests {
     fn build(args: &[&str],
              extra_host: &[&str],
              extra_target: &[&str]) -> Build {
+        build_(args, extra_host, extra_target, true)
+    }
+
+    fn build_(args: &[&str],
+              extra_host: &[&str],
+              extra_target: &[&str],
+              docs: bool) -> Build {
         let mut args = args.iter().map(|s| s.to_string()).collect::<Vec<_>>();
         args.push("--build".to_string());
         args.push("A".to_string());
         let flags = Flags::parse(&args);
 
         let mut config = Config::default();
-        config.docs = true;
+        config.docs = docs;
         config.build = "A".to_string();
         config.host = vec![config.build.clone()];
         config.host.extend(extra_host.iter().map(|s| s.to_string()));
@@ -1465,8 +1496,8 @@ mod tests {
         let step = super::Step {
             name: "",
             stage: 2,
-            host: &build.config.build,
+            host: &build.build,
-            target: &build.config.build,
+            target: &build.build,
         };
 
         assert!(plan.contains(&step.name("dist-docs")));
@@ -1488,8 +1519,8 @@ mod tests {
         let step = super::Step {
             name: "",
             stage: 2,
-            host: &build.config.build,
+            host: &build.build,
-            target: &build.config.build,
+            target: &build.build,
         };
 
         assert!(plan.contains(&step.name("dist-docs")));
@@ -1516,8 +1547,8 @@ mod tests {
         let step = super::Step {
             name: "",
             stage: 2,
-            host: &build.config.build,
+            host: &build.build,
-            target: &build.config.build,
+            target: &build.build,
         };
 
         assert!(!plan.iter().any(|s| s.host == "B"));
@@ -1546,8 +1577,8 @@ mod tests {
         let step = super::Step {
             name: "",
             stage: 2,
-            host: &build.config.build,
+            host: &build.build,
-            target: &build.config.build,
+            target: &build.build,
         };
 
         assert!(!plan.iter().any(|s| s.host == "B"));
@@ -1583,8 +1614,8 @@ mod tests {
         let step = super::Step {
             name: "",
             stage: 2,
-            host: &build.config.build,
+            host: &build.build,
-            target: &build.config.build,
+            target: &build.build,
         };
 
         assert!(!plan.iter().any(|s| s.target == "A"));
@@ -1610,8 +1641,8 @@ mod tests {
         let step = super::Step {
             name: "",
             stage: 2,
-            host: &build.config.build,
+            host: &build.build,
-            target: &build.config.build,
+            target: &build.build,
         };
 
         assert!(!plan.iter().any(|s| s.target == "A"));
@@ -1654,8 +1685,8 @@ mod tests {
         let step = super::Step {
             name: "",
             stage: 2,
-            host: &build.config.build,
+            host: &build.build,
-            target: &build.config.build,
+            target: &build.build,
         };
 
         // rustc built for all for of (A, B) x (A, B)
@@ -1765,4 +1796,22 @@ mod tests {
         assert!(!plan.iter().any(|s| s.name.contains("tidy")));
         assert!(plan.iter().any(|s| s.name.contains("valgrind")));
     }
+
+    #[test]
+    fn test_disable_docs() {
+        let build = build_(&["test"], &[], &[], false);
+        let rules = super::build_rules(&build);
+        let plan = rules.plan();
+        println!("rules: {:#?}", plan);
+        assert!(!plan.iter().any(|s| {
+            s.name.contains("doc-") || s.name.contains("default:doc")
+        }));
+        // none of the dependencies should be a doc rule either
+        assert!(!plan.iter().any(|s| {
+            rules.rules[s.name].deps.iter().any(|dep| {
+                let dep = dep(&rules.sbuild.name(s.name));
+                dep.name.contains("doc-") || dep.name.contains("default:doc")
+            })
+        }));
+    }
 }

src/bootstrap/util.rs

@@ -14,7 +14,6 @@
 //! not a lot of interesting happenings here unfortunately.
 
 use std::env;
-use std::ffi::OsString;
 use std::fs;
 use std::io::{self, Write};
 use std::path::{Path, PathBuf};
@@ -32,16 +31,9 @@ pub fn staticlib(name: &str, target: &str) -> String {
     }
 }
 
-/// Copies a file from `src` to `dst`, attempting to use hard links and then
+/// Copies a file from `src` to `dst`
-/// falling back to an actually filesystem copy if necessary.
 pub fn copy(src: &Path, dst: &Path) {
-    // A call to `hard_link` will fail if `dst` exists, so remove it if it
-    // already exists so we can try to help `hard_link` succeed.
     let _ = fs::remove_file(&dst);
-
-    // Attempt to "easy copy" by creating a hard link (symlinks don't work on
-    // windows), but if that fails just fall back to a slow `copy` operation.
-    // let res = fs::hard_link(src, dst);
     let res = fs::copy(src, dst);
     if let Err(e) = res {
         panic!("failed to copy `{}` to `{}`: {}", src.display(),
@@ -149,8 +141,7 @@ pub fn dylib_path_var() -> &'static str {
 /// Parses the `dylib_path_var()` environment variable, returning a list of
 /// paths that are members of this lookup path.
 pub fn dylib_path() -> Vec<PathBuf> {
-    env::split_paths(&env::var_os(dylib_path_var()).unwrap_or(OsString::new()))
+    env::split_paths(&env::var_os(dylib_path_var()).unwrap_or_default()).collect()
-        .collect()
 }
 
 /// `push` all components to `buf`. On windows, append `.exe` to the last component.
@@ -422,4 +413,4 @@ impl CiEnv {
             cmd.env("TERM", "xterm").args(&["--color", "always"]);
         }
     }
-}
+}

src/build_helper/lib.rs

@@ -239,7 +239,10 @@ pub fn sanitizer_lib_boilerplate(sanitizer_name: &str) -> Result<NativeLibBoiler
         ),
         _ => return Err(()),
     };
-    native_lib_boilerplate("compiler-rt", sanitizer_name, &link_name, search_path)
+    native_lib_boilerplate("libcompiler_builtins/compiler-rt",
+                           sanitizer_name,
+                           &link_name,
+                           search_path)
 }
 
 fn dir_up_to_date(src: &Path, threshold: &FileTime) -> bool {

src/ci/docker/arm-android/Dockerfile

@@ -1,31 +1,15 @@
 FROM ubuntu:16.04
 
-RUN apt-get update && \
+COPY scripts/android-base-apt-get.sh /scripts/
-    apt-get install -y --no-install-recommends \
+RUN sh /scripts/android-base-apt-get.sh
-  ca-certificates \
-  cmake \
-  curl \
-  file \
-  g++ \
-  git \
-  libssl-dev \
-  make \
-  pkg-config \
-  python2.7 \
-  sudo \
-  unzip \
-  xz-utils
 
-# dumb-init
 COPY scripts/dumb-init.sh /scripts/
 RUN sh /scripts/dumb-init.sh
 
-# ndk
 COPY scripts/android-ndk.sh /scripts/
 RUN . /scripts/android-ndk.sh && \
     download_and_make_toolchain android-ndk-r13b-linux-x86_64.zip arm 9
 
-# sdk
 RUN dpkg --add-architecture i386 && \
     apt-get update && \
     apt-get install -y --no-install-recommends \
@@ -39,7 +23,6 @@ COPY scripts/android-sdk.sh /scripts/
 RUN . /scripts/android-sdk.sh && \
     download_and_create_avd tools_r25.2.5-linux.zip armeabi-v7a 18
 
-# env
 ENV PATH=$PATH:/android/sdk/tools
 ENV PATH=$PATH:/android/sdk/platform-tools
 
@@ -51,10 +34,8 @@ ENV RUST_CONFIGURE_ARGS \
 
 ENV SCRIPT python2.7 ../x.py test --target $TARGETS
 
-# sccache
 COPY scripts/sccache.sh /scripts/
 RUN sh /scripts/sccache.sh
 
-# init
 COPY scripts/android-start-emulator.sh /scripts/
 ENTRYPOINT ["/usr/bin/dumb-init", "--", "/scripts/android-start-emulator.sh"]

src/ci/docker/armhf-gnu/Dockerfile

@@ -73,13 +73,12 @@ RUN arm-linux-gnueabihf-gcc addentropy.c -o rootfs/addentropy -static
 # TODO: What is this?!
 RUN curl -O http://ftp.nl.debian.org/debian/dists/jessie/main/installer-armhf/current/images/device-tree/vexpress-v2p-ca15-tc1.dtb
 
-RUN curl -o /usr/local/bin/sccache \
+COPY scripts/dumb-init.sh /scripts/
-      https://s3.amazonaws.com/rust-lang-ci/rust-ci-mirror/2017-05-12-sccache-x86_64-unknown-linux-musl && \
+RUN sh /scripts/dumb-init.sh
-      chmod +x /usr/local/bin/sccache
+
+COPY scripts/sccache.sh /scripts/
+RUN sh /scripts/sccache.sh
 
-RUN curl -OL https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64.deb && \
-    dpkg -i dumb-init_*.deb && \
-    rm dumb-init_*.deb
 ENTRYPOINT ["/usr/bin/dumb-init", "--"]
 
 ENV RUST_CONFIGURE_ARGS \

src/ci/docker/asmjs/Dockerfile

@@ -0,0 +1,39 @@
+FROM ubuntu:16.04
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+  g++ \
+  make \
+  file \
+  curl \
+  ca-certificates \
+  python \
+  git \
+  cmake \
+  sudo \
+  gdb \
+  xz-utils
+
+COPY scripts/dumb-init.sh /scripts/
+RUN sh /scripts/dumb-init.sh
+
+COPY scripts/emscripten.sh /scripts/
+RUN bash /scripts/emscripten.sh
+
+ENV PATH=$PATH:/emsdk-portable
+ENV PATH=$PATH:/emsdk-portable/clang/e1.37.13_64bit/
+ENV PATH=$PATH:/emsdk-portable/emscripten/1.37.13/
+ENV PATH=$PATH:/emsdk-portable/node/4.1.1_64bit/bin/
+ENV EMSCRIPTEN=/emsdk-portable/emscripten/1.37.13/
+ENV BINARYEN_ROOT=/emsdk-portable/clang/e1.37.13_64bit/binaryen/
+ENV EM_CONFIG=/emsdk-portable/.emscripten
+
+ENV TARGETS=asmjs-unknown-emscripten
+
+ENV RUST_CONFIGURE_ARGS --target=$TARGETS
+
+ENV SCRIPT python2.7 ../x.py test --target $TARGETS
+
+COPY scripts/sccache.sh /scripts/
+RUN sh /scripts/sccache.sh
+
+ENTRYPOINT ["/usr/bin/dumb-init", "--"]

src/ci/docker/cross/Dockerfile

@@ -21,7 +21,6 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
   libssl-dev \
   pkg-config
 
-# dumb-init
 COPY scripts/dumb-init.sh /scripts/
 RUN sh /scripts/dumb-init.sh
 
@@ -68,5 +67,4 @@ ENV SCRIPT python2.7 ../x.py dist --target $TARGETS
 COPY scripts/sccache.sh /scripts/
 RUN sh /scripts/sccache.sh
 
-# init
 ENTRYPOINT ["/usr/bin/dumb-init", "--"]

src/ci/docker/disabled/dist-aarch64-android/Dockerfile

@@ -1,31 +1,15 @@
 FROM ubuntu:16.04
 
-RUN apt-get update && \
+COPY scripts/android-base-apt-get.sh /scripts/
-    apt-get install -y --no-install-recommends \
+RUN sh /scripts/android-base-apt-get.sh
-  ca-certificates \
-  cmake \
-  curl \
-  file \
-  g++ \
-  git \
-  libssl-dev \
-  make \
-  pkg-config \
-  python2.7 \
-  sudo \
-  unzip \
-  xz-utils
 
-# dumb-init
 COPY scripts/dumb-init.sh /scripts/
 RUN sh /scripts/dumb-init.sh
 
-# ndk
 COPY scripts/android-ndk.sh /scripts/
 RUN . /scripts/android-ndk.sh && \
     download_and_make_toolchain android-ndk-r13b-linux-x86_64.zip arm64 21
 
-# env
 ENV PATH=$PATH:/android/ndk/arm64-21/bin
 
 ENV DEP_Z_ROOT=/android/ndk/arm64-21/sysroot/usr/
@@ -42,9 +26,7 @@ ENV RUST_CONFIGURE_ARGS \
 
 ENV SCRIPT python2.7 ../x.py dist --target $HOSTS --host $HOSTS
 
-# sccache
 COPY scripts/sccache.sh /scripts/
 RUN sh /scripts/sccache.sh
 
-# init
 ENTRYPOINT ["/usr/bin/dumb-init", "--"]

src/ci/docker/disabled/dist-armv7-android/Dockerfile

@@ -1,26 +1,11 @@
 FROM ubuntu:16.04
 
-RUN apt-get update && \
+COPY scripts/android-base-apt-get.sh /scripts/
-    apt-get install -y --no-install-recommends \
+RUN sh /scripts/android-base-apt-get.sh
-  ca-certificates \
-  cmake \
-  curl \
-  file \
-  g++ \
-  git \
-  libssl-dev \
-  make \
-  pkg-config \
-  python2.7 \
-  sudo \
-  unzip \
-  xz-utils
 
-# dumb-init
 COPY scripts/dumb-init.sh /scripts/
 RUN sh /scripts/dumb-init.sh
 
-# ndk
 COPY scripts/android-ndk.sh /scripts/
 RUN . /scripts/android-ndk.sh && \
     download_ndk android-ndk-r13b-linux-x86_64.zip && \
@@ -31,7 +16,6 @@ RUN . /scripts/android-ndk.sh && \
 RUN chmod 777 /android/ndk && \
     ln -s /android/ndk/arm-21 /android/ndk/arm
 
-# env
 ENV PATH=$PATH:/android/ndk/arm-9/bin
 
 ENV DEP_Z_ROOT=/android/ndk/arm-9/sysroot/usr/
@@ -60,9 +44,7 @@ ENV SCRIPT \
     ln -s /android/ndk/arm-9 /android/ndk/arm && \
     python2.7 ../x.py dist --host $HOSTS --target $HOSTS)
 
-# sccache
 COPY scripts/sccache.sh /scripts/
 RUN sh /scripts/sccache.sh
 
-# init
 ENTRYPOINT ["/usr/bin/dumb-init", "--"]

src/ci/docker/disabled/dist-i686-android/Dockerfile

@@ -1,26 +1,11 @@
 FROM ubuntu:16.04
 
-RUN apt-get update && \
+COPY scripts/android-base-apt-get.sh /scripts/
-    apt-get install -y --no-install-recommends \
+RUN sh /scripts/android-base-apt-get.sh
-  ca-certificates \
-  cmake \
-  curl \
-  file \
-  g++ \
-  git \
-  libssl-dev \
-  make \
-  pkg-config \
-  python2.7 \
-  sudo \
-  unzip \
-  xz-utils
 
-# dumb-init
 COPY scripts/dumb-init.sh /scripts/
 RUN sh /scripts/dumb-init.sh
 
-# ndk
 COPY scripts/android-ndk.sh /scripts/
 RUN . /scripts/android-ndk.sh && \
     download_ndk android-ndk-r13b-linux-x86_64.zip && \
@@ -31,7 +16,6 @@ RUN . /scripts/android-ndk.sh && \
 RUN chmod 777 /android/ndk && \
     ln -s /android/ndk/x86-21 /android/ndk/x86
 
-# env
 ENV PATH=$PATH:/android/ndk/x86-9/bin
 
 ENV DEP_Z_ROOT=/android/ndk/x86-9/sysroot/usr/
@@ -60,9 +44,7 @@ ENV SCRIPT \
     ln -s /android/ndk/x86-9 /android/ndk/x86 && \
     python2.7 ../x.py dist --host $HOSTS --target $HOSTS)
 
-# sccache
 COPY scripts/sccache.sh /scripts/
 RUN sh /scripts/sccache.sh
 
-# init
 ENTRYPOINT ["/usr/bin/dumb-init", "--"]

src/ci/docker/disabled/dist-x86_64-android/Dockerfile

@@ -1,31 +1,15 @@
 FROM ubuntu:16.04
 
-RUN apt-get update && \
+COPY scripts/android-base-apt-get.sh /scripts/
-    apt-get install -y --no-install-recommends \
+RUN sh /scripts/android-base-apt-get.sh
-  ca-certificates \
-  cmake \
-  curl \
-  file \
-  g++ \
-  git \
-  libssl-dev \
-  make \
-  pkg-config \
-  python2.7 \
-  sudo \
-  unzip \
-  xz-utils
 
-# dumb-init
 COPY scripts/dumb-init.sh /scripts/
 RUN sh /scripts/dumb-init.sh
 
-# ndk
 COPY scripts/android-ndk.sh /scripts/
 RUN . /scripts/android-ndk.sh && \
     download_and_make_toolchain android-ndk-r13b-linux-x86_64.zip x86_64 21
 
-# env
 ENV PATH=$PATH:/android/ndk/x86_64-21/bin
 
 ENV DEP_Z_ROOT=/android/ndk/x86_64-21/sysroot/usr/
@@ -42,9 +26,7 @@ ENV RUST_CONFIGURE_ARGS \
 
 ENV SCRIPT python2.7 ../x.py dist --target $HOSTS --host $HOSTS
 
-# sccache
 COPY scripts/sccache.sh /scripts/
 RUN sh /scripts/sccache.sh
 
-# init
 ENTRYPOINT ["/usr/bin/dumb-init", "--"]

src/ci/docker/disabled/wasm32-exp/Dockerfile

@@ -0,0 +1,42 @@
+FROM ubuntu:16.04
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+  g++ \
+  make \
+  file \
+  curl \
+  ca-certificates \
+  python \
+  git \
+  cmake \
+  sudo \
+  gdb \
+  xz-utils \
+  jq \
+  bzip2
+
+# dumb-init
+COPY scripts/dumb-init.sh /scripts/
+RUN sh /scripts/dumb-init.sh
+
+# emscripten
+COPY scripts/emscripten-wasm.sh /scripts/
+COPY disabled/wasm32-exp/node.sh /usr/local/bin/node
+RUN bash /scripts/emscripten-wasm.sh
+
+# cache
+COPY scripts/sccache.sh /scripts/
+RUN sh /scripts/sccache.sh
+
+# env
+ENV PATH=/wasm-install/emscripten:/wasm-install/bin:$PATH
+ENV EM_CONFIG=/root/.emscripten
+
+ENV TARGETS=wasm32-experimental-emscripten
+
+ENV RUST_CONFIGURE_ARGS --target=$TARGETS --experimental-targets=WebAssembly
+
+ENV SCRIPT python2.7 ../x.py test --target $TARGETS
+
+# init
+ENTRYPOINT ["/usr/bin/dumb-init", "--"]

src/ci/docker/disabled/wasm32-exp/node.sh

@@ -0,0 +1,18 @@
+#!/bin/bash
+# Copyright 2017 The Rust Project Developers. See the COPYRIGHT
+# file at the top-level directory of this distribution and at
+# http://rust-lang.org/COPYRIGHT.
+#
+# Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
+# http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
+# <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
+# option. This file may not be copied, modified, or distributed
+# except according to those terms.
+
+path="$(dirname $1)"
+file="$(basename $1)"
+
+shift
+
+cd "$path"
+exec /node-v8.0.0-linux-x64/bin/node "$file" "$@"

src/ci/docker/disabled/wasm32/Dockerfile

@@ -0,0 +1,40 @@
+FROM ubuntu:16.04
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+  g++ \
+  make \
+  file \
+  curl \
+  ca-certificates \
+  python \
+  git \
+  cmake \
+  sudo \
+  gdb \
+  xz-utils
+
+COPY scripts/dumb-init.sh /scripts/
+RUN sh /scripts/dumb-init.sh
+
+# emscripten
+COPY scripts/emscripten.sh /scripts/
+RUN bash /scripts/emscripten.sh
+COPY disabled/wasm32/node.sh /usr/local/bin/node
+
+COPY scripts/sccache.sh /scripts/
+RUN sh /scripts/sccache.sh
+
+ENV PATH=$PATH:/emsdk-portable
+ENV PATH=$PATH:/emsdk-portable/clang/e1.37.13_64bit/
+ENV PATH=$PATH:/emsdk-portable/emscripten/1.37.13/
+ENV EMSCRIPTEN=/emsdk-portable/emscripten/1.37.13/
+ENV BINARYEN_ROOT=/emsdk-portable/clang/e1.37.13_64bit/binaryen/
+ENV EM_CONFIG=/emsdk-portable/.emscripten
+
+ENV TARGETS=wasm32-unknown-emscripten
+
+ENV RUST_CONFIGURE_ARGS --target=$TARGETS
+
+ENV SCRIPT python2.7 ../x.py test --target $TARGETS
+
+ENTRYPOINT ["/usr/bin/dumb-init", "--"]

src/ci/docker/disabled/wasm32/node.sh

@@ -0,0 +1,18 @@
+#!/bin/bash
+# Copyright 2017 The Rust Project Developers. See the COPYRIGHT
+# file at the top-level directory of this distribution and at
+# http://rust-lang.org/COPYRIGHT.
+#
+# Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
+# http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
+# <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
+# option. This file may not be copied, modified, or distributed
+# except according to those terms.
+
+path="$(dirname $1)"
+file="$(basename $1)"
+
+shift
+
+cd "$path"
+exec /node-v8.0.0-linux-x64/bin/node "$file" "$@"

src/ci/docker/dist-aarch64-linux/Dockerfile

@@ -1,58 +1,23 @@
 FROM ubuntu:16.04
 
-RUN apt-get update && apt-get install -y --no-install-recommends \
+COPY scripts/cross-apt-packages.sh /scripts/
-  automake \
+RUN sh /scripts/cross-apt-packages.sh
-  bison \
+
-  bzip2 \
+COPY scripts/dumb-init.sh /scripts/
-  ca-certificates \
+RUN sh /scripts/dumb-init.sh
-  cmake \
-  curl \
-  file \
-  flex \
-  g++ \
-  gawk \
-  gdb \
-  git \
-  gperf \
-  help2man \
-  libncurses-dev \
-  libtool-bin \
-  make \
-  patch \
-  python2.7 \
-  sudo \
-  texinfo \
-  wget \
-  xz-utils \
-  libssl-dev \
-  pkg-config
 
-RUN curl -OL https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64.deb && \
-    dpkg -i dumb-init_*.deb && \
-    rm dumb-init_*.deb
 ENTRYPOINT ["/usr/bin/dumb-init", "--"]
 
-# Ubuntu 16.04 (this contianer) ships with make 4, but something in the
+# Ubuntu 16.04 (this container) ships with make 4, but something in the
 # toolchains we build below chokes on that, so go back to make 3
-RUN curl https://ftp.gnu.org/gnu/make/make-3.81.tar.gz | tar xzf - && \
+COPY scripts/make3.sh /scripts/
-      cd make-3.81 && \
+RUN sh /scripts/make3.sh
-      ./configure --prefix=/usr && \
-      make && \
-      make install && \
-      cd .. && \
-      rm -rf make-3.81
 
-RUN curl http://crosstool-ng.org/download/crosstool-ng/crosstool-ng-1.22.0.tar.bz2 | \
+COPY scripts/crosstool-ng.sh /scripts/
-      tar xjf - && \
+RUN sh /scripts/crosstool-ng.sh
-      cd crosstool-ng && \
-      ./configure --prefix=/usr/local && \
-      make -j$(nproc) && \
-      make install && \
-      cd .. && \
-      rm -rf crosstool-ng
 
-RUN groupadd -r rustbuild && useradd -m -r -g rustbuild rustbuild
+COPY scripts/rustbuild-setup.sh /scripts/
-RUN mkdir /x-tools && chown rustbuild:rustbuild /x-tools
+RUN sh /scripts/rustbuild-setup.sh
 USER rustbuild
 WORKDIR /tmp
 
@@ -61,9 +26,8 @@ RUN ./build-toolchains.sh
 
 USER root
 
-RUN curl -o /usr/local/bin/sccache \
+COPY scripts/sccache.sh /scripts/
-      https://s3.amazonaws.com/rust-lang-ci/rust-ci-mirror/2017-05-12-sccache-x86_64-unknown-linux-musl && \
+RUN sh /scripts/sccache.sh
-      chmod +x /usr/local/bin/sccache
 
 ENV PATH=$PATH:/x-tools/aarch64-unknown-linux-gnueabi/bin
 

src/ci/docker/dist-android/Dockerfile

@@ -1,22 +1,8 @@
 FROM ubuntu:16.04
 
-RUN apt-get update && \
+COPY scripts/android-base-apt-get.sh /scripts/
-    apt-get install -y --no-install-recommends \
+RUN sh /scripts/android-base-apt-get.sh
-  ca-certificates \
-  cmake \
-  curl \
-  file \
-  g++ \
-  git \
-  libssl-dev \
-  make \
-  pkg-config \
-  python2.7 \
-  sudo \
-  unzip \
-  xz-utils
 
-# dumb-init
 COPY scripts/dumb-init.sh /scripts/
 RUN sh /scripts/dumb-init.sh
 
@@ -48,9 +34,7 @@ ENV RUST_CONFIGURE_ARGS \
 
 ENV SCRIPT python2.7 ../x.py dist --target $TARGETS
 
-# cache
 COPY scripts/sccache.sh /scripts/
 RUN sh /scripts/sccache.sh
 
-# init
 ENTRYPOINT ["/usr/bin/dumb-init", "--"]

src/ci/docker/dist-arm-linux/Dockerfile

@@ -1,58 +1,23 @@
 FROM ubuntu:16.04
 
-RUN apt-get update && apt-get install -y --no-install-recommends \
+COPY scripts/cross-apt-packages.sh /scripts/
-  automake \
+RUN sh /scripts/cross-apt-packages.sh
-  bison \
+
-  bzip2 \
+COPY scripts/dumb-init.sh /scripts/
-  ca-certificates \
+RUN sh /scripts/dumb-init.sh
-  cmake \
-  curl \
-  file \
-  flex \
-  g++ \
-  gawk \
-  gdb \
-  git \
-  gperf \
-  help2man \
-  libncurses-dev \
-  libtool-bin \
-  make \
-  patch \
-  python2.7 \
-  sudo \
-  texinfo \
-  wget \
-  xz-utils \
-  libssl-dev \
-  pkg-config
 
-RUN curl -OL https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64.deb && \
-    dpkg -i dumb-init_*.deb && \
-    rm dumb-init_*.deb
 ENTRYPOINT ["/usr/bin/dumb-init", "--"]
 
-# Ubuntu 16.04 (this contianer) ships with make 4, but something in the
+# Ubuntu 16.04 (this container) ships with make 4, but something in the
 # toolchains we build below chokes on that, so go back to make 3
-RUN curl https://ftp.gnu.org/gnu/make/make-3.81.tar.gz | tar xzf - && \
+COPY scripts/make3.sh /scripts/
-      cd make-3.81 && \
+RUN sh /scripts/make3.sh
-      ./configure --prefix=/usr && \
-      make && \
-      make install && \
-      cd .. && \
-      rm -rf make-3.81
 
-RUN curl http://crosstool-ng.org/download/crosstool-ng/crosstool-ng-1.22.0.tar.bz2 | \
+COPY scripts/crosstool-ng.sh /scripts/
-      tar xjf - && \
+RUN sh /scripts/crosstool-ng.sh
-      cd crosstool-ng && \
-      ./configure --prefix=/usr/local && \
-      make -j$(nproc) && \
-      make install && \
-      cd .. && \
-      rm -rf crosstool-ng
 
-RUN groupadd -r rustbuild && useradd -m -r -g rustbuild rustbuild
+COPY scripts/rustbuild-setup.sh /scripts/
-RUN mkdir /x-tools && chown rustbuild:rustbuild /x-tools
+RUN sh /scripts/rustbuild-setup.sh
 USER rustbuild
 WORKDIR /tmp
 
@@ -61,9 +26,8 @@ RUN ./build-toolchains.sh
 
 USER root
 
-RUN curl -o /usr/local/bin/sccache \
+COPY scripts/sccache.sh /scripts/
-      https://s3.amazonaws.com/rust-lang-ci/rust-ci-mirror/2017-05-12-sccache-x86_64-unknown-linux-musl && \
+RUN sh /scripts/sccache.sh
-      chmod +x /usr/local/bin/sccache
 
 ENV PATH=$PATH:/x-tools/arm-unknown-linux-gnueabi/bin
 

src/ci/docker/dist-armhf-linux/Dockerfile

@@ -1,58 +1,23 @@
 FROM ubuntu:16.04
 
-RUN apt-get update && apt-get install -y --no-install-recommends \
+COPY scripts/cross-apt-packages.sh /scripts/
-  automake \
+RUN sh /scripts/cross-apt-packages.sh
-  bison \
+
-  bzip2 \
+COPY scripts/dumb-init.sh /scripts/
-  ca-certificates \
+RUN sh /scripts/dumb-init.sh
-  cmake \
-  curl \
-  file \
-  flex \
-  g++ \
-  gawk \
-  gdb \
-  git \
-  gperf \
-  help2man \
-  libncurses-dev \
-  libtool-bin \
-  make \
-  patch \
-  python2.7 \
-  sudo \
-  texinfo \
-  wget \
-  xz-utils \
-  libssl-dev \
-  pkg-config
 
-RUN curl -OL https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64.deb && \
-    dpkg -i dumb-init_*.deb && \
-    rm dumb-init_*.deb
 ENTRYPOINT ["/usr/bin/dumb-init", "--"]
 
-# Ubuntu 16.04 (this contianer) ships with make 4, but something in the
+# Ubuntu 16.04 (this container) ships with make 4, but something in the
 # toolchains we build below chokes on that, so go back to make 3
-RUN curl https://ftp.gnu.org/gnu/make/make-3.81.tar.gz | tar xzf - && \
+COPY scripts/make3.sh /scripts/
-      cd make-3.81 && \
+RUN sh /scripts/make3.sh
-      ./configure --prefix=/usr && \
-      make && \
-      make install && \
-      cd .. && \
-      rm -rf make-3.81
 
-RUN curl http://crosstool-ng.org/download/crosstool-ng/crosstool-ng-1.22.0.tar.bz2 | \
+COPY scripts/crosstool-ng.sh /scripts/
-      tar xjf - && \
+RUN sh /scripts/crosstool-ng.sh
-      cd crosstool-ng && \
-      ./configure --prefix=/usr/local && \
-      make -j$(nproc) && \
-      make install && \
-      cd .. && \
-      rm -rf crosstool-ng
 
-RUN groupadd -r rustbuild && useradd -m -r -g rustbuild rustbuild
+COPY scripts/rustbuild-setup.sh /scripts/
-RUN mkdir /x-tools && chown rustbuild:rustbuild /x-tools
+RUN sh /scripts/rustbuild-setup.sh
 USER rustbuild
 WORKDIR /tmp
 
@@ -61,9 +26,8 @@ RUN ./build-toolchains.sh
 
 USER root
 
-RUN curl -o /usr/local/bin/sccache \
+COPY scripts/sccache.sh /scripts/
-      https://s3.amazonaws.com/rust-lang-ci/rust-ci-mirror/2017-05-12-sccache-x86_64-unknown-linux-musl && \
+RUN sh /scripts/sccache.sh
-      chmod +x /usr/local/bin/sccache
 
 ENV PATH=$PATH:/x-tools/arm-unknown-linux-gnueabihf/bin
 

src/ci/docker/dist-armv7-linux/Dockerfile

@@ -1,58 +1,23 @@
 FROM ubuntu:16.04
 
-RUN apt-get update && apt-get install -y --no-install-recommends \
+COPY scripts/cross-apt-packages.sh /scripts/
-  automake \
+RUN sh /scripts/cross-apt-packages.sh
-  bison \
+
-  bzip2 \
+COPY scripts/dumb-init.sh /scripts/
-  ca-certificates \
+RUN sh /scripts/dumb-init.sh
-  cmake \
-  curl \
-  file \
-  flex \
-  g++ \
-  gawk \
-  gdb \
-  git \
-  gperf \
-  help2man \
-  libncurses-dev \
-  libtool-bin \
-  make \
-  patch \
-  python2.7 \
-  sudo \
-  texinfo \
-  wget \
-  xz-utils \
-  libssl-dev \
-  pkg-config
 
-RUN curl -OL https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64.deb && \
-    dpkg -i dumb-init_*.deb && \
-    rm dumb-init_*.deb
 ENTRYPOINT ["/usr/bin/dumb-init", "--"]
 
-# Ubuntu 16.04 (this contianer) ships with make 4, but something in the
+# Ubuntu 16.04 (this container) ships with make 4, but something in the
 # toolchains we build below chokes on that, so go back to make 3
-RUN curl https://ftp.gnu.org/gnu/make/make-3.81.tar.gz | tar xzf - && \
+COPY scripts/make3.sh /scripts/
-      cd make-3.81 && \
+RUN sh /scripts/make3.sh
-      ./configure --prefix=/usr && \
-      make && \
-      make install && \
-      cd .. && \
-      rm -rf make-3.81
 
-RUN curl http://crosstool-ng.org/download/crosstool-ng/crosstool-ng-1.22.0.tar.bz2 | \
+COPY scripts/crosstool-ng.sh /scripts/
-      tar xjf - && \
+RUN sh /scripts/crosstool-ng.sh
-      cd crosstool-ng && \
-      ./configure --prefix=/usr/local && \
-      make -j$(nproc) && \
-      make install && \
-      cd .. && \
-      rm -rf crosstool-ng
 
-RUN groupadd -r rustbuild && useradd -m -r -g rustbuild rustbuild
+COPY scripts/rustbuild-setup.sh /scripts/
-RUN mkdir /x-tools && chown rustbuild:rustbuild /x-tools
+RUN sh /scripts/rustbuild-setup.sh
 USER rustbuild
 WORKDIR /tmp
 
@@ -61,9 +26,8 @@ RUN ./build-toolchains.sh
 
 USER root
 
-RUN curl -o /usr/local/bin/sccache \
+COPY scripts/sccache.sh /scripts/
-      https://s3.amazonaws.com/rust-lang-ci/rust-ci-mirror/2017-05-12-sccache-x86_64-unknown-linux-musl && \
+RUN sh /scripts/sccache.sh
-      chmod +x /usr/local/bin/sccache
 
 ENV PATH=$PATH:/x-tools/armv7-unknown-linux-gnueabihf/bin
 

src/ci/docker/dist-fuchsia/Dockerfile

@@ -24,14 +24,13 @@ WORKDIR /tmp
 COPY dist-fuchsia/shared.sh dist-fuchsia/build-toolchain.sh dist-fuchsia/compiler-rt-dso-handle.patch /tmp/
 RUN /tmp/build-toolchain.sh
 
-RUN curl -OL https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64.deb && \
+COPY scripts/dumb-init.sh /scripts/
-    dpkg -i dumb-init_*.deb && \
+RUN sh /scripts/dumb-init.sh
-    rm dumb-init_*.deb
+
 ENTRYPOINT ["/usr/bin/dumb-init", "--"]
 
-RUN curl -o /usr/local/bin/sccache \
+COPY scripts/sccache.sh /scripts/
-      https://s3.amazonaws.com/rust-lang-ci/rust-ci-mirror/2017-05-12-sccache-x86_64-unknown-linux-musl && \
+RUN sh /scripts/sccache.sh
-      chmod +x /usr/local/bin/sccache
 
 ENV \
     AR_x86_64_unknown_fuchsia=x86_64-unknown-fuchsia-ar \

src/ci/docker/dist-i586-gnu-i686-musl/Dockerfile

@@ -20,14 +20,13 @@ WORKDIR /build/
 COPY dist-i586-gnu-i686-musl/musl-libunwind-patch.patch dist-i586-gnu-i686-musl/build-musl.sh /build/
 RUN sh /build/build-musl.sh && rm -rf /build
 
-RUN curl -OL https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64.deb && \
+COPY scripts/dumb-init.sh /scripts/
-    dpkg -i dumb-init_*.deb && \
+RUN sh /scripts/dumb-init.sh
-    rm dumb-init_*.deb
+
 ENTRYPOINT ["/usr/bin/dumb-init", "--"]
 
-RUN curl -o /usr/local/bin/sccache \
+COPY scripts/sccache.sh /scripts/
-      https://s3.amazonaws.com/rust-lang-ci/rust-ci-mirror/2017-05-12-sccache-x86_64-unknown-linux-musl && \
+RUN sh /scripts/sccache.sh
-      chmod +x /usr/local/bin/sccache
 
 ENV RUST_CONFIGURE_ARGS \
       --target=i686-unknown-linux-musl,i586-unknown-linux-gnu \

src/ci/docker/dist-i686-freebsd/Dockerfile

@@ -19,14 +19,13 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
 COPY dist-i686-freebsd/build-toolchain.sh /tmp/
 RUN /tmp/build-toolchain.sh i686
 
-RUN curl -OL https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64.deb && \
+COPY scripts/dumb-init.sh /scripts/
-    dpkg -i dumb-init_*.deb && \
+RUN sh /scripts/dumb-init.sh
-    rm dumb-init_*.deb
+
 ENTRYPOINT ["/usr/bin/dumb-init", "--"]
 
-RUN curl -o /usr/local/bin/sccache \
+COPY scripts/sccache.sh /scripts/
-      https://s3.amazonaws.com/rust-lang-ci/rust-ci-mirror/2017-05-12-sccache-x86_64-unknown-linux-musl && \
+RUN sh /scripts/sccache.sh
-      chmod +x /usr/local/bin/sccache
 
 ENV \
     AR_i686_unknown_freebsd=i686-unknown-freebsd10-ar \

src/ci/docker/dist-i686-linux/Dockerfile

@@ -81,16 +81,16 @@ RUN curl -Lo /rustroot/dumb-init \
       chmod +x /rustroot/dumb-init
 ENTRYPOINT ["/rustroot/dumb-init", "--"]
 
-RUN curl -o /usr/local/bin/sccache \
+COPY scripts/sccache.sh /scripts/
-      https://s3.amazonaws.com/rust-lang-ci/rust-ci-mirror/2017-05-12-sccache-x86_64-unknown-linux-musl && \
+RUN sh /scripts/sccache.sh
-      chmod +x /usr/local/bin/sccache
 
 ENV HOSTS=i686-unknown-linux-gnu
 
 ENV RUST_CONFIGURE_ARGS \
       --host=$HOSTS \
       --enable-extended \
-      --enable-sanitizers
+      --enable-sanitizers \
+      --enable-profiler
 ENV SCRIPT python2.7 ../x.py dist --host $HOSTS --target $HOSTS
 
 # This is the only builder which will create source tarballs

src/ci/docker/dist-mips-linux/Dockerfile

@@ -16,13 +16,13 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
   libssl-dev \
   pkg-config
 
-RUN curl -o /usr/local/bin/sccache \
-      https://s3.amazonaws.com/rust-lang-ci/rust-ci-mirror/2017-05-12-sccache-x86_64-unknown-linux-musl && \
-      chmod +x /usr/local/bin/sccache
 
-RUN curl -OL https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64.deb && \
+COPY scripts/dumb-init.sh /scripts/
-    dpkg -i dumb-init_*.deb && \
+RUN sh /scripts/dumb-init.sh
-    rm dumb-init_*.deb
+
+COPY scripts/sccache.sh /scripts/
+RUN sh /scripts/sccache.sh
+
 ENTRYPOINT ["/usr/bin/dumb-init", "--"]
 
 ENV HOSTS=mips-unknown-linux-gnu

src/ci/docker/dist-mips64-linux/Dockerfile

@@ -16,13 +16,12 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
   libssl-dev \
   pkg-config
 
-RUN curl -o /usr/local/bin/sccache \
+COPY scripts/dumb-init.sh /scripts/
-      https://s3.amazonaws.com/rust-lang-ci/rust-ci-mirror/2017-05-12-sccache-x86_64-unknown-linux-musl && \
+RUN sh /scripts/dumb-init.sh
-      chmod +x /usr/local/bin/sccache
+
+COPY scripts/sccache.sh /scripts/
+RUN sh /scripts/sccache.sh
 
-RUN curl -OL https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64.deb && \
-    dpkg -i dumb-init_*.deb && \
-    rm dumb-init_*.deb
 ENTRYPOINT ["/usr/bin/dumb-init", "--"]
 
 ENV HOSTS=mips64-unknown-linux-gnuabi64

src/ci/docker/dist-mips64el-linux/Dockerfile

@@ -16,13 +16,13 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
   libssl-dev \
   pkg-config
 
-RUN curl -o /usr/local/bin/sccache \
-      https://s3.amazonaws.com/rust-lang-ci/rust-ci-mirror/2017-05-12-sccache-x86_64-unknown-linux-musl && \
-      chmod +x /usr/local/bin/sccache
 
-RUN curl -OL https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64.deb && \
+COPY scripts/dumb-init.sh /scripts/
-    dpkg -i dumb-init_*.deb && \
+RUN sh /scripts/dumb-init.sh
-    rm dumb-init_*.deb
+
+COPY scripts/sccache.sh /scripts/
+RUN sh /scripts/sccache.sh
+
 ENTRYPOINT ["/usr/bin/dumb-init", "--"]
 
 ENV HOSTS=mips64el-unknown-linux-gnuabi64

src/ci/docker/dist-mipsel-linux/Dockerfile

@@ -16,13 +16,12 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
   libssl-dev \
   pkg-config
 
-RUN curl -o /usr/local/bin/sccache \
+COPY scripts/dumb-init.sh /scripts/
-      https://s3.amazonaws.com/rust-lang-ci/rust-ci-mirror/2017-05-12-sccache-x86_64-unknown-linux-musl && \
+RUN sh /scripts/dumb-init.sh
-      chmod +x /usr/local/bin/sccache
+
+COPY scripts/sccache.sh /scripts/
+RUN sh /scripts/sccache.sh
 
-RUN curl -OL https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64.deb && \
-    dpkg -i dumb-init_*.deb && \
-    rm dumb-init_*.deb
 ENTRYPOINT ["/usr/bin/dumb-init", "--"]
 
 ENV HOSTS=mipsel-unknown-linux-gnu

src/ci/docker/dist-powerpc-linux/Dockerfile

@@ -1,58 +1,23 @@
 FROM ubuntu:16.04
 
-RUN apt-get update && apt-get install -y --no-install-recommends \
+COPY scripts/cross-apt-packages.sh /scripts/
-  automake \
+RUN sh /scripts/cross-apt-packages.sh
-  bison \
+
-  bzip2 \
+COPY scripts/dumb-init.sh /scripts/
-  ca-certificates \
+RUN sh /scripts/dumb-init.sh
-  cmake \
-  curl \
-  file \
-  flex \
-  g++ \
-  gawk \
-  gdb \
-  git \
-  gperf \
-  help2man \
-  libncurses-dev \
-  libtool-bin \
-  make \
-  patch \
-  python2.7 \
-  sudo \
-  texinfo \
-  wget \
-  xz-utils \
-  libssl-dev \
-  pkg-config
 
-RUN curl -OL https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64.deb && \
-    dpkg -i dumb-init_*.deb && \
-    rm dumb-init_*.deb
 ENTRYPOINT ["/usr/bin/dumb-init", "--"]
 
-# Ubuntu 16.04 (this contianer) ships with make 4, but something in the
+# Ubuntu 16.04 (this container) ships with make 4, but something in the
 # toolchains we build below chokes on that, so go back to make 3
-RUN curl https://ftp.gnu.org/gnu/make/make-3.81.tar.gz | tar xzf - && \
+COPY scripts/make3.sh /scripts/
-      cd make-3.81 && \
+RUN sh /scripts/make3.sh
-      ./configure --prefix=/usr && \
-      make && \
-      make install && \
-      cd .. && \
-      rm -rf make-3.81
 
-RUN curl http://crosstool-ng.org/download/crosstool-ng/crosstool-ng-1.22.0.tar.bz2 | \
+COPY scripts/crosstool-ng.sh /scripts/
-      tar xjf - && \
+RUN sh /scripts/crosstool-ng.sh
-      cd crosstool-ng && \
-      ./configure --prefix=/usr/local && \
-      make -j$(nproc) && \
-      make install && \
-      cd .. && \
-      rm -rf crosstool-ng
 
-RUN groupadd -r rustbuild && useradd -m -r -g rustbuild rustbuild
+COPY scripts/rustbuild-setup.sh /scripts/
-RUN mkdir /x-tools && chown rustbuild:rustbuild /x-tools
+RUN sh /scripts/rustbuild-setup.sh
 USER rustbuild
 WORKDIR /tmp
 
@@ -62,9 +27,8 @@ RUN ./build-powerpc-toolchain.sh
 
 USER root
 
-RUN curl -o /usr/local/bin/sccache \
+COPY scripts/sccache.sh /scripts/
-      https://s3.amazonaws.com/rust-lang-ci/rust-ci-mirror/2017-05-12-sccache-x86_64-unknown-linux-musl && \
+RUN sh /scripts/sccache.sh
-      chmod +x /usr/local/bin/sccache
 
 ENV PATH=$PATH:/x-tools/powerpc-unknown-linux-gnu/bin
 

src/ci/docker/dist-powerpc64-linux/Dockerfile

@@ -1,58 +1,23 @@
 FROM ubuntu:16.04
 
-RUN apt-get update && apt-get install -y --no-install-recommends \
+COPY scripts/cross-apt-packages.sh /scripts/
-  automake \
+RUN sh /scripts/cross-apt-packages.sh
-  bison \
+
-  bzip2 \
+COPY scripts/dumb-init.sh /scripts/
-  ca-certificates \
+RUN sh /scripts/dumb-init.sh
-  cmake \
-  curl \
-  file \
-  flex \
-  g++ \
-  gawk \
-  gdb \
-  git \
-  gperf \
-  help2man \
-  libncurses-dev \
-  libtool-bin \
-  make \
-  patch \
-  python2.7 \
-  sudo \
-  texinfo \
-  wget \
-  xz-utils \
-  libssl-dev \
- pkg-config
 
-RUN curl -OL https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64.deb && \
-    dpkg -i dumb-init_*.deb && \
-    rm dumb-init_*.deb
 ENTRYPOINT ["/usr/bin/dumb-init", "--"]
 
-# Ubuntu 16.04 (this contianer) ships with make 4, but something in the
+# Ubuntu 16.04 (this container) ships with make 4, but something in the
 # toolchains we build below chokes on that, so go back to make 3
-RUN curl https://ftp.gnu.org/gnu/make/make-3.81.tar.gz | tar xzf - && \
+COPY scripts/make3.sh /scripts/
-      cd make-3.81 && \
+RUN sh /scripts/make3.sh
-      ./configure --prefix=/usr && \
-      make && \
-      make install && \
-      cd .. && \
-      rm -rf make-3.81
 
-RUN curl http://crosstool-ng.org/download/crosstool-ng/crosstool-ng-1.22.0.tar.bz2 | \
+COPY scripts/crosstool-ng.sh /scripts/
-      tar xjf - && \
+RUN sh /scripts/crosstool-ng.sh
-      cd crosstool-ng && \
-      ./configure --prefix=/usr/local && \
-      make -j$(nproc) && \
-      make install && \
-      cd .. && \
-      rm -rf crosstool-ng
 
-RUN groupadd -r rustbuild && useradd -m -r -g rustbuild rustbuild
+COPY scripts/rustbuild-setup.sh /scripts/
-RUN mkdir /x-tools && chown rustbuild:rustbuild /x-tools
+RUN sh /scripts/rustbuild-setup.sh
 USER rustbuild
 WORKDIR /tmp
 
@@ -62,9 +27,8 @@ RUN ./build-powerpc64-toolchain.sh
 
 USER root
 
-RUN curl -o /usr/local/bin/sccache \
+COPY scripts/sccache.sh /scripts/
-      https://s3.amazonaws.com/rust-lang-ci/rust-ci-mirror/2017-05-12-sccache-x86_64-unknown-linux-musl && \
+RUN sh /scripts/sccache.sh
-      chmod +x /usr/local/bin/sccache
 
 ENV PATH=$PATH:/x-tools/powerpc64-unknown-linux-gnu/bin
 

src/ci/docker/dist-powerpc64le-linux/Dockerfile

@@ -1,58 +1,23 @@
 FROM ubuntu:16.04
 
-RUN apt-get update && apt-get install -y --no-install-recommends \
+COPY scripts/cross-apt-packages.sh /scripts/
-  automake \
+RUN sh /scripts/cross-apt-packages.sh
-  bison \
+
-  bzip2 \
+COPY scripts/dumb-init.sh /scripts/
-  ca-certificates \
+RUN sh /scripts/dumb-init.sh
-  cmake \
-  curl \
-  file \
-  flex \
-  g++ \
-  gawk \
-  gdb \
-  git \
-  gperf \
-  help2man \
-  libncurses-dev \
-  libtool-bin \
-  make \
-  patch \
-  python2.7 \
-  sudo \
-  texinfo \
-  wget \
-  xz-utils \
-  libssl-dev \
- pkg-config
 
-RUN curl -OL https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64.deb && \
-    dpkg -i dumb-init_*.deb && \
-    rm dumb-init_*.deb
 ENTRYPOINT ["/usr/bin/dumb-init", "--"]
 
-# Ubuntu 16.04 (this contianer) ships with make 4, but something in the
+# Ubuntu 16.04 (this container) ships with make 4, but something in the
 # toolchains we build below chokes on that, so go back to make 3
-RUN curl https://ftp.gnu.org/gnu/make/make-3.81.tar.gz | tar xzf - && \
+COPY scripts/make3.sh /scripts/
-      cd make-3.81 && \
+RUN sh /scripts/make3.sh
-      ./configure --prefix=/usr && \
-      make && \
-      make install && \
-      cd .. && \
-      rm -rf make-3.81
 
-RUN curl http://crosstool-ng.org/download/crosstool-ng/crosstool-ng-1.22.0.tar.bz2 | \
+COPY scripts/crosstool-ng.sh /scripts/
-      tar xjf - && \
+RUN sh /scripts/crosstool-ng.sh
-      cd crosstool-ng && \
-      ./configure --prefix=/usr/local && \
-      make -j$(nproc) && \
-      make install && \
-      cd .. && \
-      rm -rf crosstool-ng
 
-RUN groupadd -r rustbuild && useradd -m -r -g rustbuild rustbuild
+COPY scripts/rustbuild-setup.sh /scripts/
-RUN mkdir /x-tools && chown rustbuild:rustbuild /x-tools
+RUN sh /scripts/rustbuild-setup.sh
 USER rustbuild
 WORKDIR /tmp
 
@@ -62,9 +27,8 @@ RUN apt-get install -y --no-install-recommends rpm2cpio cpio
 COPY dist-powerpc64le-linux/shared.sh dist-powerpc64le-linux/build-powerpc64le-toolchain.sh /tmp/
 RUN ./build-powerpc64le-toolchain.sh
 
-RUN curl -o /usr/local/bin/sccache \
+COPY scripts/sccache.sh /scripts/
-      https://s3.amazonaws.com/rust-lang-ci/rust-ci-mirror/2017-05-12-sccache-x86_64-unknown-linux-musl && \
+RUN sh /scripts/sccache.sh
-      chmod +x /usr/local/bin/sccache
 
 ENV \
     AR_powerpc64le_unknown_linux_gnu=powerpc64le-linux-gnu-ar \

src/ci/docker/dist-s390x-linux/Dockerfile

@@ -1,58 +1,23 @@
 FROM ubuntu:16.04
 
-RUN apt-get update && apt-get install -y --no-install-recommends \
+COPY scripts/cross-apt-packages.sh /scripts/
-  automake \
+RUN sh /scripts/cross-apt-packages.sh
-  bison \
+
-  bzip2 \
+COPY scripts/dumb-init.sh /scripts/
-  ca-certificates \
+RUN sh /scripts/dumb-init.sh
-  cmake \
-  curl \
-  file \
-  flex \
-  g++ \
-  gawk \
-  gdb \
-  git \
-  gperf \
-  help2man \
-  libncurses-dev \
-  libtool-bin \
-  make \
-  patch \
-  python2.7 \
-  sudo \
-  texinfo \
-  wget \
-  xz-utils \
-  libssl-dev \
-  pkg-config
 
-RUN curl -OL https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64.deb && \
-    dpkg -i dumb-init_*.deb && \
-    rm dumb-init_*.deb
 ENTRYPOINT ["/usr/bin/dumb-init", "--"]
 
-# Ubuntu 16.04 (this contianer) ships with make 4, but something in the
+# Ubuntu 16.04 (this container) ships with make 4, but something in the
 # toolchains we build below chokes on that, so go back to make 3
-RUN curl https://ftp.gnu.org/gnu/make/make-3.81.tar.gz | tar xzf - && \
+COPY scripts/make3.sh /scripts/
-      cd make-3.81 && \
+RUN sh /scripts/make3.sh
-      ./configure --prefix=/usr && \
-      make && \
-      make install && \
-      cd .. && \
-      rm -rf make-3.81
 
-RUN curl http://crosstool-ng.org/download/crosstool-ng/crosstool-ng-1.22.0.tar.bz2 | \
+COPY scripts/crosstool-ng.sh /scripts/
-      tar xjf - && \
+RUN sh /scripts/crosstool-ng.sh
-      cd crosstool-ng && \
-      ./configure --prefix=/usr/local && \
-      make -j$(nproc) && \
-      make install && \
-      cd .. && \
-      rm -rf crosstool-ng
 
-RUN groupadd -r rustbuild && useradd -m -r -g rustbuild rustbuild
+COPY scripts/rustbuild-setup.sh /scripts/
-RUN mkdir /x-tools && chown rustbuild:rustbuild /x-tools
+RUN sh /scripts/rustbuild-setup.sh
 USER rustbuild
 WORKDIR /tmp
 
@@ -62,9 +27,8 @@ RUN ./build-s390x-toolchain.sh
 
 USER root
 
-RUN curl -o /usr/local/bin/sccache \
+COPY scripts/sccache.sh /scripts/
-      https://s3.amazonaws.com/rust-lang-ci/rust-ci-mirror/2017-05-12-sccache-x86_64-unknown-linux-musl && \
+RUN sh /scripts/sccache.sh
-      chmod +x /usr/local/bin/sccache
 
 ENV PATH=$PATH:/x-tools/s390x-ibm-linux-gnu/bin
 

src/ci/docker/dist-x86_64-freebsd/Dockerfile

@@ -19,14 +19,13 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
 COPY dist-x86_64-freebsd/build-toolchain.sh /tmp/
 RUN /tmp/build-toolchain.sh x86_64
 
-RUN curl -OL https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64.deb && \
+COPY scripts/dumb-init.sh /scripts/
-    dpkg -i dumb-init_*.deb && \
+RUN sh /scripts/dumb-init.sh
-    rm dumb-init_*.deb
+
 ENTRYPOINT ["/usr/bin/dumb-init", "--"]
 
-RUN curl -o /usr/local/bin/sccache \
+COPY scripts/sccache.sh /scripts/
-      https://s3.amazonaws.com/rust-lang-ci/rust-ci-mirror/2017-05-12-sccache-x86_64-unknown-linux-musl && \
+RUN sh /scripts/sccache.sh
-      chmod +x /usr/local/bin/sccache
 
 ENV \
     AR_x86_64_unknown_freebsd=x86_64-unknown-freebsd10-ar \

src/ci/docker/dist-x86_64-linux/Dockerfile

@@ -81,16 +81,16 @@ RUN curl -Lo /rustroot/dumb-init \
       chmod +x /rustroot/dumb-init
 ENTRYPOINT ["/rustroot/dumb-init", "--"]
 
-RUN curl -o /usr/local/bin/sccache \
+COPY scripts/sccache.sh /scripts/
-      https://s3.amazonaws.com/rust-lang-ci/rust-ci-mirror/2017-05-12-sccache-x86_64-unknown-linux-musl && \
+RUN sh /scripts/sccache.sh
-      chmod +x /usr/local/bin/sccache
 
 ENV HOSTS=x86_64-unknown-linux-gnu
 
 ENV RUST_CONFIGURE_ARGS \
       --host=$HOSTS \
       --enable-extended \
-      --enable-sanitizers
+      --enable-sanitizers \
+      --enable-profiler
 ENV SCRIPT python2.7 ../x.py dist --host $HOSTS --target $HOSTS
 
 # This is the only builder which will create source tarballs

src/ci/docker/dist-x86_64-musl/Dockerfile

@@ -20,14 +20,13 @@ WORKDIR /build/
 COPY dist-x86_64-musl/build-musl.sh /build/
 RUN sh /build/build-musl.sh && rm -rf /build
 
-RUN curl -OL https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64.deb && \
+COPY scripts/dumb-init.sh /scripts/
-    dpkg -i dumb-init_*.deb && \
+RUN sh /scripts/dumb-init.sh
-    rm dumb-init_*.deb
+
 ENTRYPOINT ["/usr/bin/dumb-init", "--"]
 
-RUN curl -o /usr/local/bin/sccache \
+COPY scripts/sccache.sh /scripts/
-      https://s3.amazonaws.com/rust-lang-ci/rust-ci-mirror/2017-05-12-sccache-x86_64-unknown-linux-musl && \
+RUN sh /scripts/sccache.sh
-      chmod +x /usr/local/bin/sccache
 
 ENV RUST_CONFIGURE_ARGS \
       --target=x86_64-unknown-linux-musl \

src/ci/docker/dist-x86_64-netbsd/Dockerfile

@@ -1,58 +1,23 @@
 FROM ubuntu:16.04
 
-RUN apt-get update && apt-get install -y --no-install-recommends \
+COPY scripts/cross-apt-packages.sh /scripts/
-  automake \
+RUN sh /scripts/cross-apt-packages.sh
-  bison \
+
-  bzip2 \
+COPY scripts/dumb-init.sh /scripts/
-  ca-certificates \
+RUN sh /scripts/dumb-init.sh
-  cmake \
-  curl \
-  file \
-  flex \
-  g++ \
-  gawk \
-  gdb \
-  git \
-  gperf \
-  help2man \
-  libncurses-dev \
-  libtool-bin \
-  make \
-  patch \
-  python2.7 \
-  sudo \
-  texinfo \
-  wget \
-  xz-utils \
-  libssl-dev \
-  pkg-config
 
-RUN curl -OL https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64.deb && \
-    dpkg -i dumb-init_*.deb && \
-    rm dumb-init_*.deb
 ENTRYPOINT ["/usr/bin/dumb-init", "--"]
 
-# Ubuntu 16.04 (this contianer) ships with make 4, but something in the
+# Ubuntu 16.04 (this container) ships with make 4, but something in the
 # toolchains we build below chokes on that, so go back to make 3
-RUN curl https://ftp.gnu.org/gnu/make/make-3.81.tar.gz | tar xzf - && \
+COPY scripts/make3.sh /scripts/
-      cd make-3.81 && \
+RUN sh /scripts/make3.sh
-      ./configure --prefix=/usr && \
-      make && \
-      make install && \
-      cd .. && \
-      rm -rf make-3.81
 
-RUN curl http://crosstool-ng.org/download/crosstool-ng/crosstool-ng-1.22.0.tar.bz2 | \
+COPY scripts/crosstool-ng.sh /scripts/
-      tar xjf - && \
+RUN sh /scripts/crosstool-ng.sh
-      cd crosstool-ng && \
-      ./configure --prefix=/usr/local && \
-      make -j$(nproc) && \
-      make install && \
-      cd .. && \
-      rm -rf crosstool-ng
 
-RUN groupadd -r rustbuild && useradd -m -r -g rustbuild rustbuild
+COPY scripts/rustbuild-setup.sh /scripts/
-RUN mkdir /x-tools && chown rustbuild:rustbuild /x-tools
+RUN sh /scripts/rustbuild-setup.sh
 USER rustbuild
 WORKDIR /tmp
 
@@ -61,9 +26,8 @@ RUN ./build-netbsd-toolchain.sh
 
 USER root
 
-RUN curl -o /usr/local/bin/sccache \
+COPY scripts/sccache.sh /scripts/
-      https://s3.amazonaws.com/rust-lang-ci/rust-ci-mirror/2017-05-12-sccache-x86_64-unknown-linux-musl && \
+RUN sh /scripts/sccache.sh
-      chmod +x /usr/local/bin/sccache
 
 ENV PATH=$PATH:/x-tools/x86_64-unknown-netbsd/bin
 

src/ci/docker/emscripten/Dockerfile

@@ -1,41 +0,0 @@
-FROM ubuntu:16.04
-
-RUN apt-get update && apt-get install -y --no-install-recommends \
-  g++ \
-  make \
-  file \
-  curl \
-  ca-certificates \
-  python \
-  git \
-  cmake \
-  sudo \
-  gdb \
-  xz-utils \
-  lib32stdc++6
-
-RUN curl -o /usr/local/bin/sccache \
-      https://s3.amazonaws.com/rust-lang-ci/rust-ci-mirror/2017-05-12-sccache-x86_64-unknown-linux-musl && \
-      chmod +x /usr/local/bin/sccache
-
-RUN curl -OL https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64.deb && \
-    dpkg -i dumb-init_*.deb && \
-    rm dumb-init_*.deb
-ENTRYPOINT ["/usr/bin/dumb-init", "--"]
-
-WORKDIR /tmp
-COPY emscripten/build-emscripten.sh /tmp/
-RUN ./build-emscripten.sh
-ENV PATH=$PATH:/tmp/emsdk_portable
-ENV PATH=$PATH:/tmp/emsdk_portable/clang/tag-e1.37.10/build_tag-e1.37.10_32/bin
-ENV PATH=$PATH:/tmp/emsdk_portable/node/4.1.1_32bit/bin
-ENV PATH=$PATH:/tmp/emsdk_portable/emscripten/tag-1.37.10
-ENV EMSCRIPTEN=/tmp/emsdk_portable/emscripten/tag-1.37.10
-
-ENV RUST_CONFIGURE_ARGS --target=asmjs-unknown-emscripten
-
-# Run `emcc` first as it's got a prompt and doesn't actually do anything, after
-# that's done with do the real build.
-ENV SCRIPT emcc && \
-      python2.7 ../x.py test --target asmjs-unknown-emscripten
-

src/ci/docker/emscripten/build-emscripten.sh

@@ -1,53 +0,0 @@
-#!/bin/bash
-# Copyright 2017 The Rust Project Developers. See the COPYRIGHT
-# file at the top-level directory of this distribution and at
-# http://rust-lang.org/COPYRIGHT.
-#
-# Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
-# http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
-# <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
-# option. This file may not be copied, modified, or distributed
-# except according to those terms.
-
-set -ex
-
-hide_output() {
-  set +x
-  on_err="
-echo ERROR: An error was encountered with the build.
-cat /tmp/build.log
-exit 1
-"
-  trap "$on_err" ERR
-  bash -c "while true; do sleep 30; echo \$(date) - building ...; done" &
-  PING_LOOP_PID=$!
-  $@ &> /tmp/build.log
-  trap - ERR
-  kill $PING_LOOP_PID
-  rm /tmp/build.log
-  set -x
-}
-
-curl https://s3.amazonaws.com/mozilla-games/emscripten/releases/emsdk-portable.tar.gz | \
-    tar xzf -
-
-# Some versions of the EMSDK archive have their contents in .emsdk-portable
-# and others in emsdk_portable. Make sure the EMSDK ends up in a fixed path.
-if [ -d emsdk-portable ]; then
-    mv emsdk-portable emsdk_portable
-fi
-
-if [ ! -d emsdk_portable ]; then
-    echo "ERROR: Invalid emsdk archive. Dumping working directory." >&2
-    ls -l
-    exit 1
-fi
-
-# Some versions of the EMSDK set the permissions of the root directory to
-# 0700. Ensure the directory is readable by all users.
-chmod 755 emsdk_portable
-
-source emsdk_portable/emsdk_env.sh
-hide_output emsdk update
-hide_output emsdk install --build=Release sdk-tag-1.37.10-32bit
-hide_output emsdk activate --build=Release sdk-tag-1.37.10-32bit

src/ci/docker/i686-gnu-nopt/Dockerfile

@@ -13,13 +13,13 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
   gdb \
   xz-utils
 
-RUN curl -o /usr/local/bin/sccache \
-      https://s3.amazonaws.com/rust-lang-ci/rust-ci-mirror/2017-05-12-sccache-x86_64-unknown-linux-musl && \
-      chmod +x /usr/local/bin/sccache
 
-RUN curl -OL https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64.deb && \
+COPY scripts/dumb-init.sh /scripts/
-    dpkg -i dumb-init_*.deb && \
+RUN sh /scripts/dumb-init.sh
-    rm dumb-init_*.deb
+
+COPY scripts/sccache.sh /scripts/
+RUN sh /scripts/sccache.sh
+
 ENTRYPOINT ["/usr/bin/dumb-init", "--"]
 
 ENV RUST_CONFIGURE_ARGS --build=i686-unknown-linux-gnu --disable-optimize-tests

src/ci/docker/i686-gnu/Dockerfile

@@ -13,13 +13,13 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
   gdb \
   xz-utils
 
-RUN curl -o /usr/local/bin/sccache \
-      https://s3.amazonaws.com/rust-lang-ci/rust-ci-mirror/2017-05-12-sccache-x86_64-unknown-linux-musl && \
-      chmod +x /usr/local/bin/sccache
 
-RUN curl -OL https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64.deb && \
+COPY scripts/dumb-init.sh /scripts/
-    dpkg -i dumb-init_*.deb && \
+RUN sh /scripts/dumb-init.sh
-    rm dumb-init_*.deb
+
+COPY scripts/sccache.sh /scripts/
+RUN sh /scripts/sccache.sh
+
 ENTRYPOINT ["/usr/bin/dumb-init", "--"]
 
 ENV RUST_CONFIGURE_ARGS --build=i686-unknown-linux-gnu

src/ci/docker/run.sh

@@ -82,7 +82,6 @@ exec docker \
   --env TRAVIS_BRANCH \
   --volume "$HOME/.cargo:/cargo" \
   --volume "$HOME/rustsrc:$HOME/rustsrc" \
-  --privileged \
   --rm \
   rust-ci \
   /checkout/src/ci/run.sh

src/ci/docker/scripts/android-base-apt-get.sh

@@ -0,0 +1,27 @@
+# Copyright 2017 The Rust Project Developers. See the COPYRIGHT
+# file at the top-level directory of this distribution and at
+# http://rust-lang.org/COPYRIGHT.
+#
+# Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
+# http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
+# <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
+# option. This file may not be copied, modified, or distributed
+# except according to those terms.
+
+set -ex
+
+apt-get update
+apt-get install -y --no-install-recommends \
+  ca-certificates \
+  cmake \
+  curl \
+  file \
+  g++ \
+  git \
+  libssl-dev \
+  make \
+  pkg-config \
+  python2.7 \
+  sudo \
+  unzip \
+  xz-utils

src/ci/docker/scripts/cross-apt-packages.sh

@@ -0,0 +1,36 @@
+# Copyright 2017 The Rust Project Developers. See the COPYRIGHT
+# file at the top-level directory of this distribution and at
+# http://rust-lang.org/COPYRIGHT.
+#
+# Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
+# http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
+# <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
+# option. This file may not be copied, modified, or distributed
+# except according to those terms.
+
+apt-get update && apt-get install -y --no-install-recommends \
+  automake \
+  bison \
+  bzip2 \
+  ca-certificates \
+  cmake \
+  curl \
+  file \
+  flex \
+  g++ \
+  gawk \
+  gdb \
+  git \
+  gperf \
+  help2man \
+  libncurses-dev \
+  libssl-dev \
+  libtool-bin \
+  make \
+  patch \
+  pkg-config \
+  python2.7 \
+  sudo \
+  texinfo \
+  wget \
+  xz-utils

src/ci/docker/scripts/crosstool-ng.sh

@@ -0,0 +1,20 @@
+# Copyright 2017 The Rust Project Developers. See the COPYRIGHT
+# file at the top-level directory of this distribution and at
+# http://rust-lang.org/COPYRIGHT.
+#
+# Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
+# http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
+# <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
+# option. This file may not be copied, modified, or distributed
+# except according to those terms.
+
+set -ex
+
+url="http://crosstool-ng.org/download/crosstool-ng/crosstool-ng-1.22.0.tar.bz2"
+curl $url | tar xjf -
+cd crosstool-ng
+./configure --prefix=/usr/local
+make -j$(nproc)
+make install
+cd ..
+rm -rf crosstool-ng

src/ci/docker/scripts/emscripten-wasm.sh

@@ -0,0 +1,47 @@
+# Copyright 2017 The Rust Project Developers. See the COPYRIGHT
+# file at the top-level directory of this distribution and at
+# http://rust-lang.org/COPYRIGHT.
+#
+# Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
+# http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
+# <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
+# option. This file may not be copied, modified, or distributed
+# except according to those terms.
+
+set -ex
+
+hide_output() {
+  set +x
+  on_err="
+echo ERROR: An error was encountered with the build.
+cat /tmp/build.log
+exit 1
+"
+  trap "$on_err" ERR
+  bash -c "while true; do sleep 30; echo \$(date) - building ...; done" &
+  PING_LOOP_PID=$!
+  $@ &> /tmp/build.log
+  trap - ERR
+  kill $PING_LOOP_PID
+  rm -f /tmp/build.log
+  set -x
+}
+
+# Download last known good emscripten from WebAssembly waterfall
+BUILD=$(curl -L https://storage.googleapis.com/wasm-llvm/builds/linux/lkgr.json | \
+    jq '.build | tonumber')
+curl -L https://storage.googleapis.com/wasm-llvm/builds/linux/$BUILD/wasm-binaries.tbz2 | \
+    hide_output tar xvkj
+
+# node 8 is required to run wasm
+cd /
+curl -L https://nodejs.org/dist/v8.0.0/node-v8.0.0-linux-x64.tar.xz | \
+    tar -xJ
+
+# Make emscripten use wasm-ready node and LLVM tools
+echo "EMSCRIPTEN_ROOT = '/wasm-install/emscripten'" >> /root/.emscripten
+echo "NODE_JS='/usr/local/bin/node'" >> /root/.emscripten
+echo "LLVM_ROOT='/wasm-install/bin'" >> /root/.emscripten
+echo "BINARYEN_ROOT = '/wasm-install'" >> /root/.emscripten
+echo "COMPILER_ENGINE = NODE_JS" >> /root/.emscripten
+echo "JS_ENGINES = [NODE_JS]" >> /root/.emscripten

src/ci/docker/scripts/emscripten.sh

@@ -0,0 +1,53 @@
+# Copyright 2017 The Rust Project Developers. See the COPYRIGHT
+# file at the top-level directory of this distribution and at
+# http://rust-lang.org/COPYRIGHT.
+#
+# Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
+# http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
+# <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
+# option. This file may not be copied, modified, or distributed
+# except according to those terms.
+
+set -ex
+
+hide_output() {
+  set +x
+  on_err="
+echo ERROR: An error was encountered with the build.
+cat /tmp/build.log
+exit 1
+"
+  trap "$on_err" ERR
+  bash -c "while true; do sleep 30; echo \$(date) - building ...; done" &
+  PING_LOOP_PID=$!
+  $@ &> /tmp/build.log
+  trap - ERR
+  kill $PING_LOOP_PID
+  rm -f /tmp/build.log
+  set -x
+}
+
+cd /
+curl -L https://s3.amazonaws.com/mozilla-games/emscripten/releases/emsdk-portable.tar.gz | \
+    tar -xz
+
+cd /emsdk-portable
+./emsdk update
+hide_output ./emsdk install sdk-1.37.13-64bit
+./emsdk activate sdk-1.37.13-64bit
+
+# Compile and cache libc
+source ./emsdk_env.sh
+echo "main(){}" > a.c
+HOME=/emsdk-portable/ emcc a.c
+HOME=/emsdk-portable/ emcc -s BINARYEN=1 a.c
+rm -f a.*
+
+# Make emsdk usable by any user
+cp /root/.emscripten /emsdk-portable
+chmod a+rxw -R /emsdk-portable
+
+# node 8 is required to run wasm
+cd /
+curl -L https://nodejs.org/dist/v8.0.0/node-v8.0.0-linux-x64.tar.xz | \
+  tar -xJ

src/ci/docker/scripts/make3.sh

@@ -0,0 +1,19 @@
+# Copyright 2017 The Rust Project Developers. See the COPYRIGHT
+# file at the top-level directory of this distribution and at
+# http://rust-lang.org/COPYRIGHT.
+#
+# Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
+# http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
+# <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
+# option. This file may not be copied, modified, or distributed
+# except according to those terms.
+
+set -ex
+
+curl https://ftp.gnu.org/gnu/make/make-3.81.tar.gz | tar xzf -
+cd make-3.81
+./configure --prefix=/usr
+make
+make install
+cd ..
+rm -rf make-3.81

src/ci/docker/scripts/rustbuild-setup.sh

@@ -0,0 +1,14 @@
+# Copyright 2017 The Rust Project Developers. See the COPYRIGHT
+# file at the top-level directory of this distribution and at
+# http://rust-lang.org/COPYRIGHT.
+#
+# Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
+# http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
+# <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
+# option. This file may not be copied, modified, or distributed
+# except according to those terms.
+
+set -ex
+
+groupadd -r rustbuild && useradd -m -r -g rustbuild rustbuild
+mkdir /x-tools && chown rustbuild:rustbuild /x-tools

src/ci/docker/x86_64-gnu-aux/Dockerfile

@@ -14,13 +14,12 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
   xz-utils \
   pkg-config
 
-RUN curl -o /usr/local/bin/sccache \
+COPY scripts/dumb-init.sh /scripts/
-      https://s3.amazonaws.com/rust-lang-ci/rust-ci-mirror/2017-05-12-sccache-x86_64-unknown-linux-musl && \
+RUN sh /scripts/dumb-init.sh
-      chmod +x /usr/local/bin/sccache
+
+COPY scripts/sccache.sh /scripts/
+RUN sh /scripts/sccache.sh
 
-RUN curl -OL https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64.deb && \
-    dpkg -i dumb-init_*.deb && \
-    rm dumb-init_*.deb
 ENTRYPOINT ["/usr/bin/dumb-init", "--"]
 
 ENV RUST_CONFIGURE_ARGS --build=x86_64-unknown-linux-gnu

src/ci/docker/x86_64-gnu-debug/Dockerfile

@@ -13,13 +13,12 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
   gdb \
   xz-utils
 
-RUN curl -o /usr/local/bin/sccache \
+COPY scripts/dumb-init.sh /scripts/
-      https://s3.amazonaws.com/rust-lang-ci/rust-ci-mirror/2017-05-12-sccache-x86_64-unknown-linux-musl && \
+RUN sh /scripts/dumb-init.sh
-      chmod +x /usr/local/bin/sccache
+
+COPY scripts/sccache.sh /scripts/
+RUN sh /scripts/sccache.sh
 
-RUN curl -OL https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64.deb && \
-    dpkg -i dumb-init_*.deb && \
-    rm dumb-init_*.deb
 ENTRYPOINT ["/usr/bin/dumb-init", "--"]
 
 ENV RUST_CONFIGURE_ARGS \

src/ci/docker/x86_64-gnu-distcheck/Dockerfile

@@ -15,13 +15,12 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
   libssl-dev \
   pkg-config
 
-RUN curl -o /usr/local/bin/sccache \
+COPY scripts/dumb-init.sh /scripts/
-      https://s3.amazonaws.com/rust-lang-ci/rust-ci-mirror/2017-05-12-sccache-x86_64-unknown-linux-musl && \
+RUN sh /scripts/dumb-init.sh
-      chmod +x /usr/local/bin/sccache
+
+COPY scripts/sccache.sh /scripts/
+RUN sh /scripts/sccache.sh
 
-RUN curl -OL https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64.deb && \
-    dpkg -i dumb-init_*.deb && \
-    rm dumb-init_*.deb
 ENTRYPOINT ["/usr/bin/dumb-init", "--"]
 
 ENV RUST_CONFIGURE_ARGS --build=x86_64-unknown-linux-gnu

src/ci/docker/x86_64-gnu-full-bootstrap/Dockerfile

@@ -13,13 +13,12 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
   gdb \
   xz-utils
 
-RUN curl -o /usr/local/bin/sccache \
+COPY scripts/dumb-init.sh /scripts/
-      https://s3.amazonaws.com/rust-lang-ci/rust-ci-mirror/2017-05-12-sccache-x86_64-unknown-linux-musl && \
+RUN sh /scripts/dumb-init.sh
-      chmod +x /usr/local/bin/sccache
+
+COPY scripts/sccache.sh /scripts/
+RUN sh /scripts/sccache.sh
 
-RUN curl -OL https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64.deb && \
-    dpkg -i dumb-init_*.deb && \
-    rm dumb-init_*.deb
 ENTRYPOINT ["/usr/bin/dumb-init", "--"]
 
 ENV RUST_CONFIGURE_ARGS \

src/ci/docker/x86_64-gnu-incremental/Dockerfile

@@ -13,13 +13,12 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
   gdb \
   xz-utils
 
-RUN curl -o /usr/local/bin/sccache \
+COPY scripts/dumb-init.sh /scripts/
-      https://s3.amazonaws.com/rust-lang-ci/rust-ci-mirror/2017-05-12-sccache-x86_64-unknown-linux-musl && \
+RUN sh /scripts/dumb-init.sh
-      chmod +x /usr/local/bin/sccache
+
+COPY scripts/sccache.sh /scripts/
+RUN sh /scripts/sccache.sh
 
-RUN curl -OL https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64.deb && \
-    dpkg -i dumb-init_*.deb && \
-    rm dumb-init_*.deb
 ENTRYPOINT ["/usr/bin/dumb-init", "--"]
 
 ENV RUST_CONFIGURE_ARGS --build=x86_64-unknown-linux-gnu

src/ci/docker/x86_64-gnu-llvm-3.7/Dockerfile

@@ -16,13 +16,12 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
   zlib1g-dev \
   xz-utils
 
-RUN curl -o /usr/local/bin/sccache \
+COPY scripts/dumb-init.sh /scripts/
-      https://s3.amazonaws.com/rust-lang-ci/rust-ci-mirror/2017-05-12-sccache-x86_64-unknown-linux-musl && \
+RUN sh /scripts/dumb-init.sh
-      chmod +x /usr/local/bin/sccache
+
+COPY scripts/sccache.sh /scripts/
+RUN sh /scripts/sccache.sh
 
-RUN curl -OL https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64.deb && \
-    dpkg -i dumb-init_*.deb && \
-    rm dumb-init_*.deb
 ENTRYPOINT ["/usr/bin/dumb-init", "--"]
 
 ENV RUST_CONFIGURE_ARGS \

src/ci/docker/x86_64-gnu-nopt/Dockerfile

@@ -13,13 +13,12 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
   gdb \
   xz-utils
 
-RUN curl -o /usr/local/bin/sccache \
+COPY scripts/dumb-init.sh /scripts/
-      https://s3.amazonaws.com/rust-lang-ci/rust-ci-mirror/2017-05-12-sccache-x86_64-unknown-linux-musl && \
+RUN sh /scripts/dumb-init.sh
-      chmod +x /usr/local/bin/sccache
+
+COPY scripts/sccache.sh /scripts/
+RUN sh /scripts/sccache.sh
 
-RUN curl -OL https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64.deb && \
-    dpkg -i dumb-init_*.deb && \
-    rm dumb-init_*.deb
 ENTRYPOINT ["/usr/bin/dumb-init", "--"]
 
 ENV RUST_CONFIGURE_ARGS --build=x86_64-unknown-linux-gnu --disable-optimize-tests

src/ci/docker/x86_64-gnu/Dockerfile

@@ -13,14 +13,13 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
   gdb \
   xz-utils
 
-RUN curl -o /usr/local/bin/sccache \
+COPY scripts/dumb-init.sh /scripts/
-      https://s3.amazonaws.com/rust-lang-ci/rust-ci-mirror/2017-05-12-sccache-x86_64-unknown-linux-musl && \
+RUN sh /scripts/dumb-init.sh
-      chmod +x /usr/local/bin/sccache
+
+COPY scripts/sccache.sh /scripts/
+RUN sh /scripts/sccache.sh
 
-RUN curl -OL https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64.deb && \
-    dpkg -i dumb-init_*.deb && \
-    rm dumb-init_*.deb
 ENTRYPOINT ["/usr/bin/dumb-init", "--"]
 
-ENV RUST_CONFIGURE_ARGS --build=x86_64-unknown-linux-gnu --enable-sanitizers
+ENV RUST_CONFIGURE_ARGS --build=x86_64-unknown-linux-gnu --enable-sanitizers --enable-profiler
 ENV SCRIPT python2.7 ../x.py test

src/ci/init_repo.sh

@@ -22,9 +22,6 @@ REPO_DIR="$1"
 CACHE_DIR="$2"
 
 cache_src_dir="$CACHE_DIR/src"
-# If the layout of the cache directory changes, bump the number here
-# (and anywhere else this file is referenced) so the cache is wiped
-cache_valid_file="$CACHE_DIR/cache_valid1"
 
 if [ ! -d "$REPO_DIR" -o ! -d "$REPO_DIR/.git" ]; then
     echo "Error: $REPO_DIR does not exist or is not a git repo"
@@ -36,47 +33,19 @@ if [ ! -d "$CACHE_DIR" ]; then
     exit 1
 fi
 
-# Wipe the cache if it's not valid, or mark it as invalid while we update it
+rm -rf "$CACHE_DIR"
-if [ ! -f "$cache_valid_file" ]; then
+mkdir "$CACHE_DIR"
-    echo "Invalid cache, wiping ($cache_valid_file missing)"
-    rm -rf "$CACHE_DIR"
-    mkdir "$CACHE_DIR"
-else
-    # Ignore errors while gathering information about the possible brokenness
-    # of the git repo since our gathered info will tell us something is wrong
-    set +o errexit
-    stat_lines=$(cd "$cache_src_dir" && git status --porcelain | wc -l)
-    stat_ec=$(cd "$cache_src_dir" && git status >/dev/null 2>&1; echo $?)
-    set -o errexit
-    if [ ! -d "$cache_src_dir/.git" -o $stat_lines != 0 -o $stat_ec != 0 ]; then
-        # Something is badly wrong - the cache valid file is here, but something
-        # about the git repo is fishy. Nuke it all, just in case
-        echo "WARNING: $cache_valid_file exists but bad repo: l:$stat_lines, ec:$stat_ec"
-        rm -rf "$CACHE_DIR"
-        mkdir "$CACHE_DIR"
-    else
-        echo "Valid cache ($cache_valid_file exists)"
-        rm "$cache_valid_file"
-    fi
-fi
 
 travis_fold start update_cache
 travis_time_start
 
 # Update the cache (a pristine copy of the rust source master)
-if [ ! -d "$cache_src_dir/.git" ]; then
+retry sh -c "rm -rf $cache_src_dir && mkdir -p $cache_src_dir && \
-    retry sh -c "rm -rf $cache_src_dir && mkdir -p $cache_src_dir && \
+    git clone --depth 1 https://github.com/rust-lang/rust.git $cache_src_dir"
-        git clone https://github.com/rust-lang/rust.git $cache_src_dir"
-fi
-retry sh -c "cd $cache_src_dir && git reset --hard && git pull"
 (cd $cache_src_dir && git rm src/llvm)
 retry sh -c "cd $cache_src_dir && \
     git submodule deinit -f . && git submodule sync && git submodule update --init"
 
-# Cache was updated without errors, mark it as valid
-echo "Refreshed cache (touch $cache_valid_file)"
-touch "$cache_valid_file"
-
 travis_fold end update_cache
 travis_time_finish
 
@@ -92,19 +61,21 @@ for module in $modules; do
     if [ "$module" = src/llvm ]; then
         commit="$(git ls-tree HEAD src/llvm | awk '{print $3}')"
         git rm src/llvm
-        curl -sSL -O "https://github.com/rust-lang/llvm/archive/$commit.tar.gz"
+        retry sh -c "rm -f $commit.tar.gz && \
+            curl -sSL -O https://github.com/rust-lang/llvm/archive/$commit.tar.gz"
         tar -C src/ -xf "$commit.tar.gz"
         rm "$commit.tar.gz"
         mv "src/llvm-$commit" src/llvm
         continue
     fi
-    if [ ! -d "$cache_src_dir/$module" ]; then
+    if [ ! -e "$cache_src_dir/$module/.git" ]; then
         echo "WARNING: $module not found in pristine repo"
-        retry sh -c "git submodule deinit -f $module && git submodule update --init $module"
+        retry sh -c "git submodule deinit -f $module && \
+            git submodule update --init --recursive $module"
         continue
     fi
     retry sh -c "git submodule deinit -f $module && \
-        git submodule update --init --reference $cache_src_dir/$module $module"
+        git submodule update --init --recursive --reference $cache_src_dir/$module $module"
 done
 
 travis_fold end update_submodules

src/doc/book/second-edition/dictionary.txt

@@ -102,6 +102,7 @@ Enum
 enums
 enum's
 Enums
+eprintln
 ErrorKind
 Executables
 extern
@@ -127,7 +128,6 @@ GitHub
 gitignore
 grapheme
 Grapheme
-greprs
 growable
 gzip
 hardcode
@@ -192,6 +192,7 @@ Metadata
 metaprogramming
 mibbit
 Mibbit
+minigrep
 mixup
 mkdir
 modifiability

src/doc/book/second-edition/nostarch/chapter05.md

@@ -62,14 +62,27 @@ Listing 5-2: Creating an instance of the `User` struct
 To get a specific value from a struct, we can use dot notation. If we wanted
 just this user’s email address, we can use `user1.email` wherever we want to
 use this value. To change a value in a struct, if the instance is mutable, we
-can use the dot notation and assign into a particular field, such as
+can use the dot notation and assign into a particular field. Listing 5-3 shows
-`user1.email = String::from("someone-else@example.com");`.
+how to change the value in the `email` field of a mutable `User` instance:
+
+```
+let mut user1 = User {
+    email: String::from("someone@example.com"),
+    username: String::from("someusername123"),
+    active: true,
+    sign_in_count: 1,
+};
+
+user1.email = String::from("anotheremail@example.com");
+```
+
+Listing 5-3: Changing the value in the `email` field of a `User` instance
 
 ### Field Init Shorthand when Variables Have the Same Name as Fields
 
 If you have variables with the same names as struct fields, you can use *field
 init shorthand*. This can make functions that create new instances of structs
-more concise. The function named `build_user` shown here in Listing 5-3 has
+more concise. The function named `build_user` shown here in Listing 5-4 has
 parameters named `email` and `username`. The function creates and returns a
 `User` instance:
 
@@ -84,13 +97,13 @@ fn build_user(email: String, username: String) -> User {
 }
 ```
 
-Listing 5-3: A `build_user` function that takes an email and username and
+Listing 5-4: A `build_user` function that takes an email and username and
 returns a `User` instance
 
 Because the parameter names `email` and `username` are the same as the `User`
 struct's field names `email` and `username`, we can write `build_user` without
-the repetition of `email` and `username` as shown in Listing 5-4. This version
+the repetition of `email` and `username` as shown in Listing 5-5. This version
-of `build_user` behaves the same way as the one in Listing 5-3. The field init
+of `build_user` behaves the same way as the one in Listing 5-4. The field init
 syntax can make cases like this shorter to write, especially when structs have
 many fields.
 
@@ -105,13 +118,13 @@ fn build_user(email: String, username: String) -> User {
 }
 ```
 
-Listing 5-4: A `build_user` function that uses field init syntax since the
+Listing 5-5: A `build_user` function that uses field init syntax since the
 `email` and `username` parameters have the same name as struct fields
 
 ### Creating Instances From Other Instances With Struct Update Syntax
 
 It's often useful to create a new instance from an old instance, using most of
-the old instance's values but changing some. Listing 5-5 shows an example of
+the old instance's values but changing some. Listing 5-6 shows an example of
 creating a new `User` instance in `user2` by setting the values of `email` and
 `username` but using the same values for the rest of the fields from the
 `user1` instance we created in Listing 5-2:
@@ -125,13 +138,13 @@ let user2 = User {
 };
 ```
 
-Listing 5-5: Creating a new `User` instance, `user2`, and setting some fields
+Listing 5-6: Creating a new `User` instance, `user2`, and setting some fields
 to the values of the same fields from `user1`
 
   The *struct update syntax* achieves the same effect as the code in Listing
-5-5 using less code. The struct update syntax uses `..` to specify that the
+5-6 using less code. The struct update syntax uses `..` to specify that the
 remaining fields not set explicitly should have the same value as the fields in
-the given instance. The code in Listing 5-6 also creates an instance in `user2`
+the given instance. The code in Listing 5-7 also creates an instance in `user2`
 that has a different value for `email` and `username` but has the same values
 for the `active` and `sign_in_count` fields that `user1` has:
 
@@ -143,7 +156,7 @@ let user2 = User {
 };
 ```
 
-Listing 5-6: Using struct update syntax to set a new `email` and `username`
+Listing 5-7: Using struct update syntax to set a new `email` and `username`
 values for a `User` instance but use the rest of the values from the fields of
 the instance in the `user1` variable
 
@@ -242,7 +255,7 @@ refactor the program until we’re using structs instead.
 
 Let’s make a new binary project with Cargo called *rectangles* that will take
 the length and width of a rectangle specified in pixels and will calculate the
-area of the rectangle. Listing 5-7 shows a short program with one way of doing
+area of the rectangle. Listing 5-8 shows a short program with one way of doing
 just that in our project’s *src/main.rs*:
 
 Filename: src/main.rs
@@ -263,7 +276,7 @@ fn area(length: u32, width: u32) -> u32 {
 }
 ```
 
-Listing 5-7: Calculating the area of a rectangle specified by its length and
+Listing 5-8: Calculating the area of a rectangle specified by its length and
 width in separate variables
 
 Now, run this program using `cargo run`:
@@ -274,7 +287,7 @@ The area of the rectangle is 1500 square pixels.
 
 ### Refactoring with Tuples
 
-Even though Listing 5-7 works and figures out the area of the rectangle by
+Even though Listing 5-8 works and figures out the area of the rectangle by
 calling the `area` function with each dimension, we can do better. The length
 and the width are related to each other because together they describe one
 rectangle.
@@ -290,7 +303,7 @@ function we wrote has two parameters. The parameters are related, but that’s
 not expressed anywhere in our program. It would be more readable and more
 manageable to group length and width together. We’ve already discussed one way
 we might do that in the Grouping Values into Tuples section of Chapter 3 on
-page XX: by using tuples. Listing 5-8 shows another version of our program that
+page XX: by using tuples. Listing 5-9 shows another version of our program that
 uses tuples:
 
 Filename: src/main.rs
@@ -329,7 +342,7 @@ our code.
 
 We use structs to add meaning by labeling the data. We can transform the tuple
 we’re using into a data type with a name for the whole as well as names for the
-parts, as shown in Listing 5-9:
+parts, as shown in Listing 5-10:
 
 Filename: src/main.rs
 
@@ -353,7 +366,7 @@ fn area(rectangle: &Rectangle) -> u32 {
 }
 ```
 
-Listing 5-9: Defining a `Rectangle` struct
+Listing 5-10: Defining a `Rectangle` struct
 
 Here we’ve defined a struct and named it `Rectangle`. Inside the `{}` we
 defined the fields as `length` and `width`, both of which have type `u32`. Then
@@ -378,7 +391,7 @@ of `0` and `1`—a win for clarity.
 
 It would be helpful to be able to print out an instance of the `Rectangle`
 while we’re debugging our program in order to see the values for all its
-fields. Listing 5-10 uses the `println!` macro as we have been in earlier
+fields. Listing 5-11 uses the `println!` macro as we have been in earlier
 chapters:
 
 Filename: src/main.rs
@@ -396,7 +409,7 @@ fn main() {
 }
 ```
 
-Listing 5-10: Attempting to print a `Rectangle` instance
+Listing 5-11: Attempting to print a `Rectangle` instance
 
 When we run this code, we get an error with this core message:
 
@@ -443,7 +456,7 @@ crate, add `#[derive(Debug)]` or manually implement it
 Rust *does* include functionality to print out debugging information, but we
 have to explicitly opt-in to make that functionality available for our struct.
 To do that, we add the annotation `#[derive(Debug)]` just before the struct
-definition, as shown in Listing 5-11:
+definition, as shown in Listing 5-12:
 
 Filename: src/main.rs
 
@@ -461,7 +474,7 @@ fn main() {
 }
 ```
 
-Listing 5-11: Adding the annotation to derive the `Debug` trait and printing
+Listing 5-12: Adding the annotation to derive the `Debug` trait and printing
 the `Rectangle` instance using debug formatting
 
 Now when we run the program, we won’t get any errors and we’ll see the
@@ -498,7 +511,7 @@ continue to refactor this code by turning the `area` function into an `area`
 ## Method Syntax
 
 *Methods* are similar to functions: they’re declared with the `fn` keyword and
-their name, they can have parameters and return values, and they contain some
+their name, they can have parameters and a return value, and they contain some
 code that is run when they’re called from somewhere else. However, methods are
 different from functions in that they’re defined within the context of a struct
 (or an enum or a trait object, which we cover in Chapters 6 and 17,
@@ -509,7 +522,7 @@ instance of the struct the method is being called on.
 
 Let’s change the `area` function that has a `Rectangle` instance as a parameter
 and instead make an `area` method defined on the `Rectangle` struct, as shown
-in Listing 5-12:
+in Listing 5-13:
 
 Filename: src/main.rs
 
@@ -536,7 +549,7 @@ fn main() {
 }
 ```
 
-Listing 5-12: Defining an `area` method on the `Rectangle` struct
+Listing 5-13: Defining an `area` method on the `Rectangle` struct
 
 To define the function within the context of `Rectangle`, we start an `impl`
 (*implementation*) block. Then we move the `area` function within the `impl`
@@ -608,7 +621,7 @@ Let’s practice using methods by implementing a second method on the `Rectangle
 struct. This time, we want an instance of `Rectangle` to take another instance
 of `Rectangle` and return `true` if the second `Rectangle` can fit completely
 within `self`; otherwise it should return `false`. That is, we want to be able
-to write the program shown in Listing 5-13, once we’ve defined the `can_hold`
+to write the program shown in Listing 5-14, once we’ve defined the `can_hold`
 method:
 
 Filename: src/main.rs
@@ -624,7 +637,7 @@ fn main() {
 }
 ```
 
-Listing 5-13: Demonstration of using the as-yet-unwritten `can_hold` method
+Listing 5-14: Demonstration of using the as-yet-unwritten `can_hold` method
 
 And the expected output would look like the following, because both dimensions
 of `rect2` are smaller than the dimensions of `rect1`, but `rect3` is wider
@@ -647,7 +660,7 @@ calling the `can_hold` method. The return value of `can_hold` will be a
 boolean, and the implementation will check whether the length and width of
 `self` are both greater than the length and width of the other `Rectangle`,
 respectively. Let’s add the new `can_hold` method to the `impl` block from
-Listing 5-12, shown in Listing 5-14:
+Listing 5-13, shown in Listing 5-15:
 
 Filename: src/main.rs
 
@@ -663,10 +676,10 @@ impl Rectangle {
 }
 ```
 
-Listing 5-14: Implementing the `can_hold` method on `Rectangle` that takes
+Listing 5-15: Implementing the `can_hold` method on `Rectangle` that takes
 another `Rectangle` instance as a parameter
 
-When we run this code with the `main` function in Listing 5-13, we’ll get our
+When we run this code with the `main` function in Listing 5-14, we’ll get our
 desired output. Methods can take multiple parameters that we add to the
 signature after the `self` parameter, and those parameters work just like
 parameters in functions.

src/doc/book/second-edition/nostarch/chapter11.md

@@ -1,53 +1,52 @@
 
 [TOC]
 
-# Testing
+# Writing Automated Tests
 
 > Program testing can be a very effective way to show the presence of bugs, but
 > it is hopelessly inadequate for showing their absence.
->
+> Edsger W. Dijkstra, “The Humble Programmer” (1972)
-> Edsger W. Dijkstra, "The Humble Programmer" (1972)
 
 Correctness in our programs means that our code does what we intend for it to
 do. Rust is a programming language that cares a lot about correctness, but
-correctness is a complex topic and isn't easy to prove. Rust's type system
+correctness is a complex topic and isn’t easy to prove. Rust’s type system
 shoulders a huge part of this burden, but the type system cannot catch every
 kind of incorrectness. As such, Rust includes support for writing software
 tests within the language itself.
 
-As an example, say we write a function called `add_two` that adds two to a
+As an example, say we write a function called `add_two` that adds two to
-number passed to it. This function's signature accepts an integer as a
+whatever number is passed to it. This function’s signature accepts an integer
-parameter and returns an integer as a result. When we implement and compile
+as a parameter and returns an integer as a result. When we implement and
-that function, Rust will do all the type checking and borrow checking that
+compile that function, Rust will do all the type checking and borrow checking
-we've seen so far. Those checks will make sure that, for instance, we aren't
+that we’ve seen so far to make sure that, for instance, we aren’t passing a
-passing a `String` value or an invalid reference to this function. What Rust
+`String` value or an invalid reference to this function. What Rust *can’t*
-*can't* check is that this function will do precisely what we intend: return
+check is that this function will do precisely what we intend: return the
-the parameter plus two, rather than, say, the parameter plus 10 or the
+parameter plus two, rather than, say, the parameter plus 10 or the parameter
-parameter minus 50! That's where tests come in.
+minus 50! That’s where tests come in.
 
 We can write tests that assert, for example, that when we pass `3` to the
 `add_two` function, we get `5` back. We can run these tests whenever we make
 changes to our code to make sure any existing correct behavior has not changed.
 
 Testing is a complex skill, and we cannot hope to cover everything about how to
-write good tests in one chapter of a book, so here we'll just discuss the
+write good tests in one chapter of a book, so here we’ll just discuss the
-mechanics of Rust's testing facilities. We'll talk about the annotations and
+mechanics of Rust’s testing facilities. We’ll talk about the annotations and
 macros available to you when writing your tests, the default behavior and
 options provided for running your tests, and how to organize tests into unit
 tests and integration tests.
 
 ## How to Write Tests
 
-Tests are Rust functions that verify non-test code is functioning in the
+Tests are Rust functions that verify that the non-test code in the program is
-program in the expected manner. The bodies of test functions typically contain
+functioning in the expected manner. The bodies of test functions typically run
-some setup, running the code we want to test, then asserting that the results
+some setup code, then run the code we want to test, then assert whether the
-are what we expect. Let's look at the features Rust provides specifically for
+results are what we expect. Let’s look at the features Rust provides
-writing tests: the `test` attribute, a few macros, and the `should_panic`
+specifically for writing tests: the `test` attribute, a few macros, and the
-attribute.
+`should_panic` attribute.
 
 ### The Anatomy of a Test Function
 
-At its simplest, a test in Rust is a function that's annotated with the `test`
+At its simplest, a test in Rust is a function that’s annotated with the `test`
 attribute. Attributes are metadata about pieces of Rust code: the `derive`
 attribute that we used with structs in Chapter 5 is one example. To make a
 function into a test function, we add `#[test]` on the line before `fn`. When
@@ -55,28 +54,19 @@ we run our tests with the `cargo test` command, Rust will build a test runner
 binary that runs the functions annotated with the `test` attribute and reports
 on whether each test function passes or fails.
 
-<!-- is it annotated with `test` by the user, or only automatically? I think
-it's the latter, and has edited with a more active tone to make that clear, but
-please change if I'm wrong -->
-<!-- What do you mean by "only automatically"? The reader should be typing in
-`#test on their own when they add new test functions; there's nothing special
-about that text. I'm not sure what part of this chapter implied "only
-automatically", can you point out where that's happening if we haven't taken
-care of it? /Carol -->
-
 We saw in Chapter 7 that when you make a new library project with Cargo, a test
 module with a test function in it is automatically generated for us. This is to
-help us get started writing our tests, since we don't have to go look up the
+help us get started writing our tests, since we don’t have to go look up the
 exact structure and syntax of test functions every time we start a new project.
 We can add as many additional test functions and as many test modules as we
 want, though!
 
-We're going to explore some aspects of how tests work by experimenting with the
+We’re going to explore some aspects of how tests work by experimenting with the
-template test generated for us, without actually testing any code. Then we'll
+template test generated for us, without actually testing any code. Then we’ll
-write some real-world tests that call some code that we've written and assert
+write some real-world tests that call some code that we’ve written and assert
 that its behavior is correct.
 
-Let's create a new library project called `adder`:
+Let’s create a new library project called `adder`:
 
 ```
 $ cargo new adder
@@ -101,7 +91,7 @@ mod tests {
 Listing 11-1: The test module and function generated automatically for us by
 `cargo new`
 
-For now, let's ignore the top two lines and focus on the function to see how it
+For now, let’s ignore the top two lines and focus on the function to see how it
 works. Note the `#[test]` annotation before the `fn` line: this attribute
 indicates this is a test function, so that the test runner knows to treat this
 function as a test. We could also have non-test functions in the `tests` module
@@ -109,7 +99,7 @@ to help set up common scenarios or perform common operations, so we need to
 indicate which functions are tests with the `#[test]` attribute.
 
 The function currently has no body, which means there is no code to fail the
-test; an empty test is a passing test! Let's run it and see that this test
+test; an empty test is a passing test! Let’s run it and see that this test
 passes.
 
 The `cargo test` command runs all tests we have in our project, as shown in
@@ -118,7 +108,7 @@ Listing 11-2:
 ```
 $ cargo test
    Compiling adder v0.1.0 (file:///projects/adder)
-    Finished debug [unoptimized + debuginfo] target(s) in 0.22 secs
+    Finished dev [unoptimized + debuginfo] target(s) in 0.22 secs
      Running target/debug/deps/adder-ce99bcc2479f4607
 
 running 1 test
@@ -142,30 +132,21 @@ that test, `ok`. Then we see the overall summary of running the tests: `test
 result: ok.` means all the tests passed. `1 passed; 0 failed` adds up the
 number of tests that passed or failed.
 
-We don't have any tests we've marked as ignored, so the summary says `0
+We don’t have any tests we’ve marked as ignored, so the summary says `0
-ignored`. We're going to talk about ignoring tests in the next section on
+ignored`. We’re going to talk about ignoring tests in the next section on
 different ways to run tests. The `0 measured` statistic is for benchmark tests
 that measure performance. Benchmark tests are, as of this writing, only
 available in nightly Rust. See Appendix D for more information about nightly
 Rust.
 
 The next part of the test output that starts with `Doc-tests adder` is for the
-results of any documentation tests. We don't have any documentation tests yet,
+results of any documentation tests. We don’t have any documentation tests yet,
 but Rust can compile any code examples that appear in our API documentation.
-This feature helps us keep our docs and our code in sync! We'll be talking
+This feature helps us keep our docs and our code in sync! We’ll be talking
-about how to write documentation tests in the "Documentation Comments" section
+about how to write documentation tests in the “Documentation Comments” section
-of Chapter 14. We're going to ignore the `Doc-tests` output for now.
+of Chapter 14. We’re going to ignore the `Doc-tests` output for now.
 
-<!-- I might suggest changing the name of the function, could be misconstrued
+Let’s change the name of our test and see how that changes the test output.
-as part of the test output! -->
-<!-- `it_works` is always the name that `cargo new` generates for the first
-test function, though. We wanted to show the reader what happens when you run
-the tests immediately after generating a new project; they pass without you
-needing to change anything. I've added a bit to walk through changing the
-function name and seeing how the output changes; I hope that's sufficient.
-/Carol -->
-
-Let's change the name of our test and see how that changes the test output.
 Give the `it_works` function a different name, such as `exploration`, like so:
 
 Filename: src/lib.rs
@@ -179,7 +160,7 @@ mod tests {
 }
 ```
 
-And run `cargo test` again. In the output, we'll now see `exploration` instead
+And run `cargo test` again. In the output, we’ll now see `exploration` instead
 of `it_works`:
 
 ```
@@ -189,7 +170,7 @@ test tests::exploration ... ok
 test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured
 ```
 
-Let's add another test, but this time we'll make a test that fails! Tests fail
+Let’s add another test, but this time we’ll make a test that fails! Tests fail
 when something in the test function panics. We talked about the simplest way to
 cause a panic in Chapter 9: call the `panic!` macro! Type in the new test so
 that your `src/lib.rs` now looks like Listing 11-3:
@@ -244,14 +225,14 @@ failed because it `panicked at 'Make this test fail'`, which happened on
 *src/lib.rs* line 9. The next section lists just the names of all the failing
 tests, which is useful when there are lots of tests and lots of detailed
 failing test output. We can use the name of a failing test to run just that
-test in order to more easily debug it; we'll talk more about ways to run tests
+test in order to more easily debug it; we’ll talk more about ways to run tests
 in the next section.
 
 Finally, we have the summary line: overall, our test result is `FAILED`. We had
 1 test pass and 1 test fail.
 
-Now that we've seen what the test results look like in different scenarios,
+Now that we’ve seen what the test results look like in different scenarios,
-let's look at some macros other than `panic!` that are useful in tests.
+let’s look at some macros other than `panic!` that are useful in tests.
 
 ### Checking Results with the `assert!` Macro
 
@@ -262,24 +243,11 @@ to ensure that some condition in a test evaluates to `true`. We give the
 calls the `panic!` macro, which causes the test to fail. This is one macro that
 helps us check that our code is functioning in the way we intend.
 
-<!-- what kind of thing can be passed as an argument? Presumably when we use it
-for real we won't pass it `true` or `false` as an argument, but some condition
-that will evaluate to true or false? In which case, should below be phrased "If
-the argument evaluates to true" and an exaplanation of that? Or maybe even a
-working example would be better, this could be misleading -->
-<!-- We were trying to really break it down, to show just how the `assert!`
-macro works and what it looks like for it to pass or fail, before we got into
-calling actual code. We've changed this section to move a bit faster and just
-write actual tests instead. /Carol -->
-
 Remember all the way back in Chapter 5, Listing 5-9, where we had a `Rectangle`
-struct and a `can_hold` method, repeated here in Listing 11-5. Let's put this
+struct and a `can_hold` method, repeated here in Listing 11-5. Let’s put this
 code in *src/lib.rs* instead of *src/main.rs* and write some tests for it using
 the `assert!` macro.
 
-<!-- Listing 5-9 wasn't marked as such; I'll fix it the next time I get Chapter
-5 for editing. /Carol -->
-
 Filename: src/lib.rs
 
 ```
@@ -298,8 +266,8 @@ impl Rectangle {
 
 Listing 11-5: The `Rectangle` struct and its `can_hold` method from Chapter 5
 
-The `can_hold` method returns a boolean, which means it's a perfect use case
+The `can_hold` method returns a boolean, which means it’s a perfect use case
-for the `assert!` macro. In Listing 11-6, let's write a test that exercises the
+for the `assert!` macro. In Listing 11-6, let’s write a test that exercises the
 `can_hold` method by creating a `Rectangle` instance that has a length of 8 and
 a width of 7, and asserting that it can hold another `Rectangle` instance that
 has a length of 5 and a width of 1:
@@ -324,17 +292,17 @@ mod tests {
 Listing 11-6: A test for `can_hold` that checks that a larger rectangle indeed
 holds a smaller rectangle
 
-Note that we've added a new line inside the `tests` module: `use super::*;`.
+Note that we’ve added a new line inside the `tests` module: `use super::*;`.
 The `tests` module is a regular module that follows the usual visibility rules
-we covered in Chapter 7. Because we're in an inner module, we need to bring the
+we covered in Chapter 7. Because we’re in an inner module, we need to bring the
-code under test in the outer module into the scope of the inner module. We've
+code under test in the outer module into the scope of the inner module. We’ve
 chosen to use a glob here so that anything we define in the outer module is
 available to this `tests` module.
 
-We've named our test `larger_can_hold_smaller`, and we've created the two
+We’ve named our test `larger_can_hold_smaller`, and we’ve created the two
 `Rectangle` instances that we need. Then we called the `assert!` macro and
 passed it the result of calling `larger.can_hold(&smaller)`. This expression is
-supposed to return `true`, so our test should pass. Let's find out!
+supposed to return `true`, so our test should pass. Let’s find out!
 
 ```
 running 1 test
@@ -343,7 +311,7 @@ test tests::larger_can_hold_smaller ... ok
 test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured
 ```
 
-It does pass! Let's add another test, this time asserting that a smaller
+It does pass! Let’s add another test, this time asserting that a smaller
 rectangle cannot hold a larger rectangle:
 
 Filename: src/lib.rs
@@ -362,7 +330,7 @@ mod tests {
     }
 
     #[test]
-    fn smaller_can_hold_larger() {
+    fn smaller_cannot_hold_larger() {
         let larger = Rectangle { length: 8, width: 7 };
         let smaller = Rectangle { length: 5, width: 1 };
 
@@ -377,15 +345,15 @@ way, our test will pass if `can_hold` returns `false`:
 
 ```
 running 2 tests
-test tests::smaller_can_hold_larger ... ok
+test tests::smaller_cannot_hold_larger ... ok
 test tests::larger_can_hold_smaller ... ok
 
 test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured
 ```
 
-Two passing tests! Now let's see what happens to our test results if we
+Two passing tests! Now let’s see what happens to our test results if we
-introduce a bug in our code. Let's change the implementation of the `can_hold`
+introduce a bug in our code. Let’s change the implementation of the `can_hold`
-method to have a less-than sign when it compares the lengths where it's
+method to have a less-than sign when it compares the lengths where it’s
 supposed to have a greater-than sign:
 
 ```
@@ -406,7 +374,7 @@ Running the tests now produces:
 
 ```
 running 2 tests
-test tests::smaller_can_hold_larger ... ok
+test tests::smaller_cannot_hold_larger ... ok
 test tests::larger_can_hold_smaller ... FAILED
 
 failures:
@@ -429,18 +397,18 @@ less than 5.
 ### Testing Equality with the `assert_eq!` and `assert_ne!` Macros
 
 A common way to test functionality is to take the result of the code under test
-and the value we expect the code to return and check that they're equal. We
+and the value we expect the code to return and check that they’re equal. We
 could do this using the `assert!` macro and passing it an expression using the
 `==` operator. However, this is such a common test that the standard library
 provides a pair of macros to perform this test more conveniently: `assert_eq!`
 and `assert_ne!`. These macros compare two arguments for equality or
-inequality, respectively. They'll also print out the two values if the
+inequality, respectively. They’ll also print out the two values if the
-assertion fails, so that it's easier to see *why* the test failed, while the
+assertion fails, so that it’s easier to see *why* the test failed, while the
 `assert!` macro only tells us that it got a `false` value for the `==`
 expression, not the values that lead to the `false` value.
 
-In Listing 11-7, let's write a function named `add_two` that adds two to its
+In Listing 11-7, let’s write a function named `add_two` that adds two to its
-parameter and returns the result. Then let's test this function using the
+parameter and returns the result. Then let’s test this function using the
 `assert_eq!` macro:
 
 Filename: src/lib.rs
@@ -463,7 +431,7 @@ mod tests {
 
 Listing 11-7: Testing the function `add_two` using the `assert_eq!` macro
 
-Let's check that it passes!
+Let’s check that it passes!
 
 ```
 running 1 test
@@ -476,7 +444,7 @@ The first argument we gave to the `assert_eq!` macro, 4, is equal to the result
 of calling `add_two(2)`. We see a line for this test that says `test
 tests::it_adds_two ... ok`, and the `ok` text indicates that our test passed!
 
-Let's introduce a bug into our code to see what it looks like when a test that
+Let’s introduce a bug into our code to see what it looks like when a test that
 uses `assert_eq!` fails. Change the implementation of the `add_two` function to
 instead add 3:
 
@@ -513,16 +481,16 @@ useful and helps us get started debugging: it says the `left` argument to
 Note that in some languages and test frameworks, the parameters to the
 functions that assert two values are equal are called `expected` and `actual`
 and the order in which we specify the arguments matters. However, in Rust,
-they're called `left` and `right` instead, and the order in which we specify
+they’re called `left` and `right` instead, and the order in which we specify
-the value we expect and the value that the code under test produces doesn't
+the value we expect and the value that the code under test produces doesn’t
 matter. We could have written the assertion in this test as
 `assert_eq!(add_two(2), 4)`, which would result in a failure message that says
 `` assertion failed: `(left == right)` (left: `5`, right: `4`) ``.
 
 The `assert_ne!` macro will pass if the two values we give to it are not equal
-and fail if they are equal. This macro is most useful for cases when we're not
+and fail if they are equal. This macro is most useful for cases when we’re not
 sure exactly what a value *will* be, but we know what the value definitely
-*won't* be, if our code is functioning as we intend. For example, if we have a
+*won’t* be, if our code is functioning as we intend. For example, if we have a
 function that is guaranteed to change its input in some way, but the way in
 which the input is changed depends on the day of the week that we run our
 tests, the best thing to assert might be that the output of the function is not
@@ -533,8 +501,8 @@ Under the surface, the `assert_eq!` and `assert_ne!` macros use the operators
 arguments using debug formatting, which means the values being compared must
 implement the `PartialEq` and `Debug` traits. All of the primitive types and
 most of the standard library types implement these traits. For structs and
-enums that you define, you'll need to implement `PartialEq` in order to be able
+enums that you define, you’ll need to implement `PartialEq` in order to be able
-to assert that values of those types are equal or not equal. You'll need to
+to assert that values of those types are equal or not equal. You’ll need to
 implement `Debug` in order to be able to print out the values in the case that
 the assertion fails. Because both of these traits are derivable traits, as we
 mentioned in Chapter 5, this is usually as straightforward as adding the
@@ -552,7 +520,7 @@ contains `{}` placeholders and values to go in the placeholders. Custom
 messages are useful in order to document what an assertion means, so that when
 the test fails, we have a better idea of what the problem is with the code.
 
-For example, let's say we have a function that greets people by name, and we
+For example, let’s say we have a function that greets people by name, and we
 want to test that the name we pass into the function appears in the output:
 
 Filename: src/lib.rs
@@ -574,14 +542,14 @@ mod tests {
 }
 ```
 
-The requirements for this program haven't been agreed upon yet, and we're
+The requirements for this program haven’t been agreed upon yet, and we’re
 pretty sure the `Hello` text at the beginning of the greeting will change. We
-decided we don't want to have to update the test for the name when that
+decided we don’t want to have to update the test for the name when that
 happens, so instead of checking for exact equality to the value returned from
-the `greeting` function, we're just going to assert that the output contains
+the `greeting` function, we’re just going to assert that the output contains
 the text of the input parameter.
 
-Let's introduce a bug into this code to see what this test failure looks like,
+Let’s introduce a bug into this code to see what this test failure looks like,
 by changing `greeting` to not include `name`:
 
 ```
@@ -609,7 +577,7 @@ failures:
 
 This just tells us that the assertion failed and which line the assertion is
 on. A more useful failure message in this case would print the value we did get
-from the `greeting` function. Let's change the test function to have a custom
+from the `greeting` function. Let’s change the test function to have a custom
 failure message made from a format string with a placeholder filled in with the
 actual value we got from the `greeting` function:
 
@@ -624,11 +592,11 @@ fn greeting_contains_name() {
 }
 ```
 
-Now if we run the test again, we'll get a much more informative error message:
+Now if we run the test again, we’ll get a much more informative error message:
 
 ```
 ---- tests::greeting_contains_name stdout ----
-	thread 'tests::greeting_contains_name' panicked at 'Result did not contain
+    thread 'tests::greeting_contains_name' panicked at 'Greeting did not contain
     name, value was `Hello`', src/lib.rs:12
 note: Run with `RUST_BACKTRACE=1` for a backtrace.
 ```
@@ -639,7 +607,7 @@ debug what happened instead of what we were expecting to happen.
 ### Checking for Panics with `should_panic`
 
 In addition to checking that our code returns the correct values we expect,
-it's also important to check that our code handles error conditions as we
+it’s also important to check that our code handles error conditions as we
 expect. For example, consider the `Guess` type that we created in Chapter 9 in
 Listing 9-8. Other code that uses `Guess` is depending on the guarantee that
 `Guess` instances will only contain values between 1 and 100. We can write a
@@ -648,9 +616,9 @@ outside that range panics.
 
 We can do this by adding another attribute, `should_panic`, to our test
 function. This attribute makes a test pass if the code inside the function
-panics, and the test will fail if the code inside the function does non panic.
+panics, and the test will fail if the code inside the function doesn't panic.
 
-Listing 11-8 shows how we'd write a test that checks the error conditions of
+Listing 11-8 shows how we’d write a test that checks the error conditions of
 `Guess::new` happen when we expect:
 
 Filename: src/lib.rs
@@ -667,7 +635,7 @@ impl Guess {
         }
 
         Guess {
-            value: value,
+            value
         }
     }
 }
@@ -687,7 +655,7 @@ mod tests {
 Listing 11-8: Testing that a condition will cause a `panic!`
 
 The `#[should_panic]` attribute goes after the `#[test]` attribute and before
-the test function it applies to. Let's see what it looks like when this test
+the test function it applies to. Let’s see what it looks like when this test
 passes:
 
 ```
@@ -697,7 +665,7 @@ test tests::greater_than_100 ... ok
 test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured
 ```
 
-Looks good! Now let's introduce a bug in our code, by removing the condition
+Looks good! Now let’s introduce a bug in our code, by removing the condition
 that the `new` function will panic if the value is greater than 100:
 
 ```
@@ -708,7 +676,7 @@ impl Guess {
         }
 
         Guess {
-            value: value,
+            value
         }
     }
 }
@@ -728,8 +696,8 @@ failures:
 test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured
 ```
 
-We don't get a very helpful message in this case, but once we look at the test
+We don’t get a very helpful message in this case, but once we look at the test
-function, we can see that it's annotated with `#[should_panic]`. The failure we
+function, we can see that it’s annotated with `#[should_panic]`. The failure we
 got means that the code in the function, `Guess::new(200)`, did not cause a
 panic.
 
@@ -760,7 +728,7 @@ impl Guess {
         }
 
         Guess {
-            value: value,
+            value
         }
     }
 }
@@ -790,7 +758,7 @@ substring of the panic message is enough to ensure that the code in the
 function that gets run is the `else if value > 100` case.
 
 To see what happens when a `should_panic` test with an `expected` message
-fails, let's again introduce a bug into our code by swapping the bodies of the
+fails, let’s again introduce a bug into our code by swapping the bodies of the
 `if value < 1` and the `else if value > 100` blocks:
 
 ```
@@ -828,7 +796,7 @@ less than or equal to 100'`. We can see the panic message that we did get,
 which in this case was `Guess value must be greater than or equal to 1, got
 200.` We could then start figuring out where our bug was!
 
-Now that we've gone over ways to write tests, let's look at what is happening
+Now that we’ve gone over ways to write tests, let’s look at what is happening
 when we run our tests and talk about the different options we can use with
 `cargo test`.
 
@@ -853,10 +821,6 @@ separator `--`.
 
 ### Running Tests in Parallel or Consecutively
 
-<!-- Are we safe assuming the reader will know enough about threads in this
-context? -->
-<!-- Yes /Carol -->
-
 When multiple tests are run, by default they run in parallel using threads.
 This means the tests will finish running faster, so that we can get faster
 feedback on whether or not our code is working. Since the tests are running at
@@ -868,13 +832,13 @@ For example, say each of your tests runs some code that creates a file on disk
 named `test-output.txt` and writes some data to that file. Then each test reads
 the data in that file and asserts that the file contains a particular value,
 which is different in each test. Because the tests are all run at the same
-time, one test might overrwrite the file between when another test writes and
+time, one test might overwrite the file between when another test writes and
 reads the file. The second test will then fail, not because the code is
 incorrect, but because the tests have interfered with each other while running
 in parallel. One solution would be to make sure each test writes to a different
 file; another solution is to run the tests one at a time.
 
-If you don't want to run the tests in parallel, or if you want more
+If you don’t want to run the tests in parallel, or if you want more
 fine-grained control over the number of threads used, you can send the
 `--test-threads` flag and the number of threads you want to use to the test
 binary. For example:
@@ -885,14 +849,14 @@ $ cargo test -- --test-threads=1
 
 We set the number of test threads to 1, telling the program not to use any
 parallelism. This will take longer than running them in parallel, but the tests
-won't be potentially interfering with each other if they share state.
+won’t be potentially interfering with each other if they share state.
 
 ### Showing Function Output
 
-By default, if a test passes, Rust's test library captures anything printed to
+By default, if a test passes, Rust’s test library captures anything printed to
 standard output. For example, if we call `println!` in a test and the test
-passes, we won't see the `println!` output in the terminal: we'll only see the
+passes, we won’t see the `println!` output in the terminal: we’ll only see the
-line that says the test passed. If a test fails, we'll see whatever was printed
+line that says the test passed. If a test fails, we’ll see whatever was printed
 to standard output with the rest of the failure message.
 
 For example, Listing 11-10 has a silly function that prints out the value of
@@ -927,7 +891,7 @@ mod tests {
 
 Listing 11-10: Tests for a function that calls `println!`
 
-The output we'll see when we run these tests with `cargo test` is:
+The output we’ll see when we run these tests with `cargo test` is:
 
 ```
 running 2 tests
@@ -988,12 +952,12 @@ function and see what the output looks like then!
 
 ### Running a Subset of Tests by Name
 
-Sometimes, running a full test suite can take a long time. If you're working on
+Sometimes, running a full test suite can take a long time. If you’re working on
 code in a particular area, you might want to run only the tests pertaining to
 that code. You can choose which tests to run by passing `cargo test` the name
 or names of the test/s you want to run as an argument.
 
-To demonstrate how to run a subset of tests, we'll create three tests for our
+To demonstrate how to run a subset of tests, we’ll create three tests for our
 `add_two` function as shown in Listing 11-11 and choose which ones to run:
 
 Filename: src/lib.rs
@@ -1026,7 +990,7 @@ mod tests {
 
 Listing 11-11: Three tests with a variety of names
 
-If we run the tests without passing any arguments, as we've already seen, all
+If we run the tests without passing any arguments, as we’ve already seen, all
 the tests will run in parallel:
 
 ```
@@ -1044,7 +1008,7 @@ We can pass the name of any test function to `cargo test` to run only that test:
 
 ```
 $ cargo test one_hundred
-    Finished debug [unoptimized + debuginfo] target(s) in 0.0 secs
+    Finished dev [unoptimized + debuginfo] target(s) in 0.0 secs
      Running target/debug/deps/adder-06a75b4a1f2515e9
 
 running 1 test
@@ -1053,18 +1017,18 @@ test tests::one_hundred ... ok
 test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured
 ```
 
-We can't specify the names of multiple tests in this way, only the first value
+We can’t specify the names of multiple tests in this way, only the first value
 given to `cargo test` will be used.
 
 #### Filtering to Run Multiple Tests
 
 However, we can specify part of a test name, and any test whose name matches
-that value will get run. For example, since two of our tests' names contain
+that value will get run. For example, since two of our tests’ names contain
 `add`, we can run those two by running `cargo test add`:
 
 ```
 $ cargo test add
-    Finished debug [unoptimized + debuginfo] target(s) in 0.0 secs
+    Finished dev [unoptimized + debuginfo] target(s) in 0.0 secs
      Running target/debug/deps/adder-06a75b4a1f2515e9
 
 running 2 tests
@@ -1075,16 +1039,8 @@ test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured
 ```
 
 This ran all tests with `add` in the name. Also note that the module in which
-tests appear becomes part of the test's name, so we can run all the tests in a
+tests appear becomes part of the test’s name, so we can run all the tests in a
-module by filtering on the module's name.
+module by filtering on the module’s name.
-
-<!-- in what kind of situation might you need to run only some tests, when you
-have lots and lots in a program? -->
-<!-- We covered this in the first paragraph of the "Running a Subset of Tests
-by Name" section, do you think it should be repeated so soon? Most people who
-use tests have sufficient motivation for wanting to run a subset of the tests,
-they just need to know how to do it with Rust, so we don't think this is a
-point that needs to be emphasized multiple times. /Carol -->
 
 ### Ignore Some Tests Unless Specifically Requested
 
@@ -1109,13 +1065,13 @@ fn expensive_test() {
 ```
 
 We add the `#[ignore]` line to the test we want to exclude, after `#[test]`.
-Now if we run our tests, we'll see `it_works` runs, but `expensive_test` does
+Now if we run our tests, we’ll see `it_works` runs, but `expensive_test` does
 not:
 
 ```
 $ cargo test
    Compiling adder v0.1.0 (file:///projects/adder)
-    Finished debug [unoptimized + debuginfo] target(s) in 0.24 secs
+    Finished dev [unoptimized + debuginfo] target(s) in 0.24 secs
      Running target/debug/deps/adder-ce99bcc2479f4607
 
 running 2 tests
@@ -1134,20 +1090,9 @@ test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured
 `expensive_test` is listed as `ignored`. If we want to run only the ignored
 tests, we can ask for them to be run with `cargo test -- --ignored`:
 
-<!-- what does the double `-- --` mean? That seems interesting -->
-<!-- We covered that in the second paragraph after the "Controlling How Tests
-are Run" heading, and this section is beneath that heading, so I don't think a
-back reference is needed /Carol -->
-
-<!-- is that right, this way the program knows to run only the test with
-`ignore` if we add this, or it knows to run all tests? -->
-<!-- Is this unclear from the output that shows `expensive_test` was run and
-the `it_works` test does not appear? I'm not sure how to make this clearer.
-/Carol -->
-
 ```
 $ cargo test -- --ignored
-    Finished debug [unoptimized + debuginfo] target(s) in 0.0 secs
+    Finished dev [unoptimized + debuginfo] target(s) in 0.0 secs
      Running target/debug/deps/adder-ce99bcc2479f4607
 
 running 1 test
@@ -1157,7 +1102,7 @@ test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured
 ```
 
 By controlling which tests run, you can make sure your `cargo test` results
-will be fast. When you're at a point that it makes sense to check the results
+will be fast. When you’re at a point that it makes sense to check the results
 of the `ignored` tests and you have time to wait for the results, you can
 choose to run `cargo test -- --ignored` instead.
 
@@ -1180,7 +1125,7 @@ doing what you expect them to separately and together.
 The purpose of unit tests is to test each unit of code in isolation from the
 rest of the code, in order to be able to quickly pinpoint where code is and is
 not working as expected. We put unit tests in the *src* directory, in each file
-with the code that they're testing. The convention is that we create a module
+with the code that they’re testing. The convention is that we create a module
 named `tests` in each file to contain the test functions, and we annotate the
 module with `cfg(test)`.
 
@@ -1189,8 +1134,8 @@ module with `cfg(test)`.
 The `#[cfg(test)]` annotation on the tests module tells Rust to compile and run
 the test code only when we run `cargo test`, and not when we run `cargo build`.
 This saves compile time when we only want to build the library, and saves space
-in the resulting compiled artifact since the tests are not included. We'll see
+in the resulting compiled artifact since the tests are not included. We’ll see
-that since integration tests go in a different directory, they don't need the
+that since integration tests go in a different directory, they don’t need the
 `#[cfg(test)]` annotation. Because unit tests go in the same files as the code,
 though, we use `#[cfg(test)]`to specify that they should not be included in the
 compiled result.
@@ -1210,19 +1155,19 @@ mod tests {
 ```
 
 This is the automatically generated test module. The attribute `cfg` stands for
-*configruation*, and tells Rust that the following item should only be included
+*configuration*, and tells Rust that the following item should only be included
-given a certain configuration. In this case, the configuration is `test`,
+given a certain configuration option. In this case, the configuration option is
-provided by Rust for compiling and running tests. By using this attribute,
+`test`, provided by Rust for compiling and running tests. By using this
-Cargo only compiles our test code if we actively run the tests with `cargo
+attribute, Cargo only compiles our test code if we actively run the tests with
-test`. This includes any helper functions that might be within this module, in
+`cargo test`. This includes any helper functions that might be within this
-addition to the functions annotated with `#[test]`.
+module, in addition to the functions annotated with `#[test]`.
 
 #### Testing Private Functions
 
-There's debate within the testing community about whether private functions
+There’s debate within the testing community about whether private functions
 should be tested directly or not, and other languages make it difficult or
 impossible to test private functions. Regardless of which testing ideology you
-adhere to, Rust's privacy rules do allow you to test private functions.
+adhere to, Rust’s privacy rules do allow you to test private functions.
 Consider the code in Listing 11-12 with the private function `internal_adder`:
 
 Filename: src/lib.rs
@@ -1249,25 +1194,17 @@ mod tests {
 
 Listing 11-12: Testing a private function
 
-<!-- I'm not clear on why we would assume this might not be fine, why are we
-highlighting this specifically? -->
-<!-- We're addressing experience that the reader might bring with them from
-other languages where this is not allowed; I added a sentence mentioning "other
-languages" at the beginning of this section. Also testing private functions
-from integration tests is not allowed, so if you did want to do this, you'd
-have to do it in unit tests. /Carol -->
-
 Note that the `internal_adder` function is not marked as `pub`, but because
 tests are just Rust code and the `tests` module is just another module, we can
-import and call `internal_adder` in a test just fine. If you don't think
+import and call `internal_adder` in a test just fine. If you don’t think
-private functions should be tested, there's nothing in Rust that will compel
+private functions should be tested, there’s nothing in Rust that will compel
 you to do so.
 
 ### Integration Tests
 
 In Rust, integration tests are entirely external to your library. They use your
 library in the same way any other code would, which means they can only call
-functions that are part of your library's public API. Their purpose is to test
+functions that are part of your library’s public API. Their purpose is to test
 that many parts of your library work correctly together. Units of code that
 work correctly by themselves could have problems when integrated, so test
 coverage of the integrated code is important as well. To create integration
@@ -1278,10 +1215,10 @@ tests, you first need a *tests* directory.
 To write integration tests for our code, we need to make a *tests* directory at
 the top level of our project directory, next to *src*. Cargo knows to look for
 integration test files in this directory. We can then make as many test files
-as we'd like in this directory, and Cargo will compile each of the files as an
+as we’d like in this directory, and Cargo will compile each of the files as an
 individual crate.
 
-Let's give it a try! Keep the code from Listing 11-12 in *src/lib.rs*. Make a
+Let’s give it a try! Keep the code from Listing 11-12 in *src/lib.rs*. Make a
 *tests* directory, then make a new file named *tests/integration_test.rs*, and
 enter the code in Listing 11-13.
 
@@ -1298,21 +1235,21 @@ fn it_adds_two() {
 
 Listing 11-13: An integration test of a function in the `adder` crate
 
-We've added `extern crate adder` at the top, which we didn't need in the unit
+We’ve added `extern crate adder` at the top, which we didn’t need in the unit
 tests. This is because each test in the `tests` directory is an entirely
 separate crate, so we need to import our library into each of them. Integration
 tests use the library like any other consumer of it would, by importing the
 crate and using only the public API.
 
-We don't need to annotate any code in *tests/integration_test.rs* with
+We don’t need to annotate any code in *tests/integration_test.rs* with
 `#[cfg(test)]`. Cargo treats the `tests` directory specially and will only
-compile files in this directory if we run `cargo test`. Let's try running
+compile files in this directory if we run `cargo test`. Let’s try running
 `cargo test` now:
 
 ```
 cargo test
    Compiling adder v0.1.0 (file:///projects/adder)
-    Finished debug [unoptimized + debuginfo] target(s) in 0.31 secs
+    Finished dev [unoptimized + debuginfo] target(s) in 0.31 secs
      Running target/debug/deps/adder-abcabcabc
 
 running 1 test
@@ -1334,11 +1271,6 @@ running 0 tests
 test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured
 ```
 
-<!-- what are the doc tests? How do we tell the difference between unit and
-integration tests here? -->
-<!-- We mentioned documentation tests in the beginning of this chapter /Carol
--->
-
 Now we have three sections of output: the unit tests, the integration test, and
 the doc tests. The first section for the unit tests is the same as we have been
 seeing: one line for each unit test (we have one named `internal` that we added
@@ -1346,7 +1278,7 @@ in Listing 11-12), then a summary line for the unit tests.
 
 The integration tests section starts with the line that says `Running
 target/debug/deps/integration-test-ce99bcc2479f4607` (the hash at the end of
-your output will be different). Then there's a line for each test function in
+your output will be different). Then there’s a line for each test function in
 that integration test, and a summary line for the results of the integration
 test just before the `Doc-tests adder` section starts.
 
@@ -1357,13 +1289,13 @@ section. Each integration test file gets its own section, so if we add more
 files in the *tests* directory, there will be more integration test sections.
 
 We can still run a particular integration test function by specifying the test
-function's name as an argument to `cargo test`. To run all of the tests in a
+function’s name as an argument to `cargo test`. To run all of the tests in a
 particular integration test file, use the `--test` argument of `cargo test`
 followed by the name of the file:
 
 ```
 $ cargo test --test integration_test
-    Finished debug [unoptimized + debuginfo] target(s) in 0.0 secs
+    Finished dev [unoptimized + debuginfo] target(s) in 0.0 secs
      Running target/debug/integration_test-952a27e0126bb565
 
 running 1 test
@@ -1378,12 +1310,12 @@ This tests only the file that we specified from the *tests* directory.
 
 As you add more integration tests, you may want to make more than one file in
 the *tests* directory to help organize them; for example, to group the test
-functions by the functionality they're testing. As we mentioned, each file in
+functions by the functionality they’re testing. As we mentioned, each file in
 the *tests* directory is compiled as its own separate crate.
 
 Treating each integration test file as its own crate is useful to create
 separate scopes that are more like the way end users will be using your crate.
-However, this means files in the *tests* directory don't share the same
+However, this means files in the *tests* directory don’t share the same
 behavior as files in *src* do that we learned about in Chapter 7 regarding how
 to separate code into modules and files.
 
@@ -1403,8 +1335,8 @@ pub fn setup() {
 }
 ```
 
-If we run the tests again, we'll see a new section in the test output for the
+If we run the tests again, we’ll see a new section in the test output for the
-*common.rs* file, even though this file doesn't contain any test functions, nor
+*common.rs* file, even though this file doesn’t contain any test functions, nor
 are we calling the `setup` function from anywhere:
 
 ```
@@ -1433,23 +1365,20 @@ running 0 tests
 test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured
 ```
 
-<!-- The new section is lines 6-10, will ghost everything else in libreoffice
-/Carol -->
-
 Having `common` show up in the test results with `running 0 tests` displayed
 for it is not what we wanted; we just wanted to be able to share some code with
 the other integration test files.
 
 In order to not have `common` show up in the test output, we need to use the
 other method of extracting code into a file that we learned about in Chapter 7:
-instead of creating *tests/common.rs*, we'll create *tests/common/mod.rs*. When
+instead of creating *tests/common.rs*, we’ll create *tests/common/mod.rs*. When
 we move the `setup` function code into *tests/common/mod.rs* and get rid of the
 *tests/common.rs* file, the section in the test output will no longer show up.
 Files in subdirectories of the *tests* directory do not get compiled as
 separate crates or have sections in the test output.
 
 Once we have *tests/common/mod.rs*, we can use it from any of the integration
-test files as a module. Here's an example of calling the `setup` function from
+test files as a module. Here’s an example of calling the `setup` function from
 the `it_adds_two` test in *tests/integration_test.rs*:
 
 Filename: tests/integration_test.rs
@@ -1473,7 +1402,7 @@ function.
 #### Integration Tests for Binary Crates
 
 If our project is a binary crate that only contains a *src/main.rs* and does
-not have a *src/lib.rs*, we aren't able to create integration tests in the
+not have a *src/lib.rs*, we aren’t able to create integration tests in the
 *tests* directory and use `extern crate` to import functions defined in
 *src/main.rs*. Only library crates expose functions that other crates are able
 to call and use; binary crates are meant to be run on their own.
@@ -1487,14 +1416,14 @@ small amount of code does not need to be tested.
 
 ## Summary
 
-Rust's testing features provide a way to specify how code should function to
+Rust’s testing features provide a way to specify how code should function to
 ensure it continues to work as we expect even as we make changes. Unit tests
 exercise different parts of a library separately and can test private
 implementation details. Integration tests cover the use of many parts of the
-library working together, and they use the library's public API to test the
+library working together, and they use the library’s public API to test the
-code in the same way external code will use it. Even though Rust's type system
+code in the same way external code will use it. Even though Rust’s type system
 and ownership rules help prevent some kinds of bugs, tests are still important
 to help reduce logic bugs having to do with how your code is expected to behave.
 
-Let's put together the knowledge from this chapter and other previous chapters
+Let’s put together the knowledge from this chapter and other previous chapters
 and work on a project in the next chapter!

src/doc/book/second-edition/src/SUMMARY.md

@@ -57,13 +57,13 @@
     - [Running tests](ch11-02-running-tests.md)
     - [Test Organization](ch11-03-test-organization.md)
 
-- [An I/O Project](ch12-00-an-io-project.md)
+- [An I/O Project: Building a Command Line Program](ch12-00-an-io-project.md)
     - [Accepting Command Line Arguments](ch12-01-accepting-command-line-arguments.md)
     - [Reading a File](ch12-02-reading-a-file.md)
-    - [Improving Error Handling and Modularity](ch12-03-improving-error-handling-and-modularity.md)
+    - [Refactoring to Improve Modularity and Error Handling](ch12-03-improving-error-handling-and-modularity.md)
     - [Testing the Library's Functionality](ch12-04-testing-the-librarys-functionality.md)
     - [Working with Environment Variables](ch12-05-working-with-environment-variables.md)
-    - [Writing to `stderr` instead of `stdout`](ch12-06-writing-to-stderr-instead-of-stdout.md)
+    - [Writing Error Messages to `stderr` Instead of `stdout`](ch12-06-writing-to-stderr-instead-of-stdout.md)
 
 ## Thinking in Rust
 

src/doc/book/second-edition/src/appendix-06-translation.md

@@ -6,8 +6,12 @@ For resources in languages other than English. Most are still in progress; see
 [label]: https://github.com/rust-lang/book/issues?q=is%3Aopen+is%3Aissue+label%3ATranslations
 
 - [Português](https://coreh.github.io/rust-book-pt-br/)
-- [Tiếng việt](https://rust-vietnam.github.io/book/)
+- [Tiếng việt](https://github.com/hngnaig/rust-lang-book/tree/vi-VN)
 - [简体中文](http://www.broadview.com.cn/article/144), [alternate](https://github.com/KaiserY/trpl-zh-cn)
 - [українська мова](https://github.com/pavloslav/rust-book-uk-ua)
 - [Español](https://github.com/z1mvader/book)
-- [Italiano](https://github.com/CodelessFuture/trpl2-it)
+- [Italiano](https://github.com/CodelessFuture/trpl2-it)
+- [Русский](https://github.com/iDeBugger/rust-book-ru)
+- [한국어](https://github.com/rinthel/rust-lang-book-ko)
+- [日本語](https://github.com/hazama-yuinyan/book)
+- [Français](https://github.com/quadrifoglio/rust-book-fr)

src/doc/book/second-edition/src/ch01-01-installation.md

@@ -30,6 +30,19 @@ Rust is installed now. Great!
 Of course, if you disapprove of the `curl | sh` pattern, you can download, inspect
 and run the script however you like.
 
+The installation script automatically adds Rust to your system PATH after your next login. 
+If you want to start using Rust right away, run the following command in your shell:
+
+```text
+$ source $HOME/.cargo/env
+```
+
+Alternatively, add the following line to your `~/.bash_profile`:
+
+```text
+$ export PATH="$HOME/.cargo/bin:$PATH"
+```
+
 ### Installing on Windows
 
 On Windows, go to [https://rustup.rs](https://rustup.rs/)<!-- ignore --> and

src/doc/book/second-edition/src/ch04-01-what-is-ownership.md

@@ -327,7 +327,7 @@ use `s1` after `s2` is created:
 let s1 = String::from("hello");
 let s2 = s1;
 
-println!("{}", s1);
+println!("{}, world!", s1);
 ```
 
 You’ll get an error like this because Rust prevents you from using the

src/doc/book/second-edition/src/ch05-01-defining-structs.md

@@ -55,14 +55,35 @@ struct</span>
 To get a specific value from a struct, we can use dot notation. If we wanted
 just this user’s email address, we can use `user1.email` wherever we want to
 use this value. To change a value in a struct, if the instance is mutable, we
-can use the dot notation and assign into a particular field, such as
+can use the dot notation and assign into a particular field. Listing 5-3 shows
-`user1.email = String::from("someone-else@example.com");`.
+how to change the value in the `email` field of a mutable `User` instance:
+
+```rust
+# struct User {
+#     username: String,
+#     email: String,
+#     sign_in_count: u64,
+#     active: bool,
+# }
+#
+let mut user1 = User {
+    email: String::from("someone@example.com"),
+    username: String::from("someusername123"),
+    active: true,
+    sign_in_count: 1,
+};
+
+user1.email = String::from("anotheremail@example.com");
+```
+
+<span class="caption">Listing 5-3: Changing the value in the `email` field of a
+`User` instance</span>
 
 ### Field Init Shorthand when Variables Have the Same Name as Fields
 
 If you have variables with the same names as struct fields, you can use *field
 init shorthand*. This can make functions that create new instances of structs
-more concise. The function named `build_user` shown here in Listing 5-3 has
+more concise. The function named `build_user` shown here in Listing 5-4 has
 parameters named `email` and `username`. The function creates and returns a
 `User` instance:
 
@@ -84,13 +105,13 @@ fn build_user(email: String, username: String) -> User {
 }
 ```
 
-<span class="caption">Listing 5-3: A `build_user` function that takes an email
+<span class="caption">Listing 5-4: A `build_user` function that takes an email
 and username and returns a `User` instance</span>
 
 Because the parameter names `email` and `username` are the same as the `User`
 struct's field names `email` and `username`, we can write `build_user` without
-the repetition of `email` and `username` as shown in Listing 5-4. This version
+the repetition of `email` and `username` as shown in Listing 5-5. This version
-of `build_user` behaves the same way as the one in Listing 5-3. The field init
+of `build_user` behaves the same way as the one in Listing 5-4. The field init
 syntax can make cases like this shorter to write, especially when structs have
 many fields.
 
@@ -112,14 +133,14 @@ fn build_user(email: String, username: String) -> User {
 }
 ```
 
-<span class="caption">Listing 5-4: A `build_user` function that uses field init
+<span class="caption">Listing 5-5: A `build_user` function that uses field init
 syntax since the `email` and `username` parameters have the same name as struct
 fields</span>
 
 ### Creating Instances From Other Instances With Struct Update Syntax
 
 It's often useful to create a new instance from an old instance, using most of
-the old instance's values but changing some. Listing 5-5 shows an example of
+the old instance's values but changing some. Listing 5-6 shows an example of
 creating a new `User` instance in `user2` by setting the values of `email` and
 `username` but using the same values for the rest of the fields from the
 `user1` instance we created in Listing 5-2:
@@ -147,13 +168,13 @@ let user2 = User {
 };
 ```
 
-<span class="caption">Listing 5-5: Creating a new `User` instance, `user2`, and
+<span class="caption">Listing 5-6: Creating a new `User` instance, `user2`, and
 setting some fields to the values of the same fields from `user1`</span>
 
-  The *struct update syntax* achieves the same effect as the code in Listing
+The *struct update syntax* achieves the same effect as the code in Listing 5-6
-5-5 using less code. The struct update syntax uses `..` to specify that the
+using less code. The struct update syntax uses `..` to specify that the
 remaining fields not set explicitly should have the same value as the fields in
-the given instance. The code in Listing 5-6 also creates an instance in `user2`
+the given instance. The code in Listing 5-7 also creates an instance in `user2`
 that has a different value for `email` and `username` but has the same values
 for the `active` and `sign_in_count` fields that `user1` has:
 
@@ -179,7 +200,7 @@ let user2 = User {
 };
 ```
 
-<span class="caption">Listing5-6: Using struct update syntax to set a new
+<span class="caption">Listing 5-7: Using struct update syntax to set a new
 `email` and `username` values for a `User` instance but use the rest of the
 values from the fields of the instance in the `user1` variable</span>
 

src/doc/book/second-edition/src/ch05-02-example-structs.md

@@ -6,7 +6,7 @@ refactor the program until we’re using structs instead.
 
 Let’s make a new binary project with Cargo called *rectangles* that will take
 the length and width of a rectangle specified in pixels and will calculate the
-area of the rectangle. Listing 5-7 shows a short program with one way of doing
+area of the rectangle. Listing 5-8 shows a short program with one way of doing
 just that in our project’s *src/main.rs*:
 
 <span class="filename">Filename: src/main.rs</span>
@@ -27,7 +27,7 @@ fn area(length: u32, width: u32) -> u32 {
 }
 ```
 
-<span class="caption">Listing 5-7: Calculating the area of a rectangle
+<span class="caption">Listing 5-8: Calculating the area of a rectangle
 specified by its length and width in separate variables</span>
 
 Now, run this program using `cargo run`:
@@ -38,7 +38,7 @@ The area of the rectangle is 1500 square pixels.
 
 ### Refactoring with Tuples
 
-Even though Listing 5-7 works and figures out the area of the rectangle by
+Even though Listing 5-8 works and figures out the area of the rectangle by
 calling the `area` function with each dimension, we can do better. The length
 and the width are related to each other because together they describe one
 rectangle.
@@ -54,7 +54,7 @@ function we wrote has two parameters. The parameters are related, but that’s
 not expressed anywhere in our program. It would be more readable and more
 manageable to group length and width together. We’ve already discussed one way
 we might do that in the Grouping Values into Tuples section of Chapter 3 on
-page XX: by using tuples. Listing 5-8 shows another version of our program that
+page XX: by using tuples. Listing 5-9 shows another version of our program that
 uses tuples:
 
 <span class="filename">Filename: src/main.rs</span>
@@ -94,7 +94,7 @@ our code.
 
 We use structs to add meaning by labeling the data. We can transform the tuple
 we’re using into a data type with a name for the whole as well as names for the
-parts, as shown in Listing 5-9:
+parts, as shown in Listing 5-10:
 
 <span class="filename">Filename: src/main.rs</span>
 
@@ -118,7 +118,7 @@ fn area(rectangle: &Rectangle) -> u32 {
 }
 ```
 
-<span class="caption">Listing 5-9: Defining a `Rectangle` struct</span>
+<span class="caption">Listing 5-10: Defining a `Rectangle` struct</span>
 
 Here we’ve defined a struct and named it `Rectangle`. Inside the `{}` we
 defined the fields as `length` and `width`, both of which have type `u32`. Then
@@ -143,7 +143,7 @@ and `1`—a win for clarity.
 
 It would be helpful to be able to print out an instance of the `Rectangle`
 while we’re debugging our program in order to see the values for all its
-fields. Listing 5-10 uses the `println!` macro as we have been in earlier
+fields. Listing 5-11 uses the `println!` macro as we have been in earlier
 chapters:
 
 <span class="filename">Filename: src/main.rs</span>
@@ -161,7 +161,7 @@ fn main() {
 }
 ```
 
-<span class="caption">Listing 5-10: Attempting to print a `Rectangle`
+<span class="caption">Listing 5-11: Attempting to print a `Rectangle`
 instance</span>
 
 When we run this code, we get an error with this core message:
@@ -209,7 +209,7 @@ crate, add `#[derive(Debug)]` or manually implement it
 Rust *does* include functionality to print out debugging information, but we
 have to explicitly opt-in to make that functionality available for our struct.
 To do that, we add the annotation `#[derive(Debug)]` just before the struct
-definition, as shown in Listing 5-11:
+definition, as shown in Listing 5-12:
 
 <span class="filename">Filename: src/main.rs</span>
 
@@ -227,7 +227,7 @@ fn main() {
 }
 ```
 
-<span class="caption">Listing5-11: Adding the annotation to derive the `Debug`
+<span class="caption">Listing 5-12: Adding the annotation to derive the `Debug`
 trait and printing the `Rectangle` instance using debug formatting</span>
 
 Now when we run the program, we won’t get any errors and we’ll see the

src/doc/book/second-edition/src/ch05-03-method-syntax.md

@@ -1,7 +1,7 @@
 ## Method Syntax
 
 *Methods* are similar to functions: they’re declared with the `fn` keyword and
-their name, they can have parameters and return values, and they contain some
+their name, they can have parameters and a return value, and they contain some
 code that is run when they’re called from somewhere else. However, methods are
 different from functions in that they’re defined within the context of a struct
 (or an enum or a trait object, which we cover in Chapters 6 and 17,
@@ -12,7 +12,7 @@ instance of the struct the method is being called on.
 
 Let’s change the `area` function that has a `Rectangle` instance as a parameter
 and instead make an `area` method defined on the `Rectangle` struct, as shown
-in Listing 5-12:
+in Listing 5-13:
 
 <span class="filename">Filename: src/main.rs</span>
 
@@ -39,7 +39,7 @@ fn main() {
 }
 ```
 
-<span class="caption">Listing 5-12: Defining an `area` method on the
+<span class="caption">Listing 5-13: Defining an `area` method on the
 `Rectangle` struct</span>
 
 To define the function within the context of `Rectangle`, we start an `impl`
@@ -124,7 +124,7 @@ Let’s practice using methods by implementing a second method on the `Rectangle
 struct. This time, we want an instance of `Rectangle` to take another instance
 of `Rectangle` and return `true` if the second `Rectangle` can fit completely
 within `self`; otherwise it should return `false`. That is, we want to be able
-to write the program shown in Listing 5-13, once we’ve defined the `can_hold`
+to write the program shown in Listing 5-14, once we’ve defined the `can_hold`
 method:
 
 <span class="filename">Filename: src/main.rs</span>
@@ -140,7 +140,7 @@ fn main() {
 }
 ```
 
-<span class="caption">Listing 5-13: Demonstration of using the as-yet-unwritten
+<span class="caption">Listing 5-14: Demonstration of using the as-yet-unwritten
 `can_hold` method</span>
 
 And the expected output would look like the following, because both dimensions
@@ -164,7 +164,7 @@ calling the `can_hold` method. The return value of `can_hold` will be a
 boolean, and the implementation will check whether the length and width of
 `self` are both greater than the length and width of the other `Rectangle`,
 respectively. Let’s add the new `can_hold` method to the `impl` block from
-Listing 5-12, shown in Listing 5-14:
+Listing 5-13, shown in Listing 5-15:
 
 <span class="filename">Filename: src/main.rs</span>
 
@@ -186,10 +186,10 @@ impl Rectangle {
 }
 ```
 
-<span class="caption">Listing 5-14: Implementing the `can_hold` method on
+<span class="caption">Listing 5-15: Implementing the `can_hold` method on
 `Rectangle` that takes another `Rectangle` instance as a parameter</span>
 
-When we run this code with the `main` function in Listing 5-13, we’ll get our
+When we run this code with the `main` function in Listing 5-14, we’ll get our
 desired output. Methods can take multiple parameters that we add to the
 signature after the `self` parameter, and those parameters work just like
 parameters in functions.