proxmox-mirrors/rustc
1
0
Fork 0
mirror of https://git.proxmox.com/git/rustc synced 2025-08-13 20:30:15 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

New upstream version 1.21.0+dfsg1

Browse Source
This commit is contained in:
Ximin Luo 2017-10-16 16:39:26 +02:00
parent 041b39d230
commit 3b2f29766f
2023 changed files with 114908 additions and 42293 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

15
CONTRIBUTING.md
View File

@ -99,7 +99,7 @@ Before you can start building the compiler you need to configure the build for
your system. In most cases, that will just mean using the defaults provided
for Rust.
To change configuration, you must copy the file `src/bootstrap/config.toml.example`
To change configuration, you must copy the file `config.toml.example`
to `config.toml` in the directory from which you will be running the build, and
change the settings provided.
@ -237,10 +237,13 @@ Some common invocations of `x.py` are:
## Pull Requests
Pull requests are the primary mechanism we use to change Rust. GitHub itself
has some [great documentation][pull-requests] on using the Pull Request
feature. We use the 'fork and pull' model described there.
has some [great documentation][pull-requests] on using the Pull Request feature.
We use the "fork and pull" model [described here][development-models], where
contributors push changes to their personal fork and create pull requests to
bring those changes into the source repository.
[pull-requests]: https://help.github.com/articles/using-pull-requests/
[pull-requests]: https://help.github.com/articles/about-pull-requests/
[development-models]: https://help.github.com/articles/about-collaborative-development-models/
Please make pull requests against the `master` branch.
@ -289,7 +292,7 @@ been approved. The PR then enters the [merge queue][merge-queue], where @bors
will run all the tests on every platform we support. If it all works out,
@bors will merge your code into `master` and close the pull request.
[merge-queue]: https://buildbot.rust-lang.org/homu/queue/rust
[merge-queue]: https://buildbot2.rust-lang.org/homu/queue/rust
Speaking of tests, Rust has a comprehensive test suite. More information about
it can be found
@ -412,4 +415,4 @@ are:
[tlgba]: http://tomlee.co/2014/04/a-more-detailed-tour-of-the-rust-compiler/
[ro]: http://www.rustaceans.org/
[rctd]: ./src/test/COMPILER_TESTS.md
[cheatsheet]: https://buildbot.rust-lang.org/homu/
[cheatsheet]: https://buildbot2.rust-lang.org/homu/

41
COPYRIGHT
View File

@ -6,16 +6,17 @@ terms.
Longer version:
The Rust Project is copyright 2010, The Rust Project
Developers.
Copyrights in the Rust project are retained by their contributors. No
copyright assignment is required to contribute to the Rust project.
Licensed under the Apache License, Version 2.0
<LICENSE-APACHE or
http://www.apache.org/licenses/LICENSE-2.0> or the MIT
license <LICENSE-MIT or http://opensource.org/licenses/MIT>,
at your option. All files in the project carrying such
notice may not be copied, modified, or distributed except
according to those terms.
Some files include explicit copyright notices and/or license notices.
For full authorship information, see AUTHORS.txt and the version control
history.
Except as otherwise noted (below and/or in individual files), Rust is
licensed under the Apache License, Version 2.0 <LICENSE-APACHE> or
<http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
<LICENSE-MIT> or <http://opensource.org/licenses/MIT>, at your option.
The Rust Project includes packages written by third parties.
@ -282,25 +283,3 @@ their own copyright notices and license terms:
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE
USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY
OF SUCH DAMAGE.
* Additional copyright may be retained by contributors other
than Mozilla, the Rust Project Developers, or the parties
enumerated in this file. Such copyright can be determined
on a case-by-case basis by examining the author of each
portion of a file in the revision-control commit records
of the project, or by consulting representative comments
claiming copyright ownership for a file.
For example, the text:
"Copyright (c) 2011 Google Inc."
appears in some files, and these files thereby denote
that their author and copyright-holder is Google Inc.
In all such cases, the absence of explicit licensing text
indicates that the contributor chose to license their work
for distribution under identical terms to those Mozilla
has chosen for the collective work, enumerated at the top
of this file. The only difference is the retention of
copyright itself, held by the contributor.

2
LICENSE-MIT
View File

@ -1,5 +1,3 @@
Copyright (c) 2010 The Rust Project Developers
Permission is hereby granted, free of charge, to any
person obtaining a copy of this software and associated
documentation files (the "Software"), to deal in the

4
README.md
View File

@ -39,7 +39,7 @@ Read ["Installation"] from [The Book].
```
> ***Note:*** Install locations can be adjusted by copying the config file
> from `./src/bootstrap/config.toml.example` to `./config.toml`, and
> from `./config.toml.example` to `./config.toml`, and
> adjusting the `prefix` option under `[install]`. Various other options, such
> as enabling debug information, are also supported, and are documented in
> the config file.
@ -135,7 +135,7 @@ Windows build triples are:
- `i686-pc-windows-msvc`
- `x86_64-pc-windows-msvc`
The build triple can be specified by either specifying `--build=ABI` when
The build triple can be specified by either specifying `--build=<triple>` when
invoking `x.py` commands, or by copying the `config.toml` file (as described
in Building From Source), and modifying the `build` option under the `[build]`
section.

373
RELEASES.md
View File

@ -1,3 +1,370 @@
Version 1.20.0 (2017-08-31)
===========================
Language
--------
- [Associated constants in traits is now stabilised.][42809]
- [A lot of macro bugs are now fixed.][42913]
Compiler
--------
- [Struct fields are now properly coerced to the expected field type.][42807]
- [Enabled wasm LLVM backend][42571] WASM can now be built with the
`wasm32-experimental-emscripten` target.
- [Changed some of the error messages to be more helpful.][42033]
- [Add support for RELRO(RELocation Read-Only) for platforms that support
it.][43170]
- [rustc now reports the total number of errors on compilation failure][43015]
previously this was only the number of errors in the pass that failed.
- [Expansion in rustc has been sped up 29x.][42533]
- [added `msp430-none-elf` target.][43099]
- [rustc will now suggest one-argument enum variant to fix type mismatch when
applicable][43178]
- [Fixes backtraces on Redox][43228]
- [rustc now identifies different versions of same crate when absolute paths of
different types match in an error message.][42826]
Libraries
---------
- [Relaxed Debug constraints on `{HashMap,BTreeMap}::{Keys,Values}`.][42854]
- [Impl `PartialEq`, `Eq`, `PartialOrd`, `Ord`, `Debug`, `Hash` for unsized
tuples.][43011]
- [Impl `fmt::{Display, Debug}` for `Ref`, `RefMut`, `MutexGuard`,
`RwLockReadGuard`, `RwLockWriteGuard`][42822]
- [Impl `Clone` for `DefaultHasher`.][42799]
- [Impl `Sync` for `SyncSender`.][42397]
- [Impl `FromStr` for `char`][42271]
- [Fixed how `{f32, f64}::{is_sign_negative, is_sign_positive}` handles
NaN.][42431]
- [allow messages in the `unimplemented!()` macro.][42155]
ie. `unimplemented!("Waiting for 1.21 to be stable")`
- [`pub(restricted)` is now supported in the `thread_local!` macro.][43185]
- [Upgrade to Unicode 10.0.0][42999]
- [Reimplemented `{f32, f64}::{min, max}` in Rust instead of using CMath.][42430]
- [Skip the main thread's manual stack guard on Linux][43072]
- [Iterator::nth for `ops::{Range, RangeFrom}` is now done in O(1) time][43077]
- [`#[repr(align(N))]` attribute max number is now 2^31 - 1.][43097] This was
previously 2^15.
- [`{OsStr, Path}::Display` now avoids allocations where possible][42613]
Stabilized APIs
---------------
- [`CStr::into_c_string`]
- [`CString::as_c_str`]
- [`CString::into_boxed_c_str`]
- [`Chain::get_mut`]
- [`Chain::get_ref`]
- [`Chain::into_inner`]
- [`Option::get_or_insert_with`]
- [`Option::get_or_insert`]
- [`OsStr::into_os_string`]
- [`OsString::into_boxed_os_str`]
- [`Take::get_mut`]
- [`Take::get_ref`]
- [`Utf8Error::error_len`]
- [`char::EscapeDebug`]
- [`char::escape_debug`]
- [`compile_error!`]
- [`f32::from_bits`]
- [`f32::to_bits`]
- [`f64::from_bits`]
- [`f64::to_bits`]
- [`mem::ManuallyDrop`]
- [`slice::sort_unstable_by_key`]
- [`slice::sort_unstable_by`]
- [`slice::sort_unstable`]
- [`ste::from_boxed_utf8_unchecked`]
- [`str::as_bytes_mut`]
- [`str::as_bytes_mut`]
- [`str::from_utf8_mut`]
- [`str::from_utf8_unchecked_mut`]
- [`str::get_mut`]
- [`str::get_unchecked_mut`]
- [`str::get_unchecked`]
- [`str::get`]
- [`str::into_boxed_bytes`]
Cargo
-----
- [Cargo API token location moved from `~/.cargo/config` to
`~/.cargo/credentials`.][cargo/3978]
- [Cargo will now build `main.rs` binaries that are in sub-directories of
`src/bin`.][cargo/4214] ie. Having `src/bin/server/main.rs` and
`src/bin/client/main.rs` generates `target/debug/server` and `target/debug/client`
- [You can now specify version of a binary when installed through
`cargo install` using `--vers`.][cargo/4229]
- [Added `--no-fail-fast` flag to cargo to run all benchmarks regardless of
failure.][cargo/4248]
- [Changed the convention around which file is the crate root.][cargo/4259]
- [The `include`/`exclude` property in `Cargo.toml` now accepts gitignore paths
instead of glob patterns][cargo/4270]. Glob patterns are now deprecated.
Compatibility Notes
-------------------
- [Functions with `'static` in their return types will now not be as usable as
if they were using lifetime parameters instead.][42417]
- [The reimplementation of `{f32, f64}::is_sign_{negative, positive}` now
takes the sign of NaN into account where previously didn't.][42430]
[42033]: https://github.com/rust-lang/rust/pull/42033
[42155]: https://github.com/rust-lang/rust/pull/42155
[42271]: https://github.com/rust-lang/rust/pull/42271
[42397]: https://github.com/rust-lang/rust/pull/42397
[42417]: https://github.com/rust-lang/rust/pull/42417
[42430]: https://github.com/rust-lang/rust/pull/42430
[42431]: https://github.com/rust-lang/rust/pull/42431
[42533]: https://github.com/rust-lang/rust/pull/42533
[42571]: https://github.com/rust-lang/rust/pull/42571
[42613]: https://github.com/rust-lang/rust/pull/42613
[42799]: https://github.com/rust-lang/rust/pull/42799
[42807]: https://github.com/rust-lang/rust/pull/42807
[42809]: https://github.com/rust-lang/rust/pull/42809
[42822]: https://github.com/rust-lang/rust/pull/42822
[42826]: https://github.com/rust-lang/rust/pull/42826
[42854]: https://github.com/rust-lang/rust/pull/42854
[42913]: https://github.com/rust-lang/rust/pull/42913
[42999]: https://github.com/rust-lang/rust/pull/42999
[43011]: https://github.com/rust-lang/rust/pull/43011
[43015]: https://github.com/rust-lang/rust/pull/43015
[43072]: https://github.com/rust-lang/rust/pull/43072
[43077]: https://github.com/rust-lang/rust/pull/43077
[43097]: https://github.com/rust-lang/rust/pull/43097
[43099]: https://github.com/rust-lang/rust/pull/43099
[43170]: https://github.com/rust-lang/rust/pull/43170
[43178]: https://github.com/rust-lang/rust/pull/43178
[43185]: https://github.com/rust-lang/rust/pull/43185
[43228]: https://github.com/rust-lang/rust/pull/43228
[cargo/3978]: https://github.com/rust-lang/cargo/pull/3978
[cargo/4214]: https://github.com/rust-lang/cargo/pull/4214
[cargo/4229]: https://github.com/rust-lang/cargo/pull/4229
[cargo/4248]: https://github.com/rust-lang/cargo/pull/4248
[cargo/4259]: https://github.com/rust-lang/cargo/pull/4259
[cargo/4270]: https://github.com/rust-lang/cargo/pull/4270
[`CStr::into_c_string`]: https://doc.rust-lang.org/std/ffi/struct.CStr.html#method.into_c_string
[`CString::as_c_str`]: https://doc.rust-lang.org/std/ffi/struct.CString.html#method.as_c_str
[`CString::into_boxed_c_str`]: https://doc.rust-lang.org/std/ffi/struct.CString.html#method.into_boxed_c_str
[`Chain::get_mut`]: https://doc.rust-lang.org/std/io/struct.Chain.html#method.get_mut
[`Chain::get_ref`]: https://doc.rust-lang.org/std/io/struct.Chain.html#method.get_ref
[`Chain::into_inner`]: https://doc.rust-lang.org/std/io/struct.Chain.html#method.into_inner
[`Option::get_or_insert_with`]: https://doc.rust-lang.org/std/option/enum.Option.html#method.get_or_insert_with
[`Option::get_or_insert`]: https://doc.rust-lang.org/std/option/enum.Option.html#method.get_or_insert
[`OsStr::into_os_string`]: https://doc.rust-lang.org/std/ffi/struct.OsStr.html#method.into_os_string
[`OsString::into_boxed_os_str`]: https://doc.rust-lang.org/std/ffi/struct.OsString.html#method.into_boxed_os_str
[`Take::get_mut`]: https://doc.rust-lang.org/std/io/struct.Take.html#method.get_mut
[`Take::get_ref`]: https://doc.rust-lang.org/std/io/struct.Take.html#method.get_ref
[`Utf8Error::error_len`]: https://doc.rust-lang.org/std/str/struct.Utf8Error.html#method.error_len
[`char::EscapeDebug`]: https://doc.rust-lang.org/std/char/struct.EscapeDebug.html
[`char::escape_debug`]: https://doc.rust-lang.org/std/primitive.char.html#method.escape_debug
[`compile_error!`]: https://doc.rust-lang.org/std/macro.compile_error.html
[`f32::from_bits`]: https://doc.rust-lang.org/std/primitive.f32.html#method.from_bits
[`f32::to_bits`]: https://doc.rust-lang.org/std/primitive.f32.html#method.to_bits
[`f64::from_bits`]: https://doc.rust-lang.org/std/primitive.f64.html#method.from_bits
[`f64::to_bits`]: https://doc.rust-lang.org/std/primitive.f64.html#method.to_bits
[`mem::ManuallyDrop`]: https://doc.rust-lang.org/std/mem/union.ManuallyDrop.html
[`slice::sort_unstable_by_key`]: https://doc.rust-lang.org/std/primitive.slice.html#method.sort_unstable_by_key
[`slice::sort_unstable_by`]: https://doc.rust-lang.org/std/primitive.slice.html#method.sort_unstable_by
[`slice::sort_unstable`]: https://doc.rust-lang.org/std/primitive.slice.html#method.sort_unstable
[`ste::from_boxed_utf8_unchecked`]: https://doc.rust-lang.org/std/str/fn.from_boxed_utf8_unchecked.html
[`str::as_bytes_mut`]: https://doc.rust-lang.org/std/primitive.str.html#method.as_bytes_mut
[`str::from_utf8_mut`]: https://doc.rust-lang.org/std/str/fn.from_utf8_mut.html
[`str::from_utf8_unchecked_mut`]: https://doc.rust-lang.org/std/str/fn.from_utf8_unchecked_mut.html
[`str::get_mut`]: https://doc.rust-lang.org/std/primitive.str.html#method.get_mut
[`str::get_unchecked_mut`]: https://doc.rust-lang.org/std/primitive.str.html#method.get_unchecked_mut
[`str::get_unchecked`]: https://doc.rust-lang.org/std/primitive.str.html#method.get_unchecked
[`str::get`]: https://doc.rust-lang.org/std/primitive.str.html#method.get
[`str::into_boxed_bytes`]: https://doc.rust-lang.org/std/primitive.str.html#method.into_boxed_bytes
Version 1.19.0 (2017-07-20)
===========================
Language
--------
- [Numeric fields can now be used for creating tuple structs.][41145] [RFC 1506]
For example `struct Point(u32, u32); let x = Point { 0: 7, 1: 0 };`.
- [Macro recursion limit increased to 1024 from 64.][41676]
- [Added lint for detecting unused macros.][41907]
- [`loop` can now return a value with `break`.][42016] [RFC 1624]
For example: `let x = loop { break 7; };`
- [C compatible `union`s are now available.][42068] [RFC 1444] They can only
contain `Copy` types and cannot have a `Drop` implementation.
Example: `union Foo { bar: u8, baz: usize }`
- [Non capturing closures can now be coerced into `fn`s,][42162] [RFC 1558]
Example: `let foo: fn(u8) -> u8 = |v: u8| { v };`
Compiler
--------
- [Add support for bootstrapping the Rust compiler toolchain on Android.][41370]
- [Change `arm-linux-androideabi` to correspond to the `armeabi`
official ABI.][41656] If you wish to continue targeting the `armeabi-v7a` ABI
you should use `--target armv7-linux-androideabi`.
- [Fixed ICE when removing a source file between compilation sessions.][41873]
- [Minor optimisation of string operations.][42037]
- [Compiler error message is now `aborting due to previous error(s)` instead of
`aborting due to N previous errors`][42150] This was previously inaccurate and
would only count certain kinds of errors.
- [The compiler now supports Visual Studio 2017][42225]
- [The compiler is now built against LLVM 4.0.1 by default][42948]
- [Added a lot][42264] of [new error codes][42302]
- [Added `target-feature=+crt-static` option][37406] [RFC 1721] Which allows
libraries with C Run-time Libraries(CRT) to be statically linked.
- [Fixed various ARM codegen bugs][42740]
Libraries
---------
- [`String` now implements `FromIterator<Cow<'a, str>>` and
`Extend<Cow<'a, str>>`][41449]
- [`Vec` now implements `From<&mut [T]>`][41530]
- [`Box<[u8]>` now implements `From<Box<str>>`][41258]
- [`SplitWhitespace` now implements `Clone`][41659]
- [`[u8]::reverse` is now 5x faster and `[u16]::reverse` is now
1.5x faster][41764]
- [`eprint!` and `eprintln!` macros added to prelude.][41192] Same as the `print!`
macros, but for printing to stderr.
Stabilized APIs
---------------
- [`OsString::shrink_to_fit`]
- [`cmp::Reverse`]
- [`Command::envs`]
- [`thread::ThreadId`]
Cargo
-----
- [Build scripts can now add environment variables to the environment
the crate is being compiled in.
Example: `println!("cargo:rustc-env=FOO=bar");`][cargo/3929]
- [Subcommands now replace the current process rather than spawning a new
child process][cargo/3970]
- [Workspace members can now accept glob file patterns][cargo/3979]
- [Added `--all` flag to the `cargo bench` subcommand to run benchmarks of all
the members in a given workspace.][cargo/3988]
- [Updated `libssh2-sys` to 0.2.6][cargo/4008]
- [Target directory path is now in the cargo metadata][cargo/4022]
- [Cargo no longer checks out a local working directory for the
crates.io index][cargo/4026] This should provide smaller file size for the
registry, and improve cloning times, especially on Windows machines.
- [Added an `--exclude` option for excluding certain packages when using the
`--all` option][cargo/4031]
- [Cargo will now automatically retry when receiving a 5xx error
from crates.io][cargo/4032]
- [The `--features` option now accepts multiple comma or space
delimited values.][cargo/4084]
- [Added support for custom target specific runners][cargo/3954]
Misc
----
- [Added `rust-windbg.cmd`][39983] for loading rust `.natvis` files in the
Windows Debugger.
- [Rust will now release XZ compressed packages][rust-installer/57]
- [rustup will now prefer to download rust packages with
XZ compression][rustup/1100] over GZip packages.
- [Added the ability to escape `#` in rust documentation][41785] By adding
additional `#`'s ie. `##` is now `#`
Compatibility Notes
-------------------
- [`MutexGuard<T>` may only be `Sync` if `T` is `Sync`.][41624]
- [`-Z` flags are now no longer allowed to be used on the stable
compiler.][41751] This has been a warning for a year previous to this.
- [As a result of the `-Z` flag change, the `cargo-check` plugin no
longer works][42844]. Users should migrate to the built-in `check`
command, which has been available since 1.16.
- [Ending a float literal with `._` is now a hard error.
Example: `42._` .][41946]
- [Any use of a private `extern crate` outside of its module is now a
hard error.][36886] This was previously a warning.
- [`use ::self::foo;` is now a hard error.][36888] `self` paths are always
relative while the `::` prefix makes a path absolute, but was ignored and the
path was relative regardless.
- [Floating point constants in match patterns is now a hard error][36890]
This was previously a warning.
- [Struct or enum constants that don't derive `PartialEq` & `Eq` used
match patterns is now a hard error][36891] This was previously a warning.
- [Lifetimes named `'_` are no longer allowed.][36892] This was previously
a warning.
- [From the pound escape, lines consisting of multiple `#`s are
now visible][41785]
- [It is an error to reexport private enum variants][42460]. This is
known to break a number of crates that depend on an older version of
mustache.
- [On Windows, if `VCINSTALLDIR` is set incorrectly, `rustc` will try
to use it to find the linker, and the build will fail where it did
not previously][42607]
[36886]: https://github.com/rust-lang/rust/issues/36886
[36888]: https://github.com/rust-lang/rust/issues/36888
[36890]: https://github.com/rust-lang/rust/issues/36890
[36891]: https://github.com/rust-lang/rust/issues/36891
[36892]: https://github.com/rust-lang/rust/issues/36892
[37406]: https://github.com/rust-lang/rust/issues/37406
[39983]: https://github.com/rust-lang/rust/pull/39983
[41145]: https://github.com/rust-lang/rust/pull/41145
[41192]: https://github.com/rust-lang/rust/pull/41192
[41258]: https://github.com/rust-lang/rust/pull/41258
[41370]: https://github.com/rust-lang/rust/pull/41370
[41449]: https://github.com/rust-lang/rust/pull/41449
[41530]: https://github.com/rust-lang/rust/pull/41530
[41624]: https://github.com/rust-lang/rust/pull/41624
[41656]: https://github.com/rust-lang/rust/pull/41656
[41659]: https://github.com/rust-lang/rust/pull/41659
[41676]: https://github.com/rust-lang/rust/pull/41676
[41751]: https://github.com/rust-lang/rust/pull/41751
[41764]: https://github.com/rust-lang/rust/pull/41764
[41785]: https://github.com/rust-lang/rust/pull/41785
[41873]: https://github.com/rust-lang/rust/pull/41873
[41907]: https://github.com/rust-lang/rust/pull/41907
[41946]: https://github.com/rust-lang/rust/pull/41946
[42016]: https://github.com/rust-lang/rust/pull/42016
[42037]: https://github.com/rust-lang/rust/pull/42037
[42068]: https://github.com/rust-lang/rust/pull/42068
[42150]: https://github.com/rust-lang/rust/pull/42150
[42162]: https://github.com/rust-lang/rust/pull/42162
[42225]: https://github.com/rust-lang/rust/pull/42225
[42264]: https://github.com/rust-lang/rust/pull/42264
[42302]: https://github.com/rust-lang/rust/pull/42302
[42460]: https://github.com/rust-lang/rust/issues/42460
[42607]: https://github.com/rust-lang/rust/issues/42607
[42740]: https://github.com/rust-lang/rust/pull/42740
[42844]: https://github.com/rust-lang/rust/issues/42844
[42948]: https://github.com/rust-lang/rust/pull/42948
[RFC 1444]: https://github.com/rust-lang/rfcs/pull/1444
[RFC 1506]: https://github.com/rust-lang/rfcs/pull/1506
[RFC 1558]: https://github.com/rust-lang/rfcs/pull/1558
[RFC 1624]: https://github.com/rust-lang/rfcs/pull/1624
[RFC 1721]: https://github.com/rust-lang/rfcs/pull/1721
[`Command::envs`]: https://doc.rust-lang.org/std/process/struct.Command.html#method.envs
[`OsString::shrink_to_fit`]: https://doc.rust-lang.org/std/ffi/struct.OsString.html#method.shrink_to_fit
[`cmp::Reverse`]: https://doc.rust-lang.org/std/cmp/struct.Reverse.html
[`thread::ThreadId`]: https://doc.rust-lang.org/std/thread/struct.ThreadId.html
[cargo/3929]: https://github.com/rust-lang/cargo/pull/3929
[cargo/3954]: https://github.com/rust-lang/cargo/pull/3954
[cargo/3970]: https://github.com/rust-lang/cargo/pull/3970
[cargo/3979]: https://github.com/rust-lang/cargo/pull/3979
[cargo/3988]: https://github.com/rust-lang/cargo/pull/3988
[cargo/4008]: https://github.com/rust-lang/cargo/pull/4008
[cargo/4022]: https://github.com/rust-lang/cargo/pull/4022
[cargo/4026]: https://github.com/rust-lang/cargo/pull/4026
[cargo/4031]: https://github.com/rust-lang/cargo/pull/4031
[cargo/4032]: https://github.com/rust-lang/cargo/pull/4032
[cargo/4084]: https://github.com/rust-lang/cargo/pull/4084
[rust-installer/57]: https://github.com/rust-lang/rust-installer/pull/57
[rustup/1100]: https://github.com/rust-lang-nursery/rustup.rs/pull/1100
Version 1.18.0 (2017-06-08)
===========================
@ -530,6 +897,9 @@ Compatibility Notes
* [Ctrl-Z returns from `Stdin.read()` when reading from the console on
Windows][38274]
* [Clean up semantics of `self` in an import list][38313]
* Reimplemented lifetime elision. This change was almost entirely compatible
with existing code, but it did close a number of small bugs and loopholes,
as well as being more accepting in some other [cases][41105].
[37057]: https://github.com/rust-lang/rust/pull/37057
[37761]: https://github.com/rust-lang/rust/pull/37761
@ -564,6 +934,7 @@ Compatibility Notes
[39048]: https://github.com/rust-lang/rust/pull/39048
[39282]: https://github.com/rust-lang/rust/pull/39282
[39379]: https://github.com/rust-lang/rust/pull/39379
[41105]: https://github.com/rust-lang/rust/issues/41105
[`<*const T>::wrapping_offset`]: https://doc.rust-lang.org/std/primitive.pointer.html#method.wrapping_offset
[`<*mut T>::wrapping_offset`]: https://doc.rust-lang.org/std/primitive.pointer.html#method.wrapping_offset
[`Duration::checked_add`]: https://doc.rust-lang.org/std/time/struct.Duration.html#method.checked_add
@ -1495,7 +1866,7 @@ Tooling
* [Test binaries now support a `--test-threads` argument to specify the number
of threads used to run tests, and which acts the same as the
`RUST_TEST_THREADS` environment variable](https://github.com/rust-lang/rust/pull/35414)
`RUST_TEST_THREADS` environment variable](https://github.com/rust-lang/rust/pull/35414)
* [The test runner now emits a warning when tests run over 60 seconds](https://github.com/rust-lang/rust/pull/35405)
* [rustdoc: Fix methods in search results](https://github.com/rust-lang/rust/pull/34752)
* [`rust-lldb` warns about unsupported versions of LLDB](https://github.com/rust-lang/rust/pull/34646)

10
configure vendored
View File

@ -437,7 +437,6 @@ opt local-rust 0 "use an installed rustc rather than downloading a snapshot"
opt local-rebuild 0 "assume local-rust matches the current version, for rebuilds; implies local-rust, and is implied if local-rust already matches the current version"
opt llvm-static-stdcpp 0 "statically link to libstdc++ for LLVM"
opt llvm-link-shared 0 "prefer shared linking to LLVM (llvm-config --link-shared)"
opt llvm-clean-rebuild 0 "delete LLVM build directory on rebuild"
opt rpath 1 "build rpaths into rustc itself"
opt stage0-landing-pads 1 "enable landing pads during bootstrap with stage0"
# This is used by the automation to produce single-target nightlies
@ -490,6 +489,7 @@ valopt musl-root-armhf "" "arm-unknown-linux-musleabihf install directory"
valopt musl-root-armv7 "" "armv7-unknown-linux-musleabihf install directory"
valopt extra-filename "" "Additional data that is hashed and passed to the -C extra-filename flag"
valopt qemu-armhf-rootfs "" "rootfs in qemu testing, you probably don't want to use this"
valopt qemu-aarch64-rootfs "" "rootfs in qemu testing, you probably don't want to use this"
valopt experimental-targets "" "experimental LLVM targets to build"
if [ -e ${CFG_SRC_DIR}.git ]
@ -560,8 +560,8 @@ case "$CFG_RELEASE_CHANNEL" in
*-pc-windows-gnu)
;;
*)
CFG_ENABLE_DEBUGINFO_LINES=1
CFG_ENABLE_DEBUGINFO_ONLY_STD=1
enable_if_not_disabled debuginfo-lines
enable_if_not_disabled debuginfo-only-std
;;
esac
@ -572,8 +572,8 @@ case "$CFG_RELEASE_CHANNEL" in
*-pc-windows-gnu)
;;
*)
CFG_ENABLE_DEBUGINFO_LINES=1
CFG_ENABLE_DEBUGINFO_ONLY_STD=1
enable_if_not_disabled debuginfo-lines
enable_if_not_disabled debuginfo-only-std
;;
esac
;;

774
src/Cargo.lock generated
View File

File diff suppressed because it is too large Load Diff

21
src/Cargo.toml
View File

@ -16,6 +16,27 @@ members = [
"tools/remote-test-server",
"tools/rust-installer",
"tools/cargo",
"tools/rustdoc",
"tools/rls",
# FIXME(https://github.com/rust-lang/cargo/issues/4089): move these to exclude
"tools/rls/test_data/borrow_error",
"tools/rls/test_data/completion",
"tools/rls/test_data/find_all_refs",
"tools/rls/test_data/find_all_refs_no_cfg_test",
"tools/rls/test_data/goto_def",
"tools/rls/test_data/highlight",
"tools/rls/test_data/hover",
"tools/rls/test_data/rename",
"tools/rls/test_data/reformat",
"tools/rls/test_data/bin_lib_no_cfg_test",
"tools/rls/test_data/multiple_bins",
"tools/rls/test_data/bin_lib",
"tools/rls/test_data/reformat_with_range",
"tools/rls/test_data/find_impls",
"tools/rls/test_data/infer_bin",
"tools/rls/test_data/infer_custom_bin",
"tools/rls/test_data/infer_lib",
"tools/rls/test_data/omit_init_build",
]
# Curiously, compiletest will segfault if compiled with opt-level=3 on 64-bit

7
src/bootstrap/Cargo.toml
View File

@ -33,8 +33,11 @@ build_helper = { path = "../build_helper" }
cmake = "0.1.23"
filetime = "0.1"
num_cpus = "1.0"
toml = "0.1"
getopts = "0.2"
rustc-serialize = "0.3"
gcc = "0.3.50"
libc = "0.2"
serde = "1.0.8"
serde_derive = "1.0.8"
serde_json = "1.0.2"
toml = "0.4"
lazy_static = "0.2"

36
src/bootstrap/README.md
View File

@ -73,16 +73,19 @@ The script accepts commands, flags, and arguments to determine what to do:
## Configuring rustbuild
There are currently two primary methods for configuring the rustbuild build
system. First, the `./configure` options serialized in `config.mk` will be
parsed and read. That is, if any `./configure` options are passed, they'll be
handled naturally.
There are currently two methods for configuring the rustbuild build system.
Next, rustbuild offers a TOML-based configuration system with a `config.toml`
First, rustbuild offers a TOML-based configuration system with a `config.toml`
file in the same location as `config.mk`. An example of this configuration can
be found at `src/bootstrap/config.toml.example`, and the configuration file
can also be passed as `--config path/to/config.toml` if the build system is
being invoked manually (via the python script).
be found at `config.toml.example`, and the configuration file can also be passed
as `--config path/to/config.toml` if the build system is being invoked manually
(via the python script).
Next, the `./configure` options serialized in `config.mk` will be
parsed and read. That is, if any `./configure` options are passed, they'll be
handled naturally. `./configure` should almost never be used for local
installations, and is primarily useful for CI. Prefer to customize behavior
using `config.toml`.
Finally, rustbuild makes use of the [gcc-rs crate] which has [its own
method][env-vars] of configuring C compilers and C flags via environment
@ -310,17 +313,18 @@ After that, each module in rustbuild should have enough documentation to keep
you up and running. Some general areas that you may be interested in modifying
are:
* Adding a new build tool? Take a look at `bootstrap/step.rs` for examples of
* Adding a new build tool? Take a look at `bootstrap/tool.rs` for examples of
other tools.
* Adding a new compiler crate? Look no further! Adding crates can be done by
adding a new directory with `Cargo.toml` followed by configuring all
`Cargo.toml` files accordingly.
* Adding a new dependency from crates.io? We're still working on that, so hold
off on that for now.
* Adding a new configuration option? Take a look at `bootstrap/config.rs` or
perhaps `bootstrap/flags.rs` and then modify the build elsewhere to read that
option.
* Adding a new dependency from crates.io? This should just work inside the
compiler artifacts stage (everything other than libtest and libstd).
* Adding a new configuration option? You'll want to modify `bootstrap/flags.rs`
for command line flags and then `bootstrap/config.rs` to copy the flags to the
`Config` struct.
* Adding a sanity check? Take a look at `bootstrap/sanity.rs`.
If you have any questions feel free to reach out on `#rust-internals` on IRC or
open an issue in the bug tracker!
If you have any questions feel free to reach out on `#rust-infra` on IRC or ask on
internals.rust-lang.org. When you encounter bugs, please file issues on the
rust-lang/rust issue tracker.

7
src/bootstrap/bin/main.rs
View File

@ -21,11 +21,10 @@ extern crate bootstrap;
use std::env;
use bootstrap::{Flags, Config, Build};
use bootstrap::{Config, Build};
fn main() {
let args = env::args().skip(1).collect::<Vec<_>>();
let flags = Flags::parse(&args);
let config = Config::parse(&flags.build, flags.config.clone());
Build::new(flags, config).build();
let config = Config::parse(&args);
Build::new(config).build();
}

15
src/bootstrap/bin/rustc.rs
View File

@ -185,7 +185,10 @@ fn main() {
// Emit save-analysis info.
if env::var("RUSTC_SAVE_ANALYSIS") == Ok("api".to_string()) {
cmd.arg("-Zsave-analysis-api");
cmd.arg("-Zsave-analysis");
cmd.env("RUST_SAVE_ANALYSIS_CONFIG",
"{\"output_file\": null,\"full_docs\": false,\"pub_only\": true,\
\"distro_crate\": true,\"signatures\": false,\"borrow_data\": false}");
}
// Dealing with rpath here is a little special, so let's go into some
@ -234,9 +237,13 @@ fn main() {
}
}
if target.contains("pc-windows-msvc") {
cmd.arg("-Z").arg("unstable-options");
cmd.arg("-C").arg("target-feature=+crt-static");
if let Ok(s) = env::var("RUSTC_CRT_STATIC") {
if s == "true" {
cmd.arg("-C").arg("target-feature=+crt-static");
}
if s == "false" {
cmd.arg("-C").arg("target-feature=-crt-static");
}
}
// Force all crates compiled by this compiler to (a) be unstable and (b)

505
src/bootstrap/bootstrap.py
View File

@ -37,12 +37,12 @@ def get(url, path, verbose=False):
if os.path.exists(path):
if verify(path, sha_path, False):
if verbose:
print("using already-download file " + path)
print("using already-download file", path)
return
else:
if verbose:
print("ignoring already-download file " +
path + " due to failed verification")
print("ignoring already-download file",
path, "due to failed verification")
os.unlink(path)
download(temp_path, url, True, verbose)
if not verify(temp_path, sha_path, verbose):
@ -59,12 +59,12 @@ def delete_if_present(path, verbose):
"""Remove the given file if present"""
if os.path.isfile(path):
if verbose:
print("removing " + path)
print("removing", path)
os.unlink(path)
def download(path, url, probably_big, verbose):
for x in range(0, 4):
for _ in range(0, 4):
try:
_download(path, url, probably_big, verbose, True)
return
@ -96,7 +96,7 @@ def _download(path, url, probably_big, verbose, exception):
def verify(path, sha_path, verbose):
"""Check if the sha256 sum of the given path is valid"""
if verbose:
print("verifying " + path)
print("verifying", path)
with open(path, "rb") as source:
found = hashlib.sha256(source.read()).hexdigest()
with open(sha_path, "r") as sha256sum:
@ -111,29 +111,30 @@ def verify(path, sha_path, verbose):
def unpack(tarball, dst, verbose=False, match=None):
"""Unpack the given tarball file"""
print("extracting " + tarball)
print("extracting", tarball)
fname = os.path.basename(tarball).replace(".tar.gz", "")
with contextlib.closing(tarfile.open(tarball)) as tar:
for p in tar.getnames():
if "/" not in p:
for member in tar.getnames():
if "/" not in member:
continue
name = p.replace(fname + "/", "", 1)
name = member.replace(fname + "/", "", 1)
if match is not None and not name.startswith(match):
continue
name = name[len(match) + 1:]
fp = os.path.join(dst, name)
dst_path = os.path.join(dst, name)
if verbose:
print(" extracting " + p)
tar.extract(p, dst)
tp = os.path.join(dst, p)
if os.path.isdir(tp) and os.path.exists(fp):
print(" extracting", member)
tar.extract(member, dst)
src_path = os.path.join(dst, member)
if os.path.isdir(src_path) and os.path.exists(dst_path):
continue
shutil.move(tp, fp)
shutil.move(src_path, dst_path)
shutil.rmtree(os.path.join(dst, fname))
def run(args, verbose=False, exception=False, **kwargs):
"""Run a child program in a new process"""
if verbose:
print("running: " + ' '.join(args))
sys.stdout.flush()
@ -149,97 +150,118 @@ def run(args, verbose=False, exception=False, **kwargs):
def stage0_data(rust_root):
"""Build a dictionary from stage0.txt"""
nightlies = os.path.join(rust_root, "src/stage0.txt")
data = {}
with open(nightlies, 'r') as nightlies:
for line in nightlies:
line = line.rstrip() # Strip newline character, '\n'
if line.startswith("#") or line == '':
continue
a, b = line.split(": ", 1)
data[a] = b
return data
lines = [line.rstrip() for line in nightlies
if not line.startswith("#")]
return dict([line.split(": ", 1) for line in lines if line])
def format_build_time(duration):
"""Return a nicer format for build time
>>> format_build_time('300')
'0:05:00'
"""
return str(datetime.timedelta(seconds=int(duration)))
class RustBuild(object):
"""Provide all the methods required to build Rust"""
def __init__(self):
self.cargo_channel = ''
self.date = ''
self._download_url = 'https://static.rust-lang.org'
self.rustc_channel = ''
self.build = ''
self.build_dir = os.path.join(os.getcwd(), "build")
self.clean = False
self.config_mk = ''
self.config_toml = ''
self.printed = False
self.rust_root = os.path.abspath(os.path.join(__file__, '../../..'))
self.use_locked_deps = ''
self.use_vendored_sources = ''
self.verbose = False
def download_stage0(self):
cache_dst = os.path.join(self.build_dir, "cache")
rustc_cache = os.path.join(cache_dst, self.stage0_date())
if not os.path.exists(rustc_cache):
os.makedirs(rustc_cache)
"""Fetch the build system for Rust, written in Rust
rustc_channel = self.stage0_rustc_channel()
cargo_channel = self.stage0_cargo_channel()
This method will build a cache directory, then it will fetch the
tarball which has the stage0 compiler used to then bootstrap the Rust
compiler itself.
Each downloaded tarball is extracted, after that, the script
will move all the content to the right place.
"""
rustc_channel = self.rustc_channel
cargo_channel = self.cargo_channel
if self.rustc().startswith(self.bin_root()) and \
(not os.path.exists(self.rustc()) or self.rustc_out_of_date()):
self.print_what_it_means_to_bootstrap()
(not os.path.exists(self.rustc()) or
self.program_out_of_date(self.rustc_stamp())):
self.print_what_bootstrap_means()
if os.path.exists(self.bin_root()):
shutil.rmtree(self.bin_root())
filename = "rust-std-{}-{}.tar.gz".format(
rustc_channel, self.build)
url = self._download_url + "/dist/" + self.stage0_date()
tarball = os.path.join(rustc_cache, filename)
if not os.path.exists(tarball):
get("{}/{}".format(url, filename),
tarball, verbose=self.verbose)
unpack(tarball, self.bin_root(),
match="rust-std-" + self.build,
verbose=self.verbose)
pattern = "rust-std-{}".format(self.build)
self._download_stage0_helper(filename, pattern)
filename = "rustc-{}-{}.tar.gz".format(rustc_channel, self.build)
url = self._download_url + "/dist/" + self.stage0_date()
tarball = os.path.join(rustc_cache, filename)
if not os.path.exists(tarball):
get("{}/{}".format(url, filename),
tarball, verbose=self.verbose)
unpack(tarball, self.bin_root(),
match="rustc", verbose=self.verbose)
self.fix_executable(self.bin_root() + "/bin/rustc")
self.fix_executable(self.bin_root() + "/bin/rustdoc")
with open(self.rustc_stamp(), 'w') as f:
f.write(self.stage0_date())
self._download_stage0_helper(filename, "rustc")
self.fix_executable("{}/bin/rustc".format(self.bin_root()))
self.fix_executable("{}/bin/rustdoc".format(self.bin_root()))
with open(self.rustc_stamp(), 'w') as rust_stamp:
rust_stamp.write(self.date)
if "pc-windows-gnu" in self.build:
filename = "rust-mingw-{}-{}.tar.gz".format(
rustc_channel, self.build)
url = self._download_url + "/dist/" + self.stage0_date()
tarball = os.path.join(rustc_cache, filename)
if not os.path.exists(tarball):
get("{}/{}".format(url, filename),
tarball, verbose=self.verbose)
unpack(tarball, self.bin_root(),
match="rust-mingw", verbose=self.verbose)
self._download_stage0_helper(filename, "rust-mingw")
if self.cargo().startswith(self.bin_root()) and \
(not os.path.exists(self.cargo()) or self.cargo_out_of_date()):
self.print_what_it_means_to_bootstrap()
(not os.path.exists(self.cargo()) or
self.program_out_of_date(self.cargo_stamp())):
self.print_what_bootstrap_means()
filename = "cargo-{}-{}.tar.gz".format(cargo_channel, self.build)
url = self._download_url + "/dist/" + self.stage0_date()
tarball = os.path.join(rustc_cache, filename)
if not os.path.exists(tarball):
get("{}/{}".format(url, filename),
tarball, verbose=self.verbose)
unpack(tarball, self.bin_root(),
match="cargo", verbose=self.verbose)
self.fix_executable(self.bin_root() + "/bin/cargo")
with open(self.cargo_stamp(), 'w') as f:
f.write(self.stage0_date())
self._download_stage0_helper(filename, "cargo")
self.fix_executable("{}/bin/cargo".format(self.bin_root()))
with open(self.cargo_stamp(), 'w') as cargo_stamp:
cargo_stamp.write(self.date)
def fix_executable(self, fname):
# If we're on NixOS we need to change the path to the dynamic loader
def _download_stage0_helper(self, filename, pattern):
cache_dst = os.path.join(self.build_dir, "cache")
rustc_cache = os.path.join(cache_dst, self.date)
if not os.path.exists(rustc_cache):
os.makedirs(rustc_cache)
url = "{}/dist/{}".format(self._download_url, self.date)
tarball = os.path.join(rustc_cache, filename)
if not os.path.exists(tarball):
get("{}/{}".format(url, filename), tarball, verbose=self.verbose)
unpack(tarball, self.bin_root(), match=pattern, verbose=self.verbose)
@staticmethod
def fix_executable(fname):
"""Modifies the interpreter section of 'fname' to fix the dynamic linker
This method is only required on NixOS and uses the PatchELF utility to
change the dynamic linker of ELF executables.
Please see https://nixos.org/patchelf.html for more information
"""
default_encoding = sys.getdefaultencoding()
try:
ostype = subprocess.check_output(
['uname', '-s']).strip().decode(default_encoding)
except (subprocess.CalledProcessError, WindowsError):
except subprocess.CalledProcessError:
return
except OSError as reason:
if getattr(reason, 'winerror', None) is not None:
return
raise reason
if ostype != "Linux":
return
@ -257,8 +279,8 @@ class RustBuild(object):
interpreter = subprocess.check_output(
["patchelf", "--print-interpreter", fname])
interpreter = interpreter.strip().decode(default_encoding)
except subprocess.CalledProcessError as e:
print("warning: failed to call patchelf: %s" % e)
except subprocess.CalledProcessError as reason:
print("warning: failed to call patchelf:", reason)
return
loader = interpreter.split("/")[-1]
@ -267,8 +289,8 @@ class RustBuild(object):
ldd_output = subprocess.check_output(
['ldd', '/run/current-system/sw/bin/sh'])
ldd_output = ldd_output.strip().decode(default_encoding)
except subprocess.CalledProcessError as e:
print("warning: unable to call ldd: %s" % e)
except subprocess.CalledProcessError as reason:
print("warning: unable to call ldd:", reason)
return
for line in ldd_output.splitlines():
@ -285,45 +307,66 @@ class RustBuild(object):
try:
subprocess.check_output(
["patchelf", "--set-interpreter", correct_interpreter, fname])
except subprocess.CalledProcessError as e:
print("warning: failed to call patchelf: %s" % e)
except subprocess.CalledProcessError as reason:
print("warning: failed to call patchelf:", reason)
return
def stage0_date(self):
return self._date
def stage0_rustc_channel(self):
return self._rustc_channel
def stage0_cargo_channel(self):
return self._cargo_channel
def rustc_stamp(self):
"""Return the path for .rustc-stamp"""
"""Return the path for .rustc-stamp
>>> rb = RustBuild()
>>> rb.build_dir = "build"
>>> rb.rustc_stamp() == os.path.join("build", "stage0", ".rustc-stamp")
True
"""
return os.path.join(self.bin_root(), '.rustc-stamp')
def cargo_stamp(self):
"""Return the path for .cargo-stamp"""
"""Return the path for .cargo-stamp
>>> rb = RustBuild()
>>> rb.build_dir = "build"
>>> rb.cargo_stamp() == os.path.join("build", "stage0", ".cargo-stamp")
True
"""
return os.path.join(self.bin_root(), '.cargo-stamp')
def rustc_out_of_date(self):
"""Check if rustc is out of date"""
if not os.path.exists(self.rustc_stamp()) or self.clean:
def program_out_of_date(self, stamp_path):
"""Check if the given program stamp is out of date"""
if not os.path.exists(stamp_path) or self.clean:
return True
with open(self.rustc_stamp(), 'r') as f:
return self.stage0_date() != f.read()
def cargo_out_of_date(self):
"""Check if cargo is out of date"""
if not os.path.exists(self.cargo_stamp()) or self.clean:
return True
with open(self.cargo_stamp(), 'r') as f:
return self.stage0_date() != f.read()
with open(stamp_path, 'r') as stamp:
return self.date != stamp.read()
def bin_root(self):
"""Return the binary root directory
>>> rb = RustBuild()
>>> rb.build_dir = "build"
>>> rb.bin_root() == os.path.join("build", "stage0")
True
When the 'build' property is given should be a nested directory:
>>> rb.build = "devel"
>>> rb.bin_root() == os.path.join("build", "devel", "stage0")
True
"""
return os.path.join(self.build_dir, self.build, "stage0")
def get_toml(self, key):
"""Returns the value of the given key in config.toml, otherwise returns None
>>> rb = RustBuild()
>>> rb.config_toml = 'key1 = "value1"\\nkey2 = "value2"'
>>> rb.get_toml("key2")
'value2'
If the key does not exists, the result is None:
>>> rb.get_toml("key3") == None
True
"""
for line in self.config_toml.splitlines():
match = re.match(r'^{}\s*=(.*)$'.format(key), line)
if match is not None:
@ -332,6 +375,18 @@ class RustBuild(object):
return None
def get_mk(self, key):
"""Returns the value of the given key in config.mk, otherwise returns None
>>> rb = RustBuild()
>>> rb.config_mk = 'key := value\\n'
>>> rb.get_mk('key')
'value'
If the key does not exists, the result is None:
>>> rb.get_mk('does_not_exists') == None
True
"""
for line in iter(self.config_mk.splitlines()):
if line.startswith(key + ' '):
var = line[line.find(':=') + 2:].strip()
@ -340,36 +395,64 @@ class RustBuild(object):
return None
def cargo(self):
config = self.get_toml('cargo')
if config:
return config
config = self.get_mk('CFG_LOCAL_RUST_ROOT')
if config:
return config + '/bin/cargo' + self.exe_suffix()
return os.path.join(self.bin_root(), "bin/cargo" + self.exe_suffix())
"""Return config path for cargo"""
return self.program_config('cargo')
def rustc(self):
config = self.get_toml('rustc')
"""Return config path for rustc"""
return self.program_config('rustc')
def program_config(self, program):
"""Return config path for the given program
>>> rb = RustBuild()
>>> rb.config_toml = 'rustc = "rustc"\\n'
>>> rb.config_mk = 'CFG_LOCAL_RUST_ROOT := /tmp/rust\\n'
>>> rb.program_config('rustc')
'rustc'
>>> cargo_path = rb.program_config('cargo')
>>> cargo_path.rstrip(".exe") == os.path.join("/tmp/rust",
... "bin", "cargo")
True
>>> rb.config_toml = ''
>>> rb.config_mk = ''
>>> cargo_path = rb.program_config('cargo')
>>> cargo_path.rstrip(".exe") == os.path.join(rb.bin_root(),
... "bin", "cargo")
True
"""
config = self.get_toml(program)
if config:
return config
config = self.get_mk('CFG_LOCAL_RUST_ROOT')
if config:
return config + '/bin/rustc' + self.exe_suffix()
return os.path.join(self.bin_root(), "bin/rustc" + self.exe_suffix())
return os.path.join(config, "bin", "{}{}".format(
program, self.exe_suffix()))
return os.path.join(self.bin_root(), "bin", "{}{}".format(
program, self.exe_suffix()))
def get_string(self, line):
@staticmethod
def get_string(line):
"""Return the value between double quotes
>>> RustBuild.get_string(' "devel" ')
'devel'
"""
start = line.find('"')
if start == -1:
return None
end = start + 1 + line[start + 1:].find('"')
return line[start + 1:end]
def exe_suffix(self):
@staticmethod
def exe_suffix():
"""Return a suffix for executables"""
if sys.platform == 'win32':
return '.exe'
return ''
def print_what_it_means_to_bootstrap(self):
def print_what_bootstrap_means(self):
"""Prints more information about the build system"""
if hasattr(self, 'printed'):
return
self.printed = True
@ -386,10 +469,19 @@ class RustBuild(object):
print(' src/bootstrap/README.md before the download finishes')
def bootstrap_binary(self):
return os.path.join(self.build_dir, "bootstrap/debug/bootstrap")
"""Return the path of the boostrap binary
>>> rb = RustBuild()
>>> rb.build_dir = "build"
>>> rb.bootstrap_binary() == os.path.join("build", "bootstrap",
... "debug", "bootstrap")
True
"""
return os.path.join(self.build_dir, "bootstrap", "debug", "bootstrap")
def build_bootstrap(self):
self.print_what_it_means_to_bootstrap()
"""Build bootstrap"""
self.print_what_bootstrap_means()
build_dir = os.path.join(self.build_dir, "bootstrap")
if self.clean and os.path.exists(build_dir):
shutil.rmtree(build_dir)
@ -409,7 +501,8 @@ class RustBuild(object):
env["PATH"] = os.path.join(self.bin_root(), "bin") + \
os.pathsep + env["PATH"]
if not os.path.isfile(self.cargo()):
raise Exception("no cargo executable found at `%s`" % self.cargo())
raise Exception("no cargo executable found at `{}`".format(
self.cargo()))
args = [self.cargo(), "build", "--manifest-path",
os.path.join(self.rust_root, "src/bootstrap/Cargo.toml")]
if self.verbose:
@ -423,6 +516,7 @@ class RustBuild(object):
run(args, env=env, verbose=self.verbose)
def build_triple(self):
"""Build triple as in LLVM"""
default_encoding = sys.getdefaultencoding()
config = self.get_toml('build')
if config:
@ -445,23 +539,26 @@ class RustBuild(object):
# The goal here is to come up with the same triple as LLVM would,
# at least for the subset of platforms we're willing to target.
if ostype == 'Linux':
ostype_mapper = {
'Bitrig': 'unknown-bitrig',
'Darwin': 'apple-darwin',
'DragonFly': 'unknown-dragonfly',
'FreeBSD': 'unknown-freebsd',
'Haiku': 'unknown-haiku',
'NetBSD': 'unknown-netbsd',
'OpenBSD': 'unknown-openbsd'
}
# Consider the direct transformation first and then the special cases
if ostype in ostype_mapper:
ostype = ostype_mapper[ostype]
elif ostype == 'Linux':
os_from_sp = subprocess.check_output(
['uname', '-o']).strip().decode(default_encoding)
if os_from_sp == 'Android':
ostype = 'linux-android'
else:
ostype = 'unknown-linux-gnu'
elif ostype == 'FreeBSD':
ostype = 'unknown-freebsd'
elif ostype == 'DragonFly':
ostype = 'unknown-dragonfly'
elif ostype == 'Bitrig':
ostype = 'unknown-bitrig'
elif ostype == 'OpenBSD':
ostype = 'unknown-openbsd'
elif ostype == 'NetBSD':
ostype = 'unknown-netbsd'
elif ostype == 'SunOS':
ostype = 'sun-solaris'
# On Solaris, uname -m will return a machine classification instead
@ -477,10 +574,6 @@ class RustBuild(object):
if self.verbose:
raise Exception(err)
sys.exit(err)
elif ostype == 'Darwin':
ostype = 'apple-darwin'
elif ostype == 'Haiku':
ostype = 'unknown-haiku'
elif ostype.startswith('MINGW'):
# msys' `uname` does not print gcc configuration, but prints msys
# configuration. so we cannot believe `uname -m`:
@ -499,13 +592,36 @@ class RustBuild(object):
cputype = 'x86_64'
ostype = 'pc-windows-gnu'
else:
err = "unknown OS type: " + ostype
err = "unknown OS type: {}".format(ostype)
if self.verbose:
raise ValueError(err)
sys.exit(err)
if cputype in {'i386', 'i486', 'i686', 'i786', 'x86'}:
cputype = 'i686'
cputype_mapper = {
'BePC': 'i686',
'aarch64': 'aarch64',
'amd64': 'x86_64',
'arm64': 'aarch64',
'i386': 'i686',
'i486': 'i686',
'i686': 'i686',
'i786': 'i686',
'powerpc': 'powerpc',
'powerpc64': 'powerpc64',
'powerpc64le': 'powerpc64le',
'ppc': 'powerpc',
'ppc64': 'powerpc64',
'ppc64le': 'powerpc64le',
's390x': 's390x',
'x64': 'x86_64',
'x86': 'i686',
'x86-64': 'x86_64',
'x86_64': 'x86_64'
}
# Consider the direct transformation first and then the special cases
if cputype in cputype_mapper:
cputype = cputype_mapper[cputype]
elif cputype in {'xscale', 'arm'}:
cputype = 'arm'
if ostype == 'linux-android':
@ -522,40 +638,26 @@ class RustBuild(object):
ostype = 'linux-androideabi'
else:
ostype += 'eabihf'
elif cputype in {'aarch64', 'arm64'}:
cputype = 'aarch64'
elif cputype == 'mips':
if sys.byteorder == 'big':
cputype = 'mips'
elif sys.byteorder == 'little':
cputype = 'mipsel'
else:
raise ValueError('unknown byteorder: ' + sys.byteorder)
raise ValueError("unknown byteorder: {}".format(sys.byteorder))
elif cputype == 'mips64':
if sys.byteorder == 'big':
cputype = 'mips64'
elif sys.byteorder == 'little':
cputype = 'mips64el'
else:
raise ValueError('unknown byteorder: ' + sys.byteorder)
raise ValueError('unknown byteorder: {}'.format(sys.byteorder))
# only the n64 ABI is supported, indicate it
ostype += 'abi64'
elif cputype in {'powerpc', 'ppc'}:
cputype = 'powerpc'
elif cputype in {'powerpc64', 'ppc64'}:
cputype = 'powerpc64'
elif cputype in {'powerpc64le', 'ppc64le'}:
cputype = 'powerpc64le'
elif cputype == 'sparcv9':
pass
elif cputype in {'amd64', 'x86_64', 'x86-64', 'x64'}:
cputype = 'x86_64'
elif cputype == 's390x':
cputype = 's390x'
elif cputype == 'BePC':
cputype = 'i686'
else:
err = "unknown cpu type: " + cputype
err = "unknown cpu type: {}".format(cputype)
if self.verbose:
raise ValueError(err)
sys.exit(err)
@ -563,6 +665,7 @@ class RustBuild(object):
return "{}-{}".format(cputype, ostype)
def update_submodules(self):
"""Update submodules"""
if (not os.path.exists(os.path.join(self.rust_root, ".git"))) or \
self.get_toml('submodules') == "false" or \
self.get_mk('CFG_DISABLE_MANAGE_SUBMODULES') == "1":
@ -592,10 +695,16 @@ class RustBuild(object):
"clean", "-qdfx"],
cwd=self.rust_root, verbose=self.verbose)
def set_dev_environment(self):
"""Set download URL for development environment"""
self._download_url = 'https://dev-static.rust-lang.org'
def bootstrap():
"""Configure, fetch, build and run the initial bootstrap"""
parser = argparse.ArgumentParser(description='Build rust')
parser.add_argument('--config')
parser.add_argument('--build')
parser.add_argument('--clean', action='store_true')
parser.add_argument('-v', '--verbose', action='store_true')
@ -603,107 +712,103 @@ def bootstrap():
args, _ = parser.parse_known_args(args)
# Configure initial bootstrap
rb = RustBuild()
rb.config_toml = ''
rb.config_mk = ''
rb.rust_root = os.path.abspath(os.path.join(__file__, '../../..'))
rb.build_dir = os.path.join(os.getcwd(), "build")
rb.verbose = args.verbose
rb.clean = args.clean
build = RustBuild()
build.verbose = args.verbose
build.clean = args.clean
try:
with open(args.config or 'config.toml') as config:
rb.config_toml = config.read()
build.config_toml = config.read()
except:
pass
try:
rb.config_mk = open('config.mk').read()
build.config_mk = open('config.mk').read()
except:
pass
if '\nverbose = 2' in rb.config_toml:
rb.verbose = 2
elif '\nverbose = 1' in rb.config_toml:
rb.verbose = 1
if '\nverbose = 2' in build.config_toml:
build.verbose = 2
elif '\nverbose = 1' in build.config_toml:
build.verbose = 1
rb.use_vendored_sources = '\nvendor = true' in rb.config_toml or \
'CFG_ENABLE_VENDOR' in rb.config_mk
build.use_vendored_sources = '\nvendor = true' in build.config_toml or \
'CFG_ENABLE_VENDOR' in build.config_mk
rb.use_locked_deps = '\nlocked-deps = true' in rb.config_toml or \
'CFG_ENABLE_LOCKED_DEPS' in rb.config_mk
build.use_locked_deps = '\nlocked-deps = true' in build.config_toml or \
'CFG_ENABLE_LOCKED_DEPS' in build.config_mk
if 'SUDO_USER' in os.environ and not rb.use_vendored_sources:
if 'SUDO_USER' in os.environ and not build.use_vendored_sources:
if os.environ.get('USER') != os.environ['SUDO_USER']:
rb.use_vendored_sources = True
build.use_vendored_sources = True
print('info: looks like you are running this command under `sudo`')
print(' and so in order to preserve your $HOME this will now')
print(' use vendored sources by default. Note that if this')
print(' does not work you should run a normal build first')
print(' before running a command like `sudo make install`')
if rb.use_vendored_sources:
if build.use_vendored_sources:
if not os.path.exists('.cargo'):
os.makedirs('.cargo')
with open('.cargo/config', 'w') as f:
f.write("""
with open('.cargo/config', 'w') as cargo_config:
cargo_config.write("""
[source.crates-io]
replace-with = 'vendored-sources'
registry = 'https://example.com'
[source.vendored-sources]
directory = '{}/src/vendor'
""".format(rb.rust_root))
""".format(build.rust_root))
else:
if os.path.exists('.cargo'):
shutil.rmtree('.cargo')
data = stage0_data(rb.rust_root)
rb._date = data['date']
rb._rustc_channel = data['rustc']
rb._cargo_channel = data['cargo']
if 'dev' in data:
rb._download_url = 'https://dev-static.rust-lang.org'
else:
rb._download_url = 'https://static.rust-lang.org'
data = stage0_data(build.rust_root)
build.date = data['date']
build.rustc_channel = data['rustc']
build.cargo_channel = data['cargo']
rb.update_submodules()
if 'dev' in data:
build.set_dev_environment()
build.update_submodules()
# Fetch/build the bootstrap
rb.build = rb.build_triple()
rb.download_stage0()
build.build = args.build or build.build_triple()
build.download_stage0()
sys.stdout.flush()
rb.build_bootstrap()
build.build_bootstrap()
sys.stdout.flush()
# Run the bootstrap
args = [rb.bootstrap_binary()]
args = [build.bootstrap_binary()]
args.extend(sys.argv[1:])
env = os.environ.copy()
env["BUILD"] = rb.build
env["SRC"] = rb.rust_root
env["BUILD"] = build.build
env["SRC"] = build.rust_root
env["BOOTSTRAP_PARENT_ID"] = str(os.getpid())
env["BOOTSTRAP_PYTHON"] = sys.executable
run(args, env=env, verbose=rb.verbose)
run(args, env=env, verbose=build.verbose)
def main():
"""Entry point for the bootstrap process"""
start_time = time()
help_triggered = (
'-h' in sys.argv) or ('--help' in sys.argv) or (len(sys.argv) == 1)
try:
bootstrap()
if not help_triggered:
print("Build completed successfully in %s" %
format_build_time(time() - start_time))
except (SystemExit, KeyboardInterrupt) as e:
if hasattr(e, 'code') and isinstance(e.code, int):
exit_code = e.code
print("Build completed successfully in {}".format(
format_build_time(time() - start_time)))
except (SystemExit, KeyboardInterrupt) as error:
if hasattr(error, 'code') and isinstance(error.code, int):
exit_code = error.code
else:
exit_code = 1
print(e)
print(error)
if not help_triggered:
print("Build completed unsuccessfully in %s" %
format_build_time(time() - start_time))
print("Build completed unsuccessfully in {}".format(
format_build_time(time() - start_time)))
sys.exit(exit_code)

114
src/bootstrap/bootstrap_test.py Normal file
View File

@ -0,0 +1,114 @@
# Copyright 2015-2016 The Rust Project Developers. See the COPYRIGHT
# file at the top-level directory of this distribution and at
# http://rust-lang.org/COPYRIGHT.
#
# Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
# http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
# <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
# option. This file may not be copied, modified, or distributed
# except according to those terms.
"""Bootstrap tests"""
import os
import doctest
import unittest
import tempfile
import hashlib
from shutil import rmtree
import bootstrap
class Stage0DataTestCase(unittest.TestCase):
"""Test Case for stage0_data"""
def setUp(self):
self.rust_root = tempfile.mkdtemp()
os.mkdir(os.path.join(self.rust_root, "src"))
with open(os.path.join(self.rust_root, "src",
"stage0.txt"), "w") as stage0:
stage0.write("#ignore\n\ndate: 2017-06-15\nrustc: beta\ncargo: beta")
def tearDown(self):
rmtree(self.rust_root)
def test_stage0_data(self):
"""Extract data from stage0.txt"""
expected = {"date": "2017-06-15", "rustc": "beta", "cargo": "beta"}
data = bootstrap.stage0_data(self.rust_root)
self.assertDictEqual(data, expected)
class VerifyTestCase(unittest.TestCase):
"""Test Case for verify"""
def setUp(self):
self.container = tempfile.mkdtemp()
self.src = os.path.join(self.container, "src.txt")
self.sums = os.path.join(self.container, "sums")
self.bad_src = os.path.join(self.container, "bad.txt")
content = "Hello world"
with open(self.src, "w") as src:
src.write(content)
with open(self.sums, "w") as sums:
sums.write(hashlib.sha256(content.encode("utf-8")).hexdigest())
with open(self.bad_src, "w") as bad:
bad.write("Hello!")
def tearDown(self):
rmtree(self.container)
def test_valid_file(self):
"""Check if the sha256 sum of the given file is valid"""
self.assertTrue(bootstrap.verify(self.src, self.sums, False))
def test_invalid_file(self):
"""Should verify that the file is invalid"""
self.assertFalse(bootstrap.verify(self.bad_src, self.sums, False))
class ProgramOutOfDate(unittest.TestCase):
"""Test if a program is out of date"""
def setUp(self):
self.container = tempfile.mkdtemp()
os.mkdir(os.path.join(self.container, "stage0"))
self.build = bootstrap.RustBuild()
self.build.date = "2017-06-15"
self.build.build_dir = self.container
self.rustc_stamp_path = os.path.join(self.container, "stage0",
".rustc-stamp")
def tearDown(self):
rmtree(self.container)
def test_stamp_path_does_not_exists(self):
"""Return True when the stamp file does not exists"""
if os.path.exists(self.rustc_stamp_path):
os.unlink(self.rustc_stamp_path)
self.assertTrue(self.build.program_out_of_date(self.rustc_stamp_path))
def test_dates_are_different(self):
"""Return True when the dates are different"""
with open(self.rustc_stamp_path, "w") as rustc_stamp:
rustc_stamp.write("2017-06-14")
self.assertTrue(self.build.program_out_of_date(self.rustc_stamp_path))
def test_same_dates(self):
"""Return False both dates match"""
with open(self.rustc_stamp_path, "w") as rustc_stamp:
rustc_stamp.write("2017-06-15")
self.assertFalse(self.build.program_out_of_date(self.rustc_stamp_path))
if __name__ == '__main__':
SUITE = unittest.TestSuite()
TEST_LOADER = unittest.TestLoader()
SUITE.addTest(doctest.DocTestSuite(bootstrap))
SUITE.addTests([
TEST_LOADER.loadTestsFromTestCase(Stage0DataTestCase),
TEST_LOADER.loadTestsFromTestCase(VerifyTestCase),
TEST_LOADER.loadTestsFromTestCase(ProgramOutOfDate)])
RUNNER = unittest.TextTestRunner(verbosity=2)
RUNNER.run(SUITE)

630
src/bootstrap/builder.rs Normal file
View File

@ -0,0 +1,630 @@
// Copyright 2017 The Rust Project Developers. See the COPYRIGHT
// file at the top-level directory of this distribution and at
// http://rust-lang.org/COPYRIGHT.
//
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
// http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
// option. This file may not be copied, modified, or distributed
// except according to those terms.
use std::fmt::Debug;
use std::hash::Hash;
use std::cell::RefCell;
use std::path::{Path, PathBuf};
use std::process::Command;
use std::fs;
use std::ops::Deref;
use std::any::Any;
use std::collections::BTreeSet;
use compile;
use install;
use dist;
use util::{exe, libdir, add_lib_path};
use {Build, Mode};
use cache::{INTERNER, Interned, Cache};
use check;
use flags::Subcommand;
use doc;
use tool;
use native;
pub use Compiler;
pub struct Builder<'a> {
pub build: &'a Build,
pub top_stage: u32,
pub kind: Kind,
cache: Cache,
stack: RefCell<Vec<Box<Any>>>,
}
impl<'a> Deref for Builder<'a> {
type Target = Build;
fn deref(&self) -> &Self::Target {
self.build
}
}
pub trait Step: 'static + Clone + Debug + PartialEq + Eq + Hash {
/// `PathBuf` when directories are created or to return a `Compiler` once
/// it's been assembled.
type Output: Clone;
const DEFAULT: bool = false;
/// Run this rule for all hosts without cross compiling.
const ONLY_HOSTS: bool = false;
/// Run this rule for all targets, but only with the native host.
const ONLY_BUILD_TARGETS: bool = false;
/// Only run this step with the build triple as host and target.
const ONLY_BUILD: bool = false;
/// Primary function to execute this rule. Can call `builder.ensure(...)`
/// with other steps to run those.
fn run(self, builder: &Builder) -> Self::Output;
/// When bootstrap is passed a set of paths, this controls whether this rule
/// will execute. However, it does not get called in a "default" context
/// when we are not passed any paths; in that case, make_run is called
/// directly.
fn should_run(run: ShouldRun) -> ShouldRun;
/// Build up a "root" rule, either as a default rule or from a path passed
/// to us.
///
/// When path is `None`, we are executing in a context where no paths were
/// passed. When `./x.py build` is run, for example, this rule could get
/// called if it is in the correct list below with a path of `None`.
fn make_run(_run: RunConfig) {
// It is reasonable to not have an implementation of make_run for rules
// who do not want to get called from the root context. This means that
// they are likely dependencies (e.g., sysroot creation) or similar, and
// as such calling them from ./x.py isn't logical.
unimplemented!()
}
}
pub struct RunConfig<'a> {
pub builder: &'a Builder<'a>,
pub host: Interned<String>,
pub target: Interned<String>,
pub path: Option<&'a Path>,
}
struct StepDescription {
default: bool,
only_hosts: bool,
only_build_targets: bool,
only_build: bool,
should_run: fn(ShouldRun) -> ShouldRun,
make_run: fn(RunConfig),
}
impl StepDescription {
fn from<S: Step>() -> StepDescription {
StepDescription {
default: S::DEFAULT,
only_hosts: S::ONLY_HOSTS,
only_build_targets: S::ONLY_BUILD_TARGETS,
only_build: S::ONLY_BUILD,
should_run: S::should_run,
make_run: S::make_run,
}
}
fn maybe_run(&self, builder: &Builder, path: Option<&Path>) {
let build = builder.build;
let hosts = if self.only_build_targets || self.only_build {
build.build_triple()
} else {
&build.hosts
};
// Determine the targets participating in this rule.
let targets = if self.only_hosts {
if build.config.run_host_only {
&[]
} else if self.only_build {
build.build_triple()
} else {
&build.hosts
}
} else {
&build.targets
};
for host in hosts {
for target in targets {
let run = RunConfig {
builder,
path,
host: *host,
target: *target,
};
(self.make_run)(run);
}
}
}
fn run(v: &[StepDescription], builder: &Builder, paths: &[PathBuf]) {
let should_runs = v.iter().map(|desc| {
(desc.should_run)(ShouldRun::new(builder))
}).collect::<Vec<_>>();
if paths.is_empty() {
for (desc, should_run) in v.iter().zip(should_runs) {
if desc.default && should_run.is_really_default {
desc.maybe_run(builder, None);
}
}
} else {
for path in paths {
let mut attempted_run = false;
for (desc, should_run) in v.iter().zip(&should_runs) {
if should_run.run(path) {
attempted_run = true;
desc.maybe_run(builder, Some(path));
}
}
if !attempted_run {
eprintln!("Warning: no rules matched {}.", path.display());
}
}
}
}
}
#[derive(Clone)]
pub struct ShouldRun<'a> {
pub builder: &'a Builder<'a>,
// use a BTreeSet to maintain sort order
paths: BTreeSet<PathBuf>,
// If this is a default rule, this is an additional constraint placed on
// it's run. Generally something like compiler docs being enabled.
is_really_default: bool,
}
impl<'a> ShouldRun<'a> {
fn new(builder: &'a Builder) -> ShouldRun<'a> {
ShouldRun {
builder,
paths: BTreeSet::new(),
is_really_default: true, // by default no additional conditions
}
}
pub fn default_condition(mut self, cond: bool) -> Self {
self.is_really_default = cond;
self
}
pub fn krate(mut self, name: &str) -> Self {
for (_, krate_path) in self.builder.crates(name) {
self.paths.insert(PathBuf::from(krate_path));
}
self
}
pub fn path(mut self, path: &str) -> Self {
self.paths.insert(PathBuf::from(path));
self
}
// allows being more explicit about why should_run in Step returns the value passed to it
pub fn never(self) -> ShouldRun<'a> {
self
}
fn run(&self, path: &Path) -> bool {
self.paths.iter().any(|p| path.ends_with(p))
}
}
#[derive(Copy, Clone, PartialEq, Eq, Debug)]
pub enum Kind {
Build,
Test,
Bench,
Dist,
Doc,
Install,
}
impl<'a> Builder<'a> {
fn get_step_descriptions(kind: Kind) -> Vec<StepDescription> {
macro_rules! describe {
($($rule:ty),+ $(,)*) => {{
vec![$(StepDescription::from::<$rule>()),+]
}};
}
match kind {
Kind::Build => describe!(compile::Std, compile::Test, compile::Rustc,
compile::StartupObjects, tool::BuildManifest, tool::Rustbook, tool::ErrorIndex,
tool::UnstableBookGen, tool::Tidy, tool::Linkchecker, tool::CargoTest,
tool::Compiletest, tool::RemoteTestServer, tool::RemoteTestClient,
tool::RustInstaller, tool::Cargo, tool::Rls, tool::Rustdoc,
native::Llvm),
Kind::Test => describe!(check::Tidy, check::Bootstrap, check::DefaultCompiletest,
check::HostCompiletest, check::Crate, check::CrateLibrustc, check::Linkcheck,
check::Cargotest, check::Cargo, check::Rls, check::Docs, check::ErrorIndex,
check::Distcheck),
Kind::Bench => describe!(check::Crate, check::CrateLibrustc),
Kind::Doc => describe!(doc::UnstableBook, doc::UnstableBookGen, doc::TheBook,
doc::Standalone, doc::Std, doc::Test, doc::Rustc, doc::ErrorIndex, doc::Nomicon,
doc::Reference, doc::Rustdoc, doc::CargoBook),
Kind::Dist => describe!(dist::Docs, dist::Mingw, dist::Rustc, dist::DebuggerScripts,
dist::Std, dist::Analysis, dist::Src, dist::PlainSourceTarball, dist::Cargo,
dist::Rls, dist::Extended, dist::HashSign),
Kind::Install => describe!(install::Docs, install::Std, install::Cargo, install::Rls,
install::Analysis, install::Src, install::Rustc),
}
}
pub fn get_help(build: &Build, subcommand: &str) -> Option<String> {
let kind = match subcommand {
"build" => Kind::Build,
"doc" => Kind::Doc,
"test" => Kind::Test,
"bench" => Kind::Bench,
"dist" => Kind::Dist,
"install" => Kind::Install,
_ => return None,
};
let builder = Builder {
build,
top_stage: build.config.stage.unwrap_or(2),
kind,
cache: Cache::new(),
stack: RefCell::new(Vec::new()),
};
let builder = &builder;
let mut should_run = ShouldRun::new(builder);
for desc in Builder::get_step_descriptions(builder.kind) {
should_run = (desc.should_run)(should_run);
}
let mut help = String::from("Available paths:\n");
for path in should_run.paths {
help.push_str(format!(" ./x.py {} {}\n", subcommand, path.display()).as_str());
}
Some(help)
}
pub fn run(build: &Build) {
let (kind, paths) = match build.config.cmd {
Subcommand::Build { ref paths } => (Kind::Build, &paths[..]),
Subcommand::Doc { ref paths } => (Kind::Doc, &paths[..]),
Subcommand::Test { ref paths, .. } => (Kind::Test, &paths[..]),
Subcommand::Bench { ref paths, .. } => (Kind::Bench, &paths[..]),
Subcommand::Dist { ref paths } => (Kind::Dist, &paths[..]),
Subcommand::Install { ref paths } => (Kind::Install, &paths[..]),
Subcommand::Clean => panic!(),
};
let builder = Builder {
build,
top_stage: build.config.stage.unwrap_or(2),
kind,
cache: Cache::new(),
stack: RefCell::new(Vec::new()),
};
StepDescription::run(&Builder::get_step_descriptions(builder.kind), &builder, paths);
}
pub fn default_doc(&self, paths: Option<&[PathBuf]>) {
let paths = paths.unwrap_or(&[]);
StepDescription::run(&Builder::get_step_descriptions(Kind::Doc), self, paths);
}
/// Obtain a compiler at a given stage and for a given host. Explicitly does
/// not take `Compiler` since all `Compiler` instances are meant to be
/// obtained through this function, since it ensures that they are valid
/// (i.e., built and assembled).
pub fn compiler(&self, stage: u32, host: Interned<String>) -> Compiler {
self.ensure(compile::Assemble { target_compiler: Compiler { stage, host } })
}
pub fn sysroot(&self, compiler: Compiler) -> Interned<PathBuf> {
self.ensure(compile::Sysroot { compiler })
}
/// Returns the libdir where the standard library and other artifacts are
/// found for a compiler's sysroot.
pub fn sysroot_libdir(
&self, compiler: Compiler, target: Interned<String>
) -> Interned<PathBuf> {
#[derive(Debug, Copy, Clone, Hash, PartialEq, Eq)]
struct Libdir {
compiler: Compiler,
target: Interned<String>,
}
impl Step for Libdir {
type Output = Interned<PathBuf>;
fn should_run(run: ShouldRun) -> ShouldRun {
run.never()
}
fn run(self, builder: &Builder) -> Interned<PathBuf> {
let compiler = self.compiler;
let lib = if compiler.stage >= 2 && builder.build.config.libdir_relative.is_some() {
builder.build.config.libdir_relative.clone().unwrap()
} else {
PathBuf::from("lib")
};
let sysroot = builder.sysroot(self.compiler).join(lib)
.join("rustlib").join(self.target).join("lib");
let _ = fs::remove_dir_all(&sysroot);
t!(fs::create_dir_all(&sysroot));
INTERNER.intern_path(sysroot)
}
}
self.ensure(Libdir { compiler, target })
}
/// Returns the compiler's libdir where it stores the dynamic libraries that
/// it itself links against.
///
/// For example this returns `<sysroot>/lib` on Unix and `<sysroot>/bin` on
/// Windows.
pub fn rustc_libdir(&self, compiler: Compiler) -> PathBuf {
if compiler.is_snapshot(self) {
self.build.rustc_snapshot_libdir()
} else {
self.sysroot(compiler).join(libdir(&compiler.host))
}
}
/// Adds the compiler's directory of dynamic libraries to `cmd`'s dynamic
/// library lookup path.
pub fn add_rustc_lib_path(&self, compiler: Compiler, cmd: &mut Command) {
// Windows doesn't need dylib path munging because the dlls for the
// compiler live next to the compiler and the system will find them
// automatically.
if cfg!(windows) {
return
}
add_lib_path(vec![self.rustc_libdir(compiler)], cmd);
}
/// Get a path to the compiler specified.
pub fn rustc(&self, compiler: Compiler) -> PathBuf {
if compiler.is_snapshot(self) {
self.initial_rustc.clone()
} else {
self.sysroot(compiler).join("bin").join(exe("rustc", &compiler.host))
}
}
pub fn rustdoc(&self, host: Interned<String>) -> PathBuf {
self.ensure(tool::Rustdoc { host })
}
pub fn rustdoc_cmd(&self, host: Interned<String>) -> Command {
let mut cmd = Command::new(&self.out.join("bootstrap/debug/rustdoc"));
let compiler = self.compiler(self.top_stage, host);
cmd
.env("RUSTC_STAGE", compiler.stage.to_string())
.env("RUSTC_SYSROOT", self.sysroot(compiler))
.env("RUSTC_LIBDIR", self.sysroot_libdir(compiler, self.build.build))
.env("CFG_RELEASE_CHANNEL", &self.build.config.channel)
.env("RUSTDOC_REAL", self.rustdoc(host));
cmd
}
/// Prepares an invocation of `cargo` to be run.
///
/// This will create a `Command` that represents a pending execution of
/// Cargo. This cargo will be configured to use `compiler` as the actual
/// rustc compiler, its output will be scoped by `mode`'s output directory,
/// it will pass the `--target` flag for the specified `target`, and will be
/// executing the Cargo command `cmd`.
pub fn cargo(&self,
compiler: Compiler,
mode: Mode,
target: Interned<String>,
cmd: &str) -> Command {
let mut cargo = Command::new(&self.initial_cargo);
let out_dir = self.stage_out(compiler, mode);
cargo.env("CARGO_TARGET_DIR", out_dir)
.arg(cmd)
.arg("-j").arg(self.jobs().to_string())
.arg("--target").arg(target);
// FIXME: Temporary fix for https://github.com/rust-lang/cargo/issues/3005
// Force cargo to output binaries with disambiguating hashes in the name
cargo.env("__CARGO_DEFAULT_LIB_METADATA", &self.config.channel);
let stage;
if compiler.stage == 0 && self.local_rebuild {
// Assume the local-rebuild rustc already has stage1 features.
stage = 1;
} else {
stage = compiler.stage;
}
// Customize the compiler we're running. Specify the compiler to cargo
// as our shim and then pass it some various options used to configure
// how the actual compiler itself is called.
//
// These variables are primarily all read by
// src/bootstrap/bin/{rustc.rs,rustdoc.rs}
cargo.env("RUSTBUILD_NATIVE_DIR", self.native_dir(target))
.env("RUSTC", self.out.join("bootstrap/debug/rustc"))
.env("RUSTC_REAL", self.rustc(compiler))
.env("RUSTC_STAGE", stage.to_string())
.env("RUSTC_CODEGEN_UNITS",
self.config.rust_codegen_units.to_string())
.env("RUSTC_DEBUG_ASSERTIONS",
self.config.rust_debug_assertions.to_string())
.env("RUSTC_SYSROOT", self.sysroot(compiler))
.env("RUSTC_LIBDIR", self.rustc_libdir(compiler))
.env("RUSTC_RPATH", self.config.rust_rpath.to_string())
.env("RUSTDOC", self.out.join("bootstrap/debug/rustdoc"))
.env("RUSTDOC_REAL", if cmd == "doc" || cmd == "test" {
self.rustdoc(compiler.host)
} else {
PathBuf::from("/path/to/nowhere/rustdoc/not/required")
})
.env("RUSTC_FLAGS", self.rustc_flags(target).join(" "));
if mode != Mode::Tool {
// Tools don't get debuginfo right now, e.g. cargo and rls don't
// get compiled with debuginfo.
cargo.env("RUSTC_DEBUGINFO", self.config.rust_debuginfo.to_string())
.env("RUSTC_DEBUGINFO_LINES", self.config.rust_debuginfo_lines.to_string())
.env("RUSTC_FORCE_UNSTABLE", "1");
// Currently the compiler depends on crates from crates.io, and
// then other crates can depend on the compiler (e.g. proc-macro
// crates). Let's say, for example that rustc itself depends on the
// bitflags crate. If an external crate then depends on the
// bitflags crate as well, we need to make sure they don't
// conflict, even if they pick the same version of bitflags. We'll
// want to make sure that e.g. a plugin and rustc each get their
// own copy of bitflags.
// Cargo ensures that this works in general through the -C metadata
// flag. This flag will frob the symbols in the binary to make sure
// they're different, even though the source code is the exact
// same. To solve this problem for the compiler we extend Cargo's
// already-passed -C metadata flag with our own. Our rustc.rs
// wrapper around the actual rustc will detect -C metadata being
// passed and frob it with this extra string we're passing in.
cargo.env("RUSTC_METADATA_SUFFIX", "rustc");
}
if let Some(x) = self.crt_static(target) {
cargo.env("RUSTC_CRT_STATIC", x.to_string());
}
// Enable usage of unstable features
cargo.env("RUSTC_BOOTSTRAP", "1");
self.add_rust_test_threads(&mut cargo);
// Almost all of the crates that we compile as part of the bootstrap may
// have a build script, including the standard library. To compile a
// build script, however, it itself needs a standard library! This
// introduces a bit of a pickle when we're compiling the standard
// library itself.
//
// To work around this we actually end up using the snapshot compiler
// (stage0) for compiling build scripts of the standard library itself.
// The stage0 compiler is guaranteed to have a libstd available for use.
//
// For other crates, however, we know that we've already got a standard
// library up and running, so we can use the normal compiler to compile
// build scripts in that situation.
if mode == Mode::Libstd {
cargo.env("RUSTC_SNAPSHOT", &self.initial_rustc)
.env("RUSTC_SNAPSHOT_LIBDIR", self.rustc_snapshot_libdir());
} else {
cargo.env("RUSTC_SNAPSHOT", self.rustc(compiler))
.env("RUSTC_SNAPSHOT_LIBDIR", self.rustc_libdir(compiler));
}
// Ignore incremental modes except for stage0, since we're
// not guaranteeing correctness across builds if the compiler
// is changing under your feet.`
if self.config.incremental && compiler.stage == 0 {
let incr_dir = self.incremental_dir(compiler);
cargo.env("RUSTC_INCREMENTAL", incr_dir);
}
if let Some(ref on_fail) = self.config.on_fail {
cargo.env("RUSTC_ON_FAIL", on_fail);
}
cargo.env("RUSTC_VERBOSE", format!("{}", self.verbosity));
// Specify some various options for build scripts used throughout
// the build.
//
// FIXME: the guard against msvc shouldn't need to be here
if !target.contains("msvc") {
cargo.env(format!("CC_{}", target), self.cc(target))
.env(format!("AR_{}", target), self.ar(target).unwrap()) // only msvc is None
.env(format!("CFLAGS_{}", target), self.cflags(target).join(" "));
if let Ok(cxx) = self.cxx(target) {
cargo.env(format!("CXX_{}", target), cxx);
}
}
if mode == Mode::Libstd && self.config.extended && compiler.is_final_stage(self) {
cargo.env("RUSTC_SAVE_ANALYSIS", "api".to_string());
}
// Environment variables *required* throughout the build
//
// FIXME: should update code to not require this env var
cargo.env("CFG_COMPILER_HOST_TRIPLE", target);
// Set this for all builds to make sure doc builds also get it.
cargo.env("CFG_RELEASE_CHANNEL", &self.build.config.channel);
if self.is_verbose() {
cargo.arg("-v");
}
// FIXME: cargo bench does not accept `--release`
if self.config.rust_optimize && cmd != "bench" {
cargo.arg("--release");
}
if self.config.locked_deps {
cargo.arg("--locked");
}
if self.config.vendor || self.is_sudo {
cargo.arg("--frozen");
}
self.ci_env.force_coloring_in_ci(&mut cargo);
cargo
}
/// Ensure that a given step is built, returning it's output. This will
/// cache the step, so it is safe (and good!) to call this as often as
/// needed to ensure that all dependencies are built.
pub fn ensure<S: Step>(&'a self, step: S) -> S::Output {
{
let mut stack = self.stack.borrow_mut();
for stack_step in stack.iter() {
// should skip
if stack_step.downcast_ref::<S>().map_or(true, |stack_step| *stack_step != step) {
continue;
}
let mut out = String::new();
out += &format!("\n\nCycle in build detected when adding {:?}\n", step);
for el in stack.iter().rev() {
out += &format!("\t{:?}\n", el);
}
panic!(out);
}
if let Some(out) = self.cache.get(&step) {
self.build.verbose(&format!("{}c {:?}", " ".repeat(stack.len()), step));
return out;
}
self.build.verbose(&format!("{}> {:?}", " ".repeat(stack.len()), step));
stack.push(Box::new(step.clone()));
}
let out = step.clone().run(self);
{
let mut stack = self.stack.borrow_mut();
let cur_step = stack.pop().expect("step stack empty");
assert_eq!(cur_step.downcast_ref(), Some(&step));
}
self.build.verbose(&format!("{}< {:?}", " ".repeat(self.stack.borrow().len()), step));
self.cache.put(step, out.clone());
out
}
}

267
src/bootstrap/cache.rs Normal file
View File

@ -0,0 +1,267 @@
// Copyright 2017 The Rust Project Developers. See the COPYRIGHT
// file at the top-level directory of this distribution and at
// http://rust-lang.org/COPYRIGHT.
//
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
// http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
// option. This file may not be copied, modified, or distributed
// except according to those terms.
use std::any::{Any, TypeId};
use std::borrow::Borrow;
use std::cell::RefCell;
use std::collections::HashMap;
use std::convert::AsRef;
use std::ffi::OsStr;
use std::fmt;
use std::hash::{Hash, Hasher};
use std::marker::PhantomData;
use std::mem;
use std::ops::Deref;
use std::path::{Path, PathBuf};
use std::sync::Mutex;
use builder::Step;
pub struct Interned<T>(usize, PhantomData<*const T>);
impl Default for Interned<String> {
fn default() -> Self {
INTERNER.intern_string(String::default())
}
}
impl Default for Interned<PathBuf> {
fn default() -> Self {
INTERNER.intern_path(PathBuf::default())
}
}
impl<T> Copy for Interned<T> {}
impl<T> Clone for Interned<T> {
fn clone(&self) -> Interned<T> {
*self
}
}
impl<T> PartialEq for Interned<T> {
fn eq(&self, other: &Self) -> bool {
self.0 == other.0
}
}
impl<T> Eq for Interned<T> {}
impl PartialEq<str> for Interned<String> {
fn eq(&self, other: &str) -> bool {
*self == other
}
}
impl<'a> PartialEq<&'a str> for Interned<String> {
fn eq(&self, other: &&str) -> bool {
**self == **other
}
}
impl<'a, T> PartialEq<&'a Interned<T>> for Interned<T> {
fn eq(&self, other: &&Self) -> bool {
self.0 == other.0
}
}
impl<'a, T> PartialEq<Interned<T>> for &'a Interned<T> {
fn eq(&self, other: &Interned<T>) -> bool {
self.0 == other.0
}
}
unsafe impl<T> Send for Interned<T> {}
unsafe impl<T> Sync for Interned<T> {}
impl fmt::Display for Interned<String> {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
let s: &str = &*self;
f.write_str(s)
}
}
impl fmt::Debug for Interned<String> {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
let s: &str = &*self;
f.write_fmt(format_args!("{:?}", s))
}
}
impl fmt::Debug for Interned<PathBuf> {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
let s: &Path = &*self;
f.write_fmt(format_args!("{:?}", s))
}
}
impl Hash for Interned<String> {
fn hash<H: Hasher>(&self, state: &mut H) {
let l = INTERNER.strs.lock().unwrap();
l.get(*self).hash(state)
}
}
impl Hash for Interned<PathBuf> {
fn hash<H: Hasher>(&self, state: &mut H) {
let l = INTERNER.paths.lock().unwrap();
l.get(*self).hash(state)
}
}
impl Deref for Interned<String> {
type Target = str;
fn deref(&self) -> &'static str {
let l = INTERNER.strs.lock().unwrap();
unsafe { mem::transmute::<&str, &'static str>(l.get(*self)) }
}
}
impl Deref for Interned<PathBuf> {
type Target = Path;
fn deref(&self) -> &'static Path {
let l = INTERNER.paths.lock().unwrap();
unsafe { mem::transmute::<&Path, &'static Path>(l.get(*self)) }
}
}
impl AsRef<Path> for Interned<PathBuf> {
fn as_ref(&self) -> &'static Path {
let l = INTERNER.paths.lock().unwrap();
unsafe { mem::transmute::<&Path, &'static Path>(l.get(*self)) }
}
}
impl AsRef<Path> for Interned<String> {
fn as_ref(&self) -> &'static Path {
let l = INTERNER.strs.lock().unwrap();
unsafe { mem::transmute::<&Path, &'static Path>(l.get(*self).as_ref()) }
}
}
impl AsRef<OsStr> for Interned<PathBuf> {
fn as_ref(&self) -> &'static OsStr {
let l = INTERNER.paths.lock().unwrap();
unsafe { mem::transmute::<&OsStr, &'static OsStr>(l.get(*self).as_ref()) }
}
}
impl AsRef<OsStr> for Interned<String> {
fn as_ref(&self) -> &'static OsStr {
let l = INTERNER.strs.lock().unwrap();
unsafe { mem::transmute::<&OsStr, &'static OsStr>(l.get(*self).as_ref()) }
}
}
struct TyIntern<T> {
items: Vec<T>,
set: HashMap<T, Interned<T>>,
}
impl<T: Hash + Clone + Eq> TyIntern<T> {
fn new() -> TyIntern<T> {
TyIntern {
items: Vec::new(),
set: HashMap::new(),
}
}
fn intern_borrow<B>(&mut self, item: &B) -> Interned<T>
where
B: Eq + Hash + ToOwned<Owned=T> + ?Sized,
T: Borrow<B>,
{
if let Some(i) = self.set.get(&item) {
return *i;
}
let item = item.to_owned();
let interned = Interned(self.items.len(), PhantomData::<*const T>);
self.set.insert(item.clone(), interned);
self.items.push(item);
interned
}
fn intern(&mut self, item: T) -> Interned<T> {
if let Some(i) = self.set.get(&item) {
return *i;
}
let interned = Interned(self.items.len(), PhantomData::<*const T>);
self.set.insert(item.clone(), interned);
self.items.push(item);
interned
}
fn get(&self, i: Interned<T>) -> &T {
&self.items[i.0]
}
}
pub struct Interner {
strs: Mutex<TyIntern<String>>,
paths: Mutex<TyIntern<PathBuf>>,
}
impl Interner {
fn new() -> Interner {
Interner {
strs: Mutex::new(TyIntern::new()),
paths: Mutex::new(TyIntern::new()),
}
}
pub fn intern_str(&self, s: &str) -> Interned<String> {
self.strs.lock().unwrap().intern_borrow(s)
}
pub fn intern_string(&self, s: String) -> Interned<String> {
self.strs.lock().unwrap().intern(s)
}
pub fn intern_path(&self, s: PathBuf) -> Interned<PathBuf> {
self.paths.lock().unwrap().intern(s)
}
}
lazy_static! {
pub static ref INTERNER: Interner = Interner::new();
}
/// This is essentially a HashMap which allows storing any type in its input and
/// any type in its output. It is a write-once cache; values are never evicted,
/// which means that references to the value can safely be returned from the
/// get() method.
#[derive(Debug)]
pub struct Cache(
RefCell<HashMap<
TypeId,
Box<Any>, // actually a HashMap<Step, Interned<Step::Output>>
>>
);
impl Cache {
pub fn new() -> Cache {
Cache(RefCell::new(HashMap::new()))
}
pub fn put<S: Step>(&self, step: S, value: S::Output) {
let mut cache = self.0.borrow_mut();
let type_id = TypeId::of::<S>();
let stepcache = cache.entry(type_id)
.or_insert_with(|| Box::new(HashMap::<S, S::Output>::new()))
.downcast_mut::<HashMap<S, S::Output>>()
.expect("invalid type mapped");
assert!(!stepcache.contains_key(&step), "processing {:?} a second time", step);
stepcache.insert(step, value);
}
pub fn get<S: Step>(&self, step: &S) -> Option<S::Output> {
let mut cache = self.0.borrow_mut();
let type_id = TypeId::of::<S>();
let stepcache = cache.entry(type_id)
.or_insert_with(|| Box::new(HashMap::<S, S::Output>::new()))
.downcast_mut::<HashMap<S, S::Output>>()
.expect("invalid type mapped");
stepcache.get(step).cloned()
}
}

34
src/bootstrap/cc.rs
View File

@ -32,25 +32,24 @@
//! everything.
use std::process::Command;
use std::iter;
use build_helper::{cc2ar, output};
use gcc;
use Build;
use config::Target;
use cache::Interned;
pub fn find(build: &mut Build) {
// For all targets we're going to need a C compiler for building some shims
// and such as well as for being a linker for Rust code.
//
// This includes targets that aren't necessarily passed on the commandline
// (FIXME: Perhaps it shouldn't?)
for target in &build.config.target {
for target in build.targets.iter().chain(&build.hosts).cloned().chain(iter::once(build.build)) {
let mut cfg = gcc::Config::new();
cfg.cargo_metadata(false).opt_level(0).debug(false)
.target(target).host(&build.build);
.target(&target).host(&build.build);
let config = build.config.target_config.get(target);
let config = build.config.target_config.get(&target);
if let Some(cc) = config.and_then(|c| c.cc.as_ref()) {
cfg.compiler(cc);
} else {
@ -58,23 +57,20 @@ pub fn find(build: &mut Build) {
}
let compiler = cfg.get_compiler();
let ar = cc2ar(compiler.path(), target);
build.verbose(&format!("CC_{} = {:?}", target, compiler.path()));
let ar = cc2ar(compiler.path(), &target);
build.verbose(&format!("CC_{} = {:?}", &target, compiler.path()));
if let Some(ref ar) = ar {
build.verbose(&format!("AR_{} = {:?}", target, ar));
build.verbose(&format!("AR_{} = {:?}", &target, ar));
}
build.cc.insert(target.to_string(), (compiler, ar));
build.cc.insert(target, (compiler, ar));
}
// For all host triples we need to find a C++ compiler as well
//
// This includes hosts that aren't necessarily passed on the commandline
// (FIXME: Perhaps it shouldn't?)
for host in &build.config.host {
for host in build.hosts.iter().cloned().chain(iter::once(build.build)) {
let mut cfg = gcc::Config::new();
cfg.cargo_metadata(false).opt_level(0).debug(false).cpp(true)
.target(host).host(&build.build);
let config = build.config.target_config.get(host);
.target(&host).host(&build.build);
let config = build.config.target_config.get(&host);
if let Some(cxx) = config.and_then(|c| c.cxx.as_ref()) {
cfg.compiler(cxx);
} else {
@ -82,16 +78,16 @@ pub fn find(build: &mut Build) {
}
let compiler = cfg.get_compiler();
build.verbose(&format!("CXX_{} = {:?}", host, compiler.path()));
build.cxx.insert(host.to_string(), compiler);
build.cxx.insert(host, compiler);
}
}
fn set_compiler(cfg: &mut gcc::Config,
gnu_compiler: &str,
target: &str,
target: Interned<String>,
config: Option<&Target>,
build: &Build) {
match target {
match &*target {
// When compiling for android we may have the NDK configured in the
// config.toml in which case we look there. Otherwise the default
// compiler already takes into account the triple in question.

9
src/bootstrap/channel.rs
View File

@ -21,14 +21,15 @@ use std::process::Command;
use build_helper::output;
use Build;
use config::Config;
// The version number
pub const CFG_RELEASE_NUM: &str = "1.20.0";
pub const CFG_RELEASE_NUM: &str = "1.21.0";
// An optional number to put after the label, e.g. '.2' -> '-beta.2'
// Be sure to make this starts with a dot to conform to semver pre-release
// versions (section 9)
pub const CFG_PRERELEASE_VERSION: &str = ".3";
pub const CFG_PRERELEASE_VERSION: &str = ".4";
pub struct GitInfo {
inner: Option<Info>,
@ -41,9 +42,9 @@ struct Info {
}
impl GitInfo {
pub fn new(dir: &Path) -> GitInfo {
pub fn new(config: &Config, dir: &Path) -> GitInfo {
// See if this even begins to look like a git dir
if !dir.join(".git").exists() {
if config.ignore_git || !dir.join(".git").exists() {
return GitInfo { inner: None }
}

1598
src/bootstrap/check.rs
View File

File diff suppressed because it is too large Load Diff

2
src/bootstrap/clean.rs
View File

@ -26,7 +26,7 @@ pub fn clean(build: &Build) {
rm_rf(&build.out.join("tmp"));
rm_rf(&build.out.join("dist"));
for host in build.config.host.iter() {
for host in &build.hosts {
let entries = match build.out.join(host).read_dir() {
Ok(iter) => iter,
Err(_) => continue,

815
src/bootstrap/compile.rs
View File

@ -23,31 +23,121 @@ use std::io::prelude::*;
use std::path::{Path, PathBuf};
use std::process::{Command, Stdio};
use std::str;
use std::cmp::min;
use build_helper::{output, mtime, up_to_date};
use filetime::FileTime;
use rustc_serialize::json;
use serde_json;
use channel::GitInfo;
use util::{exe, libdir, is_dylib, copy};
use {Build, Compiler, Mode};
use native;
use tool;
/// Build the standard library.
use cache::{INTERNER, Interned};
use builder::{Step, RunConfig, ShouldRun, Builder};
#[derive(Debug, Copy, Clone, PartialEq, Eq, Hash)]
pub struct Std {
pub target: Interned<String>,
pub compiler: Compiler,
}
impl Step for Std {
type Output = ();
const DEFAULT: bool = true;
fn should_run(run: ShouldRun) -> ShouldRun {
run.path("src/libstd").krate("std")
}
fn make_run(run: RunConfig) {
run.builder.ensure(Std {
compiler: run.builder.compiler(run.builder.top_stage, run.host),
target: run.target,
});
}
/// Build the standard library.
///
/// This will build the standard library for a particular stage of the build
/// using the `compiler` targeting the `target` architecture. The artifacts
/// created will also be linked into the sysroot directory.
fn run(self, builder: &Builder) {
let build = builder.build;
let target = self.target;
let compiler = self.compiler;
builder.ensure(StartupObjects { compiler, target });
if build.force_use_stage1(compiler, target) {
let from = builder.compiler(1, build.build);
builder.ensure(Std {
compiler: from,
target,
});
println!("Uplifting stage1 std ({} -> {})", from.host, target);
// Even if we're not building std this stage, the new sysroot must
// still contain the musl startup objects.
if target.contains("musl") && !target.contains("mips") {
let libdir = builder.sysroot_libdir(compiler, target);
copy_musl_third_party_objects(build, target, &libdir);
}
builder.ensure(StdLink {
compiler: from,
target_compiler: compiler,
target,
});
return;
}
let _folder = build.fold_output(|| format!("stage{}-std", compiler.stage));
println!("Building stage{} std artifacts ({} -> {})", compiler.stage,
&compiler.host, target);
if target.contains("musl") && !target.contains("mips") {
let libdir = builder.sysroot_libdir(compiler, target);
copy_musl_third_party_objects(build, target, &libdir);
}
let out_dir = build.cargo_out(compiler, Mode::Libstd, target);
build.clear_if_dirty(&out_dir, &builder.rustc(compiler));
let mut cargo = builder.cargo(compiler, Mode::Libstd, target, "build");
std_cargo(build, &compiler, target, &mut cargo);
run_cargo(build,
&mut cargo,
&libstd_stamp(build, compiler, target));
builder.ensure(StdLink {
compiler: builder.compiler(compiler.stage, build.build),
target_compiler: compiler,
target,
});
}
}
/// Copies the crt(1,i,n).o startup objects
///
/// This will build the standard library for a particular stage of the build
/// using the `compiler` targeting the `target` architecture. The artifacts
/// created will also be linked into the sysroot directory.
pub fn std(build: &Build, target: &str, compiler: &Compiler) {
let libdir = build.sysroot_libdir(compiler, target);
t!(fs::create_dir_all(&libdir));
/// Since musl supports fully static linking, we can cross link for it even
/// with a glibc-targeting toolchain, given we have the appropriate startup
/// files. As those shipped with glibc won't work, copy the ones provided by
/// musl so we have them on linux-gnu hosts.
fn copy_musl_third_party_objects(build: &Build,
target: Interned<String>,
into: &Path) {
for &obj in &["crt1.o", "crti.o", "crtn.o"] {
copy(&build.musl_root(target).unwrap().join("lib").join(obj), &into.join(obj));
}
}
let _folder = build.fold_output(|| format!("stage{}-std", compiler.stage));
println!("Building stage{} std artifacts ({} -> {})", compiler.stage,
compiler.host, target);
let out_dir = build.cargo_out(compiler, Mode::Libstd, target);
build.clear_if_dirty(&out_dir, &build.compiler_path(compiler));
let mut cargo = build.cargo(compiler, Mode::Libstd, target, "build");
/// Configure cargo to compile the standard library, adding appropriate env vars
/// and such.
pub fn std_cargo(build: &Build,
compiler: &Compiler,
target: Interned<String>,
cargo: &mut Command) {
let mut features = build.std_features();
if let Some(target) = env::var_os("MACOSX_STD_DEPLOYMENT_TARGET") {
@ -73,11 +163,12 @@ pub fn std(build: &Build, target: &str, compiler: &Compiler) {
// config.toml equivalent) is used
cargo.env("LLVM_CONFIG", build.llvm_config(target));
}
cargo.arg("--features").arg(features)
.arg("--manifest-path")
.arg(build.src.join("src/libstd/Cargo.toml"));
if let Some(target) = build.config.target_config.get(target) {
cargo.arg("--features").arg(features)
.arg("--manifest-path")
.arg(build.src.join("src/libstd/Cargo.toml"));
if let Some(target) = build.config.target_config.get(&target) {
if let Some(ref jemalloc) = target.jemalloc {
cargo.env("JEMALLOC_OVERRIDE", jemalloc);
}
@ -87,51 +178,56 @@ pub fn std(build: &Build, target: &str, compiler: &Compiler) {
cargo.env("MUSL_ROOT", p);
}
}
run_cargo(build,
&mut cargo,
&libstd_stamp(build, &compiler, target));
}
/// Link all libstd rlibs/dylibs into the sysroot location.
///
/// Links those artifacts generated by `compiler` to a the `stage` compiler's
/// sysroot for the specified `host` and `target`.
///
/// Note that this assumes that `compiler` has already generated the libstd
/// libraries for `target`, and this method will find them in the relevant
/// output directory.
pub fn std_link(build: &Build,
compiler: &Compiler,
target_compiler: &Compiler,
target: &str) {
println!("Copying stage{} std from stage{} ({} -> {} / {})",
target_compiler.stage,
compiler.stage,
compiler.host,
target_compiler.host,
target);
let libdir = build.sysroot_libdir(target_compiler, target);
add_to_sysroot(&libdir, &libstd_stamp(build, compiler, target));
if target.contains("musl") && !target.contains("mips") {
copy_musl_third_party_objects(build, target, &libdir);
}
if build.config.sanitizers && compiler.stage != 0 && target == "x86_64-apple-darwin" {
// The sanitizers are only built in stage1 or above, so the dylibs will
// be missing in stage0 and causes panic. See the `std()` function above
// for reason why the sanitizers are not built in stage0.
copy_apple_sanitizer_dylibs(&build.native_dir(target), "osx", &libdir);
}
#[derive(Debug, Copy, Clone, PartialEq, Eq, Hash)]
struct StdLink {
pub compiler: Compiler,
pub target_compiler: Compiler,
pub target: Interned<String>,
}
/// Copies the crt(1,i,n).o startup objects
///
/// Only required for musl targets that statically link to libc
fn copy_musl_third_party_objects(build: &Build, target: &str, into: &Path) {
for &obj in &["crt1.o", "crti.o", "crtn.o"] {
copy(&build.musl_root(target).unwrap().join("lib").join(obj), &into.join(obj));
impl Step for StdLink {
type Output = ();
fn should_run(run: ShouldRun) -> ShouldRun {
run.never()
}
/// Link all libstd rlibs/dylibs into the sysroot location.
///
/// Links those artifacts generated by `compiler` to a the `stage` compiler's
/// sysroot for the specified `host` and `target`.
///
/// Note that this assumes that `compiler` has already generated the libstd
/// libraries for `target`, and this method will find them in the relevant
/// output directory.
fn run(self, builder: &Builder) {
let build = builder.build;
let compiler = self.compiler;
let target_compiler = self.target_compiler;
let target = self.target;
println!("Copying stage{} std from stage{} ({} -> {} / {})",
target_compiler.stage,
compiler.stage,
&compiler.host,
target_compiler.host,
target);
let libdir = builder.sysroot_libdir(target_compiler, target);
add_to_sysroot(&libdir, &libstd_stamp(build, compiler, target));
if build.config.sanitizers && compiler.stage != 0 && target == "x86_64-apple-darwin" {
// The sanitizers are only built in stage1 or above, so the dylibs will
// be missing in stage0 and causes panic. See the `std()` function above
// for reason why the sanitizers are not built in stage0.
copy_apple_sanitizer_dylibs(&build.native_dir(target), "osx", &libdir);
}
builder.ensure(tool::CleanTools {
compiler: target_compiler,
target,
mode: Mode::Libstd,
});
}
}
@ -147,97 +243,265 @@ fn copy_apple_sanitizer_dylibs(native_dir: &Path, platform: &str, into: &Path) {
}
}
/// Build and prepare startup objects like rsbegin.o and rsend.o
///
/// These are primarily used on Windows right now for linking executables/dlls.
/// They don't require any library support as they're just plain old object
/// files, so we just use the nightly snapshot compiler to always build them (as
/// no other compilers are guaranteed to be available).
pub fn build_startup_objects(build: &Build, for_compiler: &Compiler, target: &str) {
if !target.contains("pc-windows-gnu") {
return
#[derive(Debug, Copy, Clone, PartialEq, Eq, Hash)]
pub struct StartupObjects {
pub compiler: Compiler,
pub target: Interned<String>,
}
impl Step for StartupObjects {
type Output = ();
fn should_run(run: ShouldRun) -> ShouldRun {
run.path("src/rtstartup")
}
let compiler = Compiler::new(0, &build.build);
let compiler_path = build.compiler_path(&compiler);
let src_dir = &build.src.join("src/rtstartup");
let dst_dir = &build.native_dir(target).join("rtstartup");
let sysroot_dir = &build.sysroot_libdir(for_compiler, target);
t!(fs::create_dir_all(dst_dir));
t!(fs::create_dir_all(sysroot_dir));
fn make_run(run: RunConfig) {
run.builder.ensure(StartupObjects {
compiler: run.builder.compiler(run.builder.top_stage, run.host),
target: run.target,
});
}
for file in &["rsbegin", "rsend"] {
let src_file = &src_dir.join(file.to_string() + ".rs");
let dst_file = &dst_dir.join(file.to_string() + ".o");
if !up_to_date(src_file, dst_file) {
let mut cmd = Command::new(&compiler_path);
build.run(cmd.env("RUSTC_BOOTSTRAP", "1")
.arg("--cfg").arg(format!("stage{}", compiler.stage))
.arg("--target").arg(target)
.arg("--emit=obj")
.arg("--out-dir").arg(dst_dir)
.arg(src_file));
/// Build and prepare startup objects like rsbegin.o and rsend.o
///
/// These are primarily used on Windows right now for linking executables/dlls.
/// They don't require any library support as they're just plain old object
/// files, so we just use the nightly snapshot compiler to always build them (as
/// no other compilers are guaranteed to be available).
fn run(self, builder: &Builder) {
let build = builder.build;
let for_compiler = self.compiler;
let target = self.target;
if !target.contains("pc-windows-gnu") {
return
}
copy(dst_file, &sysroot_dir.join(file.to_string() + ".o"));
}
let src_dir = &build.src.join("src/rtstartup");
let dst_dir = &build.native_dir(target).join("rtstartup");
let sysroot_dir = &builder.sysroot_libdir(for_compiler, target);
t!(fs::create_dir_all(dst_dir));
for obj in ["crt2.o", "dllcrt2.o"].iter() {
copy(&compiler_file(build.cc(target), obj), &sysroot_dir.join(obj));
for file in &["rsbegin", "rsend"] {
let src_file = &src_dir.join(file.to_string() + ".rs");
let dst_file = &dst_dir.join(file.to_string() + ".o");
if !up_to_date(src_file, dst_file) {
let mut cmd = Command::new(&build.initial_rustc);
build.run(cmd.env("RUSTC_BOOTSTRAP", "1")
.arg("--cfg").arg("stage0")
.arg("--target").arg(target)
.arg("--emit=obj")
.arg("-o").arg(dst_file)
.arg(src_file));
}
copy(dst_file, &sysroot_dir.join(file.to_string() + ".o"));
}
for obj in ["crt2.o", "dllcrt2.o"].iter() {
copy(&compiler_file(build.cc(target), obj), &sysroot_dir.join(obj));
}
}
}
/// Build libtest.
///
/// This will build libtest and supporting libraries for a particular stage of
/// the build using the `compiler` targeting the `target` architecture. The
/// artifacts created will also be linked into the sysroot directory.
pub fn test(build: &Build, target: &str, compiler: &Compiler) {
let _folder = build.fold_output(|| format!("stage{}-test", compiler.stage));
println!("Building stage{} test artifacts ({} -> {})", compiler.stage,
compiler.host, target);
let out_dir = build.cargo_out(compiler, Mode::Libtest, target);
build.clear_if_dirty(&out_dir, &libstd_stamp(build, compiler, target));
let mut cargo = build.cargo(compiler, Mode::Libtest, target, "build");
#[derive(Debug, Copy, Clone, PartialEq, Eq, Hash)]
pub struct Test {
pub compiler: Compiler,
pub target: Interned<String>,
}
impl Step for Test {
type Output = ();
const DEFAULT: bool = true;
fn should_run(run: ShouldRun) -> ShouldRun {
run.path("src/libtest").krate("test")
}
fn make_run(run: RunConfig) {
run.builder.ensure(Test {
compiler: run.builder.compiler(run.builder.top_stage, run.host),
target: run.target,
});
}
/// Build libtest.
///
/// This will build libtest and supporting libraries for a particular stage of
/// the build using the `compiler` targeting the `target` architecture. The
/// artifacts created will also be linked into the sysroot directory.
fn run(self, builder: &Builder) {
let build = builder.build;
let target = self.target;
let compiler = self.compiler;
builder.ensure(Std { compiler, target });
if build.force_use_stage1(compiler, target) {
builder.ensure(Test {
compiler: builder.compiler(1, build.build),
target,
});
println!("Uplifting stage1 test ({} -> {})", &build.build, target);
builder.ensure(TestLink {
compiler: builder.compiler(1, build.build),
target_compiler: compiler,
target,
});
return;
}
let _folder = build.fold_output(|| format!("stage{}-test", compiler.stage));
println!("Building stage{} test artifacts ({} -> {})", compiler.stage,
&compiler.host, target);
let out_dir = build.cargo_out(compiler, Mode::Libtest, target);
build.clear_if_dirty(&out_dir, &libstd_stamp(build, compiler, target));
let mut cargo = builder.cargo(compiler, Mode::Libtest, target, "build");
test_cargo(build, &compiler, target, &mut cargo);
run_cargo(build,
&mut cargo,
&libtest_stamp(build, compiler, target));
builder.ensure(TestLink {
compiler: builder.compiler(compiler.stage, build.build),
target_compiler: compiler,
target,
});
}
}
/// Same as `std_cargo`, but for libtest
pub fn test_cargo(build: &Build,
_compiler: &Compiler,
_target: Interned<String>,
cargo: &mut Command) {
if let Some(target) = env::var_os("MACOSX_STD_DEPLOYMENT_TARGET") {
cargo.env("MACOSX_DEPLOYMENT_TARGET", target);
}
cargo.arg("--manifest-path")
.arg(build.src.join("src/libtest/Cargo.toml"));
run_cargo(build,
&mut cargo,
&libtest_stamp(build, compiler, target));
.arg(build.src.join("src/libtest/Cargo.toml"));
}
/// Same as `std_link`, only for libtest
pub fn test_link(build: &Build,
compiler: &Compiler,
target_compiler: &Compiler,
target: &str) {
println!("Copying stage{} test from stage{} ({} -> {} / {})",
target_compiler.stage,
compiler.stage,
compiler.host,
target_compiler.host,
target);
add_to_sysroot(&build.sysroot_libdir(target_compiler, target),
&libtest_stamp(build, compiler, target));
#[derive(Debug, Copy, Clone, PartialEq, Eq, Hash)]
pub struct TestLink {
pub compiler: Compiler,
pub target_compiler: Compiler,
pub target: Interned<String>,
}
/// Build the compiler.
///
/// This will build the compiler for a particular stage of the build using
/// the `compiler` targeting the `target` architecture. The artifacts
/// created will also be linked into the sysroot directory.
pub fn rustc(build: &Build, target: &str, compiler: &Compiler) {
let _folder = build.fold_output(|| format!("stage{}-rustc", compiler.stage));
println!("Building stage{} compiler artifacts ({} -> {})",
compiler.stage, compiler.host, target);
impl Step for TestLink {
type Output = ();
let out_dir = build.cargo_out(compiler, Mode::Librustc, target);
build.clear_if_dirty(&out_dir, &libtest_stamp(build, compiler, target));
fn should_run(run: ShouldRun) -> ShouldRun {
run.never()
}
let mut cargo = build.cargo(compiler, Mode::Librustc, target, "build");
/// Same as `std_link`, only for libtest
fn run(self, builder: &Builder) {
let build = builder.build;
let compiler = self.compiler;
let target_compiler = self.target_compiler;
let target = self.target;
println!("Copying stage{} test from stage{} ({} -> {} / {})",
target_compiler.stage,
compiler.stage,
&compiler.host,
target_compiler.host,
target);
add_to_sysroot(&builder.sysroot_libdir(target_compiler, target),
&libtest_stamp(build, compiler, target));
builder.ensure(tool::CleanTools {
compiler: target_compiler,
target,
mode: Mode::Libtest,
});
}
}
#[derive(Debug, Copy, Clone, PartialEq, Eq, Hash)]
pub struct Rustc {
pub compiler: Compiler,
pub target: Interned<String>,
}
impl Step for Rustc {
type Output = ();
const ONLY_HOSTS: bool = true;
const DEFAULT: bool = true;
fn should_run(run: ShouldRun) -> ShouldRun {
run.path("src/librustc").krate("rustc-main")
}
fn make_run(run: RunConfig) {
run.builder.ensure(Rustc {
compiler: run.builder.compiler(run.builder.top_stage, run.host),
target: run.target,
});
}
/// Build the compiler.
///
/// This will build the compiler for a particular stage of the build using
/// the `compiler` targeting the `target` architecture. The artifacts
/// created will also be linked into the sysroot directory.
fn run(self, builder: &Builder) {
let build = builder.build;
let compiler = self.compiler;
let target = self.target;
builder.ensure(Test { compiler, target });
// Build LLVM for our target. This will implicitly build the host LLVM
// if necessary.
builder.ensure(native::Llvm { target });
if build.force_use_stage1(compiler, target) {
builder.ensure(Rustc {
compiler: builder.compiler(1, build.build),
target,
});
println!("Uplifting stage1 rustc ({} -> {})", &build.build, target);
builder.ensure(RustcLink {
compiler: builder.compiler(1, build.build),
target_compiler: compiler,
target,
});
return;
}
// Ensure that build scripts have a std to link against.
builder.ensure(Std {
compiler: builder.compiler(self.compiler.stage, build.build),
target: build.build,
});
let _folder = build.fold_output(|| format!("stage{}-rustc", compiler.stage));
println!("Building stage{} compiler artifacts ({} -> {})",
compiler.stage, &compiler.host, target);
let out_dir = build.cargo_out(compiler, Mode::Librustc, target);
build.clear_if_dirty(&out_dir, &libtest_stamp(build, compiler, target));
let mut cargo = builder.cargo(compiler, Mode::Librustc, target, "build");
rustc_cargo(build, &compiler, target, &mut cargo);
run_cargo(build,
&mut cargo,
&librustc_stamp(build, compiler, target));
builder.ensure(RustcLink {
compiler: builder.compiler(compiler.stage, build.build),
target_compiler: compiler,
target,
});
}
}
/// Same as `std_cargo`, but for libtest
pub fn rustc_cargo(build: &Build,
compiler: &Compiler,
target: Interned<String>,
cargo: &mut Command) {
cargo.arg("--features").arg(build.rustc_features())
.arg("--manifest-path")
.arg(build.src.join("src/rustc/Cargo.toml"));
@ -252,7 +516,8 @@ pub fn rustc(build: &Build, target: &str, compiler: &Compiler) {
if compiler.stage == 0 {
cargo.env("CFG_LIBDIR_RELATIVE", "lib");
} else {
let libdir_relative = build.config.libdir_relative.clone().unwrap_or(PathBuf::from("lib"));
let libdir_relative =
build.config.libdir_relative.clone().unwrap_or(PathBuf::from("lib"));
cargo.env("CFG_LIBDIR_RELATIVE", libdir_relative);
}
@ -277,7 +542,7 @@ pub fn rustc(build: &Build, target: &str, compiler: &Compiler) {
cargo.env("LLVM_RUSTLLVM", "1");
}
cargo.env("LLVM_CONFIG", build.llvm_config(target));
let target_config = build.config.target_config.get(target);
let target_config = build.config.target_config.get(&target);
if let Some(s) = target_config.and_then(|c| c.llvm_config.as_ref()) {
cargo.env("CFG_LLVM_ROOT", s);
}
@ -298,41 +563,59 @@ pub fn rustc(build: &Build, target: &str, compiler: &Compiler) {
if let Some(ref s) = build.config.rustc_default_ar {
cargo.env("CFG_DEFAULT_AR", s);
}
run_cargo(build,
&mut cargo,
&librustc_stamp(build, compiler, target));
}
/// Same as `std_link`, only for librustc
pub fn rustc_link(build: &Build,
compiler: &Compiler,
target_compiler: &Compiler,
target: &str) {
println!("Copying stage{} rustc from stage{} ({} -> {} / {})",
target_compiler.stage,
compiler.stage,
compiler.host,
target_compiler.host,
target);
add_to_sysroot(&build.sysroot_libdir(target_compiler, target),
&librustc_stamp(build, compiler, target));
#[derive(Debug, Copy, Clone, PartialEq, Eq, Hash)]
struct RustcLink {
pub compiler: Compiler,
pub target_compiler: Compiler,
pub target: Interned<String>,
}
impl Step for RustcLink {
type Output = ();
fn should_run(run: ShouldRun) -> ShouldRun {
run.never()
}
/// Same as `std_link`, only for librustc
fn run(self, builder: &Builder) {
let build = builder.build;
let compiler = self.compiler;
let target_compiler = self.target_compiler;
let target = self.target;
println!("Copying stage{} rustc from stage{} ({} -> {} / {})",
target_compiler.stage,
compiler.stage,
&compiler.host,
target_compiler.host,
target);
add_to_sysroot(&builder.sysroot_libdir(target_compiler, target),
&librustc_stamp(build, compiler, target));
builder.ensure(tool::CleanTools {
compiler: target_compiler,
target,
mode: Mode::Librustc,
});
}
}
/// Cargo's output path for the standard library in a given stage, compiled
/// by a particular compiler for the specified target.
fn libstd_stamp(build: &Build, compiler: &Compiler, target: &str) -> PathBuf {
pub fn libstd_stamp(build: &Build, compiler: Compiler, target: Interned<String>) -> PathBuf {
build.cargo_out(compiler, Mode::Libstd, target).join(".libstd.stamp")
}
/// Cargo's output path for libtest in a given stage, compiled by a particular
/// compiler for the specified target.
fn libtest_stamp(build: &Build, compiler: &Compiler, target: &str) -> PathBuf {
pub fn libtest_stamp(build: &Build, compiler: Compiler, target: Interned<String>) -> PathBuf {
build.cargo_out(compiler, Mode::Libtest, target).join(".libtest.stamp")
}
/// Cargo's output path for librustc in a given stage, compiled by a particular
/// compiler for the specified target.
fn librustc_stamp(build: &Build, compiler: &Compiler, target: &str) -> PathBuf {
pub fn librustc_stamp(build: &Build, compiler: Compiler, target: Interned<String>) -> PathBuf {
build.cargo_out(compiler, Mode::Librustc, target).join(".librustc.stamp")
}
@ -342,60 +625,132 @@ fn compiler_file(compiler: &Path, file: &str) -> PathBuf {
PathBuf::from(out.trim())
}
pub fn create_sysroot(build: &Build, compiler: &Compiler) {
let sysroot = build.sysroot(compiler);
let _ = fs::remove_dir_all(&sysroot);
t!(fs::create_dir_all(&sysroot));
#[derive(Debug, Copy, Clone, PartialEq, Eq, Hash)]
pub struct Sysroot {
pub compiler: Compiler,
}
/// Prepare a new compiler from the artifacts in `stage`
///
/// This will assemble a compiler in `build/$host/stage$stage`. The compiler
/// must have been previously produced by the `stage - 1` build.build
/// compiler.
pub fn assemble_rustc(build: &Build, stage: u32, host: &str) {
// nothing to do in stage0
if stage == 0 {
return
impl Step for Sysroot {
type Output = Interned<PathBuf>;
fn should_run(run: ShouldRun) -> ShouldRun {
run.never()
}
println!("Copying stage{} compiler ({})", stage, host);
/// Returns the sysroot for the `compiler` specified that *this build system
/// generates*.
///
/// That is, the sysroot for the stage0 compiler is not what the compiler
/// thinks it is by default, but it's the same as the default for stages
/// 1-3.
fn run(self, builder: &Builder) -> Interned<PathBuf> {
let build = builder.build;
let compiler = self.compiler;
let sysroot = if compiler.stage == 0 {
build.out.join(&compiler.host).join("stage0-sysroot")
} else {
build.out.join(&compiler.host).join(format!("stage{}", compiler.stage))
};
let _ = fs::remove_dir_all(&sysroot);
t!(fs::create_dir_all(&sysroot));
INTERNER.intern_path(sysroot)
}
}
// The compiler that we're assembling
let target_compiler = Compiler::new(stage, host);
#[derive(Debug, Copy, Clone, PartialEq, Eq, Hash)]
pub struct Assemble {
/// The compiler which we will produce in this step. Assemble itself will
/// take care of ensuring that the necessary prerequisites to do so exist,
/// that is, this target can be a stage2 compiler and Assemble will build
/// previous stages for you.
pub target_compiler: Compiler,
}
// The compiler that compiled the compiler we're assembling
let build_compiler = Compiler::new(stage - 1, &build.build);
impl Step for Assemble {
type Output = Compiler;
// Link in all dylibs to the libdir
let sysroot = build.sysroot(&target_compiler);
let sysroot_libdir = sysroot.join(libdir(host));
t!(fs::create_dir_all(&sysroot_libdir));
let src_libdir = build.sysroot_libdir(&build_compiler, host);
for f in t!(fs::read_dir(&src_libdir)).map(|f| t!(f)) {
let filename = f.file_name().into_string().unwrap();
if is_dylib(&filename) {
copy(&f.path(), &sysroot_libdir.join(&filename));
fn should_run(run: ShouldRun) -> ShouldRun {
run.path("src/rustc")
}
/// Prepare a new compiler from the artifacts in `stage`
///
/// This will assemble a compiler in `build/$host/stage$stage`. The compiler
/// must have been previously produced by the `stage - 1` build.build
/// compiler.
fn run(self, builder: &Builder) -> Compiler {
let build = builder.build;
let target_compiler = self.target_compiler;
if target_compiler.stage == 0 {
assert_eq!(build.build, target_compiler.host,
"Cannot obtain compiler for non-native build triple at stage 0");
// The stage 0 compiler for the build triple is always pre-built.
return target_compiler;
}
}
let out_dir = build.cargo_out(&build_compiler, Mode::Librustc, host);
// Get the compiler that we'll use to bootstrap ourselves.
let build_compiler = if target_compiler.host != build.build {
// Build a compiler for the host platform. We cannot use the stage0
// compiler for the host platform for this because it doesn't have
// the libraries we need. FIXME: Perhaps we should download those
// libraries? It would make builds faster...
// FIXME: It may be faster if we build just a stage 1
// compiler and then use that to bootstrap this compiler
// forward.
builder.compiler(target_compiler.stage - 1, build.build)
} else {
// Build the compiler we'll use to build the stage requested. This
// may build more than one compiler (going down to stage 0).
builder.compiler(target_compiler.stage - 1, target_compiler.host)
};
// Link the compiler binary itself into place
let rustc = out_dir.join(exe("rustc", host));
let bindir = sysroot.join("bin");
t!(fs::create_dir_all(&bindir));
let compiler = build.compiler_path(&target_compiler);
let _ = fs::remove_file(&compiler);
copy(&rustc, &compiler);
// Build the libraries for this compiler to link to (i.e., the libraries
// it uses at runtime). NOTE: Crates the target compiler compiles don't
// link to these. (FIXME: Is that correct? It seems to be correct most
// of the time but I think we do link to these for stage2/bin compilers
// when not performing a full bootstrap).
if builder.build.config.keep_stage.map_or(false, |s| target_compiler.stage <= s) {
builder.verbose("skipping compilation of compiler due to --keep-stage");
let compiler = build_compiler;
for stage in 0..min(target_compiler.stage, builder.config.keep_stage.unwrap()) {
let target_compiler = builder.compiler(stage, target_compiler.host);
let target = target_compiler.host;
builder.ensure(StdLink { compiler, target_compiler, target });
builder.ensure(TestLink { compiler, target_compiler, target });
builder.ensure(RustcLink { compiler, target_compiler, target });
}
} else {
builder.ensure(Rustc { compiler: build_compiler, target: target_compiler.host });
}
// See if rustdoc exists to link it into place
let rustdoc = exe("rustdoc", host);
let rustdoc_src = out_dir.join(&rustdoc);
let rustdoc_dst = bindir.join(&rustdoc);
if fs::metadata(&rustdoc_src).is_ok() {
let _ = fs::remove_file(&rustdoc_dst);
copy(&rustdoc_src, &rustdoc_dst);
let stage = target_compiler.stage;
let host = target_compiler.host;
println!("Assembling stage{} compiler ({})", stage, host);
// Link in all dylibs to the libdir
let sysroot = builder.sysroot(target_compiler);
let sysroot_libdir = sysroot.join(libdir(&*host));
t!(fs::create_dir_all(&sysroot_libdir));
let src_libdir = builder.sysroot_libdir(build_compiler, host);
for f in t!(fs::read_dir(&src_libdir)).map(|f| t!(f)) {
let filename = f.file_name().into_string().unwrap();
if is_dylib(&filename) {
copy(&f.path(), &sysroot_libdir.join(&filename));
}
}
let out_dir = build.cargo_out(build_compiler, Mode::Librustc, host);
// Link the compiler binary itself into place
let rustc = out_dir.join(exe("rustc", &*host));
let bindir = sysroot.join("bin");
t!(fs::create_dir_all(&bindir));
let compiler = builder.rustc(target_compiler);
let _ = fs::remove_file(&compiler);
copy(&rustc, &compiler);
target_compiler
}
}
@ -418,64 +773,6 @@ fn add_to_sysroot(sysroot_dst: &Path, stamp: &Path) {
}
}
/// Build a tool in `src/tools`
///
/// This will build the specified tool with the specified `host` compiler in
/// `stage` into the normal cargo output directory.
pub fn maybe_clean_tools(build: &Build, stage: u32, target: &str, mode: Mode) {
let compiler = Compiler::new(stage, &build.build);
let stamp = match mode {
Mode::Libstd => libstd_stamp(build, &compiler, target),
Mode::Libtest => libtest_stamp(build, &compiler, target),
Mode::Librustc => librustc_stamp(build, &compiler, target),
_ => panic!(),
};
let out_dir = build.cargo_out(&compiler, Mode::Tool, target);
build.clear_if_dirty(&out_dir, &stamp);
}
/// Build a tool in `src/tools`
///
/// This will build the specified tool with the specified `host` compiler in
/// `stage` into the normal cargo output directory.
pub fn tool(build: &Build, stage: u32, target: &str, tool: &str) {
let _folder = build.fold_output(|| format!("stage{}-{}", stage, tool));
println!("Building stage{} tool {} ({})", stage, tool, target);
let compiler = Compiler::new(stage, &build.build);
let mut cargo = build.cargo(&compiler, Mode::Tool, target, "build");
let dir = build.src.join("src/tools").join(tool);
cargo.arg("--manifest-path").arg(dir.join("Cargo.toml"));
// We don't want to build tools dynamically as they'll be running across
// stages and such and it's just easier if they're not dynamically linked.
cargo.env("RUSTC_NO_PREFER_DYNAMIC", "1");
if let Some(dir) = build.openssl_install_dir(target) {
cargo.env("OPENSSL_STATIC", "1");
cargo.env("OPENSSL_DIR", dir);
cargo.env("LIBZ_SYS_STATIC", "1");
}
cargo.env("CFG_RELEASE_CHANNEL", &build.config.channel);
let info = GitInfo::new(&dir);
if let Some(sha) = info.sha() {
cargo.env("CFG_COMMIT_HASH", sha);
}
if let Some(sha_short) = info.sha_short() {
cargo.env("CFG_SHORT_COMMIT_HASH", sha_short);
}
if let Some(date) = info.commit_date() {
cargo.env("CFG_COMMIT_DATE", date);
}
build.run(&mut cargo);
}
// Avoiding a dependency on winapi to keep compile times down
#[cfg(unix)]
fn stderr_isatty() -> bool {
@ -535,18 +832,18 @@ fn run_cargo(build: &Build, cargo: &mut Command, stamp: &Path) {
let stdout = BufReader::new(child.stdout.take().unwrap());
for line in stdout.lines() {
let line = t!(line);
let json = if line.starts_with("{") {
t!(line.parse::<json::Json>())
let json: serde_json::Value = if line.starts_with("{") {
t!(serde_json::from_str(&line))
} else {
// If this was informational, just print it out and continue
println!("{}", line);
continue
};
if json.find("reason").and_then(|j| j.as_string()) != Some("compiler-artifact") {
if json["reason"].as_str() != Some("compiler-artifact") {
continue
}
for filename in json["filenames"].as_array().unwrap() {
let filename = filename.as_string().unwrap();
let filename = filename.as_str().unwrap();
// Skip files like executables
if !filename.ends_with(".rlib") &&
!filename.ends_with(".lib") &&

215
src/bootstrap/config.rs
View File

@ -19,11 +19,14 @@ use std::fs::{self, File};
use std::io::prelude::*;
use std::path::PathBuf;
use std::process;
use std::cmp;
use num_cpus;
use rustc_serialize::Decodable;
use toml::{Parser, Decoder, Value};
use toml;
use util::{exe, push_exe_path};
use cache::{INTERNER, Interned};
use flags::Flags;
pub use flags::Subcommand;
/// Global configuration for the entire build and/or bootstrap.
///
@ -35,7 +38,7 @@ use util::{exe, push_exe_path};
/// Note that this structure is not decoded directly into, but rather it is
/// filled out from the decoded forms of the structs below. For documentation
/// each field, see the corresponding fields in
/// `src/bootstrap/config.toml.example`.
/// `config.toml.example`.
#[derive(Default)]
pub struct Config {
pub ccache: Option<String>,
@ -46,13 +49,25 @@ pub struct Config {
pub docs: bool,
pub locked_deps: bool,
pub vendor: bool,
pub target_config: HashMap<String, Target>,
pub target_config: HashMap<Interned<String>, Target>,
pub full_bootstrap: bool,
pub extended: bool,
pub sanitizers: bool,
pub profiler: bool,
pub ignore_git: bool,
pub run_host_only: bool,
pub on_fail: Option<String>,
pub stage: Option<u32>,
pub keep_stage: Option<u32>,
pub src: PathBuf,
pub jobs: Option<u32>,
pub cmd: Subcommand,
pub incremental: bool,
// llvm codegen options
pub llvm_enabled: bool,
pub llvm_assertions: bool,
pub llvm_optimize: bool,
pub llvm_release_debuginfo: bool,
@ -62,7 +77,6 @@ pub struct Config {
pub llvm_targets: Option<String>,
pub llvm_experimental_targets: Option<String>,
pub llvm_link_jobs: Option<u32>,
pub llvm_clean_rebuild: bool,
// rust codegen options
pub rust_optimize: bool,
@ -78,9 +92,9 @@ pub struct Config {
pub rust_debuginfo_tests: bool,
pub rust_dist_src: bool,
pub build: String,
pub host: Vec<String>,
pub target: Vec<String>,
pub build: Interned<String>,
pub hosts: Vec<Interned<String>>,
pub targets: Vec<Interned<String>>,
pub local_rebuild: bool,
// dist misc
@ -129,6 +143,7 @@ pub struct Target {
pub cc: Option<PathBuf>,
pub cxx: Option<PathBuf>,
pub ndk: Option<PathBuf>,
pub crt_static: Option<bool>,
pub musl_root: Option<PathBuf>,
pub qemu_rootfs: Option<PathBuf>,
}
@ -138,7 +153,8 @@ pub struct Target {
/// This structure uses `Decodable` to automatically decode a TOML configuration
/// file into this format, and then this is traversed and written into the above
/// `Config` structure.
#[derive(RustcDecodable, Default)]
#[derive(Deserialize, Default)]
#[serde(deny_unknown_fields, rename_all = "kebab-case")]
struct TomlConfig {
build: Option<Build>,
install: Option<Install>,
@ -149,10 +165,13 @@ struct TomlConfig {
}
/// TOML representation of various global build decisions.
#[derive(RustcDecodable, Default, Clone)]
#[derive(Deserialize, Default, Clone)]
#[serde(deny_unknown_fields, rename_all = "kebab-case")]
struct Build {
build: Option<String>,
#[serde(default)]
host: Vec<String>,
#[serde(default)]
target: Vec<String>,
cargo: Option<String>,
rustc: Option<String>,
@ -174,7 +193,8 @@ struct Build {
}
/// TOML representation of various global install decisions.
#[derive(RustcDecodable, Default, Clone)]
#[derive(Deserialize, Default, Clone)]
#[serde(deny_unknown_fields, rename_all = "kebab-case")]
struct Install {
prefix: Option<String>,
sysconfdir: Option<String>,
@ -185,8 +205,10 @@ struct Install {
}
/// TOML representation of how the LLVM build is configured.
#[derive(RustcDecodable, Default)]
#[derive(Deserialize, Default)]
#[serde(deny_unknown_fields, rename_all = "kebab-case")]
struct Llvm {
enabled: Option<bool>,
ccache: Option<StringOrBool>,
ninja: Option<bool>,
assertions: Option<bool>,
@ -197,10 +219,10 @@ struct Llvm {
targets: Option<String>,
experimental_targets: Option<String>,
link_jobs: Option<u32>,
clean_rebuild: Option<bool>,
}
#[derive(RustcDecodable, Default, Clone)]
#[derive(Deserialize, Default, Clone)]
#[serde(deny_unknown_fields, rename_all = "kebab-case")]
struct Dist {
sign_folder: Option<String>,
gpg_password_file: Option<String>,
@ -208,7 +230,8 @@ struct Dist {
src_tarball: Option<bool>,
}
#[derive(RustcDecodable)]
#[derive(Deserialize)]
#[serde(untagged)]
enum StringOrBool {
String(String),
Bool(bool),
@ -221,7 +244,8 @@ impl Default for StringOrBool {
}
/// TOML representation of how the Rust build is configured.
#[derive(RustcDecodable, Default)]
#[derive(Deserialize, Default)]
#[serde(deny_unknown_fields, rename_all = "kebab-case")]
struct Rust {
optimize: Option<bool>,
codegen_units: Option<u32>,
@ -240,23 +264,29 @@ struct Rust {
optimize_tests: Option<bool>,
debuginfo_tests: Option<bool>,
codegen_tests: Option<bool>,
ignore_git: Option<bool>,
}
/// TOML representation of how each build target is configured.
#[derive(RustcDecodable, Default)]
#[derive(Deserialize, Default)]
#[serde(deny_unknown_fields, rename_all = "kebab-case")]
struct TomlTarget {
llvm_config: Option<String>,
jemalloc: Option<String>,
cc: Option<String>,
cxx: Option<String>,
android_ndk: Option<String>,
crt_static: Option<bool>,
musl_root: Option<String>,
qemu_rootfs: Option<String>,
}
impl Config {
pub fn parse(build: &str, file: Option<PathBuf>) -> Config {
pub fn parse(args: &[String]) -> Config {
let flags = Flags::parse(&args);
let file = flags.config.clone();
let mut config = Config::default();
config.llvm_enabled = true;
config.llvm_optimize = true;
config.use_jemalloc = true;
config.backtrace = true;
@ -266,52 +296,69 @@ impl Config {
config.docs = true;
config.rust_rpath = true;
config.rust_codegen_units = 1;
config.build = build.to_string();
config.channel = "dev".to_string();
config.codegen_tests = true;
config.ignore_git = false;
config.rust_dist_src = true;
config.on_fail = flags.on_fail;
config.stage = flags.stage;
config.src = flags.src;
config.jobs = flags.jobs;
config.cmd = flags.cmd;
config.incremental = flags.incremental;
config.keep_stage = flags.keep_stage;
// If --target was specified but --host wasn't specified, don't run any host-only tests.
config.run_host_only = flags.host.is_empty() && !flags.target.is_empty();
let toml = file.map(|file| {
let mut f = t!(File::open(&file));
let mut toml = String::new();
t!(f.read_to_string(&mut toml));
let mut p = Parser::new(&toml);
let table = match p.parse() {
Some(table) => table,
None => {
println!("failed to parse TOML configuration '{}':", file.to_str().unwrap());
for err in p.errors.iter() {
let (loline, locol) = p.to_linecol(err.lo);
let (hiline, hicol) = p.to_linecol(err.hi);
println!("{}:{}-{}:{}: {}", loline, locol, hiline,
hicol, err.desc);
}
process::exit(2);
}
};
let mut d = Decoder::new(Value::Table(table));
match Decodable::decode(&mut d) {
Ok(cfg) => cfg,
Err(e) => {
println!("failed to decode TOML: {}", e);
let mut contents = String::new();
t!(f.read_to_string(&mut contents));
match toml::from_str(&contents) {
Ok(table) => table,
Err(err) => {
println!("failed to parse TOML configuration '{}': {}",
file.display(), err);
process::exit(2);
}
}
}).unwrap_or_else(|| TomlConfig::default());
let build = toml.build.clone().unwrap_or(Build::default());
set(&mut config.build, build.build.clone());
config.host.push(config.build.clone());
set(&mut config.build, build.build.clone().map(|x| INTERNER.intern_string(x)));
set(&mut config.build, flags.build);
if config.build.is_empty() {
// set by bootstrap.py
config.build = INTERNER.intern_str(&env::var("BUILD").unwrap());
}
config.hosts.push(config.build.clone());
for host in build.host.iter() {
if !config.host.contains(host) {
config.host.push(host.clone());
let host = INTERNER.intern_str(host);
if !config.hosts.contains(&host) {
config.hosts.push(host);
}
}
for target in config.host.iter().chain(&build.target) {
if !config.target.contains(target) {
config.target.push(target.clone());
for target in config.hosts.iter().cloned()
.chain(build.target.iter().map(|s| INTERNER.intern_str(s)))
{
if !config.targets.contains(&target) {
config.targets.push(target);
}
}
config.hosts = if !flags.host.is_empty() {
flags.host
} else {
config.hosts
};
config.targets = if !flags.target.is_empty() {
flags.target
} else {
config.targets
};
config.nodejs = build.nodejs.map(PathBuf::from);
config.gdb = build.gdb.map(PathBuf::from);
config.python = build.python.map(PathBuf::from);
@ -327,6 +374,7 @@ impl Config {
set(&mut config.sanitizers, build.sanitizers);
set(&mut config.profiler, build.profiler);
set(&mut config.openssl_static, build.openssl_static);
config.verbose = cmp::max(config.verbose, flags.verbose);
if let Some(ref install) = toml.install {
config.prefix = install.prefix.clone().map(PathBuf::from);
@ -348,12 +396,12 @@ impl Config {
Some(StringOrBool::Bool(false)) | None => {}
}
set(&mut config.ninja, llvm.ninja);
set(&mut config.llvm_enabled, llvm.enabled);
set(&mut config.llvm_assertions, llvm.assertions);
set(&mut config.llvm_optimize, llvm.optimize);
set(&mut config.llvm_release_debuginfo, llvm.release_debuginfo);
set(&mut config.llvm_version_check, llvm.version_check);
set(&mut config.llvm_static_stdcpp, llvm.static_libstdcpp);
set(&mut config.llvm_clean_rebuild, llvm.clean_rebuild);
config.llvm_targets = llvm.targets.clone();
config.llvm_experimental_targets = llvm.experimental_targets.clone();
config.llvm_link_jobs = llvm.link_jobs;
@ -373,6 +421,7 @@ impl Config {
set(&mut config.use_jemalloc, rust.use_jemalloc);
set(&mut config.backtrace, rust.backtrace);
set(&mut config.channel, rust.channel.clone());
set(&mut config.ignore_git, rust.ignore_git);
config.rustc_default_linker = rust.default_linker.clone();
config.rustc_default_ar = rust.default_ar.clone();
config.musl_root = rust.musl_root.clone().map(PathBuf::from);
@ -399,10 +448,11 @@ impl Config {
}
target.cxx = cfg.cxx.clone().map(PathBuf::from);
target.cc = cfg.cc.clone().map(PathBuf::from);
target.crt_static = cfg.crt_static.clone();
target.musl_root = cfg.musl_root.clone().map(PathBuf::from);
target.qemu_rootfs = cfg.qemu_rootfs.clone().map(PathBuf::from);
config.target_config.insert(triple.clone(), target);
config.target_config.insert(INTERNER.intern_string(triple.clone()), target);
}
}
@ -478,7 +528,6 @@ impl Config {
("LLVM_VERSION_CHECK", self.llvm_version_check),
("LLVM_STATIC_STDCPP", self.llvm_static_stdcpp),
("LLVM_LINK_SHARED", self.llvm_link_shared),
("LLVM_CLEAN_REBUILD", self.llvm_clean_rebuild),
("OPTIMIZE", self.rust_optimize),
("DEBUG_ASSERTIONS", self.rust_debug_assertions),
("DEBUGINFO", self.rust_debuginfo),
@ -504,13 +553,13 @@ impl Config {
}
match key {
"CFG_BUILD" if value.len() > 0 => self.build = value.to_string(),
"CFG_BUILD" if value.len() > 0 => self.build = INTERNER.intern_str(value),
"CFG_HOST" if value.len() > 0 => {
self.host.extend(value.split(" ").map(|s| s.to_string()));
self.hosts.extend(value.split(" ").map(|s| INTERNER.intern_str(s)));
}
"CFG_TARGET" if value.len() > 0 => {
self.target.extend(value.split(" ").map(|s| s.to_string()));
self.targets.extend(value.split(" ").map(|s| INTERNER.intern_str(s)));
}
"CFG_EXPERIMENTAL_TARGETS" if value.len() > 0 => {
self.llvm_experimental_targets = Some(value.to_string());
@ -519,33 +568,28 @@ impl Config {
self.musl_root = Some(parse_configure_path(value));
}
"CFG_MUSL_ROOT_X86_64" if value.len() > 0 => {
let target = "x86_64-unknown-linux-musl".to_string();
let target = self.target_config.entry(target)
.or_insert(Target::default());
let target = INTERNER.intern_str("x86_64-unknown-linux-musl");
let target = self.target_config.entry(target).or_insert(Target::default());
target.musl_root = Some(parse_configure_path(value));
}
"CFG_MUSL_ROOT_I686" if value.len() > 0 => {
let target = "i686-unknown-linux-musl".to_string();
let target = self.target_config.entry(target)
.or_insert(Target::default());
let target = INTERNER.intern_str("i686-unknown-linux-musl");
let target = self.target_config.entry(target).or_insert(Target::default());
target.musl_root = Some(parse_configure_path(value));
}
"CFG_MUSL_ROOT_ARM" if value.len() > 0 => {
let target = "arm-unknown-linux-musleabi".to_string();
let target = self.target_config.entry(target)
.or_insert(Target::default());
let target = INTERNER.intern_str("arm-unknown-linux-musleabi");
let target = self.target_config.entry(target).or_insert(Target::default());
target.musl_root = Some(parse_configure_path(value));
}
"CFG_MUSL_ROOT_ARMHF" if value.len() > 0 => {
let target = "arm-unknown-linux-musleabihf".to_string();
let target = self.target_config.entry(target)
.or_insert(Target::default());
let target = INTERNER.intern_str("arm-unknown-linux-musleabihf");
let target = self.target_config.entry(target).or_insert(Target::default());
target.musl_root = Some(parse_configure_path(value));
}
"CFG_MUSL_ROOT_ARMV7" if value.len() > 0 => {
let target = "armv7-unknown-linux-musleabihf".to_string();
let target = self.target_config.entry(target)
.or_insert(Target::default());
let target = INTERNER.intern_str("armv7-unknown-linux-musleabihf");
let target = self.target_config.entry(target).or_insert(Target::default());
target.musl_root = Some(parse_configure_path(value));
}
"CFG_DEFAULT_AR" if value.len() > 0 => {
@ -593,33 +637,28 @@ impl Config {
target.jemalloc = Some(parse_configure_path(value).join("libjemalloc_pic.a"));
}
"CFG_ARM_LINUX_ANDROIDEABI_NDK" if value.len() > 0 => {
let target = "arm-linux-androideabi".to_string();
let target = self.target_config.entry(target)
.or_insert(Target::default());
let target = INTERNER.intern_str("arm-linux-androideabi");
let target = self.target_config.entry(target).or_insert(Target::default());
target.ndk = Some(parse_configure_path(value));
}
"CFG_ARMV7_LINUX_ANDROIDEABI_NDK" if value.len() > 0 => {
let target = "armv7-linux-androideabi".to_string();
let target = self.target_config.entry(target)
.or_insert(Target::default());
let target = INTERNER.intern_str("armv7-linux-androideabi");
let target = self.target_config.entry(target).or_insert(Target::default());
target.ndk = Some(parse_configure_path(value));
}
"CFG_I686_LINUX_ANDROID_NDK" if value.len() > 0 => {
let target = "i686-linux-android".to_string();
let target = self.target_config.entry(target)
.or_insert(Target::default());
let target = INTERNER.intern_str("i686-linux-android");
let target = self.target_config.entry(target).or_insert(Target::default());
target.ndk = Some(parse_configure_path(value));
}
"CFG_AARCH64_LINUX_ANDROID_NDK" if value.len() > 0 => {
let target = "aarch64-linux-android".to_string();
let target = self.target_config.entry(target)
.or_insert(Target::default());
let target = INTERNER.intern_str("aarch64-linux-android");
let target = self.target_config.entry(target).or_insert(Target::default());
target.ndk = Some(parse_configure_path(value));
}
"CFG_X86_64_LINUX_ANDROID_NDK" if value.len() > 0 => {
let target = "x86_64-linux-android".to_string();
let target = self.target_config.entry(target)
.or_insert(Target::default());
let target = INTERNER.intern_str("x86_64-linux-android");
let target = self.target_config.entry(target).or_insert(Target::default());
target.ndk = Some(parse_configure_path(value));
}
"CFG_LOCAL_RUST_ROOT" if value.len() > 0 => {
@ -643,9 +682,13 @@ impl Config {
.collect();
}
"CFG_QEMU_ARMHF_ROOTFS" if value.len() > 0 => {
let target = "arm-unknown-linux-gnueabihf".to_string();
let target = self.target_config.entry(target)
.or_insert(Target::default());
let target = INTERNER.intern_str("arm-unknown-linux-gnueabihf");
let target = self.target_config.entry(target).or_insert(Target::default());
target.qemu_rootfs = Some(parse_configure_path(value));
}
"CFG_QEMU_AARCH64_ROOTFS" if value.len() > 0 => {
let target = INTERNER.intern_str("aarch64-unknown-linux-gnu");
let target = self.target_config.entry(target).or_insert(Target::default());
target.qemu_rootfs = Some(parse_configure_path(value));
}
_ => {}

333
src/bootstrap/config.toml.example
View File

@ -1,333 +0,0 @@
# Sample TOML configuration file for building Rust.
#
# To configure rustbuild, copy this file to the directory from which you will be
# running the build, and name it config.toml.
#
# All options are commented out by default in this file, and they're commented
# out with their default values. The build system by default looks for
# `config.toml` in the current directory of a build for build configuration, but
# a custom configuration file can also be specified with `--config` to the build
# system.
# =============================================================================
# Tweaking how LLVM is compiled
# =============================================================================
[llvm]
# Indicates whether the LLVM build is a Release or Debug build
#optimize = true
# Indicates whether an LLVM Release build should include debug info
#release-debuginfo = false
# Indicates whether the LLVM assertions are enabled or not
#assertions = false
# Indicates whether ccache is used when building LLVM
#ccache = false
# or alternatively ...
#ccache = "/path/to/ccache"
# If an external LLVM root is specified, we automatically check the version by
# default to make sure it's within the range that we're expecting, but setting
# this flag will indicate that this version check should not be done.
#version-check = false
# Link libstdc++ statically into the librustc_llvm instead of relying on a
# dynamic version to be available.
#static-libstdcpp = false
# Tell the LLVM build system to use Ninja instead of the platform default for
# the generated build system. This can sometimes be faster than make, for
# example.
#ninja = false
# LLVM targets to build support for.
# Note: this is NOT related to Rust compilation targets. However, as Rust is
# dependent on LLVM for code generation, turning targets off here WILL lead to
# the resulting rustc being unable to compile for the disabled architectures.
# Also worth pointing out is that, in case support for new targets are added to
# LLVM, enabling them here doesn't mean Rust is automatically gaining said
# support. You'll need to write a target specification at least, and most
# likely, teach rustc about the C ABI of the target. Get in touch with the
# Rust team and file an issue if you need assistance in porting!
#targets = "X86;ARM;AArch64;Mips;PowerPC;SystemZ;JSBackend;MSP430;Sparc;NVPTX;Hexagon"
# LLVM experimental targets to build support for. These targets are specified in
# the same format as above, but since these targets are experimental, they are
# not built by default and the experimental Rust compilation targets that depend
# on them will not work unless the user opts in to building them. Possible
# experimental LLVM targets include WebAssembly for the
# wasm32-experimental-emscripten Rust target.
#experimental-targets = ""
# Cap the number of parallel linker invocations when compiling LLVM.
# This can be useful when building LLVM with debug info, which significantly
# increases the size of binaries and consequently the memory required by
# each linker process.
# If absent or 0, linker invocations are treated like any other job and
# controlled by rustbuild's -j parameter.
#link-jobs = 0
# Delete LLVM build directory on LLVM rebuild.
# This option defaults to `false` for local development, but CI may want to
# always perform clean full builds (possibly accelerated by (s)ccache).
#clean-rebuild = false
# =============================================================================
# General build configuration options
# =============================================================================
[build]
# Build triple for the original snapshot compiler. This must be a compiler that
# nightlies are already produced for. The current platform must be able to run
# binaries of this build triple and the nightly will be used to bootstrap the
# first compiler.
#build = "x86_64-unknown-linux-gnu" # defaults to your host platform
# In addition to the build triple, other triples to produce full compiler
# toolchains for. Each of these triples will be bootstrapped from the build
# triple and then will continue to bootstrap themselves. This platform must
# currently be able to run all of the triples provided here.
#host = ["x86_64-unknown-linux-gnu"] # defaults to just the build triple
# In addition to all host triples, other triples to produce the standard library
# for. Each host triple will be used to produce a copy of the standard library
# for each target triple.
#target = ["x86_64-unknown-linux-gnu"] # defaults to just the build triple
# Instead of downloading the src/stage0.txt version of Cargo specified, use
# this Cargo binary instead to build all Rust code
#cargo = "/path/to/bin/cargo"
# Instead of downloading the src/stage0.txt version of the compiler
# specified, use this rustc binary instead as the stage0 snapshot compiler.
#rustc = "/path/to/bin/rustc"
# Flag to specify whether any documentation is built. If false, rustdoc and
# friends will still be compiled but they will not be used to generate any
# documentation.
#docs = true
# Indicate whether the compiler should be documented in addition to the standard
# library and facade crates.
#compiler-docs = false
# Indicate whether submodules are managed and updated automatically.
#submodules = true
# The path to (or name of) the GDB executable to use. This is only used for
# executing the debuginfo test suite.
#gdb = "gdb"
# The node.js executable to use. Note that this is only used for the emscripten
# target when running tests, otherwise this can be omitted.
#nodejs = "node"
# Python interpreter to use for various tasks throughout the build, notably
# rustdoc tests, the lldb python interpreter, and some dist bits and pieces.
# Note that Python 2 is currently required.
#python = "python2.7"
# Force Cargo to check that Cargo.lock describes the precise dependency
# set that all the Cargo.toml files create, instead of updating it.
#locked-deps = false
# Indicate whether the vendored sources are used for Rust dependencies or not
#vendor = false
# Typically the build system will build the rust compiler twice. The second
# compiler, however, will simply use its own libraries to link against. If you
# would rather to perform a full bootstrap, compiling the compiler three times,
# then you can set this option to true. You shouldn't ever need to set this
# option to true.
#full-bootstrap = false
# Enable a build of the and extended rust tool set which is not only the
# compiler but also tools such as Cargo. This will also produce "combined
# installers" which are used to install Rust and Cargo together. This is
# disabled by default.
#extended = false
# Verbosity level: 0 == not verbose, 1 == verbose, 2 == very verbose
#verbose = 0
# Build the sanitizer runtimes
#sanitizers = false
# Build the profiler runtime
#profiler = false
# Indicates whether the OpenSSL linked into Cargo will be statically linked or
# not. If static linkage is specified then the build system will download a
# known-good version of OpenSSL, compile it, and link it to Cargo.
#openssl-static = false
# Run the build with low priority, by setting the process group's "nice" value
# to +10 on Unix platforms, and by using a "low priority" job object on Windows.
#low-priority = false
# =============================================================================
# General install configuration options
# =============================================================================
[install]
# Instead of installing to /usr/local, install to this path instead.
#prefix = "/usr/local"
# Where to install system configuration files
# If this is a relative path, it will get installed in `prefix` above
#sysconfdir = "/etc"
# Where to install documentation in `prefix` above
#docdir = "share/doc/rust"
# Where to install binaries in `prefix` above
#bindir = "bin"
# Where to install libraries in `prefix` above
#libdir = "lib"
# Where to install man pages in `prefix` above
#mandir = "share/man"
# =============================================================================
# Options for compiling Rust code itself
# =============================================================================
[rust]
# Whether or not to optimize the compiler and standard library
# Note: the slowness of the non optimized compiler compiling itself usually
# outweighs the time gains in not doing optimizations, therefore a
# full bootstrap takes much more time with optimize set to false.
#optimize = true
# Number of codegen units to use for each compiler invocation. A value of 0
# means "the number of cores on this machine", and 1+ is passed through to the
# compiler.
#codegen-units = 1
# Whether or not debug assertions are enabled for the compiler and standard
# library
#debug-assertions = false
# Whether or not debuginfo is emitted
#debuginfo = false
# Whether or not line number debug information is emitted
#debuginfo-lines = false
# Whether or not to only build debuginfo for the standard library if enabled.
# If enabled, this will not compile the compiler with debuginfo, just the
# standard library.
#debuginfo-only-std = false
# Whether or not jemalloc is built and enabled
#use-jemalloc = true
# Whether or not jemalloc is built with its debug option set
#debug-jemalloc = false
# Whether or not `panic!`s generate backtraces (RUST_BACKTRACE)
#backtrace = true
# The default linker that will be used by the generated compiler. Note that this
# is not the linker used to link said compiler.
#default-linker = "cc"
# The default ar utility that will be used by the generated compiler if LLVM
# cannot be used. Note that this is not used to assemble said compiler.
#default-ar = "ar"
# The "channel" for the Rust build to produce. The stable/beta channels only
# allow using stable features, whereas the nightly and dev channels allow using
# nightly features
#channel = "dev"
# By default the `rustc` executable is built with `-Wl,-rpath` flags on Unix
# platforms to ensure that the compiler is usable by default from the build
# directory (as it links to a number of dynamic libraries). This may not be
# desired in distributions, for example.
#rpath = true
# Flag indicating whether tests are compiled with optimizations (the -O flag) or
# with debuginfo (the -g flag)
#optimize-tests = true
#debuginfo-tests = true
# Flag indicating whether codegen tests will be run or not. If you get an error
# saying that the FileCheck executable is missing, you may want to disable this.
#codegen-tests = true
# =============================================================================
# Options for specific targets
#
# Each of the following options is scoped to the specific target triple in
# question and is used for determining how to compile each target.
# =============================================================================
[target.x86_64-unknown-linux-gnu]
# C compiler to be used to compiler C code and link Rust code. Note that the
# default value is platform specific, and if not specified it may also depend on
# what platform is crossing to what platform.
#cc = "cc"
# C++ compiler to be used to compiler C++ code (e.g. LLVM and our LLVM shims).
# This is only used for host targets.
#cxx = "c++"
# Path to the `llvm-config` binary of the installation of a custom LLVM to link
# against. Note that if this is specifed we don't compile LLVM at all for this
# target.
#llvm-config = "../path/to/llvm/root/bin/llvm-config"
# Path to the custom jemalloc static library to link into the standard library
# by default. This is only used if jemalloc is still enabled above
#jemalloc = "/path/to/jemalloc/libjemalloc_pic.a"
# If this target is for Android, this option will be required to specify where
# the NDK for the target lives. This is used to find the C compiler to link and
# build native code.
#android-ndk = "/path/to/ndk"
# The root location of the MUSL installation directory. The library directory
# will also need to contain libunwind.a for an unwinding implementation. Note
# that this option only makes sense for MUSL targets that produce statically
# linked binaries
#musl-root = "..."
# =============================================================================
# Distribution options
#
# These options are related to distribution, mostly for the Rust project itself.
# You probably won't need to concern yourself with any of these options
# =============================================================================
[dist]
# This is the folder of artifacts that the build system will sign. All files in
# this directory will be signed with the default gpg key using the system `gpg`
# binary. The `asc` and `sha256` files will all be output into the standard dist
# output folder (currently `build/dist`)
#
# This folder should be populated ahead of time before the build system is
# invoked.
#sign-folder = "path/to/folder/to/sign"
# This is a file which contains the password of the default gpg key. This will
# be passed to `gpg` down the road when signing all files in `sign-folder`
# above. This should be stored in plaintext.
#gpg-password-file = "path/to/gpg/password"
# The remote address that all artifacts will eventually be uploaded to. The
# build system generates manifests which will point to these urls, and for the
# manifests to be correct they'll have to have the right URLs encoded.
#
# Note that this address should not contain a trailing slash as file names will
# be appended to it.
#upload-addr = "https://example.com/folder"
# Whether to build a plain source tarball to upload
# We disable that on Windows not to override the one already uploaded on S3
# as the one built on Windows will contain backslashes in paths causing problems
# on linux
#src-tarball = true

1937
src/bootstrap/dist.rs
View File

File diff suppressed because it is too large Load Diff

939
src/bootstrap/doc.rs
View File

@ -20,87 +20,277 @@
use std::fs::{self, File};
use std::io::prelude::*;
use std::io;
use std::path::Path;
use std::process::Command;
use std::path::{PathBuf, Path};
use {Build, Compiler, Mode};
use util::{cp_r, symlink_dir};
use Mode;
use build_helper::up_to_date;
/// Invoke `rustbook` for `target` for the doc book `name`.
///
/// This will not actually generate any documentation if the documentation has
/// already been generated.
pub fn rustbook(build: &Build, target: &str, name: &str) {
let src = build.src.join("src/doc");
rustbook_src(build, target, name, &src);
}
use util::{cp_r, symlink_dir};
use builder::{Builder, Compiler, RunConfig, ShouldRun, Step};
use tool::Tool;
use compile;
use cache::{INTERNER, Interned};
/// Invoke `rustbook` for `target` for the doc book `name` from the `src` path.
///
/// This will not actually generate any documentation if the documentation has
/// already been generated.
pub fn rustbook_src(build: &Build, target: &str, name: &str, src: &Path) {
let out = build.doc_out(target);
t!(fs::create_dir_all(&out));
macro_rules! book {
($($name:ident, $path:expr, $book_name:expr;)+) => {
$(
#[derive(Debug, Copy, Clone, Hash, PartialEq, Eq)]
pub struct $name {
target: Interned<String>,
}
let out = out.join(name);
let compiler = Compiler::new(0, &build.build);
let src = src.join(name);
let index = out.join("index.html");
let rustbook = build.tool(&compiler, "rustbook");
if up_to_date(&src, &index) && up_to_date(&rustbook, &index) {
return
}
println!("Rustbook ({}) - {}", target, name);
let _ = fs::remove_dir_all(&out);
build.run(build.tool_cmd(&compiler, "rustbook")
.arg("build")
.arg(&src)
.arg("-d")
.arg(out));
}
impl Step for $name {
type Output = ();
const DEFAULT: bool = true;
/// Build the book and associated stuff.
///
/// We need to build:
///
/// * Book (first edition)
/// * Book (second edition)
/// * Index page
/// * Redirect pages
pub fn book(build: &Build, target: &str, name: &str) {
// build book first edition
rustbook(build, target, &format!("{}/first-edition", name));
fn should_run(run: ShouldRun) -> ShouldRun {
let builder = run.builder;
run.path($path).default_condition(builder.build.config.docs)
}
// build book second edition
rustbook(build, target, &format!("{}/second-edition", name));
fn make_run(run: RunConfig) {
run.builder.ensure($name {
target: run.target,
});
}
// build the index page
let index = format!("{}/index.md", name);
println!("Documenting book index ({})", target);
invoke_rustdoc(build, target, &index);
// build the redirect pages
println!("Documenting book redirect pages ({})", target);
for file in t!(fs::read_dir(build.src.join("src/doc/book/redirects"))) {
let file = t!(file);
let path = file.path();
let path = path.to_str().unwrap();
invoke_rustdoc(build, target, path);
fn run(self, builder: &Builder) {
builder.ensure(Rustbook {
target: self.target,
name: INTERNER.intern_str($book_name),
})
}
}
)+
}
}
fn invoke_rustdoc(build: &Build, target: &str, markdown: &str) {
let out = build.doc_out(target);
book!(
Nomicon, "src/doc/book", "nomicon";
Reference, "src/doc/reference", "reference";
Rustdoc, "src/doc/rustdoc", "rustdoc";
);
let compiler = Compiler::new(0, &build.build);
#[derive(Debug, Copy, Clone, Hash, PartialEq, Eq)]
struct Rustbook {
target: Interned<String>,
name: Interned<String>,
}
impl Step for Rustbook {
type Output = ();
// rustbook is never directly called, and only serves as a shim for the nomicon and the
// reference.
fn should_run(run: ShouldRun) -> ShouldRun {
run.never()
}
/// Invoke `rustbook` for `target` for the doc book `name`.
///
/// This will not actually generate any documentation if the documentation has
/// already been generated.
fn run(self, builder: &Builder) {
let src = builder.build.src.join("src/doc");
builder.ensure(RustbookSrc {
target: self.target,
name: self.name,
src: INTERNER.intern_path(src),
});
}
}
#[derive(Debug, Copy, Clone, Hash, PartialEq, Eq)]
pub struct UnstableBook {
target: Interned<String>,
}
impl Step for UnstableBook {
type Output = ();
const DEFAULT: bool = true;
fn should_run(run: ShouldRun) -> ShouldRun {
let builder = run.builder;
run.path("src/doc/unstable-book").default_condition(builder.build.config.docs)
}
fn make_run(run: RunConfig) {
run.builder.ensure(UnstableBook {
target: run.target,
});
}
fn run(self, builder: &Builder) {
builder.ensure(UnstableBookGen {
target: self.target,
});
builder.ensure(RustbookSrc {
target: self.target,
name: INTERNER.intern_str("unstable-book"),
src: builder.build.md_doc_out(self.target),
})
}
}
#[derive(Debug, Copy, Clone, Hash, PartialEq, Eq)]
struct RustbookSrc {
target: Interned<String>,
name: Interned<String>,
src: Interned<PathBuf>,
}
impl Step for RustbookSrc {
type Output = ();
fn should_run(run: ShouldRun) -> ShouldRun {
run.never()
}
/// Invoke `rustbook` for `target` for the doc book `name` from the `src` path.
///
/// This will not actually generate any documentation if the documentation has
/// already been generated.
fn run(self, builder: &Builder) {
let build = builder.build;
let target = self.target;
let name = self.name;
let src = self.src;
let out = build.doc_out(target);
t!(fs::create_dir_all(&out));
let out = out.join(name);
let src = src.join(name);
let index = out.join("index.html");
let rustbook = builder.tool_exe(Tool::Rustbook);
if up_to_date(&src, &index) && up_to_date(&rustbook, &index) {
return
}
println!("Rustbook ({}) - {}", target, name);
let _ = fs::remove_dir_all(&out);
build.run(builder.tool_cmd(Tool::Rustbook)
.arg("build")
.arg(&src)
.arg("-d")
.arg(out));
}
}
#[derive(Debug, Copy, Clone, Hash, PartialEq, Eq)]
pub struct TheBook {
compiler: Compiler,
target: Interned<String>,
name: &'static str,
}
impl Step for TheBook {
type Output = ();
const DEFAULT: bool = true;
fn should_run(run: ShouldRun) -> ShouldRun {
let builder = run.builder;
run.path("src/doc/book").default_condition(builder.build.config.docs)
}
fn make_run(run: RunConfig) {
run.builder.ensure(TheBook {
compiler: run.builder.compiler(run.builder.top_stage, run.builder.build.build),
target: run.target,
name: "book",
});
}
/// Build the book and associated stuff.
///
/// We need to build:
///
/// * Book (first edition)
/// * Book (second edition)
/// * Index page
/// * Redirect pages
fn run(self, builder: &Builder) {
let build = builder.build;
let target = self.target;
let name = self.name;
// build book first edition
builder.ensure(Rustbook {
target,
name: INTERNER.intern_string(format!("{}/first-edition", name)),
});
// build book second edition
builder.ensure(Rustbook {
target,
name: INTERNER.intern_string(format!("{}/second-edition", name)),
});
// build the index page
let index = format!("{}/index.md", name);
println!("Documenting book index ({})", target);
invoke_rustdoc(builder, self.compiler, target, &index);
// build the redirect pages
println!("Documenting book redirect pages ({})", target);
for file in t!(fs::read_dir(build.src.join("src/doc/book/redirects"))) {
let file = t!(file);
let path = file.path();
let path = path.to_str().unwrap();
invoke_rustdoc(builder, self.compiler, target, path);
}
}
}
#[derive(Debug, Copy, Clone, Hash, PartialEq, Eq)]
pub struct CargoBook {
target: Interned<String>,
}
impl Step for CargoBook {
type Output = ();
const DEFAULT: bool = true;
fn should_run(run: ShouldRun) -> ShouldRun {
let builder = run.builder;
run.path("src/doc/cargo").default_condition(builder.build.config.docs)
}
fn make_run(run: RunConfig) {
run.builder.ensure(CargoBook {
target: run.target,
});
}
/// Create a placeholder for the cargo documentation so that doc.rust-lang.org/cargo will
/// redirect to doc.crates.io. We want to publish doc.rust-lang.org/cargo in the paper
/// version of the book, but we don't want to rush the process of switching cargo's docs
/// over to mdbook and deploying them. When the cargo book is ready, this implementation
/// should build the mdbook instead of this redirect page.
fn run(self, builder: &Builder) {
let build = builder.build;
let out = build.doc_out(self.target);
let cargo_dir = out.join("cargo");
t!(fs::create_dir_all(&cargo_dir));
let index = cargo_dir.join("index.html");
let redirect_html = r#"
<html>
<head>
<meta http-equiv="refresh" content="0; URL='http://doc.crates.io'" />
</head>
</html>"#;
println!("Creating cargo book redirect page");
t!(t!(File::create(&index)).write_all(redirect_html.as_bytes()));
}
}
fn invoke_rustdoc(builder: &Builder, compiler: Compiler, target: Interned<String>, markdown: &str) {
let build = builder.build;
let out = build.doc_out(target);
let path = build.src.join("src/doc").join(markdown);
let rustdoc = build.rustdoc(&compiler);
let favicon = build.src.join("src/doc/favicon.inc");
let footer = build.src.join("src/doc/footer.inc");
@ -116,9 +306,7 @@ fn invoke_rustdoc(build: &Build, target: &str, markdown: &str) {
t!(t!(File::create(&version_info)).write_all(info.as_bytes()));
}
let mut cmd = Command::new(&rustdoc);
build.add_rustc_lib_path(&compiler, &mut cmd);
let mut cmd = builder.rustdoc_cmd(compiler.host);
let out = out.join("book");
@ -137,244 +325,411 @@ fn invoke_rustdoc(build: &Build, target: &str, markdown: &str) {
build.run(&mut cmd);
}
/// Generates all standalone documentation as compiled by the rustdoc in `stage`
/// for the `target` into `out`.
///
/// This will list all of `src/doc` looking for markdown files and appropriately
/// perform transformations like substituting `VERSION`, `SHORT_HASH`, and
/// `STAMP` alongw ith providing the various header/footer HTML we've cutomized.
///
/// In the end, this is just a glorified wrapper around rustdoc!
pub fn standalone(build: &Build, target: &str) {
println!("Documenting standalone ({})", target);
let out = build.doc_out(target);
t!(fs::create_dir_all(&out));
#[derive(Debug, Copy, Clone, Hash, PartialEq, Eq)]
pub struct Standalone {
compiler: Compiler,
target: Interned<String>,
}
let compiler = Compiler::new(0, &build.build);
impl Step for Standalone {
type Output = ();
const DEFAULT: bool = true;
let favicon = build.src.join("src/doc/favicon.inc");
let footer = build.src.join("src/doc/footer.inc");
let full_toc = build.src.join("src/doc/full-toc.inc");
t!(fs::copy(build.src.join("src/doc/rust.css"), out.join("rust.css")));
let version_input = build.src.join("src/doc/version_info.html.template");
let version_info = out.join("version_info.html");
if !up_to_date(&version_input, &version_info) {
let mut info = String::new();
t!(t!(File::open(&version_input)).read_to_string(&mut info));
let info = info.replace("VERSION", &build.rust_release())
.replace("SHORT_HASH", build.rust_info.sha_short().unwrap_or(""))
.replace("STAMP", build.rust_info.sha().unwrap_or(""));
t!(t!(File::create(&version_info)).write_all(info.as_bytes()));
fn should_run(run: ShouldRun) -> ShouldRun {
let builder = run.builder;
run.path("src/doc").default_condition(builder.build.config.docs)
}
for file in t!(fs::read_dir(build.src.join("src/doc"))) {
let file = t!(file);
let path = file.path();
let filename = path.file_name().unwrap().to_str().unwrap();
if !filename.ends_with(".md") || filename == "README.md" {
continue
fn make_run(run: RunConfig) {
run.builder.ensure(Standalone {
compiler: run.builder.compiler(run.builder.top_stage, run.builder.build.build),
target: run.target,
});
}
/// Generates all standalone documentation as compiled by the rustdoc in `stage`
/// for the `target` into `out`.
///
/// This will list all of `src/doc` looking for markdown files and appropriately
/// perform transformations like substituting `VERSION`, `SHORT_HASH`, and
/// `STAMP` along with providing the various header/footer HTML we've customized.
///
/// In the end, this is just a glorified wrapper around rustdoc!
fn run(self, builder: &Builder) {
let build = builder.build;
let target = self.target;
let compiler = self.compiler;
println!("Documenting standalone ({})", target);
let out = build.doc_out(target);
t!(fs::create_dir_all(&out));
let favicon = build.src.join("src/doc/favicon.inc");
let footer = build.src.join("src/doc/footer.inc");
let full_toc = build.src.join("src/doc/full-toc.inc");
t!(fs::copy(build.src.join("src/doc/rust.css"), out.join("rust.css")));
let version_input = build.src.join("src/doc/version_info.html.template");
let version_info = out.join("version_info.html");
if !up_to_date(&version_input, &version_info) {
let mut info = String::new();
t!(t!(File::open(&version_input)).read_to_string(&mut info));
let info = info.replace("VERSION", &build.rust_release())
.replace("SHORT_HASH", build.rust_info.sha_short().unwrap_or(""))
.replace("STAMP", build.rust_info.sha().unwrap_or(""));
t!(t!(File::create(&version_info)).write_all(info.as_bytes()));
}
let html = out.join(filename).with_extension("html");
let rustdoc = build.rustdoc(&compiler);
if up_to_date(&path, &html) &&
up_to_date(&footer, &html) &&
up_to_date(&favicon, &html) &&
up_to_date(&full_toc, &html) &&
up_to_date(&version_info, &html) &&
up_to_date(&rustdoc, &html) {
continue
for file in t!(fs::read_dir(build.src.join("src/doc"))) {
let file = t!(file);
let path = file.path();
let filename = path.file_name().unwrap().to_str().unwrap();
if !filename.ends_with(".md") || filename == "README.md" {
continue
}
let html = out.join(filename).with_extension("html");
let rustdoc = builder.rustdoc(compiler.host);
if up_to_date(&path, &html) &&
up_to_date(&footer, &html) &&
up_to_date(&favicon, &html) &&
up_to_date(&full_toc, &html) &&
up_to_date(&version_info, &html) &&
up_to_date(&rustdoc, &html) {
continue
}
let mut cmd = builder.rustdoc_cmd(compiler.host);
cmd.arg("--html-after-content").arg(&footer)
.arg("--html-before-content").arg(&version_info)
.arg("--html-in-header").arg(&favicon)
.arg("--markdown-playground-url")
.arg("https://play.rust-lang.org/")
.arg("-o").arg(&out)
.arg(&path);
if filename == "not_found.md" {
cmd.arg("--markdown-no-toc")
.arg("--markdown-css")
.arg("https://doc.rust-lang.org/rust.css");
} else {
cmd.arg("--markdown-css").arg("rust.css");
}
build.run(&mut cmd);
}
}
}
let mut cmd = Command::new(&rustdoc);
build.add_rustc_lib_path(&compiler, &mut cmd);
cmd.arg("--html-after-content").arg(&footer)
.arg("--html-before-content").arg(&version_info)
.arg("--html-in-header").arg(&favicon)
.arg("--markdown-playground-url")
.arg("https://play.rust-lang.org/")
.arg("-o").arg(&out)
.arg(&path);
#[derive(Debug, Copy, Clone, Hash, PartialEq, Eq)]
pub struct Std {
stage: u32,
target: Interned<String>,
}
if filename == "not_found.md" {
cmd.arg("--markdown-no-toc")
.arg("--markdown-css")
.arg("https://doc.rust-lang.org/rust.css");
impl Step for Std {
type Output = ();
const DEFAULT: bool = true;
fn should_run(run: ShouldRun) -> ShouldRun {
let builder = run.builder;
run.krate("std").default_condition(builder.build.config.docs)
}
fn make_run(run: RunConfig) {
run.builder.ensure(Std {
stage: run.builder.top_stage,
target: run.target
});
}
/// Compile all standard library documentation.
///
/// This will generate all documentation for the standard library and its
/// dependencies. This is largely just a wrapper around `cargo doc`.
fn run(self, builder: &Builder) {
let build = builder.build;
let stage = self.stage;
let target = self.target;
println!("Documenting stage{} std ({})", stage, target);
let out = build.doc_out(target);
t!(fs::create_dir_all(&out));
let compiler = builder.compiler(stage, build.build);
let rustdoc = builder.rustdoc(compiler.host);
let compiler = if build.force_use_stage1(compiler, target) {
builder.compiler(1, compiler.host)
} else {
cmd.arg("--markdown-css").arg("rust.css");
compiler
};
builder.ensure(compile::Std { compiler, target });
let out_dir = build.stage_out(compiler, Mode::Libstd)
.join(target).join("doc");
// Here what we're doing is creating a *symlink* (directory junction on
// Windows) to the final output location. This is not done as an
// optimization but rather for correctness. We've got three trees of
// documentation, one for std, one for test, and one for rustc. It's then
// our job to merge them all together.
//
// Unfortunately rustbuild doesn't know nearly as well how to merge doc
// trees as rustdoc does itself, so instead of actually having three
// separate trees we just have rustdoc output to the same location across
// all of them.
//
// This way rustdoc generates output directly into the output, and rustdoc
// will also directly handle merging.
let my_out = build.crate_doc_out(target);
build.clear_if_dirty(&my_out, &rustdoc);
t!(symlink_dir_force(&my_out, &out_dir));
let mut cargo = builder.cargo(compiler, Mode::Libstd, target, "doc");
compile::std_cargo(build, &compiler, target, &mut cargo);
// We don't want to build docs for internal std dependencies unless
// in compiler-docs mode. When not in that mode, we whitelist the crates
// for which docs must be built.
if !build.config.compiler_docs {
cargo.arg("--no-deps");
for krate in &["alloc", "collections", "core", "std", "std_unicode"] {
cargo.arg("-p").arg(krate);
// Create all crate output directories first to make sure rustdoc uses
// relative links.
// FIXME: Cargo should probably do this itself.
t!(fs::create_dir_all(out_dir.join(krate)));
}
}
build.run(&mut cargo);
cp_r(&my_out, &out);
}
}
#[derive(Debug, Copy, Clone, Hash, PartialEq, Eq)]
pub struct Test {
stage: u32,
target: Interned<String>,
}
impl Step for Test {
type Output = ();
const DEFAULT: bool = true;
fn should_run(run: ShouldRun) -> ShouldRun {
let builder = run.builder;
run.krate("test").default_condition(builder.config.compiler_docs)
}
fn make_run(run: RunConfig) {
run.builder.ensure(Test {
stage: run.builder.top_stage,
target: run.target,
});
}
/// Compile all libtest documentation.
///
/// This will generate all documentation for libtest and its dependencies. This
/// is largely just a wrapper around `cargo doc`.
fn run(self, builder: &Builder) {
let build = builder.build;
let stage = self.stage;
let target = self.target;
println!("Documenting stage{} test ({})", stage, target);
let out = build.doc_out(target);
t!(fs::create_dir_all(&out));
let compiler = builder.compiler(stage, build.build);
let rustdoc = builder.rustdoc(compiler.host);
let compiler = if build.force_use_stage1(compiler, target) {
builder.compiler(1, compiler.host)
} else {
compiler
};
// Build libstd docs so that we generate relative links
builder.ensure(Std { stage, target });
builder.ensure(compile::Test { compiler, target });
let out_dir = build.stage_out(compiler, Mode::Libtest)
.join(target).join("doc");
// See docs in std above for why we symlink
let my_out = build.crate_doc_out(target);
build.clear_if_dirty(&my_out, &rustdoc);
t!(symlink_dir_force(&my_out, &out_dir));
let mut cargo = builder.cargo(compiler, Mode::Libtest, target, "doc");
compile::test_cargo(build, &compiler, target, &mut cargo);
build.run(&mut cargo);
cp_r(&my_out, &out);
}
}
#[derive(Debug, Copy, Clone, Hash, PartialEq, Eq)]
pub struct Rustc {
stage: u32,
target: Interned<String>,
}
impl Step for Rustc {
type Output = ();
const DEFAULT: bool = true;
const ONLY_HOSTS: bool = true;
fn should_run(run: ShouldRun) -> ShouldRun {
let builder = run.builder;
run.krate("rustc-main").default_condition(builder.build.config.docs)
}
fn make_run(run: RunConfig) {
run.builder.ensure(Rustc {
stage: run.builder.top_stage,
target: run.target,
});
}
/// Generate all compiler documentation.
///
/// This will generate all documentation for the compiler libraries and their
/// dependencies. This is largely just a wrapper around `cargo doc`.
fn run(self, builder: &Builder) {
let build = builder.build;
let stage = self.stage;
let target = self.target;
println!("Documenting stage{} compiler ({})", stage, target);
let out = build.doc_out(target);
t!(fs::create_dir_all(&out));
let compiler = builder.compiler(stage, build.build);
let rustdoc = builder.rustdoc(compiler.host);
let compiler = if build.force_use_stage1(compiler, target) {
builder.compiler(1, compiler.host)
} else {
compiler
};
// Build libstd docs so that we generate relative links
builder.ensure(Std { stage, target });
builder.ensure(compile::Rustc { compiler, target });
let out_dir = build.stage_out(compiler, Mode::Librustc)
.join(target).join("doc");
// See docs in std above for why we symlink
let my_out = build.crate_doc_out(target);
build.clear_if_dirty(&my_out, &rustdoc);
t!(symlink_dir_force(&my_out, &out_dir));
let mut cargo = builder.cargo(compiler, Mode::Librustc, target, "doc");
compile::rustc_cargo(build, &compiler, target, &mut cargo);
if build.config.compiler_docs {
// src/rustc/Cargo.toml contains bin crates called rustc and rustdoc
// which would otherwise overwrite the docs for the real rustc and
// rustdoc lib crates.
cargo.arg("-p").arg("rustc_driver")
.arg("-p").arg("rustdoc");
} else {
// Like with libstd above if compiler docs aren't enabled then we're not
// documenting internal dependencies, so we have a whitelist.
cargo.arg("--no-deps");
for krate in &["proc_macro"] {
cargo.arg("-p").arg(krate);
}
}
build.run(&mut cargo);
cp_r(&my_out, &out);
}
}
#[derive(Debug, Copy, Clone, Hash, PartialEq, Eq)]
pub struct ErrorIndex {
target: Interned<String>,
}
impl Step for ErrorIndex {
type Output = ();
const DEFAULT: bool = true;
const ONLY_HOSTS: bool = true;
fn should_run(run: ShouldRun) -> ShouldRun {
let builder = run.builder;
run.path("src/tools/error_index_generator").default_condition(builder.build.config.docs)
}
fn make_run(run: RunConfig) {
run.builder.ensure(ErrorIndex {
target: run.target,
});
}
/// Generates the HTML rendered error-index by running the
/// `error_index_generator` tool.
fn run(self, builder: &Builder) {
let build = builder.build;
let target = self.target;
builder.ensure(compile::Rustc {
compiler: builder.compiler(0, build.build),
target,
});
println!("Documenting error index ({})", target);
let out = build.doc_out(target);
t!(fs::create_dir_all(&out));
let mut index = builder.tool_cmd(Tool::ErrorIndex);
index.arg("html");
index.arg(out.join("error-index.html"));
// FIXME: shouldn't have to pass this env var
index.env("CFG_BUILD", &build.build);
build.run(&mut index);
}
}
#[derive(Debug, Copy, Clone, Hash, PartialEq, Eq)]
pub struct UnstableBookGen {
target: Interned<String>,
}
impl Step for UnstableBookGen {
type Output = ();
const DEFAULT: bool = true;
const ONLY_HOSTS: bool = true;
fn should_run(run: ShouldRun) -> ShouldRun {
let builder = run.builder;
run.path("src/tools/unstable-book-gen").default_condition(builder.build.config.docs)
}
fn make_run(run: RunConfig) {
run.builder.ensure(UnstableBookGen {
target: run.target,
});
}
fn run(self, builder: &Builder) {
let build = builder.build;
let target = self.target;
builder.ensure(compile::Std {
compiler: builder.compiler(builder.top_stage, build.build),
target,
});
println!("Generating unstable book md files ({})", target);
let out = build.md_doc_out(target).join("unstable-book");
t!(fs::create_dir_all(&out));
t!(fs::remove_dir_all(&out));
let mut cmd = builder.tool_cmd(Tool::UnstableBookGen);
cmd.arg(build.src.join("src"));
cmd.arg(out);
build.run(&mut cmd);
}
}
/// Compile all standard library documentation.
///
/// This will generate all documentation for the standard library and its
/// dependencies. This is largely just a wrapper around `cargo doc`.
pub fn std(build: &Build, stage: u32, target: &str) {
println!("Documenting stage{} std ({})", stage, target);
let out = build.doc_out(target);
t!(fs::create_dir_all(&out));
let compiler = Compiler::new(stage, &build.build);
let compiler = if build.force_use_stage1(&compiler, target) {
Compiler::new(1, compiler.host)
} else {
compiler
};
let out_dir = build.stage_out(&compiler, Mode::Libstd)
.join(target).join("doc");
let rustdoc = build.rustdoc(&compiler);
// Here what we're doing is creating a *symlink* (directory junction on
// Windows) to the final output location. This is not done as an
// optimization but rather for correctness. We've got three trees of
// documentation, one for std, one for test, and one for rustc. It's then
// our job to merge them all together.
//
// Unfortunately rustbuild doesn't know nearly as well how to merge doc
// trees as rustdoc does itself, so instead of actually having three
// separate trees we just have rustdoc output to the same location across
// all of them.
//
// This way rustdoc generates output directly into the output, and rustdoc
// will also directly handle merging.
let my_out = build.crate_doc_out(target);
build.clear_if_dirty(&my_out, &rustdoc);
t!(symlink_dir_force(&my_out, &out_dir));
let mut cargo = build.cargo(&compiler, Mode::Libstd, target, "doc");
cargo.arg("--manifest-path")
.arg(build.src.join("src/libstd/Cargo.toml"))
.arg("--features").arg(build.std_features());
// We don't want to build docs for internal std dependencies unless
// in compiler-docs mode. When not in that mode, we whitelist the crates
// for which docs must be built.
if !build.config.compiler_docs {
cargo.arg("--no-deps");
for krate in &["alloc", "collections", "core", "std", "std_unicode"] {
cargo.arg("-p").arg(krate);
// Create all crate output directories first to make sure rustdoc uses
// relative links.
// FIXME: Cargo should probably do this itself.
t!(fs::create_dir_all(out_dir.join(krate)));
}
}
build.run(&mut cargo);
cp_r(&my_out, &out);
}
/// Compile all libtest documentation.
///
/// This will generate all documentation for libtest and its dependencies. This
/// is largely just a wrapper around `cargo doc`.
pub fn test(build: &Build, stage: u32, target: &str) {
println!("Documenting stage{} test ({})", stage, target);
let out = build.doc_out(target);
t!(fs::create_dir_all(&out));
let compiler = Compiler::new(stage, &build.build);
let compiler = if build.force_use_stage1(&compiler, target) {
Compiler::new(1, compiler.host)
} else {
compiler
};
let out_dir = build.stage_out(&compiler, Mode::Libtest)
.join(target).join("doc");
let rustdoc = build.rustdoc(&compiler);
// See docs in std above for why we symlink
let my_out = build.crate_doc_out(target);
build.clear_if_dirty(&my_out, &rustdoc);
t!(symlink_dir_force(&my_out, &out_dir));
let mut cargo = build.cargo(&compiler, Mode::Libtest, target, "doc");
cargo.arg("--manifest-path")
.arg(build.src.join("src/libtest/Cargo.toml"));
build.run(&mut cargo);
cp_r(&my_out, &out);
}
/// Generate all compiler documentation.
///
/// This will generate all documentation for the compiler libraries and their
/// dependencies. This is largely just a wrapper around `cargo doc`.
pub fn rustc(build: &Build, stage: u32, target: &str) {
println!("Documenting stage{} compiler ({})", stage, target);
let out = build.doc_out(target);
t!(fs::create_dir_all(&out));
let compiler = Compiler::new(stage, &build.build);
let compiler = if build.force_use_stage1(&compiler, target) {
Compiler::new(1, compiler.host)
} else {
compiler
};
let out_dir = build.stage_out(&compiler, Mode::Librustc)
.join(target).join("doc");
let rustdoc = build.rustdoc(&compiler);
// See docs in std above for why we symlink
let my_out = build.crate_doc_out(target);
build.clear_if_dirty(&my_out, &rustdoc);
t!(symlink_dir_force(&my_out, &out_dir));
let mut cargo = build.cargo(&compiler, Mode::Librustc, target, "doc");
cargo.arg("--manifest-path")
.arg(build.src.join("src/rustc/Cargo.toml"))
.arg("--features").arg(build.rustc_features());
if build.config.compiler_docs {
// src/rustc/Cargo.toml contains bin crates called rustc and rustdoc
// which would otherwise overwrite the docs for the real rustc and
// rustdoc lib crates.
cargo.arg("-p").arg("rustc_driver")
.arg("-p").arg("rustdoc");
} else {
// Like with libstd above if compiler docs aren't enabled then we're not
// documenting internal dependencies, so we have a whitelist.
cargo.arg("--no-deps");
for krate in &["proc_macro"] {
cargo.arg("-p").arg(krate);
}
}
build.run(&mut cargo);
cp_r(&my_out, &out);
}
/// Generates the HTML rendered error-index by running the
/// `error_index_generator` tool.
pub fn error_index(build: &Build, target: &str) {
println!("Documenting error index ({})", target);
let out = build.doc_out(target);
t!(fs::create_dir_all(&out));
let compiler = Compiler::new(0, &build.build);
let mut index = build.tool_cmd(&compiler, "error_index_generator");
index.arg("html");
index.arg(out.join("error-index.html"));
// FIXME: shouldn't have to pass this env var
index.env("CFG_BUILD", &build.build);
build.run(&mut index);
}
pub fn unstable_book_gen(build: &Build, target: &str) {
println!("Generating unstable book md files ({})", target);
let out = build.md_doc_out(target).join("unstable-book");
t!(fs::create_dir_all(&out));
t!(fs::remove_dir_all(&out));
let compiler = Compiler::new(0, &build.build);
let mut cmd = build.tool_cmd(&compiler, "unstable-book-gen");
cmd.arg(build.src.join("src"));
cmd.arg(out);
build.run(&mut cmd);
}
fn symlink_dir_force(src: &Path, dst: &Path) -> io::Result<()> {
if let Ok(m) = fs::symlink_metadata(dst) {
if m.file_type().is_dir() {

56
src/bootstrap/flags.rs
View File

@ -23,7 +23,9 @@ use getopts::Options;
use Build;
use config::Config;
use metadata;
use step;
use builder::Builder;
use cache::{Interned, INTERNER};
/// Deserialized version of all flags for this compile.
pub struct Flags {
@ -31,9 +33,10 @@ pub struct Flags {
pub on_fail: Option<String>,
pub stage: Option<u32>,
pub keep_stage: Option<u32>,
pub build: String,
pub host: Vec<String>,
pub target: Vec<String>,
pub build: Option<Interned<String>>,
pub host: Vec<Interned<String>>,
pub target: Vec<Interned<String>>,
pub config: Option<PathBuf>,
pub src: PathBuf,
pub jobs: Option<u32>,
@ -66,6 +69,14 @@ pub enum Subcommand {
},
}
impl Default for Subcommand {
fn default() -> Subcommand {
Subcommand::Build {
paths: vec![PathBuf::from("nowhere")],
}
}
}
impl Flags {
pub fn parse(args: &[String]) -> Flags {
let mut extra_help = String::new();
@ -241,15 +252,12 @@ Arguments:
// All subcommands can have an optional "Available paths" section
if matches.opt_present("verbose") {
let flags = Flags::parse(&["build".to_string()]);
let mut config = Config::parse(&flags.build, cfg_file.clone());
config.build = flags.build.clone();
let mut build = Build::new(flags, config);
let config = Config::parse(&["build".to_string()]);
let mut build = Build::new(config);
metadata::build(&mut build);
let maybe_rules_help = step::build_rules(&build).get_help(subcommand);
if maybe_rules_help.is_some() {
extra_help.push_str(maybe_rules_help.unwrap().as_str());
}
let maybe_rules_help = Builder::get_help(&build, subcommand.as_str());
extra_help.push_str(maybe_rules_help.unwrap_or_default().as_str());
} else {
extra_help.push_str(format!("Run `./x.py {} -h -v` to see a list of available paths.",
subcommand).as_str());
@ -266,14 +274,14 @@ Arguments:
}
"test" => {
Subcommand::Test {
paths: paths,
paths,
test_args: matches.opt_strs("test-args"),
fail_fast: !matches.opt_present("no-fail-fast"),
}
}
"bench" => {
Subcommand::Bench {
paths: paths,
paths,
test_args: matches.opt_strs("test-args"),
}
}
@ -289,12 +297,12 @@ Arguments:
}
"dist" => {
Subcommand::Dist {
paths: paths,
paths,
}
}
"install" => {
Subcommand::Install {
paths: paths,
paths,
}
}
_ => {
@ -316,18 +324,18 @@ Arguments:
Flags {
verbose: matches.opt_count("verbose"),
stage: stage,
stage,
on_fail: matches.opt_str("on-fail"),
keep_stage: matches.opt_str("keep-stage").map(|j| j.parse().unwrap()),
build: matches.opt_str("build").unwrap_or_else(|| {
env::var("BUILD").unwrap()
}),
host: split(matches.opt_strs("host")),
target: split(matches.opt_strs("target")),
build: matches.opt_str("build").map(|s| INTERNER.intern_string(s)),
host: split(matches.opt_strs("host"))
.into_iter().map(|x| INTERNER.intern_string(x)).collect::<Vec<_>>(),
target: split(matches.opt_strs("target"))
.into_iter().map(|x| INTERNER.intern_string(x)).collect::<Vec<_>>(),
config: cfg_file,
src: src,
src,
jobs: matches.opt_str("jobs").map(|j| j.parse().unwrap()),
cmd: cmd,
cmd,
incremental: matches.opt_present("incremental"),
}
}

255
src/bootstrap/install.rs
View File

@ -18,121 +18,99 @@ use std::fs;
use std::path::{Path, PathBuf, Component};
use std::process::Command;
use Build;
use dist::{pkgname, sanitize_sh, tmpdir};
use dist::{self, pkgname, sanitize_sh, tmpdir};
pub struct Installer<'a> {
build: &'a Build,
prefix: PathBuf,
sysconfdir: PathBuf,
docdir: PathBuf,
bindir: PathBuf,
libdir: PathBuf,
mandir: PathBuf,
empty_dir: PathBuf,
use builder::{Builder, RunConfig, ShouldRun, Step};
use cache::Interned;
pub fn install_docs(builder: &Builder, stage: u32, host: Interned<String>) {
install_sh(builder, "docs", "rust-docs", stage, Some(host));
}
impl<'a> Drop for Installer<'a> {
fn drop(&mut self) {
t!(fs::remove_dir_all(&self.empty_dir));
pub fn install_std(builder: &Builder, stage: u32) {
for target in &builder.build.targets {
install_sh(builder, "std", "rust-std", stage, Some(*target));
}
}
impl<'a> Installer<'a> {
pub fn new(build: &'a Build) -> Installer<'a> {
let prefix_default = PathBuf::from("/usr/local");
let sysconfdir_default = PathBuf::from("/etc");
let docdir_default = PathBuf::from("share/doc/rust");
let bindir_default = PathBuf::from("bin");
let libdir_default = PathBuf::from("lib");
let mandir_default = PathBuf::from("share/man");
let prefix = build.config.prefix.as_ref().unwrap_or(&prefix_default);
let sysconfdir = build.config.sysconfdir.as_ref().unwrap_or(&sysconfdir_default);
let docdir = build.config.docdir.as_ref().unwrap_or(&docdir_default);
let bindir = build.config.bindir.as_ref().unwrap_or(&bindir_default);
let libdir = build.config.libdir.as_ref().unwrap_or(&libdir_default);
let mandir = build.config.mandir.as_ref().unwrap_or(&mandir_default);
pub fn install_cargo(builder: &Builder, stage: u32, host: Interned<String>) {
install_sh(builder, "cargo", "cargo", stage, Some(host));
}
let sysconfdir = prefix.join(sysconfdir);
let docdir = prefix.join(docdir);
let bindir = prefix.join(bindir);
let libdir = prefix.join(libdir);
let mandir = prefix.join(mandir);
pub fn install_rls(builder: &Builder, stage: u32, host: Interned<String>) {
install_sh(builder, "rls", "rls", stage, Some(host));
}
let destdir = env::var_os("DESTDIR").map(PathBuf::from);
pub fn install_analysis(builder: &Builder, stage: u32, host: Interned<String>) {
install_sh(builder, "analysis", "rust-analysis", stage, Some(host));
}
let prefix = add_destdir(&prefix, &destdir);
let sysconfdir = add_destdir(&sysconfdir, &destdir);
let docdir = add_destdir(&docdir, &destdir);
let bindir = add_destdir(&bindir, &destdir);
let libdir = add_destdir(&libdir, &destdir);
let mandir = add_destdir(&mandir, &destdir);
pub fn install_src(builder: &Builder, stage: u32) {
install_sh(builder, "src", "rust-src", stage, None);
}
pub fn install_rustc(builder: &Builder, stage: u32, host: Interned<String>) {
install_sh(builder, "rustc", "rustc", stage, Some(host));
}
let empty_dir = build.out.join("tmp/empty_dir");
fn install_sh(
builder: &Builder,
package: &str,
name: &str,
stage: u32,
host: Option<Interned<String>>
) {
let build = builder.build;
println!("Install {} stage{} ({:?})", package, stage, host);
t!(fs::create_dir_all(&empty_dir));
let prefix_default = PathBuf::from("/usr/local");
let sysconfdir_default = PathBuf::from("/etc");
let docdir_default = PathBuf::from("share/doc/rust");
let bindir_default = PathBuf::from("bin");
let libdir_default = PathBuf::from("lib");
let mandir_default = PathBuf::from("share/man");
let prefix = build.config.prefix.as_ref().unwrap_or(&prefix_default);
let sysconfdir = build.config.sysconfdir.as_ref().unwrap_or(&sysconfdir_default);
let docdir = build.config.docdir.as_ref().unwrap_or(&docdir_default);
let bindir = build.config.bindir.as_ref().unwrap_or(&bindir_default);
let libdir = build.config.libdir.as_ref().unwrap_or(&libdir_default);
let mandir = build.config.mandir.as_ref().unwrap_or(&mandir_default);
Installer {
build,
prefix,
sysconfdir,
docdir,
bindir,
libdir,
mandir,
empty_dir,
}
}
let sysconfdir = prefix.join(sysconfdir);
let docdir = prefix.join(docdir);
let bindir = prefix.join(bindir);
let libdir = prefix.join(libdir);
let mandir = prefix.join(mandir);
pub fn install_docs(&self, stage: u32, host: &str) {
self.install_sh("docs", "rust-docs", stage, Some(host));
}
let destdir = env::var_os("DESTDIR").map(PathBuf::from);
pub fn install_std(&self, stage: u32) {
for target in self.build.config.target.iter() {
self.install_sh("std", "rust-std", stage, Some(target));
}
}
let prefix = add_destdir(&prefix, &destdir);
let sysconfdir = add_destdir(&sysconfdir, &destdir);
let docdir = add_destdir(&docdir, &destdir);
let bindir = add_destdir(&bindir, &destdir);
let libdir = add_destdir(&libdir, &destdir);
let mandir = add_destdir(&mandir, &destdir);
pub fn install_cargo(&self, stage: u32, host: &str) {
self.install_sh("cargo", "cargo", stage, Some(host));
}
let empty_dir = build.out.join("tmp/empty_dir");
pub fn install_rls(&self, stage: u32, host: &str) {
self.install_sh("rls", "rls", stage, Some(host));
}
t!(fs::create_dir_all(&empty_dir));
let package_name = if let Some(host) = host {
format!("{}-{}", pkgname(build, name), host)
} else {
pkgname(build, name)
};
pub fn install_analysis(&self, stage: u32, host: &str) {
self.install_sh("analysis", "rust-analysis", stage, Some(host));
}
pub fn install_src(&self, stage: u32) {
self.install_sh("src", "rust-src", stage, None);
}
pub fn install_rustc(&self, stage: u32, host: &str) {
self.install_sh("rustc", "rustc", stage, Some(host));
}
fn install_sh(&self, package: &str, name: &str, stage: u32, host: Option<&str>) {
println!("Install {} stage{} ({:?})", package, stage, host);
let package_name = if let Some(host) = host {
format!("{}-{}", pkgname(self.build, name), host)
} else {
pkgname(self.build, name)
};
let mut cmd = Command::new("sh");
cmd.current_dir(&self.empty_dir)
.arg(sanitize_sh(&tmpdir(self.build).join(&package_name).join("install.sh")))
.arg(format!("--prefix={}", sanitize_sh(&self.prefix)))
.arg(format!("--sysconfdir={}", sanitize_sh(&self.sysconfdir)))
.arg(format!("--docdir={}", sanitize_sh(&self.docdir)))
.arg(format!("--bindir={}", sanitize_sh(&self.bindir)))
.arg(format!("--libdir={}", sanitize_sh(&self.libdir)))
.arg(format!("--mandir={}", sanitize_sh(&self.mandir)))
.arg("--disable-ldconfig");
self.build.run(&mut cmd);
}
let mut cmd = Command::new("sh");
cmd.current_dir(&empty_dir)
.arg(sanitize_sh(&tmpdir(build).join(&package_name).join("install.sh")))
.arg(format!("--prefix={}", sanitize_sh(&prefix)))
.arg(format!("--sysconfdir={}", sanitize_sh(&sysconfdir)))
.arg(format!("--docdir={}", sanitize_sh(&docdir)))
.arg(format!("--bindir={}", sanitize_sh(&bindir)))
.arg(format!("--libdir={}", sanitize_sh(&libdir)))
.arg(format!("--mandir={}", sanitize_sh(&mandir)))
.arg("--disable-ldconfig");
build.run(&mut cmd);
t!(fs::remove_dir_all(&empty_dir));
}
fn add_destdir(path: &Path, destdir: &Option<PathBuf>) -> PathBuf {
@ -148,3 +126,84 @@ fn add_destdir(path: &Path, destdir: &Option<PathBuf>) -> PathBuf {
}
ret
}
macro_rules! install {
(($sel:ident, $builder:ident, $_config:ident),
$($name:ident,
$path:expr,
$default_cond:expr,
only_hosts: $only_hosts:expr,
$run_item:block $(, $c:ident)*;)+) => {
$(
#[derive(Debug, Copy, Clone, Hash, PartialEq, Eq)]
pub struct $name {
pub stage: u32,
pub target: Interned<String>,
pub host: Interned<String>,
}
impl Step for $name {
type Output = ();
const DEFAULT: bool = true;
const ONLY_BUILD_TARGETS: bool = true;
const ONLY_HOSTS: bool = $only_hosts;
$(const $c: bool = true;)*
fn should_run(run: ShouldRun) -> ShouldRun {
let $_config = &run.builder.config;
run.path($path).default_condition($default_cond)
}
fn make_run(run: RunConfig) {
run.builder.ensure($name {
stage: run.builder.top_stage,
target: run.target,
host: run.host,
});
}
fn run($sel, $builder: &Builder) {
$run_item
}
})+
}
}
install!((self, builder, _config),
Docs, "src/doc", _config.docs, only_hosts: false, {
builder.ensure(dist::Docs { stage: self.stage, host: self.target });
install_docs(builder, self.stage, self.target);
};
Std, "src/libstd", true, only_hosts: true, {
builder.ensure(dist::Std {
compiler: builder.compiler(self.stage, self.host),
target: self.target
});
install_std(builder, self.stage);
};
Cargo, "cargo", _config.extended, only_hosts: true, {
builder.ensure(dist::Cargo { stage: self.stage, target: self.target });
install_cargo(builder, self.stage, self.target);
};
Rls, "rls", _config.extended, only_hosts: true, {
builder.ensure(dist::Rls { stage: self.stage, target: self.target });
install_rls(builder, self.stage, self.target);
};
Analysis, "analysis", _config.extended, only_hosts: false, {
builder.ensure(dist::Analysis {
compiler: builder.compiler(self.stage, self.host),
target: self.target
});
install_analysis(builder, self.stage, self.target);
};
Src, "src", _config.extended, only_hosts: true, {
builder.ensure(dist::Src);
install_src(builder, self.stage);
}, ONLY_BUILD;
Rustc, "src/librustc", true, only_hosts: true, {
builder.ensure(dist::Rustc {
compiler: builder.compiler(self.stage, self.target),
});
install_rustc(builder, self.stage, self.target);
};
);

635
src/bootstrap/lib.rs
View File

@ -23,38 +23,87 @@
//!
//! ## Architecture
//!
//! Although this build system defers most of the complicated logic to Cargo
//! itself, it still needs to maintain a list of targets and dependencies which
//! it can itself perform. Rustbuild is made up of a list of rules with
//! dependencies amongst them (created in the `step` module) and then knows how
//! to execute each in sequence. Each time rustbuild is invoked, it will simply
//! iterate through this list of steps and execute each serially in turn. For
//! each step rustbuild relies on the step internally being incremental and
//! The build system defers most of the complicated logic managing invocations
//! of rustc and rustdoc to Cargo itself. However, moving through various stages
//! and copying artifacts is still necessary for it to do. Each time rustbuild
//! is invoked, it will iterate through the list of predefined steps and execute
//! each serially in turn if it matches the paths passed or is a default rule.
//! For each step rustbuild relies on the step internally being incremental and
//! parallel. Note, though, that the `-j` parameter to rustbuild gets forwarded
//! to appropriate test harnesses and such.
//!
//! Most of the "meaty" steps that matter are backed by Cargo, which does indeed
//! have its own parallelism and incremental management. Later steps, like
//! tests, aren't incremental and simply run the entire suite currently.
//! However, compiletest itself tries to avoid running tests when the artifacts
//! that are involved (mainly the compiler) haven't changed.
//!
//! When you execute `x.py build`, the steps which are executed are:
//!
//! * First, the python script is run. This will automatically download the
//! stage0 rustc and cargo according to `src/stage0.txt`, or using the cached
//! stage0 rustc and cargo according to `src/stage0.txt`, or use the cached
//! versions if they're available. These are then used to compile rustbuild
//! itself (using Cargo). Finally, control is then transferred to rustbuild.
//!
//! * Rustbuild takes over, performs sanity checks, probes the environment,
//! reads configuration, builds up a list of steps, and then starts executing
//! them.
//! reads configuration, and starts executing steps as it reads the command
//! line arguments (paths) or going through the default rules.
//!
//! * The stage0 libstd is compiled
//! * The stage0 libtest is compiled
//! * The stage0 librustc is compiled
//! * The stage1 compiler is assembled
//! * The stage1 libstd, libtest, librustc are compiled
//! * The stage2 compiler is assembled
//! * The stage2 libstd, libtest, librustc are compiled
//! The build output will be something like the following:
//!
//! Building stage0 std artifacts
//! Copying stage0 std
//! Building stage0 test artifacts
//! Copying stage0 test
//! Building stage0 compiler artifacts
//! Copying stage0 rustc
//! Assembling stage1 compiler
//! Building stage1 std artifacts
//! Copying stage1 std
//! Building stage1 test artifacts
//! Copying stage1 test
//! Building stage1 compiler artifacts
//! Copying stage1 rustc
//! Assembling stage2 compiler
//! Uplifting stage1 std
//! Uplifting stage1 test
//! Uplifting stage1 rustc
//!
//! Let's disect that a little:
//!
//! ## Building stage0 {std,test,compiler} artifacts
//!
//! These steps use the provided (downloaded, usually) compiler to compile the
//! local Rust source into libraries we can use.
//!
//! ## Copying stage0 {std,test,rustc}
//!
//! This copies the build output from Cargo into
//! `build/$HOST/stage0-sysroot/lib/rustlib/$ARCH/lib`. FIXME: This step's
//! documentation should be expanded -- the information already here may be
//! incorrect.
//!
//! ## Assembling stage1 compiler
//!
//! This copies the libraries we built in "building stage0 ... artifacts" into
//! the stage1 compiler's lib directory. These are the host libraries that the
//! compiler itself uses to run. These aren't actually used by artifacts the new
//! compiler generates. This step also copies the rustc and rustdoc binaries we
//! generated into build/$HOST/stage/bin.
//!
//! The stage1/bin/rustc is a fully functional compiler, but it doesn't yet have
//! any libraries to link built binaries or libraries to. The next 3 steps will
//! provide those libraries for it; they are mostly equivalent to constructing
//! the stage1/bin compiler so we don't go through them individually.
//!
//! ## Uplifting stage1 {std,test,rustc}
//!
//! This step copies the libraries from the stage1 compiler sysroot into the
//! stage2 compiler. This is done to avoid rebuilding the compiler; libraries
//! we'd build in this step should be identical (in function, if not necessarily
//! identical on disk) so there's no need to recompile the compiler again. Note
//! that if you want to, you can enable the full-bootstrap option to change this
//! behavior.
//!
//! Each step is driven by a separate Cargo project and rustbuild orchestrates
//! copying files between steps and otherwise preparing for Cargo to run.
@ -65,33 +114,38 @@
//! also check out the `src/bootstrap/README.md` file for more information.
#![deny(warnings)]
#![allow(stable_features)]
#![feature(associated_consts)]
#[macro_use]
extern crate build_helper;
#[macro_use]
extern crate serde_derive;
#[macro_use]
extern crate lazy_static;
extern crate serde_json;
extern crate cmake;
extern crate filetime;
extern crate gcc;
extern crate getopts;
extern crate num_cpus;
extern crate rustc_serialize;
extern crate toml;
#[cfg(unix)]
extern crate libc;
use std::cell::Cell;
use std::cmp;
use std::collections::HashMap;
use std::collections::{HashSet, HashMap};
use std::env;
use std::ffi::OsString;
use std::fs::{self, File};
use std::io::Read;
use std::path::{PathBuf, Path};
use std::process::Command;
use std::slice;
use build_helper::{run_silent, run_suppressed, try_run_silent, try_run_suppressed, output, mtime};
use util::{exe, libdir, add_lib_path, OutputFolder, CiEnv};
use util::{exe, libdir, OutputFolder, CiEnv};
mod cc;
mod channel;
@ -106,8 +160,10 @@ mod flags;
mod install;
mod native;
mod sanity;
mod step;
pub mod util;
mod builder;
mod cache;
mod tool;
#[cfg(windows)]
mod job;
@ -130,7 +186,8 @@ mod job {
}
pub use config::Config;
pub use flags::{Flags, Subcommand};
use flags::Subcommand;
use cache::{Interned, INTERNER};
/// A structure representing a Rust compiler.
///
@ -138,9 +195,9 @@ pub use flags::{Flags, Subcommand};
/// corresponds to the platform the compiler runs on. This structure is used as
/// a parameter to many methods below.
#[derive(Eq, PartialEq, Clone, Copy, Hash, Debug)]
pub struct Compiler<'a> {
pub struct Compiler {
stage: u32,
host: &'a str,
host: Interned<String>,
}
/// Global configuration for the build system.
@ -157,9 +214,6 @@ pub struct Build {
// User-specified configuration via config.toml
config: Config,
// User-specified configuration via CLI flags
flags: Flags,
// Derived properties from the above two configurations
src: PathBuf,
out: PathBuf,
@ -171,9 +225,9 @@ pub struct Build {
verbosity: usize,
// Targets for which to build.
build: String,
hosts: Vec<String>,
targets: Vec<String>,
build: Interned<String>,
hosts: Vec<Interned<String>>,
targets: Vec<Interned<String>>,
// Stage 0 (downloaded) compiler and cargo or their local rust equivalents.
initial_rustc: PathBuf,
@ -185,10 +239,10 @@ pub struct Build {
// Runtime state filled in later on
// target -> (cc, ar)
cc: HashMap<String, (gcc::Tool, Option<PathBuf>)>,
cc: HashMap<Interned<String>, (gcc::Tool, Option<PathBuf>)>,
// host -> (cc, ar)
cxx: HashMap<String, gcc::Tool>,
crates: HashMap<String, Crate>,
cxx: HashMap<Interned<String>, gcc::Tool>,
crates: HashMap<Interned<String>, Crate>,
is_sudo: bool,
ci_env: CiEnv,
delayed_failures: Cell<usize>,
@ -196,9 +250,9 @@ pub struct Build {
#[derive(Debug)]
struct Crate {
name: String,
name: Interned<String>,
version: String,
deps: Vec<String>,
deps: Vec<Interned<String>>,
path: PathBuf,
doc_step: String,
build_step: String,
@ -210,7 +264,7 @@ struct Crate {
///
/// These entries currently correspond to the various output directories of the
/// build system, with each mod generating output in a different directory.
#[derive(Clone, Copy, PartialEq, Eq)]
#[derive(Debug, Hash, Clone, Copy, PartialEq, Eq)]
pub enum Mode {
/// Build the standard library, placing output in the "stageN-std" directory.
Libstd,
@ -230,9 +284,9 @@ impl Build {
/// line and the filesystem `config`.
///
/// By default all build output will be placed in the current directory.
pub fn new(flags: Flags, config: Config) -> Build {
pub fn new(config: Config) -> Build {
let cwd = t!(env::current_dir());
let src = flags.src.clone();
let src = config.src.clone();
let out = cwd.join("build");
let is_sudo = match env::var_os("SUDO_USER") {
@ -244,64 +298,42 @@ impl Build {
}
None => false,
};
let rust_info = channel::GitInfo::new(&src);
let cargo_info = channel::GitInfo::new(&src.join("src/tools/cargo"));
let rls_info = channel::GitInfo::new(&src.join("src/tools/rls"));
let hosts = if !flags.host.is_empty() {
for host in flags.host.iter() {
if !config.host.contains(host) {
panic!("specified host `{}` is not in configuration", host);
}
}
flags.host.clone()
} else {
config.host.clone()
};
let targets = if !flags.target.is_empty() {
for target in flags.target.iter() {
if !config.target.contains(target) {
panic!("specified target `{}` is not in configuration", target);
}
}
flags.target.clone()
} else {
config.target.clone()
};
let rust_info = channel::GitInfo::new(&config, &src);
let cargo_info = channel::GitInfo::new(&config, &src.join("src/tools/cargo"));
let rls_info = channel::GitInfo::new(&config, &src.join("src/tools/rls"));
Build {
initial_rustc: config.initial_rustc.clone(),
initial_cargo: config.initial_cargo.clone(),
local_rebuild: config.local_rebuild,
fail_fast: flags.cmd.fail_fast(),
verbosity: cmp::max(flags.verbose, config.verbose),
fail_fast: config.cmd.fail_fast(),
verbosity: config.verbose,
build: config.host[0].clone(),
hosts: hosts,
targets: targets,
build: config.build,
hosts: config.hosts.clone(),
targets: config.targets.clone(),
flags: flags,
config: config,
src: src,
out: out,
config,
src,
out,
rust_info: rust_info,
cargo_info: cargo_info,
rls_info: rls_info,
rust_info,
cargo_info,
rls_info,
cc: HashMap::new(),
cxx: HashMap::new(),
crates: HashMap::new(),
lldb_version: None,
lldb_python_dir: None,
is_sudo: is_sudo,
is_sudo,
ci_env: CiEnv::current(),
delayed_failures: Cell::new(0),
}
}
fn build_slice(&self) -> &[String] {
pub fn build_triple(&self) -> &[Interned<String>] {
unsafe {
std::slice::from_raw_parts(&self.build, 1)
slice::from_raw_parts(&self.build, 1)
}
}
@ -311,7 +343,7 @@ impl Build {
job::setup(self);
}
if let Subcommand::Clean = self.flags.cmd {
if let Subcommand::Clean = self.config.cmd {
return clean::clean(self);
}
@ -333,7 +365,7 @@ impl Build {
self.verbose("learning about cargo");
metadata::build(self);
step::run(self);
builder::Builder::run(&self);
}
/// Clear out `dir` if `input` is newer.
@ -351,242 +383,6 @@ impl Build {
t!(File::create(stamp));
}
/// Prepares an invocation of `cargo` to be run.
///
/// This will create a `Command` that represents a pending execution of
/// Cargo. This cargo will be configured to use `compiler` as the actual
/// rustc compiler, its output will be scoped by `mode`'s output directory,
/// it will pass the `--target` flag for the specified `target`, and will be
/// executing the Cargo command `cmd`.
fn cargo(&self,
compiler: &Compiler,
mode: Mode,
target: &str,
cmd: &str) -> Command {
let mut cargo = Command::new(&self.initial_cargo);
let out_dir = self.stage_out(compiler, mode);
cargo.env("CARGO_TARGET_DIR", out_dir)
.arg(cmd)
.arg("-j").arg(self.jobs().to_string())
.arg("--target").arg(target);
// FIXME: Temporary fix for https://github.com/rust-lang/cargo/issues/3005
// Force cargo to output binaries with disambiguating hashes in the name
cargo.env("__CARGO_DEFAULT_LIB_METADATA", &self.config.channel);
let stage;
if compiler.stage == 0 && self.local_rebuild {
// Assume the local-rebuild rustc already has stage1 features.
stage = 1;
} else {
stage = compiler.stage;
}
// Customize the compiler we're running. Specify the compiler to cargo
// as our shim and then pass it some various options used to configure
// how the actual compiler itself is called.
//
// These variables are primarily all read by
// src/bootstrap/bin/{rustc.rs,rustdoc.rs}
cargo.env("RUSTBUILD_NATIVE_DIR", self.native_dir(target))
.env("RUSTC", self.out.join("bootstrap/debug/rustc"))
.env("RUSTC_REAL", self.compiler_path(compiler))
.env("RUSTC_STAGE", stage.to_string())
.env("RUSTC_CODEGEN_UNITS",
self.config.rust_codegen_units.to_string())
.env("RUSTC_DEBUG_ASSERTIONS",
self.config.rust_debug_assertions.to_string())
.env("RUSTC_SYSROOT", self.sysroot(compiler))
.env("RUSTC_LIBDIR", self.rustc_libdir(compiler))
.env("RUSTC_RPATH", self.config.rust_rpath.to_string())
.env("RUSTDOC", self.out.join("bootstrap/debug/rustdoc"))
.env("RUSTDOC_REAL", self.rustdoc(compiler))
.env("RUSTC_FLAGS", self.rustc_flags(target).join(" "));
if mode != Mode::Tool {
// Tools don't get debuginfo right now, e.g. cargo and rls don't
// get compiled with debuginfo.
cargo.env("RUSTC_DEBUGINFO", self.config.rust_debuginfo.to_string())
.env("RUSTC_DEBUGINFO_LINES", self.config.rust_debuginfo_lines.to_string())
.env("RUSTC_FORCE_UNSTABLE", "1");
// Currently the compiler depends on crates from crates.io, and
// then other crates can depend on the compiler (e.g. proc-macro
// crates). Let's say, for example that rustc itself depends on the
// bitflags crate. If an external crate then depends on the
// bitflags crate as well, we need to make sure they don't
// conflict, even if they pick the same verison of bitflags. We'll
// want to make sure that e.g. a plugin and rustc each get their
// own copy of bitflags.
// Cargo ensures that this works in general through the -C metadata
// flag. This flag will frob the symbols in the binary to make sure
// they're different, even though the source code is the exact
// same. To solve this problem for the compiler we extend Cargo's
// already-passed -C metadata flag with our own. Our rustc.rs
// wrapper around the actual rustc will detect -C metadata being
// passed and frob it with this extra string we're passing in.
cargo.env("RUSTC_METADATA_SUFFIX", "rustc");
}
// Enable usage of unstable features
cargo.env("RUSTC_BOOTSTRAP", "1");
self.add_rust_test_threads(&mut cargo);
// Almost all of the crates that we compile as part of the bootstrap may
// have a build script, including the standard library. To compile a
// build script, however, it itself needs a standard library! This
// introduces a bit of a pickle when we're compiling the standard
// library itself.
//
// To work around this we actually end up using the snapshot compiler
// (stage0) for compiling build scripts of the standard library itself.
// The stage0 compiler is guaranteed to have a libstd available for use.
//
// For other crates, however, we know that we've already got a standard
// library up and running, so we can use the normal compiler to compile
// build scripts in that situation.
if mode == Mode::Libstd {
cargo.env("RUSTC_SNAPSHOT", &self.initial_rustc)
.env("RUSTC_SNAPSHOT_LIBDIR", self.rustc_snapshot_libdir());
} else {
cargo.env("RUSTC_SNAPSHOT", self.compiler_path(compiler))
.env("RUSTC_SNAPSHOT_LIBDIR", self.rustc_libdir(compiler));
}
// Ignore incremental modes except for stage0, since we're
// not guaranteeing correctness across builds if the compiler
// is changing under your feet.`
if self.flags.incremental && compiler.stage == 0 {
let incr_dir = self.incremental_dir(compiler);
cargo.env("RUSTC_INCREMENTAL", incr_dir);
}
if let Some(ref on_fail) = self.flags.on_fail {
cargo.env("RUSTC_ON_FAIL", on_fail);
}
cargo.env("RUSTC_VERBOSE", format!("{}", self.verbosity));
// Specify some various options for build scripts used throughout
// the build.
//
// FIXME: the guard against msvc shouldn't need to be here
if !target.contains("msvc") {
cargo.env(format!("CC_{}", target), self.cc(target))
.env(format!("AR_{}", target), self.ar(target).unwrap()) // only msvc is None
.env(format!("CFLAGS_{}", target), self.cflags(target).join(" "));
if let Ok(cxx) = self.cxx(target) {
cargo.env(format!("CXX_{}", target), cxx);
}
}
if mode == Mode::Libstd &&
self.config.extended &&
compiler.is_final_stage(self) {
cargo.env("RUSTC_SAVE_ANALYSIS", "api".to_string());
}
// When being built Cargo will at some point call `nmake.exe` on Windows
// MSVC. Unfortunately `nmake` will read these two environment variables
// below and try to intepret them. We're likely being run, however, from
// MSYS `make` which uses the same variables.
//
// As a result, to prevent confusion and errors, we remove these
// variables from our environment to prevent passing MSYS make flags to
// nmake, causing it to blow up.
if cfg!(target_env = "msvc") {
cargo.env_remove("MAKE");
cargo.env_remove("MAKEFLAGS");
}
// Environment variables *required* throughout the build
//
// FIXME: should update code to not require this env var
cargo.env("CFG_COMPILER_HOST_TRIPLE", target);
if self.is_verbose() {
cargo.arg("-v");
}
// FIXME: cargo bench does not accept `--release`
if self.config.rust_optimize && cmd != "bench" {
cargo.arg("--release");
}
if self.config.locked_deps {
cargo.arg("--locked");
}
if self.config.vendor || self.is_sudo {
cargo.arg("--frozen");
}
self.ci_env.force_coloring_in_ci(&mut cargo);
cargo
}
/// Get a path to the compiler specified.
fn compiler_path(&self, compiler: &Compiler) -> PathBuf {
if compiler.is_snapshot(self) {
self.initial_rustc.clone()
} else {
self.sysroot(compiler).join("bin").join(exe("rustc", compiler.host))
}
}
/// Get the specified tool built by the specified compiler
fn tool(&self, compiler: &Compiler, tool: &str) -> PathBuf {
self.cargo_out(compiler, Mode::Tool, compiler.host)
.join(exe(tool, compiler.host))
}
/// Get the `rustdoc` executable next to the specified compiler
fn rustdoc(&self, compiler: &Compiler) -> PathBuf {
let mut rustdoc = self.compiler_path(compiler);
rustdoc.pop();
rustdoc.push(exe("rustdoc", compiler.host));
rustdoc
}
/// Get a `Command` which is ready to run `tool` in `stage` built for
/// `host`.
fn tool_cmd(&self, compiler: &Compiler, tool: &str) -> Command {
let mut cmd = Command::new(self.tool(&compiler, tool));
self.prepare_tool_cmd(compiler, &mut cmd);
cmd
}
/// Prepares the `cmd` provided to be able to run the `compiler` provided.
///
/// Notably this munges the dynamic library lookup path to point to the
/// right location to run `compiler`.
fn prepare_tool_cmd(&self, compiler: &Compiler, cmd: &mut Command) {
let host = compiler.host;
let mut paths = vec![
self.sysroot_libdir(compiler, compiler.host),
self.cargo_out(compiler, Mode::Tool, host).join("deps"),
];
// On MSVC a tool may invoke a C compiler (e.g. compiletest in run-make
// mode) and that C compiler may need some extra PATH modification. Do
// so here.
if compiler.host.contains("msvc") {
let curpaths = env::var_os("PATH").unwrap_or(OsString::new());
let curpaths = env::split_paths(&curpaths).collect::<Vec<_>>();
for &(ref k, ref v) in self.cc[compiler.host].0.env() {
if k != "PATH" {
continue
}
for path in env::split_paths(v) {
if !curpaths.contains(&path) {
paths.push(path);
}
}
}
}
add_lib_path(paths, cmd);
}
/// Get the space-separated set of activated features for the standard
/// library.
fn std_features(&self) -> String {
@ -613,6 +409,9 @@ impl Build {
if self.config.use_jemalloc {
features.push_str(" jemalloc");
}
if self.config.llvm_enabled {
features.push_str(" llvm");
}
features
}
@ -622,94 +421,67 @@ impl Build {
if self.config.rust_optimize {"release"} else {"debug"}
}
/// Returns the sysroot for the `compiler` specified that *this build system
/// generates*.
///
/// That is, the sysroot for the stage0 compiler is not what the compiler
/// thinks it is by default, but it's the same as the default for stages
/// 1-3.
fn sysroot(&self, compiler: &Compiler) -> PathBuf {
if compiler.stage == 0 {
self.out.join(compiler.host).join("stage0-sysroot")
} else {
self.out.join(compiler.host).join(format!("stage{}", compiler.stage))
}
}
/// Get the directory for incremental by-products when using the
/// given compiler.
fn incremental_dir(&self, compiler: &Compiler) -> PathBuf {
self.out.join(compiler.host).join(format!("stage{}-incremental", compiler.stage))
}
/// Returns the libdir where the standard library and other artifacts are
/// found for a compiler's sysroot.
fn sysroot_libdir(&self, compiler: &Compiler, target: &str) -> PathBuf {
if compiler.stage >= 2 {
if let Some(ref libdir_relative) = self.config.libdir_relative {
return self.sysroot(compiler).join(libdir_relative)
.join("rustlib").join(target).join("lib")
}
}
self.sysroot(compiler).join("lib").join("rustlib")
.join(target).join("lib")
fn incremental_dir(&self, compiler: Compiler) -> PathBuf {
self.out.join(&*compiler.host).join(format!("stage{}-incremental", compiler.stage))
}
/// Returns the root directory for all output generated in a particular
/// stage when running with a particular host compiler.
///
/// The mode indicates what the root directory is for.
fn stage_out(&self, compiler: &Compiler, mode: Mode) -> PathBuf {
fn stage_out(&self, compiler: Compiler, mode: Mode) -> PathBuf {
let suffix = match mode {
Mode::Libstd => "-std",
Mode::Libtest => "-test",
Mode::Tool => "-tools",
Mode::Librustc => "-rustc",
};
self.out.join(compiler.host)
self.out.join(&*compiler.host)
.join(format!("stage{}{}", compiler.stage, suffix))
}
/// Returns the root output directory for all Cargo output in a given stage,
/// running a particular compiler, wehther or not we're building the
/// running a particular compiler, whether or not we're building the
/// standard library, and targeting the specified architecture.
fn cargo_out(&self,
compiler: &Compiler,
compiler: Compiler,
mode: Mode,
target: &str) -> PathBuf {
self.stage_out(compiler, mode).join(target).join(self.cargo_dir())
target: Interned<String>) -> PathBuf {
self.stage_out(compiler, mode).join(&*target).join(self.cargo_dir())
}
/// Root output directory for LLVM compiled for `target`
///
/// Note that if LLVM is configured externally then the directory returned
/// will likely be empty.
fn llvm_out(&self, target: &str) -> PathBuf {
self.out.join(target).join("llvm")
fn llvm_out(&self, target: Interned<String>) -> PathBuf {
self.out.join(&*target).join("llvm")
}
/// Output directory for all documentation for a target
fn doc_out(&self, target: &str) -> PathBuf {
self.out.join(target).join("doc")
fn doc_out(&self, target: Interned<String>) -> PathBuf {
self.out.join(&*target).join("doc")
}
/// Output directory for some generated md crate documentation for a target (temporary)
fn md_doc_out(&self, target: &str) -> PathBuf {
self.out.join(target).join("md-doc")
fn md_doc_out(&self, target: Interned<String>) -> Interned<PathBuf> {
INTERNER.intern_path(self.out.join(&*target).join("md-doc"))
}
/// Output directory for all crate documentation for a target (temporary)
///
/// The artifacts here are then copied into `doc_out` above.
fn crate_doc_out(&self, target: &str) -> PathBuf {
self.out.join(target).join("crate-docs")
fn crate_doc_out(&self, target: Interned<String>) -> PathBuf {
self.out.join(&*target).join("crate-docs")
}
/// Returns true if no custom `llvm-config` is set for the specified target.
///
/// If no custom `llvm-config` was specified then Rust's llvm will be used.
fn is_rust_llvm(&self, target: &str) -> bool {
match self.config.target_config.get(target) {
fn is_rust_llvm(&self, target: Interned<String>) -> bool {
match self.config.target_config.get(&target) {
Some(ref c) => c.llvm_config.is_none(),
None => true
}
@ -719,25 +491,25 @@ impl Build {
///
/// If a custom `llvm-config` was specified for target then that's returned
/// instead.
fn llvm_config(&self, target: &str) -> PathBuf {
let target_config = self.config.target_config.get(target);
fn llvm_config(&self, target: Interned<String>) -> PathBuf {
let target_config = self.config.target_config.get(&target);
if let Some(s) = target_config.and_then(|c| c.llvm_config.as_ref()) {
s.clone()
} else {
self.llvm_out(&self.config.build).join("bin")
.join(exe("llvm-config", target))
self.llvm_out(self.config.build).join("bin")
.join(exe("llvm-config", &*target))
}
}
/// Returns the path to `FileCheck` binary for the specified target
fn llvm_filecheck(&self, target: &str) -> PathBuf {
let target_config = self.config.target_config.get(target);
fn llvm_filecheck(&self, target: Interned<String>) -> PathBuf {
let target_config = self.config.target_config.get(&target);
if let Some(s) = target_config.and_then(|c| c.llvm_config.as_ref()) {
let llvm_bindir = output(Command::new(s).arg("--bindir"));
Path::new(llvm_bindir.trim()).join(exe("FileCheck", target))
Path::new(llvm_bindir.trim()).join(exe("FileCheck", &*target))
} else {
let base = self.llvm_out(&self.config.build).join("build");
let exe = exe("FileCheck", target);
let base = self.llvm_out(self.config.build).join("build");
let exe = exe("FileCheck", &*target);
if !self.config.ninja && self.config.build.contains("msvc") {
base.join("Release/bin").join(exe)
} else {
@ -747,29 +519,16 @@ impl Build {
}
/// Directory for libraries built from C/C++ code and shared between stages.
fn native_dir(&self, target: &str) -> PathBuf {
self.out.join(target).join("native")
fn native_dir(&self, target: Interned<String>) -> PathBuf {
self.out.join(&*target).join("native")
}
/// Root output directory for rust_test_helpers library compiled for
/// `target`
fn test_helpers_out(&self, target: &str) -> PathBuf {
fn test_helpers_out(&self, target: Interned<String>) -> PathBuf {
self.native_dir(target).join("rust-test-helpers")
}
/// Adds the compiler's directory of dynamic libraries to `cmd`'s dynamic
/// library lookup path.
fn add_rustc_lib_path(&self, compiler: &Compiler, cmd: &mut Command) {
// Windows doesn't need dylib path munging because the dlls for the
// compiler live next to the compiler and the system will find them
// automatically.
if cfg!(windows) {
return
}
add_lib_path(vec![self.rustc_libdir(compiler)], cmd);
}
/// Adds the `RUST_TEST_THREADS` env var if necessary
fn add_rust_test_threads(&self, cmd: &mut Command) {
if env::var_os("RUST_TEST_THREADS").is_none() {
@ -777,19 +536,6 @@ impl Build {
}
}
/// Returns the compiler's libdir where it stores the dynamic libraries that
/// it itself links against.
///
/// For example this returns `<sysroot>/lib` on Unix and `<sysroot>/bin` on
/// Windows.
fn rustc_libdir(&self, compiler: &Compiler) -> PathBuf {
if compiler.is_snapshot(self) {
self.rustc_snapshot_libdir()
} else {
self.sysroot(compiler).join(libdir(compiler.host))
}
}
/// Returns the libdir of the snapshot compiler.
fn rustc_snapshot_libdir(&self) -> PathBuf {
self.initial_rustc.parent().unwrap().parent().unwrap()
@ -842,20 +588,20 @@ impl Build {
/// Returns the number of parallel jobs that have been configured for this
/// build.
fn jobs(&self) -> u32 {
self.flags.jobs.unwrap_or_else(|| num_cpus::get() as u32)
self.config.jobs.unwrap_or_else(|| num_cpus::get() as u32)
}
/// Returns the path to the C compiler for the target specified.
fn cc(&self, target: &str) -> &Path {
self.cc[target].0.path()
fn cc(&self, target: Interned<String>) -> &Path {
self.cc[&target].0.path()
}
/// Returns a list of flags to pass to the C compiler for the target
/// specified.
fn cflags(&self, target: &str) -> Vec<String> {
fn cflags(&self, target: Interned<String>) -> Vec<String> {
// Filter out -O and /O (the optimization flags) that we picked up from
// gcc-rs because the build scripts will determine that for themselves.
let mut base = self.cc[target].0.args().iter()
let mut base = self.cc[&target].0.args().iter()
.map(|s| s.to_string_lossy().into_owned())
.filter(|s| !s.starts_with("-O") && !s.starts_with("/O"))
.collect::<Vec<_>>();
@ -871,20 +617,20 @@ impl Build {
// Work around an apparently bad MinGW / GCC optimization,
// See: http://lists.llvm.org/pipermail/cfe-dev/2016-December/051980.html
// See: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=78936
if target == "i686-pc-windows-gnu" {
if &*target == "i686-pc-windows-gnu" {
base.push("-fno-omit-frame-pointer".into());
}
base
}
/// Returns the path to the `ar` archive utility for the target specified.
fn ar(&self, target: &str) -> Option<&Path> {
self.cc[target].1.as_ref().map(|p| &**p)
fn ar(&self, target: Interned<String>) -> Option<&Path> {
self.cc[&target].1.as_ref().map(|p| &**p)
}
/// Returns the path to the C++ compiler for the target specified.
fn cxx(&self, target: &str) -> Result<&Path, String> {
match self.cxx.get(target) {
fn cxx(&self, target: Interned<String>) -> Result<&Path, String> {
match self.cxx.get(&target) {
Some(p) => Ok(p.path()),
None => Err(format!(
"target `{}` is not configured as a host, only as a target",
@ -893,7 +639,7 @@ impl Build {
}
/// Returns flags to pass to the compiler to generate code for `target`.
fn rustc_flags(&self, target: &str) -> Vec<String> {
fn rustc_flags(&self, target: Interned<String>) -> Vec<String> {
// New flags should be added here with great caution!
//
// It's quite unfortunate to **require** flags to generate code for a
@ -909,9 +655,19 @@ impl Build {
base
}
/// Returns if this target should statically link the C runtime, if specified
fn crt_static(&self, target: Interned<String>) -> Option<bool> {
if target.contains("pc-windows-msvc") {
Some(true)
} else {
self.config.target_config.get(&target)
.and_then(|t| t.crt_static)
}
}
/// Returns the "musl root" for this `target`, if defined
fn musl_root(&self, target: &str) -> Option<&Path> {
self.config.target_config.get(target)
fn musl_root(&self, target: Interned<String>) -> Option<&Path> {
self.config.target_config.get(&target)
.and_then(|t| t.musl_root.as_ref())
.or(self.config.musl_root.as_ref())
.map(|p| &**p)
@ -919,8 +675,9 @@ impl Build {
/// Returns whether the target will be tested using the `remote-test-client`
/// and `remote-test-server` binaries.
fn remote_tested(&self, target: &str) -> bool {
self.qemu_rootfs(target).is_some() || target.contains("android")
fn remote_tested(&self, target: Interned<String>) -> bool {
self.qemu_rootfs(target).is_some() || target.contains("android") ||
env::var_os("TEST_DEVICE_ADDR").is_some()
}
/// Returns the root of the "rootfs" image that this target will be using,
@ -928,8 +685,8 @@ impl Build {
///
/// If `Some` is returned then that means that tests for this target are
/// emulated with QEMU and binaries will need to be shipped to the emulator.
fn qemu_rootfs(&self, target: &str) -> Option<&Path> {
self.config.target_config.get(target)
fn qemu_rootfs(&self, target: Interned<String>) -> Option<&Path> {
self.config.target_config.get(&target)
.and_then(|t| t.qemu_rootfs.as_ref())
.map(|p| &**p)
}
@ -957,20 +714,20 @@ impl Build {
///
/// When all of these conditions are met the build will lift artifacts from
/// the previous stage forward.
fn force_use_stage1(&self, compiler: &Compiler, target: &str) -> bool {
fn force_use_stage1(&self, compiler: Compiler, target: Interned<String>) -> bool {
!self.config.full_bootstrap &&
compiler.stage >= 2 &&
self.config.host.iter().any(|h| h == target)
self.hosts.iter().any(|h| *h == target)
}
/// Returns the directory that OpenSSL artifacts are compiled into if
/// configured to do so.
fn openssl_dir(&self, target: &str) -> Option<PathBuf> {
fn openssl_dir(&self, target: Interned<String>) -> Option<PathBuf> {
// OpenSSL not used on Windows
if target.contains("windows") {
None
} else if self.config.openssl_static {
Some(self.out.join(target).join("openssl"))
Some(self.out.join(&*target).join("openssl"))
} else {
None
}
@ -978,7 +735,7 @@ impl Build {
/// Returns the directory that OpenSSL artifacts are installed into if
/// configured as such.
fn openssl_install_dir(&self, target: &str) -> Option<PathBuf> {
fn openssl_install_dir(&self, target: Interned<String>) -> Option<PathBuf> {
self.openssl_dir(target).map(|p| p.join("install"))
}
@ -1077,16 +834,38 @@ impl Build {
None
}
}
/// Get a list of crates from a root crate.
///
/// Returns Vec<(crate, path to crate, is_root_crate)>
fn crates(&self, root: &str) -> Vec<(Interned<String>, &Path)> {
let interned = INTERNER.intern_string(root.to_owned());
let mut ret = Vec::new();
let mut list = vec![interned];
let mut visited = HashSet::new();
while let Some(krate) = list.pop() {
let krate = &self.crates[&krate];
// If we can't strip prefix, then out-of-tree path
let path = krate.path.strip_prefix(&self.src).unwrap_or(&krate.path);
ret.push((krate.name, path));
for dep in &krate.deps {
if visited.insert(dep) && dep != "build_helper" {
list.push(*dep);
}
}
}
ret
}
}
impl<'a> Compiler<'a> {
/// Creates a new complier for the specified stage/host
fn new(stage: u32, host: &'a str) -> Compiler<'a> {
Compiler { stage: stage, host: host }
impl<'a> Compiler {
pub fn with_stage(mut self, stage: u32) -> Compiler {
self.stage = stage;
self
}
/// Returns whether this is a snapshot compiler for `build`'s configuration
fn is_snapshot(&self, build: &Build) -> bool {
pub fn is_snapshot(&self, build: &Build) -> bool {
self.stage == 0 && self.host == build.build
}
@ -1094,7 +873,7 @@ impl<'a> Compiler<'a> {
/// current build session.
/// This takes into account whether we're performing a full bootstrap or
/// not; don't directly compare the stage with `2`!
fn is_final_stage(&self, build: &Build) -> bool {
pub fn is_final_stage(&self, build: &Build) -> bool {
let final_stage = if build.config.full_bootstrap { 2 } else { 1 };
self.stage >= final_stage
}

32
src/bootstrap/metadata.rs
View File

@ -13,17 +13,18 @@ use std::process::Command;
use std::path::PathBuf;
use build_helper::output;
use rustc_serialize::json;
use serde_json;
use {Build, Crate};
use cache::INTERNER;
#[derive(RustcDecodable)]
#[derive(Deserialize)]
struct Output {
packages: Vec<Package>,
resolve: Resolve,
}
#[derive(RustcDecodable)]
#[derive(Deserialize)]
struct Package {
id: String,
name: String,
@ -32,12 +33,12 @@ struct Package {
manifest_path: String,
}
#[derive(RustcDecodable)]
#[derive(Deserialize)]
struct Resolve {
nodes: Vec<ResolveNode>,
}
#[derive(RustcDecodable)]
#[derive(Deserialize)]
struct ResolveNode {
id: String,
dependencies: Vec<String>,
@ -61,22 +62,23 @@ fn build_krate(build: &mut Build, krate: &str) {
.arg("--format-version").arg("1")
.arg("--manifest-path").arg(build.src.join(krate).join("Cargo.toml"));
let output = output(&mut cargo);
let output: Output = json::decode(&output).unwrap();
let output: Output = serde_json::from_str(&output).unwrap();
let mut id2name = HashMap::new();
for package in output.packages {
if package.source.is_none() {
id2name.insert(package.id, package.name.clone());
let name = INTERNER.intern_string(package.name);
id2name.insert(package.id, name);
let mut path = PathBuf::from(package.manifest_path);
path.pop();
build.crates.insert(package.name.clone(), Crate {
build_step: format!("build-crate-{}", package.name),
doc_step: format!("doc-crate-{}", package.name),
test_step: format!("test-crate-{}", package.name),
bench_step: format!("bench-crate-{}", package.name),
name: package.name,
build.crates.insert(name, Crate {
build_step: format!("build-crate-{}", name),
doc_step: format!("doc-crate-{}", name),
test_step: format!("test-crate-{}", name),
bench_step: format!("bench-crate-{}", name),
name,
version: package.version,
deps: Vec::new(),
path: path,
path,
});
}
}
@ -93,7 +95,7 @@ fn build_krate(build: &mut Build, krate: &str) {
Some(dep) => dep,
None => continue,
};
krate.deps.push(dep.clone());
krate.deps.push(*dep);
}
}
}

3
src/bootstrap/mk/Makefile.in
View File

@ -56,6 +56,7 @@ check-aux:
$(Q)$(BOOTSTRAP) test \
src/tools/cargotest \
src/tools/cargo \
src/tools/rls \
src/test/pretty \
src/test/run-pass/pretty \
src/test/run-fail/pretty \
@ -63,6 +64,8 @@ check-aux:
src/test/run-pass-fulldeps/pretty \
src/test/run-fail-fulldeps/pretty \
$(BOOTSTRAP_ARGS)
check-bootstrap:
$(Q)$(CFG_PYTHON) $(CFG_SRC_DIR)src/bootstrap/bootstrap_test.py
dist:
$(Q)$(BOOTSTRAP) dist $(BOOTSTRAP_ARGS)
distcheck:

684
src/bootstrap/native.rs
View File

@ -11,7 +11,7 @@
//! Compilation of native dependencies like LLVM.
//!
//! Native projects like LLVM unfortunately aren't suited just yet for
//! compilation in build scripts that Cargo has. This is because thie
//! compilation in build scripts that Cargo has. This is because the
//! compilation takes a *very* long time but also because we don't want to
//! compile LLVM 3 times as part of a normal bootstrap (we want it cached).
//!
@ -32,174 +32,217 @@ use gcc;
use Build;
use util;
use build_helper::up_to_date;
use builder::{Builder, RunConfig, ShouldRun, Step};
use cache::Interned;
/// Compile LLVM for `target`.
pub fn llvm(build: &Build, target: &str) {
// If we're using a custom LLVM bail out here, but we can only use a
// custom LLVM for the build triple.
if let Some(config) = build.config.target_config.get(target) {
if let Some(ref s) = config.llvm_config {
return check_llvm_version(build, s);
#[derive(Debug, Copy, Clone, Hash, PartialEq, Eq)]
pub struct Llvm {
pub target: Interned<String>,
}
impl Step for Llvm {
type Output = ();
const ONLY_HOSTS: bool = true;
fn should_run(run: ShouldRun) -> ShouldRun {
run.path("src/llvm")
}
fn make_run(run: RunConfig) {
run.builder.ensure(Llvm { target: run.target })
}
/// Compile LLVM for `target`.
fn run(self, builder: &Builder) {
let build = builder.build;
let target = self.target;
// If we're not compiling for LLVM bail out here.
if !build.config.llvm_enabled {
return;
}
}
let rebuild_trigger = build.src.join("src/rustllvm/llvm-rebuild-trigger");
let mut rebuild_trigger_contents = String::new();
t!(t!(File::open(&rebuild_trigger)).read_to_string(&mut rebuild_trigger_contents));
let out_dir = build.llvm_out(target);
let done_stamp = out_dir.join("llvm-finished-building");
if done_stamp.exists() {
let mut done_contents = String::new();
t!(t!(File::open(&done_stamp)).read_to_string(&mut done_contents));
// If LLVM was already built previously and contents of the rebuild-trigger file
// didn't change from the previous build, then no action is required.
if done_contents == rebuild_trigger_contents {
return
// If we're using a custom LLVM bail out here, but we can only use a
// custom LLVM for the build triple.
if let Some(config) = build.config.target_config.get(&target) {
if let Some(ref s) = config.llvm_config {
return check_llvm_version(build, s);
}
}
}
if build.config.llvm_clean_rebuild {
drop(fs::remove_dir_all(&out_dir));
}
let _folder = build.fold_output(|| "llvm");
println!("Building LLVM for {}", target);
let _time = util::timeit();
t!(fs::create_dir_all(&out_dir));
let rebuild_trigger = build.src.join("src/rustllvm/llvm-rebuild-trigger");
let mut rebuild_trigger_contents = String::new();
t!(t!(File::open(&rebuild_trigger)).read_to_string(&mut rebuild_trigger_contents));
// http://llvm.org/docs/CMake.html
let mut cfg = cmake::Config::new(build.src.join("src/llvm"));
if build.config.ninja {
cfg.generator("Ninja");
}
let out_dir = build.llvm_out(target);
let done_stamp = out_dir.join("llvm-finished-building");
if done_stamp.exists() {
let mut done_contents = String::new();
t!(t!(File::open(&done_stamp)).read_to_string(&mut done_contents));
let profile = match (build.config.llvm_optimize, build.config.llvm_release_debuginfo) {
(false, _) => "Debug",
(true, false) => "Release",
(true, true) => "RelWithDebInfo",
};
// NOTE: remember to also update `config.toml.example` when changing the defaults!
let llvm_targets = match build.config.llvm_targets {
Some(ref s) => s,
None => "X86;ARM;AArch64;Mips;PowerPC;SystemZ;JSBackend;MSP430;Sparc;NVPTX;Hexagon",
};
let llvm_exp_targets = match build.config.llvm_experimental_targets {
Some(ref s) => s,
None => "",
};
let assertions = if build.config.llvm_assertions {"ON"} else {"OFF"};
cfg.target(target)
.host(&build.build)
.out_dir(&out_dir)
.profile(profile)
.define("LLVM_ENABLE_ASSERTIONS", assertions)
.define("LLVM_TARGETS_TO_BUILD", llvm_targets)
.define("LLVM_EXPERIMENTAL_TARGETS_TO_BUILD", llvm_exp_targets)
.define("LLVM_INCLUDE_EXAMPLES", "OFF")
.define("LLVM_INCLUDE_TESTS", "OFF")
.define("LLVM_INCLUDE_DOCS", "OFF")
.define("LLVM_ENABLE_ZLIB", "OFF")
.define("WITH_POLLY", "OFF")
.define("LLVM_ENABLE_TERMINFO", "OFF")
.define("LLVM_ENABLE_LIBEDIT", "OFF")
.define("LLVM_PARALLEL_COMPILE_JOBS", build.jobs().to_string())
.define("LLVM_TARGET_ARCH", target.split('-').next().unwrap())
.define("LLVM_DEFAULT_TARGET_TRIPLE", target);
if target.contains("msvc") {
cfg.define("LLVM_USE_CRT_DEBUG", "MT");
cfg.define("LLVM_USE_CRT_RELEASE", "MT");
cfg.define("LLVM_USE_CRT_RELWITHDEBINFO", "MT");
cfg.static_crt(true);
}
if target.starts_with("i686") {
cfg.define("LLVM_BUILD_32_BITS", "ON");
}
if let Some(num_linkers) = build.config.llvm_link_jobs {
if num_linkers > 0 {
cfg.define("LLVM_PARALLEL_LINK_JOBS", num_linkers.to_string());
// If LLVM was already built previously and contents of the rebuild-trigger file
// didn't change from the previous build, then no action is required.
if done_contents == rebuild_trigger_contents {
return
}
}
}
// http://llvm.org/docs/HowToCrossCompileLLVM.html
if target != build.build {
// FIXME: if the llvm root for the build triple is overridden then we
// should use llvm-tblgen from there, also should verify that it
// actually exists most of the time in normal installs of LLVM.
let host = build.llvm_out(&build.build).join("bin/llvm-tblgen");
cfg.define("CMAKE_CROSSCOMPILING", "True")
.define("LLVM_TABLEGEN", &host);
}
let _folder = build.fold_output(|| "llvm");
println!("Building LLVM for {}", target);
let _time = util::timeit();
t!(fs::create_dir_all(&out_dir));
// http://llvm.org/docs/CMake.html
let mut cfg = cmake::Config::new(build.src.join("src/llvm"));
if build.config.ninja {
cfg.generator("Ninja");
}
let profile = match (build.config.llvm_optimize, build.config.llvm_release_debuginfo) {
(false, _) => "Debug",
(true, false) => "Release",
(true, true) => "RelWithDebInfo",
};
// NOTE: remember to also update `config.toml.example` when changing the defaults!
let llvm_targets = match build.config.llvm_targets {
Some(ref s) => s,
None => "X86;ARM;AArch64;Mips;PowerPC;SystemZ;JSBackend;MSP430;Sparc;NVPTX;Hexagon",
};
let llvm_exp_targets = match build.config.llvm_experimental_targets {
Some(ref s) => s,
None => "",
};
let assertions = if build.config.llvm_assertions {"ON"} else {"OFF"};
cfg.target(&target)
.host(&build.build)
.out_dir(&out_dir)
.profile(profile)
.define("LLVM_ENABLE_ASSERTIONS", assertions)
.define("LLVM_TARGETS_TO_BUILD", llvm_targets)
.define("LLVM_EXPERIMENTAL_TARGETS_TO_BUILD", llvm_exp_targets)
.define("LLVM_INCLUDE_EXAMPLES", "OFF")
.define("LLVM_INCLUDE_TESTS", "OFF")
.define("LLVM_INCLUDE_DOCS", "OFF")
.define("LLVM_ENABLE_ZLIB", "OFF")
.define("WITH_POLLY", "OFF")
.define("LLVM_ENABLE_TERMINFO", "OFF")
.define("LLVM_ENABLE_LIBEDIT", "OFF")
.define("LLVM_PARALLEL_COMPILE_JOBS", build.jobs().to_string())
.define("LLVM_TARGET_ARCH", target.split('-').next().unwrap())
.define("LLVM_DEFAULT_TARGET_TRIPLE", target);
// This setting makes the LLVM tools link to the dynamic LLVM library,
// which saves both memory during parallel links and overall disk space
// for the tools. We don't distribute any of those tools, so this is
// just a local concern. However, it doesn't work well everywhere.
if target.contains("linux-gnu") || target.contains("apple-darwin") {
cfg.define("LLVM_LINK_LLVM_DYLIB", "ON");
}
let sanitize_cc = |cc: &Path| {
if target.contains("msvc") {
OsString::from(cc.to_str().unwrap().replace("\\", "/"))
} else {
cc.as_os_str().to_owned()
}
};
let configure_compilers = |cfg: &mut cmake::Config| {
// MSVC with CMake uses msbuild by default which doesn't respect these
// vars that we'd otherwise configure. In that case we just skip this
// entirely.
if target.contains("msvc") && !build.config.ninja {
return
cfg.define("LLVM_USE_CRT_DEBUG", "MT");
cfg.define("LLVM_USE_CRT_RELEASE", "MT");
cfg.define("LLVM_USE_CRT_RELWITHDEBINFO", "MT");
cfg.static_crt(true);
}
let cc = build.cc(target);
let cxx = build.cxx(target).unwrap();
// Handle msvc + ninja + ccache specially (this is what the bots use)
if target.contains("msvc") &&
build.config.ninja &&
build.config.ccache.is_some() {
let mut cc = env::current_exe().expect("failed to get cwd");
cc.set_file_name("sccache-plus-cl.exe");
cfg.define("CMAKE_C_COMPILER", sanitize_cc(&cc))
.define("CMAKE_CXX_COMPILER", sanitize_cc(&cc));
cfg.env("SCCACHE_PATH",
build.config.ccache.as_ref().unwrap())
.env("SCCACHE_TARGET", target);
// If ccache is configured we inform the build a little differently hwo
// to invoke ccache while also invoking our compilers.
} else if let Some(ref ccache) = build.config.ccache {
cfg.define("CMAKE_C_COMPILER", ccache)
.define("CMAKE_C_COMPILER_ARG1", sanitize_cc(cc))
.define("CMAKE_CXX_COMPILER", ccache)
.define("CMAKE_CXX_COMPILER_ARG1", sanitize_cc(cxx));
} else {
cfg.define("CMAKE_C_COMPILER", sanitize_cc(cc))
.define("CMAKE_CXX_COMPILER", sanitize_cc(cxx));
if target.starts_with("i686") {
cfg.define("LLVM_BUILD_32_BITS", "ON");
}
cfg.build_arg("-j").build_arg(build.jobs().to_string());
cfg.define("CMAKE_C_FLAGS", build.cflags(target).join(" "));
cfg.define("CMAKE_CXX_FLAGS", build.cflags(target).join(" "));
};
if let Some(num_linkers) = build.config.llvm_link_jobs {
if num_linkers > 0 {
cfg.define("LLVM_PARALLEL_LINK_JOBS", num_linkers.to_string());
}
}
configure_compilers(&mut cfg);
// http://llvm.org/docs/HowToCrossCompileLLVM.html
if target != build.build {
builder.ensure(Llvm { target: build.build });
// FIXME: if the llvm root for the build triple is overridden then we
// should use llvm-tblgen from there, also should verify that it
// actually exists most of the time in normal installs of LLVM.
let host = build.llvm_out(build.build).join("bin/llvm-tblgen");
cfg.define("CMAKE_CROSSCOMPILING", "True")
.define("LLVM_TABLEGEN", &host);
if env::var_os("SCCACHE_ERROR_LOG").is_some() {
cfg.env("RUST_LOG", "sccache=warn");
if target.contains("netbsd") {
cfg.define("CMAKE_SYSTEM_NAME", "NetBSD");
} else if target.contains("freebsd") {
cfg.define("CMAKE_SYSTEM_NAME", "FreeBSD");
}
cfg.define("LLVM_NATIVE_BUILD", build.llvm_out(build.build).join("build"));
}
let sanitize_cc = |cc: &Path| {
if target.contains("msvc") {
OsString::from(cc.to_str().unwrap().replace("\\", "/"))
} else {
cc.as_os_str().to_owned()
}
};
let configure_compilers = |cfg: &mut cmake::Config| {
// MSVC with CMake uses msbuild by default which doesn't respect these
// vars that we'd otherwise configure. In that case we just skip this
// entirely.
if target.contains("msvc") && !build.config.ninja {
return
}
let cc = build.cc(target);
let cxx = build.cxx(target).unwrap();
// Handle msvc + ninja + ccache specially (this is what the bots use)
if target.contains("msvc") &&
build.config.ninja &&
build.config.ccache.is_some() {
let mut cc = env::current_exe().expect("failed to get cwd");
cc.set_file_name("sccache-plus-cl.exe");
cfg.define("CMAKE_C_COMPILER", sanitize_cc(&cc))
.define("CMAKE_CXX_COMPILER", sanitize_cc(&cc));
cfg.env("SCCACHE_PATH",
build.config.ccache.as_ref().unwrap())
.env("SCCACHE_TARGET", target);
// If ccache is configured we inform the build a little differently hwo
// to invoke ccache while also invoking our compilers.
} else if let Some(ref ccache) = build.config.ccache {
cfg.define("CMAKE_C_COMPILER", ccache)
.define("CMAKE_C_COMPILER_ARG1", sanitize_cc(cc))
.define("CMAKE_CXX_COMPILER", ccache)
.define("CMAKE_CXX_COMPILER_ARG1", sanitize_cc(cxx));
} else {
cfg.define("CMAKE_C_COMPILER", sanitize_cc(cc))
.define("CMAKE_CXX_COMPILER", sanitize_cc(cxx));
}
cfg.build_arg("-j").build_arg(build.jobs().to_string());
cfg.define("CMAKE_C_FLAGS", build.cflags(target).join(" "));
cfg.define("CMAKE_CXX_FLAGS", build.cflags(target).join(" "));
};
configure_compilers(&mut cfg);
if env::var_os("SCCACHE_ERROR_LOG").is_some() {
cfg.env("RUST_LOG", "sccache=warn");
}
// FIXME: we don't actually need to build all LLVM tools and all LLVM
// libraries here, e.g. we just want a few components and a few
// tools. Figure out how to filter them down and only build the right
// tools and libs on all platforms.
cfg.build();
t!(t!(File::create(&done_stamp)).write_all(rebuild_trigger_contents.as_bytes()));
}
// FIXME: we don't actually need to build all LLVM tools and all LLVM
// libraries here, e.g. we just want a few components and a few
// tools. Figure out how to filter them down and only build the right
// tools and libs on all platforms.
cfg.build();
t!(t!(File::create(&done_stamp)).write_all(rebuild_trigger_contents.as_bytes()));
}
fn check_llvm_version(build: &Build, llvm_config: &Path) {
@ -216,161 +259,196 @@ fn check_llvm_version(build: &Build, llvm_config: &Path) {
panic!("\n\nbad LLVM version: {}, need >=3.5\n\n", version)
}
/// Compiles the `rust_test_helpers.c` library which we used in various
/// `run-pass` test suites for ABI testing.
pub fn test_helpers(build: &Build, target: &str) {
let dst = build.test_helpers_out(target);
let src = build.src.join("src/rt/rust_test_helpers.c");
if up_to_date(&src, &dst.join("librust_test_helpers.a")) {
return
}
let _folder = build.fold_output(|| "build_test_helpers");
println!("Building test helpers");
t!(fs::create_dir_all(&dst));
let mut cfg = gcc::Config::new();
// We may have found various cross-compilers a little differently due to our
// extra configuration, so inform gcc of these compilers. Note, though, that
// on MSVC we still need gcc's detection of env vars (ugh).
if !target.contains("msvc") {
if let Some(ar) = build.ar(target) {
cfg.archiver(ar);
}
cfg.compiler(build.cc(target));
}
cfg.cargo_metadata(false)
.out_dir(&dst)
.target(target)
.host(&build.build)
.opt_level(0)
.debug(false)
.file(build.src.join("src/rt/rust_test_helpers.c"))
.compile("librust_test_helpers.a");
#[derive(Debug, Copy, Clone, PartialEq, Eq, Hash)]
pub struct TestHelpers {
pub target: Interned<String>,
}
impl Step for TestHelpers {
type Output = ();
fn should_run(run: ShouldRun) -> ShouldRun {
run.path("src/rt/rust_test_helpers.c")
}
fn make_run(run: RunConfig) {
run.builder.ensure(TestHelpers { target: run.target })
}
/// Compiles the `rust_test_helpers.c` library which we used in various
/// `run-pass` test suites for ABI testing.
fn run(self, builder: &Builder) {
let build = builder.build;
let target = self.target;
let dst = build.test_helpers_out(target);
let src = build.src.join("src/rt/rust_test_helpers.c");
if up_to_date(&src, &dst.join("librust_test_helpers.a")) {
return
}
let _folder = build.fold_output(|| "build_test_helpers");
println!("Building test helpers");
t!(fs::create_dir_all(&dst));
let mut cfg = gcc::Config::new();
// We may have found various cross-compilers a little differently due to our
// extra configuration, so inform gcc of these compilers. Note, though, that
// on MSVC we still need gcc's detection of env vars (ugh).
if !target.contains("msvc") {
if let Some(ar) = build.ar(target) {
cfg.archiver(ar);
}
cfg.compiler(build.cc(target));
}
cfg.cargo_metadata(false)
.out_dir(&dst)
.target(&target)
.host(&build.build)
.opt_level(0)
.debug(false)
.file(build.src.join("src/rt/rust_test_helpers.c"))
.compile("librust_test_helpers.a");
}
}
const OPENSSL_VERS: &'static str = "1.0.2k";
const OPENSSL_SHA256: &'static str =
"6b3977c61f2aedf0f96367dcfb5c6e578cf37e7b8d913b4ecb6643c3cb88d8c0";
pub fn openssl(build: &Build, target: &str) {
let out = match build.openssl_dir(target) {
Some(dir) => dir,
None => return,
};
let stamp = out.join(".stamp");
let mut contents = String::new();
drop(File::open(&stamp).and_then(|mut f| f.read_to_string(&mut contents)));
if contents == OPENSSL_VERS {
return
}
t!(fs::create_dir_all(&out));
let name = format!("openssl-{}.tar.gz", OPENSSL_VERS);
let tarball = out.join(&name);
if !tarball.exists() {
let tmp = tarball.with_extension("tmp");
// originally from https://www.openssl.org/source/...
let url = format!("https://s3.amazonaws.com/rust-lang-ci/rust-ci-mirror/{}",
name);
let mut ok = false;
for _ in 0..3 {
let status = Command::new("curl")
.arg("-o").arg(&tmp)
.arg(&url)
.status()
.expect("failed to spawn curl");
if status.success() {
ok = true;
break
}
}
if !ok {
panic!("failed to download openssl source")
}
let mut shasum = if target.contains("apple") {
let mut cmd = Command::new("shasum");
cmd.arg("-a").arg("256");
cmd
} else {
Command::new("sha256sum")
};
let output = output(&mut shasum.arg(&tmp));
let found = output.split_whitespace().next().unwrap();
if found != OPENSSL_SHA256 {
panic!("downloaded openssl sha256 different\n\
expected: {}\n\
found: {}\n", OPENSSL_SHA256, found);
}
t!(fs::rename(&tmp, &tarball));
}
let obj = out.join(format!("openssl-{}", OPENSSL_VERS));
let dst = build.openssl_install_dir(target).unwrap();
drop(fs::remove_dir_all(&obj));
drop(fs::remove_dir_all(&dst));
build.run(Command::new("tar").arg("xf").arg(&tarball).current_dir(&out));
let mut configure = Command::new(obj.join("Configure"));
configure.arg(format!("--prefix={}", dst.display()));
configure.arg("no-dso");
configure.arg("no-ssl2");
configure.arg("no-ssl3");
let os = match target {
"aarch64-linux-android" => "linux-aarch64",
"aarch64-unknown-linux-gnu" => "linux-aarch64",
"arm-linux-androideabi" => "android",
"arm-unknown-linux-gnueabi" => "linux-armv4",
"arm-unknown-linux-gnueabihf" => "linux-armv4",
"armv7-linux-androideabi" => "android-armv7",
"armv7-unknown-linux-gnueabihf" => "linux-armv4",
"i686-apple-darwin" => "darwin-i386-cc",
"i686-linux-android" => "android-x86",
"i686-unknown-freebsd" => "BSD-x86-elf",
"i686-unknown-linux-gnu" => "linux-elf",
"i686-unknown-linux-musl" => "linux-elf",
"mips-unknown-linux-gnu" => "linux-mips32",
"mips64-unknown-linux-gnuabi64" => "linux64-mips64",
"mips64el-unknown-linux-gnuabi64" => "linux64-mips64",
"mipsel-unknown-linux-gnu" => "linux-mips32",
"powerpc-unknown-linux-gnu" => "linux-ppc",
"powerpc64-unknown-linux-gnu" => "linux-ppc64",
"powerpc64le-unknown-linux-gnu" => "linux-ppc64le",
"s390x-unknown-linux-gnu" => "linux64-s390x",
"x86_64-apple-darwin" => "darwin64-x86_64-cc",
"x86_64-linux-android" => "linux-x86_64",
"x86_64-unknown-freebsd" => "BSD-x86_64",
"x86_64-unknown-linux-gnu" => "linux-x86_64",
"x86_64-unknown-linux-musl" => "linux-x86_64",
"x86_64-unknown-netbsd" => "BSD-x86_64",
_ => panic!("don't know how to configure OpenSSL for {}", target),
};
configure.arg(os);
configure.env("CC", build.cc(target));
for flag in build.cflags(target) {
configure.arg(flag);
}
// There is no specific os target for android aarch64 or x86_64,
// so we need to pass some extra cflags
if target == "aarch64-linux-android" || target == "x86_64-linux-android" {
configure.arg("-mandroid");
configure.arg("-fomit-frame-pointer");
}
// Make PIE binaries
// Non-PIE linker support was removed in Lollipop
// https://source.android.com/security/enhancements/enhancements50
if target == "i686-linux-android" {
configure.arg("no-asm");
}
configure.current_dir(&obj);
println!("Configuring openssl for {}", target);
build.run_quiet(&mut configure);
println!("Building openssl for {}", target);
build.run_quiet(Command::new("make").arg("-j1").current_dir(&obj));
println!("Installing openssl for {}", target);
build.run_quiet(Command::new("make").arg("install").current_dir(&obj));
let mut f = t!(File::create(&stamp));
t!(f.write_all(OPENSSL_VERS.as_bytes()));
#[derive(Debug, Copy, Clone, Hash, PartialEq, Eq)]
pub struct Openssl {
pub target: Interned<String>,
}
impl Step for Openssl {
type Output = ();
fn should_run(run: ShouldRun) -> ShouldRun {
run.never()
}
fn run(self, builder: &Builder) {
let build = builder.build;
let target = self.target;
let out = match build.openssl_dir(target) {
Some(dir) => dir,
None => return,
};
let stamp = out.join(".stamp");
let mut contents = String::new();
drop(File::open(&stamp).and_then(|mut f| f.read_to_string(&mut contents)));
if contents == OPENSSL_VERS {
return
}
t!(fs::create_dir_all(&out));
let name = format!("openssl-{}.tar.gz", OPENSSL_VERS);
let tarball = out.join(&name);
if !tarball.exists() {
let tmp = tarball.with_extension("tmp");
// originally from https://www.openssl.org/source/...
let url = format!("https://s3-us-west-1.amazonaws.com/rust-lang-ci2/rust-ci-mirror/{}",
name);
let mut ok = false;
for _ in 0..3 {
let status = Command::new("curl")
.arg("-o").arg(&tmp)
.arg(&url)
.status()
.expect("failed to spawn curl");
if status.success() {
ok = true;
break
}
}
if !ok {
panic!("failed to download openssl source")
}
let mut shasum = if target.contains("apple") {
let mut cmd = Command::new("shasum");
cmd.arg("-a").arg("256");
cmd
} else {
Command::new("sha256sum")
};
let output = output(&mut shasum.arg(&tmp));
let found = output.split_whitespace().next().unwrap();
if found != OPENSSL_SHA256 {
panic!("downloaded openssl sha256 different\n\
expected: {}\n\
found: {}\n", OPENSSL_SHA256, found);
}
t!(fs::rename(&tmp, &tarball));
}
let obj = out.join(format!("openssl-{}", OPENSSL_VERS));
let dst = build.openssl_install_dir(target).unwrap();
drop(fs::remove_dir_all(&obj));
drop(fs::remove_dir_all(&dst));
build.run(Command::new("tar").arg("xf").arg(&tarball).current_dir(&out));
let mut configure = Command::new(obj.join("Configure"));
configure.arg(format!("--prefix={}", dst.display()));
configure.arg("no-dso");
configure.arg("no-ssl2");
configure.arg("no-ssl3");
let os = match &*target {
"aarch64-linux-android" => "linux-aarch64",
"aarch64-unknown-linux-gnu" => "linux-aarch64",
"arm-linux-androideabi" => "android",
"arm-unknown-linux-gnueabi" => "linux-armv4",
"arm-unknown-linux-gnueabihf" => "linux-armv4",
"armv7-linux-androideabi" => "android-armv7",
"armv7-unknown-linux-gnueabihf" => "linux-armv4",
"i686-apple-darwin" => "darwin-i386-cc",
"i686-linux-android" => "android-x86",
"i686-unknown-freebsd" => "BSD-x86-elf",
"i686-unknown-linux-gnu" => "linux-elf",
"i686-unknown-linux-musl" => "linux-elf",
"mips-unknown-linux-gnu" => "linux-mips32",
"mips64-unknown-linux-gnuabi64" => "linux64-mips64",
"mips64el-unknown-linux-gnuabi64" => "linux64-mips64",
"mipsel-unknown-linux-gnu" => "linux-mips32",
"powerpc-unknown-linux-gnu" => "linux-ppc",
"powerpc64-unknown-linux-gnu" => "linux-ppc64",
"powerpc64le-unknown-linux-gnu" => "linux-ppc64le",
"s390x-unknown-linux-gnu" => "linux64-s390x",
"x86_64-apple-darwin" => "darwin64-x86_64-cc",
"x86_64-linux-android" => "linux-x86_64",
"x86_64-unknown-freebsd" => "BSD-x86_64",
"x86_64-unknown-linux-gnu" => "linux-x86_64",
"x86_64-unknown-linux-musl" => "linux-x86_64",
"x86_64-unknown-netbsd" => "BSD-x86_64",
_ => panic!("don't know how to configure OpenSSL for {}", target),
};
configure.arg(os);
configure.env("CC", build.cc(target));
for flag in build.cflags(target) {
configure.arg(flag);
}
// There is no specific os target for android aarch64 or x86_64,
// so we need to pass some extra cflags
if target == "aarch64-linux-android" || target == "x86_64-linux-android" {
configure.arg("-mandroid");
configure.arg("-fomit-frame-pointer");
}
// Make PIE binaries
// Non-PIE linker support was removed in Lollipop
// https://source.android.com/security/enhancements/enhancements50
if target == "i686-linux-android" {
configure.arg("no-asm");
}
configure.current_dir(&obj);
println!("Configuring openssl for {}", target);
build.run_quiet(&mut configure);
println!("Building openssl for {}", target);
build.run_quiet(Command::new("make").arg("-j1").current_dir(&obj));
println!("Installing openssl for {}", target);
build.run_quiet(Command::new("make").arg("install").current_dir(&obj));
let mut f = t!(File::create(&stamp));
t!(f.write_all(OPENSSL_VERS.as_bytes()));
}
}

52
src/bootstrap/sanity.rs
View File

@ -85,7 +85,7 @@ pub fn check(build: &mut Build) {
}
// We need cmake, but only if we're actually building LLVM or sanitizers.
let building_llvm = build.config.host.iter()
let building_llvm = build.hosts.iter()
.filter_map(|host| build.config.target_config.get(host))
.any(|config| config.llvm_config.is_none());
if building_llvm || build.config.sanitizers {
@ -93,10 +93,27 @@ pub fn check(build: &mut Build) {
}
// Ninja is currently only used for LLVM itself.
// Some Linux distros rename `ninja` to `ninja-build`.
// CMake can work with either binary name.
if building_llvm && build.config.ninja && cmd_finder.maybe_have("ninja-build").is_none() {
cmd_finder.must_have("ninja");
if building_llvm {
if build.config.ninja {
// Some Linux distros rename `ninja` to `ninja-build`.
// CMake can work with either binary name.
if cmd_finder.maybe_have("ninja-build").is_none() {
cmd_finder.must_have("ninja");
}
}
// If ninja isn't enabled but we're building for MSVC then we try
// doubly hard to enable it. It was realized in #43767 that the msbuild
// CMake generator for MSVC doesn't respect configuration options like
// disabling LLVM assertions, which can often be quite important!
//
// In these cases we automatically enable Ninja if we find it in the
// environment.
if !build.config.ninja && build.config.build.contains("msvc") {
if cmd_finder.maybe_have("ninja").is_some() {
build.config.ninja = true;
}
}
}
build.config.python = build.config.python.take().map(|p| cmd_finder.must_have(p))
@ -114,7 +131,7 @@ pub fn check(build: &mut Build) {
// We're gonna build some custom C code here and there, host triples
// also build some C++ shims for LLVM so we need a C++ compiler.
for target in &build.config.target {
for target in &build.targets {
// On emscripten we don't actually need the C compiler to just
// build the target artifacts, only for testing. For the sake
// of easier bot configuration, just skip detection.
@ -122,14 +139,14 @@ pub fn check(build: &mut Build) {
continue;
}
cmd_finder.must_have(build.cc(target));
if let Some(ar) = build.ar(target) {
cmd_finder.must_have(build.cc(*target));
if let Some(ar) = build.ar(*target) {
cmd_finder.must_have(ar);
}
}
for host in build.config.host.iter() {
cmd_finder.must_have(build.cxx(host).unwrap());
for host in &build.hosts {
cmd_finder.must_have(build.cxx(*host).unwrap());
// The msvc hosts don't use jemalloc, turn it off globally to
// avoid packaging the dummy liballoc_jemalloc on that platform.
@ -139,21 +156,28 @@ pub fn check(build: &mut Build) {
}
// Externally configured LLVM requires FileCheck to exist
let filecheck = build.llvm_filecheck(&build.build);
let filecheck = build.llvm_filecheck(build.build);
if !filecheck.starts_with(&build.out) && !filecheck.exists() && build.config.codegen_tests {
panic!("FileCheck executable {:?} does not exist", filecheck);
}
for target in &build.config.target {
for target in &build.targets {
// Can't compile for iOS unless we're on macOS
if target.contains("apple-ios") &&
!build.build.contains("apple-darwin") {
panic!("the iOS target is only supported on macOS");
}
// Make sure musl-root is valid if specified
// Make sure musl-root is valid
if target.contains("musl") && !target.contains("mips") {
match build.musl_root(target) {
// If this is a native target (host is also musl) and no musl-root is given,
// fall back to the system toolchain in /usr before giving up
if build.musl_root(*target).is_none() && build.config.build == *target {
let target = build.config.target_config.entry(target.clone())
.or_insert(Default::default());
target.musl_root = Some("/usr".into());
}
match build.musl_root(*target) {
Some(root) => {
if fs::metadata(root.join("lib/libc.a")).is_err() {
panic!("couldn't find libc.a in musl dir: {}",

1817
src/bootstrap/step.rs
View File

File diff suppressed because it is too large Load Diff

425
src/bootstrap/tool.rs Normal file
View File

@ -0,0 +1,425 @@
// Copyright 2017 The Rust Project Developers. See the COPYRIGHT
// file at the top-level directory of this distribution and at
// http://rust-lang.org/COPYRIGHT.
//
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
// http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
// option. This file may not be copied, modified, or distributed
// except according to those terms.
use std::fs;
use std::env;
use std::path::PathBuf;
use std::process::Command;
use Mode;
use Compiler;
use builder::{Step, RunConfig, ShouldRun, Builder};
use util::{copy, exe, add_lib_path};
use compile::{self, libtest_stamp, libstd_stamp, librustc_stamp};
use native;
use channel::GitInfo;
use cache::Interned;
#[derive(Debug, Copy, Clone, Hash, PartialEq, Eq)]
pub struct CleanTools {
pub compiler: Compiler,
pub target: Interned<String>,
pub mode: Mode,
}
impl Step for CleanTools {
type Output = ();
fn should_run(run: ShouldRun) -> ShouldRun {
run.never()
}
/// Build a tool in `src/tools`
///
/// This will build the specified tool with the specified `host` compiler in
/// `stage` into the normal cargo output directory.
fn run(self, builder: &Builder) {
let build = builder.build;
let compiler = self.compiler;
let target = self.target;
let mode = self.mode;
let stamp = match mode {
Mode::Libstd => libstd_stamp(build, compiler, target),
Mode::Libtest => libtest_stamp(build, compiler, target),
Mode::Librustc => librustc_stamp(build, compiler, target),
_ => panic!(),
};
let out_dir = build.cargo_out(compiler, Mode::Tool, target);
build.clear_if_dirty(&out_dir, &stamp);
}
}
#[derive(Debug, Copy, Clone, Hash, PartialEq, Eq)]
struct ToolBuild {
compiler: Compiler,
target: Interned<String>,
tool: &'static str,
mode: Mode,
}
impl Step for ToolBuild {
type Output = PathBuf;
fn should_run(run: ShouldRun) -> ShouldRun {
run.never()
}
/// Build a tool in `src/tools`
///
/// This will build the specified tool with the specified `host` compiler in
/// `stage` into the normal cargo output directory.
fn run(self, builder: &Builder) -> PathBuf {
let build = builder.build;
let compiler = self.compiler;
let target = self.target;
let tool = self.tool;
match self.mode {
Mode::Libstd => builder.ensure(compile::Std { compiler, target }),
Mode::Libtest => builder.ensure(compile::Test { compiler, target }),
Mode::Librustc => builder.ensure(compile::Rustc { compiler, target }),
Mode::Tool => panic!("unexpected Mode::Tool for tool build")
}
let _folder = build.fold_output(|| format!("stage{}-{}", compiler.stage, tool));
println!("Building stage{} tool {} ({})", compiler.stage, tool, target);
let mut cargo = prepare_tool_cargo(builder, compiler, target, tool);
build.run(&mut cargo);
build.cargo_out(compiler, Mode::Tool, target).join(exe(tool, &compiler.host))
}
}
fn prepare_tool_cargo(
builder: &Builder,
compiler: Compiler,
target: Interned<String>,
tool: &'static str,
) -> Command {
let build = builder.build;
let mut cargo = builder.cargo(compiler, Mode::Tool, target, "build");
let dir = build.src.join("src/tools").join(tool);
cargo.arg("--manifest-path").arg(dir.join("Cargo.toml"));
// We don't want to build tools dynamically as they'll be running across
// stages and such and it's just easier if they're not dynamically linked.
cargo.env("RUSTC_NO_PREFER_DYNAMIC", "1");
if let Some(dir) = build.openssl_install_dir(target) {
cargo.env("OPENSSL_STATIC", "1");
cargo.env("OPENSSL_DIR", dir);
cargo.env("LIBZ_SYS_STATIC", "1");
}
cargo.env("CFG_RELEASE_CHANNEL", &build.config.channel);
let info = GitInfo::new(&build.config, &dir);
if let Some(sha) = info.sha() {
cargo.env("CFG_COMMIT_HASH", sha);
}
if let Some(sha_short) = info.sha_short() {
cargo.env("CFG_SHORT_COMMIT_HASH", sha_short);
}
if let Some(date) = info.commit_date() {
cargo.env("CFG_COMMIT_DATE", date);
}
cargo
}
macro_rules! tool {
($($name:ident, $path:expr, $tool_name:expr, $mode:expr;)+) => {
#[derive(Copy, Clone)]
pub enum Tool {
$(
$name,
)+
}
impl<'a> Builder<'a> {
pub fn tool_exe(&self, tool: Tool) -> PathBuf {
match tool {
$(Tool::$name =>
self.ensure($name {
compiler: self.compiler(0, self.build.build),
target: self.build.build,
}),
)+
}
}
}
$(
#[derive(Debug, Copy, Clone, Hash, PartialEq, Eq)]
pub struct $name {
pub compiler: Compiler,
pub target: Interned<String>,
}
impl Step for $name {
type Output = PathBuf;
fn should_run(run: ShouldRun) -> ShouldRun {
run.path($path)
}
fn make_run(run: RunConfig) {
run.builder.ensure($name {
compiler: run.builder.compiler(run.builder.top_stage, run.builder.build.build),
target: run.target,
});
}
fn run(self, builder: &Builder) -> PathBuf {
builder.ensure(ToolBuild {
compiler: self.compiler,
target: self.target,
tool: $tool_name,
mode: $mode,
})
}
}
)+
}
}
tool!(
Rustbook, "src/tools/rustbook", "rustbook", Mode::Librustc;
ErrorIndex, "src/tools/error_index_generator", "error_index_generator", Mode::Librustc;
UnstableBookGen, "src/tools/unstable-book-gen", "unstable-book-gen", Mode::Libstd;
Tidy, "src/tools/tidy", "tidy", Mode::Libstd;
Linkchecker, "src/tools/linkchecker", "linkchecker", Mode::Libstd;
CargoTest, "src/tools/cargotest", "cargotest", Mode::Libstd;
Compiletest, "src/tools/compiletest", "compiletest", Mode::Libtest;
BuildManifest, "src/tools/build-manifest", "build-manifest", Mode::Libstd;
RemoteTestClient, "src/tools/remote-test-client", "remote-test-client", Mode::Libstd;
RustInstaller, "src/tools/rust-installer", "rust-installer", Mode::Libstd;
);
#[derive(Debug, Copy, Clone, Hash, PartialEq, Eq)]
pub struct RemoteTestServer {
pub compiler: Compiler,
pub target: Interned<String>,
}
impl Step for RemoteTestServer {
type Output = PathBuf;
fn should_run(run: ShouldRun) -> ShouldRun {
run.path("src/tools/remote-test-server")
}
fn make_run(run: RunConfig) {
run.builder.ensure(RemoteTestServer {
compiler: run.builder.compiler(run.builder.top_stage, run.builder.build.build),
target: run.target,
});
}
fn run(self, builder: &Builder) -> PathBuf {
builder.ensure(ToolBuild {
compiler: self.compiler,
target: self.target,
tool: "remote-test-server",
mode: Mode::Libstd,
})
}
}
#[derive(Debug, Copy, Clone, Hash, PartialEq, Eq)]
pub struct Rustdoc {
pub host: Interned<String>,
}
impl Step for Rustdoc {
type Output = PathBuf;
const DEFAULT: bool = true;
const ONLY_HOSTS: bool = true;
fn should_run(run: ShouldRun) -> ShouldRun {
run.path("src/tools/rustdoc")
}
fn make_run(run: RunConfig) {
run.builder.ensure(Rustdoc {
host: run.host,
});
}
fn run(self, builder: &Builder) -> PathBuf {
let build = builder.build;
let target_compiler = builder.compiler(builder.top_stage, self.host);
let target = target_compiler.host;
let build_compiler = if target_compiler.stage == 0 {
builder.compiler(0, builder.build.build)
} else if target_compiler.stage >= 2 {
// Past stage 2, we consider the compiler to be ABI-compatible and hence capable of
// building rustdoc itself.
builder.compiler(target_compiler.stage, builder.build.build)
} else {
// Similar to `compile::Assemble`, build with the previous stage's compiler. Otherwise
// we'd have stageN/bin/rustc and stageN/bin/rustdoc be effectively different stage
// compilers, which isn't what we want.
builder.compiler(target_compiler.stage - 1, builder.build.build)
};
builder.ensure(compile::Rustc { compiler: build_compiler, target });
let _folder = build.fold_output(|| format!("stage{}-rustdoc", target_compiler.stage));
println!("Building rustdoc for stage{} ({})", target_compiler.stage, target_compiler.host);
let mut cargo = prepare_tool_cargo(builder, build_compiler, target, "rustdoc");
build.run(&mut cargo);
// Cargo adds a number of paths to the dylib search path on windows, which results in
// the wrong rustdoc being executed. To avoid the conflicting rustdocs, we name the "tool"
// rustdoc a different name.
let tool_rustdoc = build.cargo_out(build_compiler, Mode::Tool, target)
.join(exe("rustdoc-tool-binary", &target_compiler.host));
// don't create a stage0-sysroot/bin directory.
if target_compiler.stage > 0 {
let sysroot = builder.sysroot(target_compiler);
let bindir = sysroot.join("bin");
t!(fs::create_dir_all(&bindir));
let bin_rustdoc = bindir.join(exe("rustdoc", &*target_compiler.host));
let _ = fs::remove_file(&bin_rustdoc);
copy(&tool_rustdoc, &bin_rustdoc);
bin_rustdoc
} else {
tool_rustdoc
}
}
}
#[derive(Debug, Copy, Clone, Hash, PartialEq, Eq)]
pub struct Cargo {
pub compiler: Compiler,
pub target: Interned<String>,
}
impl Step for Cargo {
type Output = PathBuf;
const DEFAULT: bool = true;
const ONLY_HOSTS: bool = true;
fn should_run(run: ShouldRun) -> ShouldRun {
let builder = run.builder;
run.path("src/tools/cargo").default_condition(builder.build.config.extended)
}
fn make_run(run: RunConfig) {
run.builder.ensure(Cargo {
compiler: run.builder.compiler(run.builder.top_stage, run.builder.build.build),
target: run.target,
});
}
fn run(self, builder: &Builder) -> PathBuf {
builder.ensure(native::Openssl {
target: self.target,
});
// Cargo depends on procedural macros, which requires a full host
// compiler to be available, so we need to depend on that.
builder.ensure(compile::Rustc {
compiler: self.compiler,
target: builder.build.build,
});
builder.ensure(ToolBuild {
compiler: self.compiler,
target: self.target,
tool: "cargo",
mode: Mode::Librustc,
})
}
}
#[derive(Debug, Copy, Clone, Hash, PartialEq, Eq)]
pub struct Rls {
pub compiler: Compiler,
pub target: Interned<String>,
}
impl Step for Rls {
type Output = PathBuf;
const DEFAULT: bool = true;
const ONLY_HOSTS: bool = true;
fn should_run(run: ShouldRun) -> ShouldRun {
let builder = run.builder;
run.path("src/tools/rls").default_condition(builder.build.config.extended)
}
fn make_run(run: RunConfig) {
run.builder.ensure(Rls {
compiler: run.builder.compiler(run.builder.top_stage, run.builder.build.build),
target: run.target,
});
}
fn run(self, builder: &Builder) -> PathBuf {
builder.ensure(native::Openssl {
target: self.target,
});
// RLS depends on procedural macros, which requires a full host
// compiler to be available, so we need to depend on that.
builder.ensure(compile::Rustc {
compiler: self.compiler,
target: builder.build.build,
});
builder.ensure(ToolBuild {
compiler: self.compiler,
target: self.target,
tool: "rls",
mode: Mode::Librustc,
})
}
}
impl<'a> Builder<'a> {
/// Get a `Command` which is ready to run `tool` in `stage` built for
/// `host`.
pub fn tool_cmd(&self, tool: Tool) -> Command {
let mut cmd = Command::new(self.tool_exe(tool));
let compiler = self.compiler(0, self.build.build);
self.prepare_tool_cmd(compiler, &mut cmd);
cmd
}
/// Prepares the `cmd` provided to be able to run the `compiler` provided.
///
/// Notably this munges the dynamic library lookup path to point to the
/// right location to run `compiler`.
fn prepare_tool_cmd(&self, compiler: Compiler, cmd: &mut Command) {
let host = &compiler.host;
let mut paths: Vec<PathBuf> = vec![
PathBuf::from(&self.sysroot_libdir(compiler, compiler.host)),
self.cargo_out(compiler, Mode::Tool, *host).join("deps"),
];
// On MSVC a tool may invoke a C compiler (e.g. compiletest in run-make
// mode) and that C compiler may need some extra PATH modification. Do
// so here.
if compiler.host.contains("msvc") {
let curpaths = env::var_os("PATH").unwrap_or_default();
let curpaths = env::split_paths(&curpaths).collect::<Vec<_>>();
for &(ref k, ref v) in self.cc[&compiler.host].0.env() {
if k != "PATH" {
continue
}
for path in env::split_paths(v) {
if !curpaths.contains(&path) {
paths.push(path);
}
}
}
}
add_lib_path(paths, cmd);
}
}

21
src/build_helper/lib.rs
View File

@ -13,7 +13,6 @@
extern crate filetime;
use std::fs::File;
use std::io;
use std::path::{Path, PathBuf};
use std::process::{Command, Stdio};
use std::{fs, env};
@ -211,7 +210,7 @@ pub fn native_lib_boilerplate(src_name: &str,
let out_dir = env::var_os("RUSTBUILD_NATIVE_DIR").unwrap_or(env::var_os("OUT_DIR").unwrap());
let out_dir = PathBuf::from(out_dir).join(out_name);
t!(create_dir_racy(&out_dir));
t!(fs::create_dir_all(&out_dir));
if link_name.contains('=') {
println!("cargo:rustc-link-lib={}", link_name);
} else {
@ -260,21 +259,3 @@ fn fail(s: &str) -> ! {
println!("\n\n{}\n\n", s);
std::process::exit(1);
}
fn create_dir_racy(path: &Path) -> io::Result<()> {
match fs::create_dir(path) {
Ok(()) => return Ok(()),
Err(ref e) if e.kind() == io::ErrorKind::AlreadyExists => return Ok(()),
Err(ref e) if e.kind() == io::ErrorKind::NotFound => {}
Err(e) => return Err(e),
}
match path.parent() {
Some(p) => try!(create_dir_racy(p)),
None => return Err(io::Error::new(io::ErrorKind::Other, "failed to create whole tree")),
}
match fs::create_dir(path) {
Ok(()) => Ok(()),
Err(ref e) if e.kind() == io::ErrorKind::AlreadyExists => Ok(()),
Err(e) => Err(e),
}
}

5
src/ci/docker/arm-android/Dockerfile
View File

@ -3,9 +3,6 @@ FROM ubuntu:16.04
COPY scripts/android-base-apt-get.sh /scripts/
RUN sh /scripts/android-base-apt-get.sh
COPY scripts/dumb-init.sh /scripts/
RUN sh /scripts/dumb-init.sh
COPY scripts/android-ndk.sh /scripts/
RUN . /scripts/android-ndk.sh && \
download_and_make_toolchain android-ndk-r13b-linux-x86_64.zip arm 9
@ -38,4 +35,4 @@ COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh
COPY scripts/android-start-emulator.sh /scripts/
ENTRYPOINT ["/usr/bin/dumb-init", "--", "/scripts/android-start-emulator.sh"]
ENTRYPOINT ["/scripts/android-start-emulator.sh"]

9
src/ci/docker/armhf-gnu/Dockerfile
View File

@ -63,24 +63,19 @@ RUN curl http://cdimage.ubuntu.com/ubuntu-base/releases/16.04/release/ubuntu-bas
# Copy over our init script, which starts up our test server and also a few
# other misc tasks.
COPY armhf-gnu/rcS rootfs/etc/init.d/rcS
COPY scripts/qemu-bare-bones-rcS rootfs/etc/init.d/rcS
RUN chmod +x rootfs/etc/init.d/rcS
# Helper to quickly fill the entropy pool in the kernel.
COPY armhf-gnu/addentropy.c /tmp/
COPY scripts/qemu-bare-bones-addentropy.c /tmp/addentropy.c
RUN arm-linux-gnueabihf-gcc addentropy.c -o rootfs/addentropy -static
# TODO: What is this?!
RUN curl -O http://ftp.nl.debian.org/debian/dists/jessie/main/installer-armhf/current/images/device-tree/vexpress-v2p-ca15-tc1.dtb
COPY scripts/dumb-init.sh /scripts/
RUN sh /scripts/dumb-init.sh
COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh
ENTRYPOINT ["/usr/bin/dumb-init", "--"]
ENV RUST_CONFIGURE_ARGS \
--target=arm-unknown-linux-gnueabihf \
--qemu-armhf-rootfs=/tmp/rootfs

5
src/ci/docker/asmjs/Dockerfile
View File

@ -13,9 +13,6 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
gdb \
xz-utils
COPY scripts/dumb-init.sh /scripts/
RUN sh /scripts/dumb-init.sh
COPY scripts/emscripten.sh /scripts/
RUN bash /scripts/emscripten.sh
@ -35,5 +32,3 @@ ENV SCRIPT python2.7 ../x.py test --target $TARGETS
COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh
ENTRYPOINT ["/usr/bin/dumb-init", "--"]

12
src/ci/docker/cross/Dockerfile
View File

@ -21,9 +21,6 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
libssl-dev \
pkg-config
COPY scripts/dumb-init.sh /scripts/
RUN sh /scripts/dumb-init.sh
WORKDIR /tmp
COPY cross/build-rumprun.sh /tmp/
@ -38,6 +35,9 @@ RUN ./install-mips-musl.sh
COPY cross/install-mipsel-musl.sh /tmp/
RUN ./install-mipsel-musl.sh
COPY cross/install-x86_64-redox.sh /tmp/
RUN ./install-x86_64-redox.sh
ENV TARGETS=asmjs-unknown-emscripten
ENV TARGETS=$TARGETS,wasm32-unknown-emscripten
ENV TARGETS=$TARGETS,x86_64-rumprun-netbsd
@ -47,10 +47,12 @@ ENV TARGETS=$TARGETS,arm-unknown-linux-musleabi
ENV TARGETS=$TARGETS,arm-unknown-linux-musleabihf
ENV TARGETS=$TARGETS,armv7-unknown-linux-musleabihf
ENV TARGETS=$TARGETS,sparc64-unknown-linux-gnu
ENV TARGETS=$TARGETS,x86_64-unknown-redox
ENV CC_mipsel_unknown_linux_musl=mipsel-openwrt-linux-gcc \
CC_mips_unknown_linux_musl=mips-openwrt-linux-gcc \
CC_sparc64_unknown_linux_gnu=sparc64-linux-gnu-gcc
CC_sparc64_unknown_linux_gnu=sparc64-linux-gnu-gcc \
CC_x86_64_unknown_redox=x86_64-unknown-redox-gcc
# Suppress some warnings in the openwrt toolchains we downloaded
ENV STAGING_DIR=/tmp
@ -66,5 +68,3 @@ ENV SCRIPT python2.7 ../x.py dist --target $TARGETS
# sccache
COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh
ENTRYPOINT ["/usr/bin/dumb-init", "--"]

2
src/ci/docker/cross/install-mips-musl.sh
View File

@ -15,7 +15,7 @@ mkdir /usr/local/mips-linux-musl
# originally from
# https://downloads.openwrt.org/snapshots/trunk/ar71xx/generic/
# OpenWrt-Toolchain-ar71xx-generic_gcc-5.3.0_musl-1.1.16.Linux-x86_64.tar.bz2
URL="https://s3.amazonaws.com/rust-lang-ci/rust-ci-mirror"
URL="https://s3-us-west-1.amazonaws.com/rust-lang-ci2/rust-ci-mirror"
FILE="OpenWrt-Toolchain-ar71xx-generic_gcc-5.3.0_musl-1.1.16.Linux-x86_64.tar.bz2"
curl -L "$URL/$FILE" | tar xjf - -C /usr/local/mips-linux-musl --strip-components=2

2
src/ci/docker/cross/install-mipsel-musl.sh
View File

@ -15,7 +15,7 @@ mkdir /usr/local/mipsel-linux-musl
# Note that this originally came from:
# https://downloads.openwrt.org/snapshots/trunk/malta/generic/
# OpenWrt-Toolchain-malta-le_gcc-5.3.0_musl-1.1.15.Linux-x86_64.tar.bz2
URL="https://s3.amazonaws.com/rust-lang-ci/libc"
URL="https://s3-us-west-1.amazonaws.com/rust-lang-ci2/libc"
FILE="OpenWrt-Toolchain-malta-le_gcc-5.3.0_musl-1.1.15.Linux-x86_64.tar.bz2"
curl -L "$URL/$FILE" | tar xjf - -C /usr/local/mipsel-linux-musl --strip-components=2

23
src/ci/docker/cross/install-x86_64-redox.sh Executable file
View File

@ -0,0 +1,23 @@
#!/bin/bash
# Copyright 2017 The Rust Project Developers. See the COPYRIGHT
# file at the top-level directory of this distribution and at
# http://rust-lang.org/COPYRIGHT.
#
# Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
# http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
# <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
# option. This file may not be copied, modified, or distributed
# except according to those terms.
# ignore-tidy-linelength
set -ex
apt-get update
apt-get install -y --no-install-recommends software-properties-common apt-transport-https
apt-key adv --batch --yes --keyserver keyserver.ubuntu.com --recv-keys AA12E97F0881517F
add-apt-repository -y 'deb https://static.redox-os.org/toolchain/apt /'
apt-get update
apt-get install -y x86-64-unknown-redox-gcc

80
src/ci/docker/disabled/aarch64-gnu/Dockerfile Normal file
View File

@ -0,0 +1,80 @@
FROM ubuntu:16.04
RUN apt-get update -y && apt-get install -y --no-install-recommends \
bc \
bzip2 \
ca-certificates \
cmake \
cpio \
curl \
file \
g++ \
gcc-aarch64-linux-gnu \
git \
libc6-dev \
libc6-dev-arm64-cross \
make \
python2.7 \
qemu-system-aarch64 \
xz-utils
ENV ARCH=arm64 \
CROSS_COMPILE=aarch64-linux-gnu-
WORKDIR /build
# Compile the kernel that we're going to run and be emulating with. This is
# basically just done to be compatible with the QEMU target that we're going
# to be using when running tests. If any other kernel works or if any
# other QEMU target works with some other stock kernel, we can use that too!
#
# The `config` config file was a previously generated config file for
# the kernel. This file was generated by running `make defconfig`
# followed by `make menuconfig` and then enabling the IPv6 protocol page.
COPY disabled/aarch64-gnu/config /build/.config
RUN curl https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.4.42.tar.xz | \
tar xJf - && \
cd /build/linux-4.4.42 && \
cp /build/.config . && \
make -j$(nproc) all && \
cp arch/arm64/boot/Image /tmp && \
cd /build && \
rm -rf linux-4.4.42
# Compile an instance of busybox as this provides a lightweight system and init
# binary which we will boot into. Only trick here is configuring busybox to
# build static binaries.
RUN curl https://www.busybox.net/downloads/busybox-1.21.1.tar.bz2 | tar xjf - && \
cd busybox-1.21.1 && \
make defconfig && \
sed -i 's/.*CONFIG_STATIC.*/CONFIG_STATIC=y/' .config && \
make -j$(nproc) && \
make install && \
mv _install /tmp/rootfs && \
cd /build && \
rm -rf busybox-1.12.1
# Download the ubuntu rootfs, which we'll use as a chroot for all our tests.
WORKDIR /tmp
RUN mkdir rootfs/ubuntu
RUN curl http://cdimage.ubuntu.com/ubuntu-base/releases/16.04/release/ubuntu-base-16.04-core-arm64.tar.gz | \
tar xzf - -C rootfs/ubuntu && \
cd rootfs && mkdir proc sys dev etc etc/init.d
# Copy over our init script, which starts up our test server and also a few
# other misc tasks.
COPY scripts/qemu-bare-bones-rcS rootfs/etc/init.d/rcS
RUN chmod +x rootfs/etc/init.d/rcS
# Helper to quickly fill the entropy pool in the kernel.
COPY scripts/qemu-bare-bones-addentropy.c /tmp/addentropy.c
RUN aarch64-linux-gnu-gcc addentropy.c -o rootfs/addentropy -static
COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh
ENV RUST_CONFIGURE_ARGS \
--target=aarch64-unknown-linux-gnu \
--qemu-aarch64-rootfs=/tmp/rootfs
ENV SCRIPT python2.7 ../x.py test --target aarch64-unknown-linux-gnu
ENV NO_CHANGE_USER=1

3100
src/ci/docker/disabled/aarch64-gnu/config Normal file
View File

File diff suppressed because it is too large Load Diff

5
src/ci/docker/disabled/dist-aarch64-android/Dockerfile
View File

@ -3,9 +3,6 @@ FROM ubuntu:16.04
COPY scripts/android-base-apt-get.sh /scripts/
RUN sh /scripts/android-base-apt-get.sh
COPY scripts/dumb-init.sh /scripts/
RUN sh /scripts/dumb-init.sh
COPY scripts/android-ndk.sh /scripts/
RUN . /scripts/android-ndk.sh && \
download_and_make_toolchain android-ndk-r13b-linux-x86_64.zip arm64 21
@ -28,5 +25,3 @@ ENV SCRIPT python2.7 ../x.py dist --target $HOSTS --host $HOSTS
COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh
ENTRYPOINT ["/usr/bin/dumb-init", "--"]

5
src/ci/docker/disabled/dist-armv7-android/Dockerfile
View File

@ -3,9 +3,6 @@ FROM ubuntu:16.04
COPY scripts/android-base-apt-get.sh /scripts/
RUN sh /scripts/android-base-apt-get.sh
COPY scripts/dumb-init.sh /scripts/
RUN sh /scripts/dumb-init.sh
COPY scripts/android-ndk.sh /scripts/
RUN . /scripts/android-ndk.sh && \
download_ndk android-ndk-r13b-linux-x86_64.zip && \
@ -46,5 +43,3 @@ ENV SCRIPT \
COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh
ENTRYPOINT ["/usr/bin/dumb-init", "--"]

5
src/ci/docker/disabled/dist-i686-android/Dockerfile
View File

@ -3,9 +3,6 @@ FROM ubuntu:16.04
COPY scripts/android-base-apt-get.sh /scripts/
RUN sh /scripts/android-base-apt-get.sh
COPY scripts/dumb-init.sh /scripts/
RUN sh /scripts/dumb-init.sh
COPY scripts/android-ndk.sh /scripts/
RUN . /scripts/android-ndk.sh && \
download_ndk android-ndk-r13b-linux-x86_64.zip && \
@ -46,5 +43,3 @@ ENV SCRIPT \
COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh
ENTRYPOINT ["/usr/bin/dumb-init", "--"]

5
src/ci/docker/disabled/dist-x86_64-android/Dockerfile
View File

@ -3,9 +3,6 @@ FROM ubuntu:16.04
COPY scripts/android-base-apt-get.sh /scripts/
RUN sh /scripts/android-base-apt-get.sh
COPY scripts/dumb-init.sh /scripts/
RUN sh /scripts/dumb-init.sh
COPY scripts/android-ndk.sh /scripts/
RUN . /scripts/android-ndk.sh && \
download_and_make_toolchain android-ndk-r13b-linux-x86_64.zip x86_64 21
@ -28,5 +25,3 @@ ENV SCRIPT python2.7 ../x.py dist --target $HOSTS --host $HOSTS
COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh
ENTRYPOINT ["/usr/bin/dumb-init", "--"]

22
src/ci/docker/disabled/dist-x86_64-redox/Dockerfile Normal file
View File

@ -0,0 +1,22 @@
FROM ubuntu:16.04
COPY scripts/cross-apt-packages.sh /scripts/
RUN sh /scripts/cross-apt-packages.sh
COPY scripts/crosstool-ng.sh /scripts/
RUN sh /scripts/crosstool-ng.sh
WORKDIR /tmp
COPY cross/install-x86_64-redox.sh /tmp/
RUN ./install-x86_64-redox.sh
COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh
ENV \
AR_x86_64_unknown_redox=x86_64-unknown-redox-ar \
CC_x86_64_unknown_redox=x86_64-unknown-redox-gcc \
CXX_x86_64_unknown_redox=x86_64-unknown-redox-g++
ENV RUST_CONFIGURE_ARGS --target=x86_64-unknown-redox --enable-extended
ENV SCRIPT python2.7 ../x.py dist --target x86_64-unknown-redox

7
src/ci/docker/disabled/wasm32-exp/Dockerfile
View File

@ -15,10 +15,6 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
jq \
bzip2
# dumb-init
COPY scripts/dumb-init.sh /scripts/
RUN sh /scripts/dumb-init.sh
# emscripten
COPY scripts/emscripten-wasm.sh /scripts/
COPY disabled/wasm32-exp/node.sh /usr/local/bin/node
@ -37,6 +33,3 @@ ENV TARGETS=wasm32-experimental-emscripten
ENV RUST_CONFIGURE_ARGS --target=$TARGETS --experimental-targets=WebAssembly
ENV SCRIPT python2.7 ../x.py test --target $TARGETS
# init
ENTRYPOINT ["/usr/bin/dumb-init", "--"]

7
src/ci/docker/disabled/wasm32/Dockerfile
View File

@ -13,13 +13,9 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
gdb \
xz-utils
COPY scripts/dumb-init.sh /scripts/
RUN sh /scripts/dumb-init.sh
# emscripten
COPY scripts/emscripten.sh /scripts/
RUN bash /scripts/emscripten.sh
COPY disabled/wasm32/node.sh /usr/local/bin/node
COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh
@ -27,6 +23,7 @@ RUN sh /scripts/sccache.sh
ENV PATH=$PATH:/emsdk-portable
ENV PATH=$PATH:/emsdk-portable/clang/e1.37.13_64bit/
ENV PATH=$PATH:/emsdk-portable/emscripten/1.37.13/
ENV PATH=$PATH:/node-v8.0.0-linux-x64/bin/
ENV EMSCRIPTEN=/emsdk-portable/emscripten/1.37.13/
ENV BINARYEN_ROOT=/emsdk-portable/clang/e1.37.13_64bit/binaryen/
ENV EM_CONFIG=/emsdk-portable/.emscripten
@ -36,5 +33,3 @@ ENV TARGETS=wasm32-unknown-emscripten
ENV RUST_CONFIGURE_ARGS --target=$TARGETS
ENV SCRIPT python2.7 ../x.py test --target $TARGETS
ENTRYPOINT ["/usr/bin/dumb-init", "--"]

5
src/ci/docker/dist-aarch64-linux/Dockerfile
View File

@ -3,11 +3,6 @@ FROM ubuntu:16.04
COPY scripts/cross-apt-packages.sh /scripts/
RUN sh /scripts/cross-apt-packages.sh
COPY scripts/dumb-init.sh /scripts/
RUN sh /scripts/dumb-init.sh
ENTRYPOINT ["/usr/bin/dumb-init", "--"]
# Ubuntu 16.04 (this container) ships with make 4, but something in the
# toolchains we build below chokes on that, so go back to make 3
COPY scripts/make3.sh /scripts/

5
src/ci/docker/dist-android/Dockerfile
View File

@ -3,9 +3,6 @@ FROM ubuntu:16.04
COPY scripts/android-base-apt-get.sh /scripts/
RUN sh /scripts/android-base-apt-get.sh
COPY scripts/dumb-init.sh /scripts/
RUN sh /scripts/dumb-init.sh
# ndk
COPY scripts/android-ndk.sh /scripts/
RUN . /scripts/android-ndk.sh && \
@ -36,5 +33,3 @@ ENV SCRIPT python2.7 ../x.py dist --target $TARGETS
COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh
ENTRYPOINT ["/usr/bin/dumb-init", "--"]

5
src/ci/docker/dist-arm-linux/Dockerfile
View File

@ -3,11 +3,6 @@ FROM ubuntu:16.04
COPY scripts/cross-apt-packages.sh /scripts/
RUN sh /scripts/cross-apt-packages.sh
COPY scripts/dumb-init.sh /scripts/
RUN sh /scripts/dumb-init.sh
ENTRYPOINT ["/usr/bin/dumb-init", "--"]
# Ubuntu 16.04 (this container) ships with make 4, but something in the
# toolchains we build below chokes on that, so go back to make 3
COPY scripts/make3.sh /scripts/

5
src/ci/docker/dist-armhf-linux/Dockerfile
View File

@ -3,11 +3,6 @@ FROM ubuntu:16.04
COPY scripts/cross-apt-packages.sh /scripts/
RUN sh /scripts/cross-apt-packages.sh
COPY scripts/dumb-init.sh /scripts/
RUN sh /scripts/dumb-init.sh
ENTRYPOINT ["/usr/bin/dumb-init", "--"]
# Ubuntu 16.04 (this container) ships with make 4, but something in the
# toolchains we build below chokes on that, so go back to make 3
COPY scripts/make3.sh /scripts/

5
src/ci/docker/dist-armv7-linux/Dockerfile
View File

@ -3,11 +3,6 @@ FROM ubuntu:16.04
COPY scripts/cross-apt-packages.sh /scripts/
RUN sh /scripts/cross-apt-packages.sh
COPY scripts/dumb-init.sh /scripts/
RUN sh /scripts/dumb-init.sh
ENTRYPOINT ["/usr/bin/dumb-init", "--"]
# Ubuntu 16.04 (this container) ships with make 4, but something in the
# toolchains we build below chokes on that, so go back to make 3
COPY scripts/make3.sh /scripts/

5
src/ci/docker/dist-fuchsia/Dockerfile
View File

@ -24,11 +24,6 @@ WORKDIR /tmp
COPY dist-fuchsia/shared.sh dist-fuchsia/build-toolchain.sh dist-fuchsia/compiler-rt-dso-handle.patch /tmp/
RUN /tmp/build-toolchain.sh
COPY scripts/dumb-init.sh /scripts/
RUN sh /scripts/dumb-init.sh
ENTRYPOINT ["/usr/bin/dumb-init", "--"]
COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh

5
src/ci/docker/dist-i586-gnu-i686-musl/Dockerfile
View File

@ -20,11 +20,6 @@ WORKDIR /build/
COPY dist-i586-gnu-i686-musl/musl-libunwind-patch.patch dist-i586-gnu-i686-musl/build-musl.sh /build/
RUN sh /build/build-musl.sh && rm -rf /build
COPY scripts/dumb-init.sh /scripts/
RUN sh /scripts/dumb-init.sh
ENTRYPOINT ["/usr/bin/dumb-init", "--"]
COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh

5
src/ci/docker/dist-i686-freebsd/Dockerfile
View File

@ -19,11 +19,6 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
COPY dist-i686-freebsd/build-toolchain.sh /tmp/
RUN /tmp/build-toolchain.sh i686
COPY scripts/dumb-init.sh /scripts/
RUN sh /scripts/dumb-init.sh
ENTRYPOINT ["/usr/bin/dumb-init", "--"]
COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh

4
src/ci/docker/dist-i686-freebsd/build-toolchain.sh
View File

@ -13,7 +13,7 @@ set -ex
ARCH=$1
BINUTILS=2.25.1
GCC=5.3.0
GCC=6.4.0
hide_output() {
set +x
@ -86,7 +86,7 @@ rm -rf freebsd
# Finally, download and build gcc to target FreeBSD
mkdir gcc
cd gcc
curl https://ftp.gnu.org/gnu/gcc/gcc-$GCC/gcc-$GCC.tar.bz2 | tar xjf -
curl https://ftp.gnu.org/gnu/gcc/gcc-$GCC/gcc-$GCC.tar.gz | tar xzf -
cd gcc-$GCC
./contrib/download_prerequisites

5
src/ci/docker/dist-i686-linux/Dockerfile
View File

@ -76,11 +76,6 @@ RUN ./build-cmake.sh
COPY dist-i686-linux/build-headers.sh /tmp/
RUN ./build-headers.sh
RUN curl -Lo /rustroot/dumb-init \
https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64 && \
chmod +x /rustroot/dumb-init
ENTRYPOINT ["/rustroot/dumb-init", "--"]
COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh

2
src/ci/docker/dist-i686-linux/build-openssl.sh
View File

@ -13,7 +13,7 @@ set -ex
source shared.sh
VERSION=1.0.2k
URL=https://s3.amazonaws.com/rust-lang-ci/rust-ci-mirror/openssl-$VERSION.tar.gz
URL=https://s3-us-west-1.amazonaws.com/rust-lang-ci2/rust-ci-mirror/openssl-$VERSION.tar.gz
curl $URL | tar xzf -

5
src/ci/docker/dist-mips-linux/Dockerfile
View File

@ -17,14 +17,9 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
pkg-config
COPY scripts/dumb-init.sh /scripts/
RUN sh /scripts/dumb-init.sh
COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh
ENTRYPOINT ["/usr/bin/dumb-init", "--"]
ENV HOSTS=mips-unknown-linux-gnu
ENV RUST_CONFIGURE_ARGS --host=$HOSTS --enable-extended

5
src/ci/docker/dist-mips64-linux/Dockerfile
View File

@ -16,14 +16,9 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
libssl-dev \
pkg-config
COPY scripts/dumb-init.sh /scripts/
RUN sh /scripts/dumb-init.sh
COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh
ENTRYPOINT ["/usr/bin/dumb-init", "--"]
ENV HOSTS=mips64-unknown-linux-gnuabi64
ENV RUST_CONFIGURE_ARGS --host=$HOSTS --enable-extended

5
src/ci/docker/dist-mips64el-linux/Dockerfile
View File

@ -17,14 +17,9 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
pkg-config
COPY scripts/dumb-init.sh /scripts/
RUN sh /scripts/dumb-init.sh
COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh
ENTRYPOINT ["/usr/bin/dumb-init", "--"]
ENV HOSTS=mips64el-unknown-linux-gnuabi64
ENV RUST_CONFIGURE_ARGS --host=$HOSTS --enable-extended

5
src/ci/docker/dist-mipsel-linux/Dockerfile
View File

@ -16,14 +16,9 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
libssl-dev \
pkg-config
COPY scripts/dumb-init.sh /scripts/
RUN sh /scripts/dumb-init.sh
COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh
ENTRYPOINT ["/usr/bin/dumb-init", "--"]
ENV HOSTS=mipsel-unknown-linux-gnu
ENV RUST_CONFIGURE_ARGS --host=$HOSTS --enable-extended

5
src/ci/docker/dist-powerpc-linux/Dockerfile
View File

@ -3,11 +3,6 @@ FROM ubuntu:16.04
COPY scripts/cross-apt-packages.sh /scripts/
RUN sh /scripts/cross-apt-packages.sh
COPY scripts/dumb-init.sh /scripts/
RUN sh /scripts/dumb-init.sh
ENTRYPOINT ["/usr/bin/dumb-init", "--"]
# Ubuntu 16.04 (this container) ships with make 4, but something in the
# toolchains we build below chokes on that, so go back to make 3
COPY scripts/make3.sh /scripts/

4
src/ci/docker/dist-powerpc64-linux/Dockerfile
View File

@ -3,10 +3,6 @@ FROM ubuntu:16.04
COPY scripts/cross-apt-packages.sh /scripts/
RUN sh /scripts/cross-apt-packages.sh
COPY scripts/dumb-init.sh /scripts/
RUN sh /scripts/dumb-init.sh
ENTRYPOINT ["/usr/bin/dumb-init", "--"]
# Ubuntu 16.04 (this container) ships with make 4, but something in the
# toolchains we build below chokes on that, so go back to make 3

5
src/ci/docker/dist-powerpc64le-linux/Dockerfile
View File

@ -3,11 +3,6 @@ FROM ubuntu:16.04
COPY scripts/cross-apt-packages.sh /scripts/
RUN sh /scripts/cross-apt-packages.sh
COPY scripts/dumb-init.sh /scripts/
RUN sh /scripts/dumb-init.sh
ENTRYPOINT ["/usr/bin/dumb-init", "--"]
# Ubuntu 16.04 (this container) ships with make 4, but something in the
# toolchains we build below chokes on that, so go back to make 3
COPY scripts/make3.sh /scripts/

5
src/ci/docker/dist-s390x-linux/Dockerfile
View File

@ -3,11 +3,6 @@ FROM ubuntu:16.04
COPY scripts/cross-apt-packages.sh /scripts/
RUN sh /scripts/cross-apt-packages.sh
COPY scripts/dumb-init.sh /scripts/
RUN sh /scripts/dumb-init.sh
ENTRYPOINT ["/usr/bin/dumb-init", "--"]
# Ubuntu 16.04 (this container) ships with make 4, but something in the
# toolchains we build below chokes on that, so go back to make 3
COPY scripts/make3.sh /scripts/

5
src/ci/docker/dist-x86_64-freebsd/Dockerfile
View File

@ -19,11 +19,6 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
COPY dist-x86_64-freebsd/build-toolchain.sh /tmp/
RUN /tmp/build-toolchain.sh x86_64
COPY scripts/dumb-init.sh /scripts/
RUN sh /scripts/dumb-init.sh
ENTRYPOINT ["/usr/bin/dumb-init", "--"]
COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh

4
src/ci/docker/dist-x86_64-freebsd/build-toolchain.sh
View File

@ -13,7 +13,7 @@ set -ex
ARCH=$1
BINUTILS=2.25.1
GCC=5.3.0
GCC=6.4.0
hide_output() {
set +x
@ -86,7 +86,7 @@ rm -rf freebsd
# Finally, download and build gcc to target FreeBSD
mkdir gcc
cd gcc
curl https://ftp.gnu.org/gnu/gcc/gcc-$GCC/gcc-$GCC.tar.bz2 | tar xjf -
curl https://ftp.gnu.org/gnu/gcc/gcc-$GCC/gcc-$GCC.tar.gz | tar xzf -
cd gcc-$GCC
./contrib/download_prerequisites

5
src/ci/docker/dist-x86_64-linux/Dockerfile
View File

@ -76,11 +76,6 @@ RUN ./build-cmake.sh
COPY dist-x86_64-linux/build-headers.sh /tmp/
RUN ./build-headers.sh
RUN curl -Lo /rustroot/dumb-init \
https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64 && \
chmod +x /rustroot/dumb-init
ENTRYPOINT ["/rustroot/dumb-init", "--"]
COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh

2
src/ci/docker/dist-x86_64-linux/build-openssl.sh
View File

@ -13,7 +13,7 @@ set -ex
source shared.sh
VERSION=1.0.2k
URL=https://s3.amazonaws.com/rust-lang-ci/rust-ci-mirror/openssl-$VERSION.tar.gz
URL=https://s3-us-west-1.amazonaws.com/rust-lang-ci2/rust-ci-mirror/openssl-$VERSION.tar.gz
curl $URL | tar xzf -

5
src/ci/docker/dist-x86_64-musl/Dockerfile
View File

@ -20,11 +20,6 @@ WORKDIR /build/
COPY dist-x86_64-musl/build-musl.sh /build/
RUN sh /build/build-musl.sh && rm -rf /build
COPY scripts/dumb-init.sh /scripts/
RUN sh /scripts/dumb-init.sh
ENTRYPOINT ["/usr/bin/dumb-init", "--"]
COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh

5
src/ci/docker/dist-x86_64-netbsd/Dockerfile
View File

@ -3,11 +3,6 @@ FROM ubuntu:16.04
COPY scripts/cross-apt-packages.sh /scripts/
RUN sh /scripts/cross-apt-packages.sh
COPY scripts/dumb-init.sh /scripts/
RUN sh /scripts/dumb-init.sh
ENTRYPOINT ["/usr/bin/dumb-init", "--"]
# Ubuntu 16.04 (this container) ships with make 4, but something in the
# toolchains we build below chokes on that, so go back to make 3
COPY scripts/make3.sh /scripts/

2
src/ci/docker/dist-x86_64-netbsd/build-netbsd-toolchain.sh
View File

@ -35,7 +35,7 @@ cd netbsd
mkdir -p /x-tools/x86_64-unknown-netbsd/sysroot
URL=https://s3.amazonaws.com/rust-lang-ci/rust-ci-mirror
URL=https://s3-us-west-1.amazonaws.com/rust-lang-ci2/rust-ci-mirror
# Originally from ftp://ftp.netbsd.org/pub/NetBSD/NetBSD-$BSD/source/sets/*.tgz
curl $URL/2017-03-17-netbsd-src.tgz | tar xzf -

5
src/ci/docker/i686-gnu-nopt/Dockerfile
View File

@ -14,13 +14,8 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
xz-utils
COPY scripts/dumb-init.sh /scripts/
RUN sh /scripts/dumb-init.sh
COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh
ENTRYPOINT ["/usr/bin/dumb-init", "--"]
ENV RUST_CONFIGURE_ARGS --build=i686-unknown-linux-gnu --disable-optimize-tests
ENV RUST_CHECK_TARGET check

5
src/ci/docker/i686-gnu/Dockerfile
View File

@ -14,13 +14,8 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
xz-utils
COPY scripts/dumb-init.sh /scripts/
RUN sh /scripts/dumb-init.sh
COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh
ENTRYPOINT ["/usr/bin/dumb-init", "--"]
ENV RUST_CONFIGURE_ARGS --build=i686-unknown-linux-gnu
ENV SCRIPT python2.7 ../x.py test

21
src/ci/docker/run.sh
View File

@ -57,9 +57,10 @@ mkdir -p $objdir/tmp
args=
if [ "$SCCACHE_BUCKET" != "" ]; then
args="$args --env SCCACHE_BUCKET=$SCCACHE_BUCKET"
args="$args --env AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID"
args="$args --env AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY"
args="$args --env SCCACHE_BUCKET"
args="$args --env SCCACHE_REGION"
args="$args --env AWS_ACCESS_KEY_ID"
args="$args --env AWS_SECRET_ACCESS_KEY"
args="$args --env SCCACHE_ERROR_LOG=/tmp/sccache/sccache.log"
args="$args --volume $objdir/tmp:/tmp/sccache"
else
@ -67,6 +68,13 @@ else
args="$args --env SCCACHE_DIR=/sccache --volume $HOME/.cache/sccache:/sccache"
fi
# Run containers as privileged as it should give them access to some more
# syscalls such as ptrace and whatnot. In the upgrade to LLVM 5.0 it was
# discovered that the leak sanitizer apparently needs these syscalls nowadays so
# we'll need `--privileged` for at least the `x86_64-gnu` builder, so this just
# goes ahead and sets it for all builders.
args="$args --privileged"
exec docker \
run \
--volume "$root_dir:/checkout:ro" \
@ -75,13 +83,14 @@ exec docker \
--env SRC=/checkout \
$args \
--env CARGO_HOME=/cargo \
--env DEPLOY=$DEPLOY \
--env DEPLOY_ALT=$DEPLOY_ALT \
--env DEPLOY \
--env DEPLOY_ALT \
--env LOCAL_USER_ID=`id -u` \
--env TRAVIS=${TRAVIS-false} \
--env TRAVIS \
--env TRAVIS_BRANCH \
--volume "$HOME/.cargo:/cargo" \
--volume "$HOME/rustsrc:$HOME/rustsrc" \
--init \
--rm \
rust-ci \
/checkout/src/ci/run.sh

2
src/ci/docker/scripts/android-ndk.sh
View File

@ -15,7 +15,7 @@ URL=https://dl.google.com/android/repository
download_ndk() {
mkdir -p /android/ndk
cd /android/ndk
curl -O $URL/$1
curl -fO $URL/$1
unzip -q $1
rm $1
mv android-ndk-* ndk

2
src/ci/docker/scripts/android-sdk.sh
View File

@ -15,7 +15,7 @@ URL=https://dl.google.com/android/repository
download_sdk() {
mkdir -p /android/sdk
cd /android/sdk
curl -O $URL/$1
curl -fO $URL/$1
unzip -q $1
rm -rf $1
}

2
src/ci/docker/scripts/crosstool-ng.sh
View File

@ -11,7 +11,7 @@
set -ex
url="http://crosstool-ng.org/download/crosstool-ng/crosstool-ng-1.22.0.tar.bz2"
curl $url | tar xjf -
curl -f $url | tar xjf -
cd crosstool-ng
./configure --prefix=/usr/local
make -j$(nproc)

6
src/ci/docker/scripts/emscripten-wasm.sh
View File

@ -28,14 +28,14 @@ exit 1
}
# Download last known good emscripten from WebAssembly waterfall
BUILD=$(curl -L https://storage.googleapis.com/wasm-llvm/builds/linux/lkgr.json | \
BUILD=$(curl -fL https://storage.googleapis.com/wasm-llvm/builds/linux/lkgr.json | \
jq '.build | tonumber')
curl -L https://storage.googleapis.com/wasm-llvm/builds/linux/$BUILD/wasm-binaries.tbz2 | \
curl -sL https://storage.googleapis.com/wasm-llvm/builds/linux/$BUILD/wasm-binaries.tbz2 | \
hide_output tar xvkj
# node 8 is required to run wasm
cd /
curl -L https://nodejs.org/dist/v8.0.0/node-v8.0.0-linux-x64.tar.xz | \
curl -sL https://nodejs.org/dist/v8.0.0/node-v8.0.0-linux-x64.tar.xz | \
tar -xJ
# Make emscripten use wasm-ready node and LLVM tools

4
src/ci/docker/scripts/emscripten.sh
View File

@ -28,7 +28,7 @@ exit 1
}
cd /
curl -L https://s3.amazonaws.com/mozilla-games/emscripten/releases/emsdk-portable.tar.gz | \
curl -fL https://s3.amazonaws.com/mozilla-games/emscripten/releases/emsdk-portable.tar.gz | \
tar -xz
cd /emsdk-portable
@ -49,5 +49,5 @@ chmod a+rxw -R /emsdk-portable
# node 8 is required to run wasm
cd /
curl -L https://nodejs.org/dist/v8.0.0/node-v8.0.0-linux-x64.tar.xz | \
curl -sL https://nodejs.org/dist/v8.0.0/node-v8.0.0-linux-x64.tar.xz | \
tar -xJ

2
src/ci/docker/scripts/make3.sh
View File

@ -10,7 +10,7 @@
set -ex
curl https://ftp.gnu.org/gnu/make/make-3.81.tar.gz | tar xzf -
curl -f https://ftp.gnu.org/gnu/make/make-3.81.tar.gz | tar xzf -
cd make-3.81
./configure --prefix=/usr
make

0
src/ci/docker/armhf-gnu/addentropy.c → src/ci/docker/scripts/qemu-bare-bones-addentropy.c
View File

0
src/ci/docker/armhf-gnu/rcS → src/ci/docker/scripts/qemu-bare-bones-rcS
View File

6
src/ci/docker/scripts/sccache.sh
View File

@ -8,9 +8,11 @@
# option. This file may not be copied, modified, or distributed
# except according to those terms.
# ignore-tidy-linelength
set -ex
curl -o /usr/local/bin/sccache \
https://s3.amazonaws.com/rust-lang-ci/rust-ci-mirror/2017-05-12-sccache-x86_64-unknown-linux-musl
curl -fo /usr/local/bin/sccache \
https://s3-us-west-1.amazonaws.com/rust-lang-ci2/rust-ci-mirror/2017-05-12-sccache-x86_64-unknown-linux-musl
chmod +x /usr/local/bin/sccache

5
src/ci/docker/x86_64-gnu-aux/Dockerfile
View File

@ -14,13 +14,8 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
xz-utils \
pkg-config
COPY scripts/dumb-init.sh /scripts/
RUN sh /scripts/dumb-init.sh
COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh
ENTRYPOINT ["/usr/bin/dumb-init", "--"]
ENV RUST_CONFIGURE_ARGS --build=x86_64-unknown-linux-gnu
ENV RUST_CHECK_TARGET check-aux

5
src/ci/docker/x86_64-gnu-debug/Dockerfile
View File

@ -13,14 +13,9 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
gdb \
xz-utils
COPY scripts/dumb-init.sh /scripts/
RUN sh /scripts/dumb-init.sh
COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh
ENTRYPOINT ["/usr/bin/dumb-init", "--"]
ENV RUST_CONFIGURE_ARGS \
--build=x86_64-unknown-linux-gnu \
--enable-debug \

5
src/ci/docker/x86_64-gnu-distcheck/Dockerfile
View File

@ -15,14 +15,9 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
libssl-dev \
pkg-config
COPY scripts/dumb-init.sh /scripts/
RUN sh /scripts/dumb-init.sh
COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh
ENTRYPOINT ["/usr/bin/dumb-init", "--"]
ENV RUST_CONFIGURE_ARGS --build=x86_64-unknown-linux-gnu
ENV SCRIPT python2.7 ../x.py test distcheck
ENV DIST_SRC 1

5
src/ci/docker/x86_64-gnu-full-bootstrap/Dockerfile
View File

@ -13,14 +13,9 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
gdb \
xz-utils
COPY scripts/dumb-init.sh /scripts/
RUN sh /scripts/dumb-init.sh
COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh
ENTRYPOINT ["/usr/bin/dumb-init", "--"]
ENV RUST_CONFIGURE_ARGS \
--build=x86_64-unknown-linux-gnu \
--enable-full-bootstrap

5
src/ci/docker/x86_64-gnu-incremental/Dockerfile
View File

@ -13,14 +13,9 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
gdb \
xz-utils
COPY scripts/dumb-init.sh /scripts/
RUN sh /scripts/dumb-init.sh
COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh
ENTRYPOINT ["/usr/bin/dumb-init", "--"]
ENV RUST_CONFIGURE_ARGS --build=x86_64-unknown-linux-gnu
ENV RUSTFLAGS -Zincremental=/tmp/rust-incr-cache
ENV RUST_CHECK_TARGET check

5
src/ci/docker/x86_64-gnu-llvm-3.7/Dockerfile
View File

@ -16,14 +16,9 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
zlib1g-dev \
xz-utils
COPY scripts/dumb-init.sh /scripts/
RUN sh /scripts/dumb-init.sh
COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh
ENTRYPOINT ["/usr/bin/dumb-init", "--"]
ENV RUST_CONFIGURE_ARGS \
--build=x86_64-unknown-linux-gnu \
--llvm-root=/usr/lib/llvm-3.7

5
src/ci/docker/x86_64-gnu-nopt/Dockerfile
View File

@ -13,13 +13,8 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
gdb \
xz-utils
COPY scripts/dumb-init.sh /scripts/
RUN sh /scripts/dumb-init.sh
COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh
ENTRYPOINT ["/usr/bin/dumb-init", "--"]
ENV RUST_CONFIGURE_ARGS --build=x86_64-unknown-linux-gnu --disable-optimize-tests
ENV RUST_CHECK_TARGET check

5
src/ci/docker/x86_64-gnu/Dockerfile
View File

@ -13,13 +13,8 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
gdb \
xz-utils
COPY scripts/dumb-init.sh /scripts/
RUN sh /scripts/dumb-init.sh
COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh
ENTRYPOINT ["/usr/bin/dumb-init", "--"]
ENV RUST_CONFIGURE_ARGS --build=x86_64-unknown-linux-gnu --enable-sanitizers --enable-profiler
ENV SCRIPT python2.7 ../x.py test

7
src/ci/run.sh
View File

@ -31,7 +31,6 @@ RUST_CONFIGURE_ARGS="$RUST_CONFIGURE_ARGS --enable-sccache"
RUST_CONFIGURE_ARGS="$RUST_CONFIGURE_ARGS --disable-manage-submodules"
RUST_CONFIGURE_ARGS="$RUST_CONFIGURE_ARGS --enable-locked-deps"
RUST_CONFIGURE_ARGS="$RUST_CONFIGURE_ARGS --enable-cargo-openssl-static"
RUST_CONFIGURE_ARGS="$RUST_CONFIGURE_ARGS --enable-llvm-clean-rebuild"
if [ "$DIST_SRC" = "" ]; then
RUST_CONFIGURE_ARGS="$RUST_CONFIGURE_ARGS --disable-dist-src"
@ -74,6 +73,12 @@ retry make prepare
travis_fold end make-prepare
travis_time_finish
travis_fold start check-bootstrap
travis_time_start
make check-bootstrap
travis_fold end check-bootstrap
travis_time_finish
if [ "$TRAVIS_OS_NAME" = "osx" ]; then
ncpus=$(sysctl -n hw.ncpu)
else

8
src/doc/book/ci/build.sh
View File

@ -20,8 +20,14 @@ cargo run -- ../../second-edition/src
cd ../..
# tests for the second edition
# tests for the first edition
cd first-edition
mdbook test
mdbook build
cd ..
# tests for the second edition
cd second-edition
bash spellcheck.sh list
mdbook test

4
src/doc/book/first-edition/src/lifetimes.md
View File

@ -53,7 +53,7 @@ Uh oh! Your reference is pointing to an invalid resource. This is called a
dangling pointer or ‘use after free’, when the resource is memory. A small
example of such a situation would be:
```rust,compile_fail
```rust,ignore
let r; // Introduce reference: `r`.
{
let i = 1; // Introduce scoped value: `i`.
@ -70,7 +70,7 @@ as it can see the lifetimes of the various values in the function.
When we have a function that takes arguments by reference the situation becomes
more complex. Consider the following example:
```rust,compile_fail,E0106
```rust,ignore
fn skip_prefix(line: &str, prefix: &str) -> &str {
// ...
# line

Some files were not shown because too many files have changed in this diff Show More