proxmox-mirrors/rustc
1
0
Fork 0
mirror of https://git.proxmox.com/git/rustc synced 2025-08-06 16:48:32 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

New upstream version 1.32.0~beta.2+dfsg1

Browse Source
This commit is contained in:
Ximin Luo 2018-12-16 10:13:16 -08:00
parent 450edc1f0b
commit a1dfa0c682
6606 changed files with 74381 additions and 40434 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
CONTRIBUTING.md
View File

@ -494,16 +494,11 @@ the version in `Cargo.lock`, so the build can no longer continue.
To resolve this, we need to update `Cargo.lock`. Luckily, cargo provides a
command to do this easily.
First, go into the `src/` directory since that is where `Cargo.toml` is in
the rust repository. Then run, `cargo update -p rustfmt-nightly` to solve
the problem.
```
$ cd src
$ cargo update -p rustfmt-nightly
```
This should change the version listed in `src/Cargo.lock` to the new version you updated
This should change the version listed in `Cargo.lock` to the new version you updated
the submodule to. Running `./x.py build` should work now.
## Writing Documentation
@ -645,7 +640,7 @@ are:
* **Google!** ([search only in Rust Documentation][gsearchdocs] to find types, traits, etc. quickly)
* Don't be afraid to ask! The Rust community is friendly and helpful.
[rustc guide]: https://rust-lang-nursery.github.io/rustc-guide/about-this-guide.html
[rustc guide]: https://rust-lang.github.io/rustc-guide/about-this-guide.html
[gdfrustc]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/
[gsearchdocs]: https://www.google.com/search?q=site:doc.rust-lang.org+your+query+here
[rif]: http://internals.rust-lang.org
@ -653,5 +648,5 @@ are:
[rustforge]: https://forge.rust-lang.org/
[tlgba]: http://tomlee.co/2014/04/a-more-detailed-tour-of-the-rust-compiler/
[ro]: http://www.rustaceans.org/
[rctd]: https://rust-lang-nursery.github.io/rustc-guide/tests/intro.html
[rctd]: https://rust-lang.github.io/rustc-guide/tests/intro.html
[cheatsheet]: https://buildbot2.rust-lang.org/homu/

247
src/Cargo.lock → Cargo.lock generated
View File

@ -15,27 +15,6 @@ dependencies = [
"rand 0.5.5 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "alloc_jemalloc"
version = "0.0.0"
dependencies = [
"build_helper 0.1.0",
"cc 1.0.25 (registry+https://github.com/rust-lang/crates.io-index)",
"compiler_builtins 0.0.0",
"core 0.0.0",
"libc 0.0.0",
]
[[package]]
name = "alloc_system"
version = "0.0.0"
dependencies = [
"compiler_builtins 0.0.0",
"core 0.0.0",
"dlmalloc 0.0.0",
"libc 0.0.0",
]
[[package]]
name = "ammonia"
version = "1.1.0"
@ -198,19 +177,19 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
[[package]]
name = "cargo"
version = "0.32.0"
version = "0.33.0"
dependencies = [
"atty 0.2.11 (registry+https://github.com/rust-lang/crates.io-index)",
"bufstream 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)",
"bytesize 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)",
"clap 2.32.0 (registry+https://github.com/rust-lang/crates.io-index)",
"core-foundation 0.6.1 (registry+https://github.com/rust-lang/crates.io-index)",
"crates-io 0.20.0",
"crossbeam-utils 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)",
"crates-io 0.21.0",
"crossbeam-utils 0.6.1 (registry+https://github.com/rust-lang/crates.io-index)",
"crypto-hash 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)",
"curl 0.4.18 (registry+https://github.com/rust-lang/crates.io-index)",
"curl-sys 0.4.13 (registry+https://github.com/rust-lang/crates.io-index)",
"env_logger 0.5.12 (registry+https://github.com/rust-lang/crates.io-index)",
"curl 0.4.19 (registry+https://github.com/rust-lang/crates.io-index)",
"curl-sys 0.4.15 (registry+https://github.com/rust-lang/crates.io-index)",
"env_logger 0.6.0 (registry+https://github.com/rust-lang/crates.io-index)",
"failure 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)",
"filetime 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)",
"flate2 1.0.3 (registry+https://github.com/rust-lang/crates.io-index)",
@ -222,6 +201,7 @@ dependencies = [
"hex 0.3.2 (registry+https://github.com/rust-lang/crates.io-index)",
"home 0.3.3 (registry+https://github.com/rust-lang/crates.io-index)",
"ignore 0.4.3 (registry+https://github.com/rust-lang/crates.io-index)",
"im-rc 12.2.0 (registry+https://github.com/rust-lang/crates.io-index)",
"jobserver 0.1.11 (registry+https://github.com/rust-lang/crates.io-index)",
"lazy_static 1.1.0 (registry+https://github.com/rust-lang/crates.io-index)",
"lazycell 1.2.0 (registry+https://github.com/rust-lang/crates.io-index)",
@ -232,6 +212,7 @@ dependencies = [
"num_cpus 1.8.0 (registry+https://github.com/rust-lang/crates.io-index)",
"opener 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)",
"openssl 0.10.13 (registry+https://github.com/rust-lang/crates.io-index)",
"pretty_env_logger 0.2.5 (registry+https://github.com/rust-lang/crates.io-index)",
"proptest 0.8.7 (registry+https://github.com/rust-lang/crates.io-index)",
"rustc-workspace-hack 1.0.0",
"rustfix 0.4.2 (registry+https://github.com/rust-lang/crates.io-index)",
@ -242,7 +223,7 @@ dependencies = [
"serde_ignored 0.0.4 (registry+https://github.com/rust-lang/crates.io-index)",
"serde_json 1.0.31 (registry+https://github.com/rust-lang/crates.io-index)",
"shell-escape 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)",
"tar 0.4.16 (registry+https://github.com/rust-lang/crates.io-index)",
"tar 0.4.19 (registry+https://github.com/rust-lang/crates.io-index)",
"tempfile 3.0.3 (registry+https://github.com/rust-lang/crates.io-index)",
"termcolor 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)",
"toml 0.4.6 (registry+https://github.com/rust-lang/crates.io-index)",
@ -279,7 +260,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
[[package]]
name = "chalk-engine"
version = "0.8.1"
version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"chalk-macros 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)",
@ -327,7 +308,7 @@ dependencies = [
"clippy-mini-macro-test 0.2.0",
"clippy_dev 0.0.1",
"clippy_lints 0.0.212",
"compiletest_rs 0.3.13 (registry+https://github.com/rust-lang/crates.io-index)",
"compiletest_rs 0.3.17 (registry+https://github.com/rust-lang/crates.io-index)",
"derive-new 0.5.4 (registry+https://github.com/rust-lang/crates.io-index)",
"lazy_static 1.1.0 (registry+https://github.com/rust-lang/crates.io-index)",
"regex 1.0.4 (registry+https://github.com/rust-lang/crates.io-index)",
@ -362,7 +343,7 @@ dependencies = [
"itertools 0.7.8 (registry+https://github.com/rust-lang/crates.io-index)",
"lazy_static 1.1.0 (registry+https://github.com/rust-lang/crates.io-index)",
"matches 0.1.8 (registry+https://github.com/rust-lang/crates.io-index)",
"pulldown-cmark 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)",
"pulldown-cmark 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)",
"quine-mc_cluskey 0.2.4 (registry+https://github.com/rust-lang/crates.io-index)",
"regex-syntax 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)",
"semver 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)",
@ -444,7 +425,7 @@ dependencies = [
[[package]]
name = "compiletest_rs"
version = "0.3.13"
version = "0.3.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"diff 0.1.11 (registry+https://github.com/rust-lang/crates.io-index)",
@ -453,6 +434,7 @@ dependencies = [
"libc 0.2.43 (registry+https://github.com/rust-lang/crates.io-index)",
"log 0.4.5 (registry+https://github.com/rust-lang/crates.io-index)",
"miow 0.3.3 (registry+https://github.com/rust-lang/crates.io-index)",
"regex 1.0.4 (registry+https://github.com/rust-lang/crates.io-index)",
"serde 1.0.75 (registry+https://github.com/rust-lang/crates.io-index)",
"serde_derive 1.0.75 (registry+https://github.com/rust-lang/crates.io-index)",
"serde_json 1.0.31 (registry+https://github.com/rust-lang/crates.io-index)",
@ -483,9 +465,9 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
[[package]]
name = "crates-io"
version = "0.20.0"
version = "0.21.0"
dependencies = [
"curl 0.4.18 (registry+https://github.com/rust-lang/crates.io-index)",
"curl 0.4.19 (registry+https://github.com/rust-lang/crates.io-index)",
"failure 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)",
"serde 1.0.75 (registry+https://github.com/rust-lang/crates.io-index)",
"serde_derive 1.0.75 (registry+https://github.com/rust-lang/crates.io-index)",
@ -559,6 +541,14 @@ name = "crossbeam-utils"
version = "0.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
[[package]]
name = "crossbeam-utils"
version = "0.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"cfg-if 0.1.5 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "crypto-hash"
version = "0.3.1"
@ -572,10 +562,10 @@ dependencies = [
[[package]]
name = "curl"
version = "0.4.18"
version = "0.4.19"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"curl-sys 0.4.13 (registry+https://github.com/rust-lang/crates.io-index)",
"curl-sys 0.4.15 (registry+https://github.com/rust-lang/crates.io-index)",
"kernel32-sys 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)",
"libc 0.2.43 (registry+https://github.com/rust-lang/crates.io-index)",
"openssl-probe 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)",
@ -587,7 +577,7 @@ dependencies = [
[[package]]
name = "curl-sys"
version = "0.4.13"
version = "0.4.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"cc 1.0.25 (registry+https://github.com/rust-lang/crates.io-index)",
@ -636,6 +626,15 @@ name = "difference"
version = "2.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
[[package]]
name = "directories"
version = "1.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"libc 0.2.43 (registry+https://github.com/rust-lang/crates.io-index)",
"winapi 0.3.6 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "dlmalloc"
version = "0.0.0"
@ -671,6 +670,14 @@ dependencies = [
"log 0.4.5 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "ena"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"log 0.4.5 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "env_logger"
version = "0.5.12"
@ -683,6 +690,18 @@ dependencies = [
"termcolor 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "env_logger"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"atty 0.2.11 (registry+https://github.com/rust-lang/crates.io-index)",
"humantime 1.1.1 (registry+https://github.com/rust-lang/crates.io-index)",
"log 0.4.5 (registry+https://github.com/rust-lang/crates.io-index)",
"regex 1.0.4 (registry+https://github.com/rust-lang/crates.io-index)",
"termcolor 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "environment"
version = "0.1.1"
@ -752,6 +771,7 @@ version = "1.0.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"libc 0.2.43 (registry+https://github.com/rust-lang/crates.io-index)",
"libz-sys 1.0.24 (registry+https://github.com/rust-lang/crates.io-index)",
"miniz-sys 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)",
]
@ -786,6 +806,11 @@ dependencies = [
"winapi 0.3.6 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "fs_extra"
version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
[[package]]
name = "fst"
version = "0.3.0"
@ -855,7 +880,7 @@ name = "git2-curl"
version = "0.8.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"curl 0.4.18 (registry+https://github.com/rust-lang/crates.io-index)",
"curl 0.4.19 (registry+https://github.com/rust-lang/crates.io-index)",
"git2 0.7.5 (registry+https://github.com/rust-lang/crates.io-index)",
"log 0.4.5 (registry+https://github.com/rust-lang/crates.io-index)",
"url 1.7.1 (registry+https://github.com/rust-lang/crates.io-index)",
@ -964,6 +989,15 @@ dependencies = [
"winapi 0.3.6 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "im-rc"
version = "12.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"rustc_version 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)",
"typenum 1.10.0 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "installer"
version = "0.0.0"
@ -973,7 +1007,7 @@ dependencies = [
"flate2 1.0.3 (registry+https://github.com/rust-lang/crates.io-index)",
"lazy_static 1.1.0 (registry+https://github.com/rust-lang/crates.io-index)",
"rayon 1.0.1 (registry+https://github.com/rust-lang/crates.io-index)",
"tar 0.4.16 (registry+https://github.com/rust-lang/crates.io-index)",
"tar 0.4.19 (registry+https://github.com/rust-lang/crates.io-index)",
"walkdir 2.2.5 (registry+https://github.com/rust-lang/crates.io-index)",
"winapi 0.3.6 (registry+https://github.com/rust-lang/crates.io-index)",
"xz2 0.1.5 (registry+https://github.com/rust-lang/crates.io-index)",
@ -997,6 +1031,16 @@ name = "itoa"
version = "0.4.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
[[package]]
name = "jemalloc-sys"
version = "0.1.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"cc 1.0.25 (registry+https://github.com/rust-lang/crates.io-index)",
"fs_extra 1.1.0 (registry+https://github.com/rust-lang/crates.io-index)",
"libc 0.2.43 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "jobserver"
version = "0.1.11"
@ -1085,7 +1129,7 @@ version = "0.7.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"cc 1.0.25 (registry+https://github.com/rust-lang/crates.io-index)",
"curl-sys 0.4.13 (registry+https://github.com/rust-lang/crates.io-index)",
"curl-sys 0.4.15 (registry+https://github.com/rust-lang/crates.io-index)",
"libc 0.2.43 (registry+https://github.com/rust-lang/crates.io-index)",
"libssh2-sys 0.2.11 (registry+https://github.com/rust-lang/crates.io-index)",
"libz-sys 1.0.24 (registry+https://github.com/rust-lang/crates.io-index)",
@ -1291,7 +1335,8 @@ dependencies = [
"byteorder 1.2.3 (registry+https://github.com/rust-lang/crates.io-index)",
"cargo_metadata 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)",
"colored 1.6.0 (registry+https://github.com/rust-lang/crates.io-index)",
"compiletest_rs 0.3.13 (registry+https://github.com/rust-lang/crates.io-index)",
"compiletest_rs 0.3.17 (registry+https://github.com/rust-lang/crates.io-index)",
"directories 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)",
"env_logger 0.5.12 (registry+https://github.com/rust-lang/crates.io-index)",
"log 0.4.5 (registry+https://github.com/rust-lang/crates.io-index)",
"vergen 3.0.3 (registry+https://github.com/rust-lang/crates.io-index)",
@ -1564,6 +1609,17 @@ dependencies = [
"difference 2.0.0 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "pretty_env_logger"
version = "0.2.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"ansi_term 0.11.0 (registry+https://github.com/rust-lang/crates.io-index)",
"chrono 0.4.4 (registry+https://github.com/rust-lang/crates.io-index)",
"env_logger 0.5.12 (registry+https://github.com/rust-lang/crates.io-index)",
"log 0.4.5 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "proc-macro2"
version = "0.3.8"
@ -1583,12 +1639,6 @@ dependencies = [
[[package]]
name = "proc_macro"
version = "0.0.0"
dependencies = [
"rustc_data_structures 0.0.0",
"rustc_errors 0.0.0",
"syntax 0.0.0",
"syntax_pos 0.0.0",
]
[[package]]
name = "profiler_builtins"
@ -1625,6 +1675,15 @@ dependencies = [
"getopts 0.2.17 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "pulldown-cmark"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"bitflags 1.0.4 (registry+https://github.com/rust-lang/crates.io-index)",
"getopts 0.2.17 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "quick-error"
version = "1.2.2"
@ -1793,7 +1852,7 @@ dependencies = [
name = "rls"
version = "1.31.6"
dependencies = [
"cargo 0.32.0",
"cargo 0.33.0",
"cargo_metadata 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)",
"clippy_lints 0.0.212",
"crossbeam-channel 0.2.6 (registry+https://github.com/rust-lang/crates.io-index)",
@ -1899,7 +1958,7 @@ dependencies = [
"backtrace 0.3.9 (registry+https://github.com/rust-lang/crates.io-index)",
"bitflags 1.0.4 (registry+https://github.com/rust-lang/crates.io-index)",
"byteorder 1.2.3 (registry+https://github.com/rust-lang/crates.io-index)",
"chalk-engine 0.8.1 (registry+https://github.com/rust-lang/crates.io-index)",
"chalk-engine 0.8.0 (registry+https://github.com/rust-lang/crates.io-index)",
"flate2 1.0.3 (registry+https://github.com/rust-lang/crates.io-index)",
"fmt_macros 0.0.0",
"graphviz 0.0.0",
@ -1908,7 +1967,6 @@ dependencies = [
"log 0.4.5 (registry+https://github.com/rust-lang/crates.io-index)",
"parking_lot 0.6.4 (registry+https://github.com/rust-lang/crates.io-index)",
"polonius-engine 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)",
"proc_macro 0.0.0",
"rustc-rayon 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)",
"rustc-rayon-core 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)",
"rustc_apfloat 0.0.0",
@ -2048,6 +2106,7 @@ dependencies = [
name = "rustc-main"
version = "0.0.0"
dependencies = [
"rustc_codegen_ssa 0.0.0",
"rustc_driver 0.0.0",
"rustc_target 0.0.0",
]
@ -2108,6 +2167,7 @@ version = "0.0.0"
dependencies = [
"bitflags 1.0.4 (registry+https://github.com/rust-lang/crates.io-index)",
"rustc_cratesio_shim 0.0.0",
"smallvec 0.6.5 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
@ -2115,7 +2175,6 @@ name = "rustc_asan"
version = "0.0.0"
dependencies = [
"alloc 0.0.0",
"alloc_system 0.0.0",
"build_helper 0.1.0",
"cmake 0.1.33 (registry+https://github.com/rust-lang/crates.io-index)",
"compiler_builtins 0.0.0",
@ -2147,6 +2206,33 @@ dependencies = [
"rustc_llvm 0.0.0",
]
[[package]]
name = "rustc_codegen_ssa"
version = "0.0.0"
dependencies = [
"bitflags 1.0.4 (registry+https://github.com/rust-lang/crates.io-index)",
"cc 1.0.25 (registry+https://github.com/rust-lang/crates.io-index)",
"jobserver 0.1.11 (registry+https://github.com/rust-lang/crates.io-index)",
"libc 0.2.43 (registry+https://github.com/rust-lang/crates.io-index)",
"log 0.4.5 (registry+https://github.com/rust-lang/crates.io-index)",
"memmap 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)",
"num_cpus 1.8.0 (registry+https://github.com/rust-lang/crates.io-index)",
"rustc 0.0.0",
"rustc-demangle 0.1.9 (registry+https://github.com/rust-lang/crates.io-index)",
"rustc_allocator 0.0.0",
"rustc_apfloat 0.0.0",
"rustc_codegen_utils 0.0.0",
"rustc_data_structures 0.0.0",
"rustc_errors 0.0.0",
"rustc_fs_util 0.0.0",
"rustc_incremental 0.0.0",
"rustc_mir 0.0.0",
"rustc_target 0.0.0",
"serialize 0.0.0",
"syntax 0.0.0",
"syntax_pos 0.0.0",
]
[[package]]
name = "rustc_codegen_utils"
version = "0.0.0"
@ -2156,7 +2242,7 @@ dependencies = [
"rustc 0.0.0",
"rustc_data_structures 0.0.0",
"rustc_incremental 0.0.0",
"rustc_metadata_utils 0.0.0",
"rustc_metadata 0.0.0",
"rustc_mir 0.0.0",
"rustc_target 0.0.0",
"syntax 0.0.0",
@ -2177,11 +2263,10 @@ name = "rustc_data_structures"
version = "0.0.0"
dependencies = [
"cfg-if 0.1.5 (registry+https://github.com/rust-lang/crates.io-index)",
"ena 0.9.3 (registry+https://github.com/rust-lang/crates.io-index)",
"ena 0.11.0 (registry+https://github.com/rust-lang/crates.io-index)",
"graphviz 0.0.0",
"log 0.4.5 (registry+https://github.com/rust-lang/crates.io-index)",
"parking_lot 0.6.4 (registry+https://github.com/rust-lang/crates.io-index)",
"parking_lot_core 0.2.14 (registry+https://github.com/rust-lang/crates.io-index)",
"rustc-hash 1.0.1 (registry+https://github.com/rust-lang/crates.io-index)",
"rustc-rayon 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)",
"rustc-rayon-core 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)",
@ -2198,6 +2283,7 @@ dependencies = [
"arena 0.0.0",
"env_logger 0.5.12 (registry+https://github.com/rust-lang/crates.io-index)",
"graphviz 0.0.0",
"jemalloc-sys 0.1.8 (registry+https://github.com/rust-lang/crates.io-index)",
"log 0.4.5 (registry+https://github.com/rust-lang/crates.io-index)",
"rustc 0.0.0",
"rustc-rayon 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)",
@ -2265,6 +2351,7 @@ version = "0.0.0"
dependencies = [
"log 0.4.5 (registry+https://github.com/rust-lang/crates.io-index)",
"rustc 0.0.0",
"rustc_data_structures 0.0.0",
"rustc_mir 0.0.0",
"rustc_target 0.0.0",
"syntax 0.0.0",
@ -2284,7 +2371,6 @@ name = "rustc_lsan"
version = "0.0.0"
dependencies = [
"alloc 0.0.0",
"alloc_system 0.0.0",
"build_helper 0.1.0",
"cmake 0.1.33 (registry+https://github.com/rust-lang/crates.io-index)",
"compiler_builtins 0.0.0",
@ -2297,27 +2383,18 @@ version = "0.0.0"
dependencies = [
"flate2 1.0.3 (registry+https://github.com/rust-lang/crates.io-index)",
"log 0.4.5 (registry+https://github.com/rust-lang/crates.io-index)",
"proc_macro 0.0.0",
"memmap 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)",
"rustc 0.0.0",
"rustc_data_structures 0.0.0",
"rustc_errors 0.0.0",
"rustc_metadata_utils 0.0.0",
"rustc_target 0.0.0",
"serialize 0.0.0",
"stable_deref_trait 1.1.0 (registry+https://github.com/rust-lang/crates.io-index)",
"syntax 0.0.0",
"syntax_ext 0.0.0",
"syntax_pos 0.0.0",
]
[[package]]
name = "rustc_metadata_utils"
version = "0.0.0"
dependencies = [
"rustc 0.0.0",
"syntax 0.0.0",
"syntax_pos 0.0.0",
]
[[package]]
name = "rustc_mir"
version = "0.0.0"
@ -2346,7 +2423,6 @@ name = "rustc_msan"
version = "0.0.0"
dependencies = [
"alloc 0.0.0",
"alloc_system 0.0.0",
"build_helper 0.1.0",
"cmake 0.1.33 (registry+https://github.com/rust-lang/crates.io-index)",
"compiler_builtins 0.0.0",
@ -2431,6 +2507,7 @@ dependencies = [
"bitflags 1.0.4 (registry+https://github.com/rust-lang/crates.io-index)",
"log 0.4.5 (registry+https://github.com/rust-lang/crates.io-index)",
"rustc_cratesio_shim 0.0.0",
"rustc_data_structures 0.0.0",
"serialize 0.0.0",
]
@ -2443,11 +2520,12 @@ name = "rustc_traits"
version = "0.0.0"
dependencies = [
"bitflags 1.0.4 (registry+https://github.com/rust-lang/crates.io-index)",
"chalk-engine 0.8.1 (registry+https://github.com/rust-lang/crates.io-index)",
"chalk-engine 0.8.0 (registry+https://github.com/rust-lang/crates.io-index)",
"graphviz 0.0.0",
"log 0.4.5 (registry+https://github.com/rust-lang/crates.io-index)",
"rustc 0.0.0",
"rustc_data_structures 0.0.0",
"rustc_target 0.0.0",
"smallvec 0.6.5 (registry+https://github.com/rust-lang/crates.io-index)",
"syntax 0.0.0",
"syntax_pos 0.0.0",
@ -2458,7 +2536,6 @@ name = "rustc_tsan"
version = "0.0.0"
dependencies = [
"alloc 0.0.0",
"alloc_system 0.0.0",
"build_helper 0.1.0",
"cmake 0.1.33 (registry+https://github.com/rust-lang/crates.io-index)",
"compiler_builtins 0.0.0",
@ -2692,12 +2769,11 @@ name = "std"
version = "0.0.0"
dependencies = [
"alloc 0.0.0",
"alloc_jemalloc 0.0.0",
"alloc_system 0.0.0",
"build_helper 0.1.0",
"cc 1.0.25 (registry+https://github.com/rust-lang/crates.io-index)",
"compiler_builtins 0.0.0",
"core 0.0.0",
"dlmalloc 0.0.0",
"libc 0.0.0",
"panic_abort 0.0.0",
"panic_unwind 0.0.0",
@ -2841,7 +2917,6 @@ version = "0.0.0"
dependencies = [
"fmt_macros 0.0.0",
"log 0.4.5 (registry+https://github.com/rust-lang/crates.io-index)",
"proc_macro 0.0.0",
"rustc_data_structures 0.0.0",
"rustc_errors 0.0.0",
"rustc_target 0.0.0",
@ -2864,7 +2939,7 @@ dependencies = [
[[package]]
name = "tar"
version = "0.4.16"
version = "0.4.19"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"filetime 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)",
@ -2931,6 +3006,7 @@ name = "test"
version = "0.0.0"
dependencies = [
"getopts 0.2.17 (registry+https://github.com/rust-lang/crates.io-index)",
"proc_macro 0.0.0",
"term 0.0.0",
]
@ -2989,6 +3065,11 @@ dependencies = [
"toml 0.4.6 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "typenum"
version = "1.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
[[package]]
name = "ucd-util"
version = "0.1.1"
@ -3216,7 +3297,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
"checksum cargo_metadata 0.6.2 (registry+https://github.com/rust-lang/crates.io-index)" = "7d8dfe3adeb30f7938e6c1dd5327f29235d8ada3e898aeb08c343005ec2915a2"
"checksum cc 1.0.25 (registry+https://github.com/rust-lang/crates.io-index)" = "f159dfd43363c4d08055a07703eb7a3406b0dac4d0584d96965a3262db3c9d16"
"checksum cfg-if 0.1.5 (registry+https://github.com/rust-lang/crates.io-index)" = "0c4e7bb64a8ebb0d856483e1e682ea3422f883c5f5615a90d51a2c82fe87fdd3"
"checksum chalk-engine 0.8.1 (registry+https://github.com/rust-lang/crates.io-index)" = "9adbe0fe1d6e937c3ee0571739a78f53c1de22f59df616060e868cf13c6c4ce5"
"checksum chalk-engine 0.8.0 (registry+https://github.com/rust-lang/crates.io-index)" = "6749eb72e7d4355d944a99f15fbaea701b978c18c5e184a025fcde942b0c9779"
"checksum chalk-macros 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)" = "295635afd6853aa9f20baeb7f0204862440c0fe994c5a253d5f479dac41d047e"
"checksum chrono 0.4.4 (registry+https://github.com/rust-lang/crates.io-index)" = "6962c635d530328acc53ac6a955e83093fedc91c5809dfac1fa60fa470830a37"
"checksum clap 2.32.0 (registry+https://github.com/rust-lang/crates.io-index)" = "b957d88f4b6a63b9d70d5f454ac8011819c6efa7727858f458ab71c756ce2d3e"
@ -3225,7 +3306,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
"checksum colored 1.6.0 (registry+https://github.com/rust-lang/crates.io-index)" = "b0aa3473e85a3161b59845d6096b289bb577874cafeaf75ea1b1beaa6572c7fc"
"checksum commoncrypto 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "d056a8586ba25a1e4d61cb090900e495952c7886786fc55f909ab2f819b69007"
"checksum commoncrypto-sys 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "1fed34f46747aa73dfaa578069fd8279d2818ade2b55f38f22a9401c7f4083e2"
"checksum compiletest_rs 0.3.13 (registry+https://github.com/rust-lang/crates.io-index)" = "d3064bc712922596dd5ab449fca9261d411893356581fe5297b96aa8f53bb1b8"
"checksum compiletest_rs 0.3.17 (registry+https://github.com/rust-lang/crates.io-index)" = "89747fe073b7838343bd2c2445e7a7c2e0d415598f8925f0fa9205b9cdfc48cb"
"checksum core-foundation 0.6.1 (registry+https://github.com/rust-lang/crates.io-index)" = "cc3532ec724375c7cb7ff0a097b714fde180bb1f6ed2ab27cfcd99ffca873cd2"
"checksum core-foundation-sys 0.6.1 (registry+https://github.com/rust-lang/crates.io-index)" = "a3fb15cdbdd9cf8b82d97d0296bb5cd3631bba58d6e31650a002a8e7fb5721f9"
"checksum crossbeam 0.3.2 (registry+https://github.com/rust-lang/crates.io-index)" = "24ce9782d4d5c53674646a6a4c1863a21a8fc0cb649b3c94dfc16e45071dea19"
@ -3235,18 +3316,22 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
"checksum crossbeam-epoch 0.6.0 (registry+https://github.com/rust-lang/crates.io-index)" = "9c90f1474584f38e270b5b613e898c8c328aa4f3dea85e0a27ac2e642f009416"
"checksum crossbeam-utils 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)" = "2760899e32a1d58d5abb31129f8fae5de75220bc2176e77ff7c627ae45c918d9"
"checksum crossbeam-utils 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)" = "677d453a17e8bd2b913fa38e8b9cf04bcdbb5be790aa294f2389661d72036015"
"checksum crossbeam-utils 0.6.1 (registry+https://github.com/rust-lang/crates.io-index)" = "c55913cc2799171a550e307918c0a360e8c16004820291bf3b638969b4a01816"
"checksum crypto-hash 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)" = "09de9ee0fc255ace04c7fa0763c9395a945c37c8292bb554f8d48361d1dcf1b4"
"checksum curl 0.4.18 (registry+https://github.com/rust-lang/crates.io-index)" = "a9e5285b49b44401518c947d3b808d14d99a538a6c9ffb3ec0205c11f9fc4389"
"checksum curl-sys 0.4.13 (registry+https://github.com/rust-lang/crates.io-index)" = "08459503c415173da1ce6b41036a37b8bfdd86af46d45abb9964d4c61fe670ef"
"checksum curl 0.4.19 (registry+https://github.com/rust-lang/crates.io-index)" = "c7c9d851c825e0c033979d4516c9173bc19a78a96eb4d6ae51d4045440eafa16"
"checksum curl-sys 0.4.15 (registry+https://github.com/rust-lang/crates.io-index)" = "721c204978be2143fab0a84b708c49d79d1f6100b8785610f456043a90708870"
"checksum datafrog 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)" = "16d724bf4ffe77cdceeecd461009b5f8d9e23c5d645d68bedb4586bf43e7e142"
"checksum derive-new 0.5.4 (registry+https://github.com/rust-lang/crates.io-index)" = "ceed73957c449214f8440eec8ad7fa282b67dc9eacbb24a3085b15d60397a17a"
"checksum derive_more 0.13.0 (registry+https://github.com/rust-lang/crates.io-index)" = "3f57d78cf3bd45270dad4e70c21ec77a960b36c7a841ff9db76aaa775a8fb871"
"checksum diff 0.1.11 (registry+https://github.com/rust-lang/crates.io-index)" = "3c2b69f912779fbb121ceb775d74d51e915af17aaebc38d28a592843a2dd0a3a"
"checksum difference 2.0.0 (registry+https://github.com/rust-lang/crates.io-index)" = "524cbf6897b527295dff137cec09ecf3a05f4fddffd7dfcd1585403449e74198"
"checksum directories 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)" = "72d337a64190607d4fcca2cb78982c5dd57f4916e19696b48a575fa746b6cb0f"
"checksum either 1.5.0 (registry+https://github.com/rust-lang/crates.io-index)" = "3be565ca5c557d7f59e7cfcf1844f9e3033650c929c6566f511e8005f205c1d0"
"checksum elasticlunr-rs 2.3.3 (registry+https://github.com/rust-lang/crates.io-index)" = "4837d77a1e157489a3933b743fd774ae75074e0e390b2b7f071530048a0d87ee"
"checksum ena 0.11.0 (registry+https://github.com/rust-lang/crates.io-index)" = "f56c93cc076508c549d9bb747f79aa9b4eb098be7b8cad8830c3137ef52d1e00"
"checksum ena 0.9.3 (registry+https://github.com/rust-lang/crates.io-index)" = "88dc8393b3c7352f94092497f6b52019643e493b6b890eb417cdb7c46117e621"
"checksum env_logger 0.5.12 (registry+https://github.com/rust-lang/crates.io-index)" = "f4d7e69c283751083d53d01eac767407343b8b69c4bd70058e08adc2637cb257"
"checksum env_logger 0.6.0 (registry+https://github.com/rust-lang/crates.io-index)" = "afb070faf94c85d17d50ca44f6ad076bce18ae92f0037d350947240a36e9d42e"
"checksum environment 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)" = "1f4b14e20978669064c33b4c1e0fb4083412e40fe56cbea2eae80fd7591503ee"
"checksum error-chain 0.11.0 (registry+https://github.com/rust-lang/crates.io-index)" = "ff511d5dc435d703f4971bc399647c9bc38e20cb41452e3b9feb4765419ed3f3"
"checksum error-chain 0.12.0 (registry+https://github.com/rust-lang/crates.io-index)" = "07e791d3be96241c77c43846b665ef1384606da2cd2a48730abe606a12906e02"
@ -3259,6 +3344,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
"checksum foreign-types 0.3.2 (registry+https://github.com/rust-lang/crates.io-index)" = "f6f339eb8adc052cd2ca78910fda869aefa38d22d5cb648e6485e4d3fc06f3b1"
"checksum foreign-types-shared 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)" = "00b0228411908ca8685dba7fc2cdd70ec9990a6e753e89b6ac91a84c40fbaf4b"
"checksum fs2 0.4.3 (registry+https://github.com/rust-lang/crates.io-index)" = "9564fc758e15025b46aa6643b1b77d047d1a56a1aea6e01002ac0c7026876213"
"checksum fs_extra 1.1.0 (registry+https://github.com/rust-lang/crates.io-index)" = "5f2a4a2034423744d2cc7ca2068453168dcdb82c438419e639a26bd87839c674"
"checksum fst 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)" = "d94485a00b1827b861dd9d1a2cc9764f9044d4c535514c0760a5a2012ef3399f"
"checksum fuchsia-zircon 0.3.3 (registry+https://github.com/rust-lang/crates.io-index)" = "2e9763c69ebaae630ba35f74888db465e49e259ba1bc0eda7d06f4a067615d82"
"checksum fuchsia-zircon-sys 0.3.3 (registry+https://github.com/rust-lang/crates.io-index)" = "3dcaa9ae7725d12cdb85b3ad99a434db70b468c09ded17e012d86b5c1010f7a7"
@ -3278,9 +3364,11 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
"checksum idna 0.1.5 (registry+https://github.com/rust-lang/crates.io-index)" = "38f09e0f0b1fb55fdee1f17470ad800da77af5186a1a76c026b679358b7e844e"
"checksum if_chain 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)" = "4bac95d9aa0624e7b78187d6fb8ab012b41d9f6f54b1bcb61e61c4845f8357ec"
"checksum ignore 0.4.3 (registry+https://github.com/rust-lang/crates.io-index)" = "3e9faa7c84064f07b40da27044af629f578bc7994b650d3e458d0c29183c1d91"
"checksum im-rc 12.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "d4591152fd573cf453a890b5f9fdc5c328a751a0785539316739d5f85e5c468c"
"checksum is-match 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)" = "7e5b386aef33a1c677be65237cb9d32c3f3ef56bd035949710c4bb13083eb053"
"checksum itertools 0.7.8 (registry+https://github.com/rust-lang/crates.io-index)" = "f58856976b776fedd95533137617a02fb25719f40e7d9b01c7043cd65474f450"
"checksum itoa 0.4.3 (registry+https://github.com/rust-lang/crates.io-index)" = "1306f3464951f30e30d12373d31c79fbd52d236e5e896fd92f96ec7babbbe60b"
"checksum jemalloc-sys 0.1.8 (registry+https://github.com/rust-lang/crates.io-index)" = "bfc62c8e50e381768ce8ee0428ee53741929f7ebd73e4d83f669bcf7693e00ae"
"checksum jobserver 0.1.11 (registry+https://github.com/rust-lang/crates.io-index)" = "60af5f849e1981434e4a31d3d782c4774ae9b434ce55b101a96ecfd09147e8be"
"checksum json 0.11.13 (registry+https://github.com/rust-lang/crates.io-index)" = "9ad0485404155f45cce53a40d4b2d6ac356418300daed05273d9e26f91c390be"
"checksum jsonrpc-core 8.0.1 (registry+https://github.com/rust-lang/crates.io-index)" = "ddf83704f4e79979a424d1082dd2c1e52683058056c9280efa19ac5f6bc9033c"
@ -3342,10 +3430,12 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
"checksum polonius-engine 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)" = "a5b6b0a7f5f4278b991ffd14abce1d01b013121ad297460237ef0a2f08d43201"
"checksum precomputed-hash 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)" = "925383efa346730478fb4838dbe9137d2a47675ad789c546d150a6e1dd4ab31c"
"checksum pretty_assertions 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)" = "3a029430f0d744bc3d15dd474d591bed2402b645d024583082b9f63bb936dac6"
"checksum pretty_env_logger 0.2.5 (registry+https://github.com/rust-lang/crates.io-index)" = "ed8d1e63042e889b85228620629b51c011d380eed2c7e0015f8a644def280c28"
"checksum proc-macro2 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)" = "1b06e2f335f48d24442b35a19df506a835fb3547bc3c06ef27340da9acf5cae7"
"checksum proc-macro2 0.4.24 (registry+https://github.com/rust-lang/crates.io-index)" = "77619697826f31a02ae974457af0b29b723e5619e113e9397b8b82c6bd253f09"
"checksum proptest 0.8.7 (registry+https://github.com/rust-lang/crates.io-index)" = "926d0604475349f463fe44130aae73f2294b5309ab2ca0310b998bd334ef191f"
"checksum pulldown-cmark 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)" = "d6fdf85cda6cadfae5428a54661d431330b312bc767ddbc57adbedc24da66e32"
"checksum pulldown-cmark 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "eef52fac62d0ea7b9b4dc7da092aa64ea7ec3d90af6679422d3d7e0e14b6ee15"
"checksum quick-error 1.2.2 (registry+https://github.com/rust-lang/crates.io-index)" = "9274b940887ce9addde99c4eee6b5c44cc494b182b97e73dc8ffdcb3397fd3f0"
"checksum quine-mc_cluskey 0.2.4 (registry+https://github.com/rust-lang/crates.io-index)" = "07589615d719a60c8dd8a4622e7946465dfef20d1a428f969e3443e7386d5f45"
"checksum quote 0.3.15 (registry+https://github.com/rust-lang/crates.io-index)" = "7a6e920b65c65f10b2ae65c831a81a073a89edd28c7cce89475bff467ab4167a"
@ -3416,7 +3506,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
"checksum syn 0.15.21 (registry+https://github.com/rust-lang/crates.io-index)" = "816b7af21405b011a23554ea2dc3f6576dc86ca557047c34098c1d741f10f823"
"checksum synom 0.11.3 (registry+https://github.com/rust-lang/crates.io-index)" = "a393066ed9010ebaed60b9eafa373d4b1baac186dd7e008555b0f702b51945b6"
"checksum synstructure 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)" = "85bb9b7550d063ea184027c9b8c20ac167cd36d3e06b3a40bceb9d746dc1a7b7"
"checksum tar 0.4.16 (registry+https://github.com/rust-lang/crates.io-index)" = "e8f41ca4a5689f06998f0247fcb60da6c760f1950cc9df2a10d71575ad0b062a"
"checksum tar 0.4.19 (registry+https://github.com/rust-lang/crates.io-index)" = "69e16840a1e0a1f1a880b739ef1cc6a4b85496c99b8aa786ccffce6e0c15624c"
"checksum tempfile 3.0.3 (registry+https://github.com/rust-lang/crates.io-index)" = "c4b103c6d08d323b92ff42c8ce62abcd83ca8efa7fd5bf7927efefec75f58c76"
"checksum tendril 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)" = "9de21546595a0873061940d994bbbc5c35f024ae4fd61ec5c5b159115684f508"
"checksum term 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)" = "5e6b677dd1e8214ea1ef4297f85dbcbed8e8cdddb561040cc998ca2551c37561"
@ -3427,6 +3517,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
"checksum time 0.1.40 (registry+https://github.com/rust-lang/crates.io-index)" = "d825be0eb33fda1a7e68012d51e9c7f451dc1a69391e7fdc197060bb8c56667b"
"checksum toml 0.4.6 (registry+https://github.com/rust-lang/crates.io-index)" = "a0263c6c02c4db6c8f7681f9fd35e90de799ebd4cfdeab77a38f4ff6b3d8c0d9"
"checksum toml-query 0.6.0 (registry+https://github.com/rust-lang/crates.io-index)" = "6854664bfc6df0360c695480836ee90e2d0c965f06db291d10be9344792d43e8"
"checksum typenum 1.10.0 (registry+https://github.com/rust-lang/crates.io-index)" = "612d636f949607bdf9b123b4a6f6d966dedf3ff669f7f045890d3a4a73948169"
"checksum ucd-util 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)" = "fd2be2d6639d0f8fe6cdda291ad456e23629558d466e2789d2c3e9892bda285d"
"checksum unicode-bidi 0.3.4 (registry+https://github.com/rust-lang/crates.io-index)" = "49f2bd0c6468a8230e1db229cff8029217cf623c767ea5d60bfbd42729ea54d5"
"checksum unicode-normalization 0.1.7 (registry+https://github.com/rust-lang/crates.io-index)" = "6a0180bc61fc5a987082bfa111f4cc95c4caff7f9799f3e46df09163a937aa25"

63
src/Cargo.toml → Cargo.toml
View File

@ -1,31 +1,34 @@
[workspace]
members = [
"bootstrap",
"rustc",
"libstd",
"libtest",
"librustc_codegen_llvm",
"tools/cargotest",
"tools/clippy",
"tools/compiletest",
"tools/error_index_generator",
"tools/linkchecker",
"tools/rustbook",
"tools/unstable-book-gen",
"tools/tidy",
"tools/build-manifest",
"tools/remote-test-client",
"tools/remote-test-server",
"tools/rust-installer",
"tools/cargo",
"tools/rustdoc",
"tools/rls",
"tools/rustfmt",
"tools/miri",
"tools/rustdoc-themes",
"src/bootstrap",
"src/rustc",
"src/libstd",
"src/libtest",
"src/librustc_codegen_llvm",
"src/tools/cargotest",
"src/tools/clippy",
"src/tools/compiletest",
"src/tools/error_index_generator",
"src/tools/linkchecker",
"src/tools/rustbook",
"src/tools/unstable-book-gen",
"src/tools/tidy",
"src/tools/build-manifest",
"src/tools/remote-test-client",
"src/tools/remote-test-server",
"src/tools/rust-installer",
"src/tools/cargo",
"src/tools/rustdoc",
"src/tools/rls",
"src/tools/rustfmt",
"src/tools/miri",
"src/tools/rustdoc-themes",
]
exclude = [
"tools/rls/test_data",
"src/tools/rls/test_data",
"build",
# HACK(eddyb) This hardcodes the fact that our CI uses `/checkout/obj`.
"obj",
]
# Curiously, LLVM 7.0 will segfault if compiled with opt-level=3
@ -50,18 +53,18 @@ debug-assertions = false
# so we use a `[patch]` here to override the github repository with our local
# vendored copy.
[patch."https://github.com/rust-lang/cargo"]
cargo = { path = "tools/cargo" }
cargo = { path = "src/tools/cargo" }
[patch.crates-io]
# Similar to Cargo above we want the RLS to use a vendored version of `rustfmt`
# that we're shipping as well (to ensure that the rustfmt in RLS and the
# `rustfmt` executable are the same exact version).
rustfmt-nightly = { path = "tools/rustfmt" }
rustfmt-nightly = { path = "src/tools/rustfmt" }
# See comments in `tools/rustc-workspace-hack/README.md` for what's going on
# See comments in `src/tools/rustc-workspace-hack/README.md` for what's going on
# here
rustc-workspace-hack = { path = 'tools/rustc-workspace-hack' }
rustc-workspace-hack = { path = 'src/tools/rustc-workspace-hack' }
[patch."https://github.com/rust-lang/rust-clippy"]
clippy_lints = { path = "tools/clippy/clippy_lints" }
rustc_tools_util = { path = "tools/clippy/rustc_tools_util" }
clippy_lints = { path = "src/tools/clippy/clippy_lints" }
rustc_tools_util = { path = "src/tools/clippy/rustc_tools_util" }

2
README.md
View File

@ -233,7 +233,7 @@ Also, you may find the [rustdocs for the compiler itself][rustdocs] useful.
[IRC]: https://en.wikipedia.org/wiki/Internet_Relay_Chat
[#rust]: irc://irc.mozilla.org/rust
[#rust-beginners]: irc://irc.mozilla.org/rust-beginners
[rustc guide]: https://rust-lang-nursery.github.io/rustc-guide/about-this-guide.html
[rustc guide]: https://rust-lang.github.io/rustc-guide/about-this-guide.html
[rustdocs]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/
## License

2
RELEASES.md
View File

@ -183,7 +183,7 @@ Misc
[cargo/5877]: https://github.com/rust-lang/cargo/pull/5877/
[cargo/5878]: https://github.com/rust-lang/cargo/pull/5878/
[cargo/5995]: https://github.com/rust-lang/cargo/pull/5995/
[proc-macros]: https://doc.rust-lang.org/stable/book/2018-edition/ch19-06-macros.html
[proc-macros]: https://doc.rust-lang.org/nightly/book/2018-edition/ch19-06-macros.html
[`Ipv4Addr::BROADCAST`]: https://doc.rust-lang.org/nightly/std/net/struct.Ipv4Addr.html#associatedconstant.BROADCAST
[`Ipv4Addr::LOCALHOST`]: https://doc.rust-lang.org/nightly/std/net/struct.Ipv4Addr.html#associatedconstant.LOCALHOST

23
config.toml.example
View File

@ -277,6 +277,10 @@
# compiler.
#codegen-units = 1
# Sets the number of codegen units to build the standard library with,
# regardless of what the codegen-unit setting for the rest of the compiler is.
#codegen-units-std = 1
# Whether or not debug assertions are enabled for the compiler and standard
# library. Also enables compilation of debug! and trace! logging macros.
#debug-assertions = false
@ -296,12 +300,6 @@
# Adding debuginfo makes them several times larger.
#debuginfo-tools = false
# Whether or not jemalloc is built and enabled
#use-jemalloc = true
# Whether or not jemalloc is built with its debug option set
#debug-jemalloc = false
# Whether or not `panic!`s generate backtraces (RUST_BACKTRACE)
#backtrace = true
@ -398,6 +396,15 @@
# generally only set for releases
#remap-debuginfo = false
# Link the compiler against `jemalloc`, where on Linux and OSX it should
# override the default allocator for rustc and LLVM.
#jemalloc = false
# Run tests in various test suites with the "nll compare mode" in addition to
# running the tests in normal mode. Largely only used on CI and during local
# development of NLL
#test-compare-mode = false
# =============================================================================
# Options for specific targets
#
@ -437,10 +444,6 @@
# not, you can specify an explicit file name for it.
#llvm-filecheck = "/path/to/FileCheck"
# Path to the custom jemalloc static library to link into the standard library
# by default. This is only used if jemalloc is still enabled above
#jemalloc = "/path/to/jemalloc/libjemalloc_pic.a"
# If this target is for Android, this option will be required to specify where
# the NDK for the target lives. This is used to find the C compiler to link and
# build native code.

2
git-commit-hash
View File

@ -1 +1 @@
abe02cefd6cd1916df62ad7dc80161bea50b72e8
a01e4761a1507939430d4044a5f0a35fdb2b146c

4
src/README.md
View File

@ -5,11 +5,11 @@ This directory contains the source code of the rust project, including:
For more information on how various parts of the compiler work, see the [rustc guide].
Their is also useful content in the following READMEs, which are gradually being moved over to the guide:
There is also useful content in the following READMEs, which are gradually being moved over to the guide:
- https://github.com/rust-lang/rust/tree/master/src/librustc/ty/query
- https://github.com/rust-lang/rust/tree/master/src/librustc/dep_graph
- https://github.com/rust-lang/rust/blob/master/src/librustc/infer/region_constraints
- https://github.com/rust-lang/rust/tree/master/src/librustc/infer/higher_ranked
- https://github.com/rust-lang/rust/tree/master/src/librustc/infer/lexical_region_resolve
[rustc guide]: https://rust-lang-nursery.github.io/rustc-guide/about-this-guide.html
[rustc guide]: https://rust-lang.github.io/rustc-guide/about-this-guide.html

23
src/bootstrap/bin/rustc.rs
View File

@ -129,10 +129,12 @@ fn main() {
// Help the libc crate compile by assisting it in finding the MUSL
// native libraries.
if let Some(s) = env::var_os("MUSL_ROOT") {
let mut root = OsString::from("native=");
root.push(&s);
root.push("/lib");
cmd.arg("-L").arg(&root);
if target.contains("musl") {
let mut root = OsString::from("native=");
root.push(&s);
root.push("/lib");
cmd.arg("-L").arg(&root);
}
}
// Override linker if necessary.
@ -232,7 +234,9 @@ fn main() {
// flesh out rpath support more fully in the future.
cmd.arg("-Z").arg("osx-rpath-install-name");
Some("-Wl,-rpath,@loader_path/../lib")
} else if !target.contains("windows") && !target.contains("wasm32") {
} else if !target.contains("windows") &&
!target.contains("wasm32") &&
!target.contains("fuchsia") {
Some("-Wl,-rpath,$ORIGIN/../lib")
} else {
None
@ -253,8 +257,15 @@ fn main() {
// When running miri tests, we need to generate MIR for all libraries
if env::var("TEST_MIRI").ok().map_or(false, |val| val == "true") {
// The flags here should be kept in sync with `add_miri_default_args`
// in miri's `src/lib.rs`.
cmd.arg("-Zalways-encode-mir");
cmd.arg("-Zmir-emit-validate=1");
// These options are preferred by miri, to be able to perform better validation,
// but the bootstrap compiler might not understand them.
if stage != "0" {
cmd.arg("-Zmir-emit-retag");
cmd.arg("-Zmir-opt-level=0");
}
}
// Force all crates compiled by this compiler to (a) be unstable and (b)

11
src/bootstrap/bootstrap.py
View File

@ -79,8 +79,8 @@ def _download(path, url, probably_big, verbose, exception):
# see http://serverfault.com/questions/301128/how-to-download
if sys.platform == 'win32':
run(["PowerShell.exe", "/nologo", "-Command",
"(New-Object System.Net.WebClient)"
".DownloadFile('{}', '{}')".format(url, path)],
"[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12;",
"(New-Object System.Net.WebClient).DownloadFile('{}', '{}')".format(url, path)],
verbose=verbose,
exception=exception)
else:
@ -715,11 +715,6 @@ class RustBuild(object):
backends = self.get_toml('codegen-backends')
if backends is None or not 'emscripten' in backends:
continue
if module.endswith("jemalloc"):
if self.get_toml('use-jemalloc') == 'false':
continue
if self.get_toml('jemalloc'):
continue
if module.endswith("lld"):
config = self.get_toml('lld')
if config is None or config == 'false':
@ -806,7 +801,7 @@ def bootstrap(help_triggered):
registry = 'https://example.com'
[source.vendored-sources]
directory = '{}/src/vendor'
directory = '{}/vendor'
""".format(build.rust_root))
else:
if os.path.exists('.cargo'):

20
src/bootstrap/builder.rs
View File

@ -390,7 +390,6 @@ impl<'a> Builder<'a> {
test::RunPassFullDeps,
test::RunFailFullDeps,
test::CompileFailFullDeps,
test::IncrementalFullDeps,
test::Rustdoc,
test::Pretty,
test::RunPassPretty,
@ -714,7 +713,7 @@ impl<'a> Builder<'a> {
"build" => self.cargo_out(compiler, mode, target),
// This is the intended out directory for crate documentation.
"doc" => self.crate_doc_out(target),
"doc" | "rustdoc" => self.crate_doc_out(target),
_ => self.stage_out(compiler, mode),
};
@ -743,7 +742,7 @@ impl<'a> Builder<'a> {
_ => compile::librustc_stamp(self, cmp, target),
};
if cmd == "doc" {
if cmd == "doc" || cmd == "rustdoc" {
if mode == Mode::Rustc || mode == Mode::ToolRustc || mode == Mode::Codegen {
// This is the intended out directory for compiler documentation.
my_out = self.compiler_doc_out(target);
@ -883,7 +882,7 @@ impl<'a> Builder<'a> {
.env("RUSTDOC", self.out.join("bootstrap/debug/rustdoc"))
.env(
"RUSTDOC_REAL",
if cmd == "doc" || (cmd == "test" && want_rustdoc) {
if cmd == "doc" || cmd == "rustdoc" || (cmd == "test" && want_rustdoc) {
self.rustdoc(compiler.host)
} else {
PathBuf::from("/path/to/nowhere/rustdoc/not/required")
@ -1120,10 +1119,15 @@ impl<'a> Builder<'a> {
cargo.arg("-v");
}
// This must be kept before the thinlto check, as we set codegen units
// to 1 forcibly there.
if let Some(n) = self.config.rust_codegen_units {
cargo.env("RUSTC_CODEGEN_UNITS", n.to_string());
match (mode, self.config.rust_codegen_units_std, self.config.rust_codegen_units) {
(Mode::Std, Some(n), _) |
(Mode::Test, Some(n), _) |
(_, _, Some(n)) => {
cargo.env("RUSTC_CODEGEN_UNITS", n.to_string());
}
_ => {
// Don't set anything
}
}
if self.config.rust_optimize {

2
src/bootstrap/channel.rs
View File

@ -24,7 +24,7 @@ use Build;
use config::Config;
// The version number
pub const CFG_RELEASE_NUM: &str = "1.31.0";
pub const CFG_RELEASE_NUM: &str = "1.32.0";
pub struct GitInfo {
inner: Option<Info>,

34
src/bootstrap/compile.rs
View File

@ -22,7 +22,7 @@ use std::fs::{self, File};
use std::io::BufReader;
use std::io::prelude::*;
use std::path::{Path, PathBuf};
use std::process::{Command, Stdio};
use std::process::{Command, Stdio, exit};
use std::str;
use build_helper::{output, mtime, up_to_date};
@ -69,7 +69,7 @@ impl Step for Std {
if builder.config.keep_stage.contains(&compiler.stage) {
builder.info("Warning: Using a potentially old libstd. This may not behave well.");
builder.ensure(StdLink {
compiler: compiler,
compiler,
target_compiler: compiler,
target,
});
@ -158,16 +158,7 @@ pub fn std_cargo(builder: &Builder,
.arg("--manifest-path")
.arg(builder.src.join("src/rustc/compiler_builtins_shim/Cargo.toml"));
} else {
let mut features = builder.std_features();
// When doing a local rebuild we tell cargo that we're stage1 rather than
// stage0. This works fine if the local rust and being-built rust have the
// same view of what the default allocator is, but fails otherwise. Since
// we don't have a way to express an allocator preference yet, work
// around the issue in the case of a local rebuild with jemalloc disabled.
if compiler.stage == 0 && builder.local_rebuild && !builder.config.use_jemalloc {
features.push_str(" force_alloc_system");
}
let features = builder.std_features();
if compiler.stage != 0 && builder.config.sanitizers {
// This variable is used by the sanitizer runtime crates, e.g.
@ -188,11 +179,6 @@ pub fn std_cargo(builder: &Builder,
.arg("--manifest-path")
.arg(builder.src.join("src/libstd/Cargo.toml"));
if let Some(target) = builder.config.target_config.get(&target) {
if let Some(ref jemalloc) = target.jemalloc {
cargo.env("JEMALLOC_OVERRIDE", jemalloc);
}
}
if target.contains("musl") {
if let Some(p) = builder.musl_root(target) {
cargo.env("MUSL_ROOT", p);
@ -217,7 +203,7 @@ impl Step for StdLink {
/// Link all libstd rlibs/dylibs into the sysroot location.
///
/// Links those artifacts generated by `compiler` to a the `stage` compiler's
/// Links those artifacts generated by `compiler` to the `stage` compiler's
/// sysroot for the specified `host` and `target`.
///
/// Note that this assumes that `compiler` has already generated the libstd
@ -358,7 +344,7 @@ impl Step for Test {
if builder.config.keep_stage.contains(&compiler.stage) {
builder.info("Warning: Using a potentially old libtest. This may not behave well.");
builder.ensure(TestLink {
compiler: compiler,
compiler,
target_compiler: compiler,
target,
});
@ -480,7 +466,7 @@ impl Step for Rustc {
if builder.config.keep_stage.contains(&compiler.stage) {
builder.info("Warning: Using a potentially old librustc. This may not behave well.");
builder.ensure(RustcLink {
compiler: compiler,
compiler,
target_compiler: compiler,
target,
});
@ -750,7 +736,7 @@ pub fn build_codegen_backend(builder: &Builder,
// Pass down configuration from the LLVM build into the build of
// librustc_llvm and librustc_codegen_llvm.
if builder.is_rust_llvm(target) {
if builder.is_rust_llvm(target) && backend != "emscripten" {
cargo.env("LLVM_RUSTLLVM", "1");
}
cargo.env("LLVM_CONFIG", &llvm_config);
@ -816,8 +802,8 @@ fn copy_codegen_backends_to_sysroot(builder: &Builder,
let filename = file.file_name().unwrap().to_str().unwrap();
// change `librustc_codegen_llvm-xxxxxx.so` to `librustc_codegen_llvm-llvm.so`
let target_filename = {
let dash = filename.find("-").unwrap();
let dot = filename.find(".").unwrap();
let dash = filename.find('-').unwrap();
let dot = filename.find('.').unwrap();
format!("{}-{}{}",
&filename[..dash],
backend,
@ -1112,7 +1098,7 @@ pub fn run_cargo(builder: &Builder,
});
if !ok {
panic!("cargo must succeed");
exit(1);
}
// Ok now we need to actually find all the files listed in `toplevel`. We've

26
src/bootstrap/config.rs
View File

@ -58,6 +58,7 @@ pub struct Config {
pub ignore_git: bool,
pub exclude: Vec<PathBuf>,
pub rustc_error_format: Option<String>,
pub test_compare_mode: bool,
pub run_host_only: bool,
@ -95,6 +96,7 @@ pub struct Config {
// rust codegen options
pub rust_optimize: bool,
pub rust_codegen_units: Option<u32>,
pub rust_codegen_units_std: Option<u32>,
pub rust_debug_assertions: bool,
pub rust_debuginfo: bool,
pub rust_debuginfo_lines: bool,
@ -115,6 +117,7 @@ pub struct Config {
pub hosts: Vec<Interned<String>>,
pub targets: Vec<Interned<String>>,
pub local_rebuild: bool,
pub jemalloc: bool,
// dist misc
pub dist_sign_folder: Option<PathBuf>,
@ -122,8 +125,6 @@ pub struct Config {
pub dist_gpg_password_file: Option<PathBuf>,
// libstd features
pub debug_jemalloc: bool,
pub use_jemalloc: bool,
pub backtrace: bool, // support for RUST_BACKTRACE
pub wasm_syscall: bool,
@ -165,7 +166,6 @@ pub struct Target {
pub llvm_config: Option<PathBuf>,
/// Some(path to FileCheck) if one was specified.
pub llvm_filecheck: Option<PathBuf>,
pub jemalloc: Option<PathBuf>,
pub cc: Option<PathBuf>,
pub cxx: Option<PathBuf>,
pub ar: Option<PathBuf>,
@ -262,7 +262,7 @@ struct Llvm {
link_jobs: Option<u32>,
link_shared: Option<bool>,
version_suffix: Option<String>,
clang_cl: Option<String>
clang_cl: Option<String>,
}
#[derive(Deserialize, Default, Clone)]
@ -294,14 +294,13 @@ impl Default for StringOrBool {
struct Rust {
optimize: Option<bool>,
codegen_units: Option<u32>,
codegen_units_std: Option<u32>,
debug_assertions: Option<bool>,
debuginfo: Option<bool>,
debuginfo_lines: Option<bool>,
debuginfo_only_std: Option<bool>,
debuginfo_tools: Option<bool>,
experimental_parallel_queries: Option<bool>,
debug_jemalloc: Option<bool>,
use_jemalloc: Option<bool>,
backtrace: Option<bool>,
default_linker: Option<String>,
channel: Option<String>,
@ -327,6 +326,8 @@ struct Rust {
backtrace_on_ice: Option<bool>,
verify_llvm_ir: Option<bool>,
remap_debuginfo: Option<bool>,
jemalloc: Option<bool>,
test_compare_mode: Option<bool>,
}
/// TOML representation of how each build target is configured.
@ -335,7 +336,6 @@ struct Rust {
struct TomlTarget {
llvm_config: Option<String>,
llvm_filecheck: Option<String>,
jemalloc: Option<String>,
cc: Option<String>,
cxx: Option<String>,
ar: Option<String>,
@ -361,7 +361,6 @@ impl Config {
config.llvm_enabled = true;
config.llvm_optimize = true;
config.llvm_version_check = true;
config.use_jemalloc = true;
config.backtrace = true;
config.rust_optimize = true;
config.rust_optimize_tests = true;
@ -497,7 +496,6 @@ impl Config {
let mut debuginfo_only_std = None;
let mut debuginfo_tools = None;
let mut debug = None;
let mut debug_jemalloc = None;
let mut debuginfo = None;
let mut debug_assertions = None;
let mut optimize = None;
@ -539,12 +537,12 @@ impl Config {
debuginfo_tools = rust.debuginfo_tools;
optimize = rust.optimize;
ignore_git = rust.ignore_git;
debug_jemalloc = rust.debug_jemalloc;
set(&mut config.rust_optimize_tests, rust.optimize_tests);
set(&mut config.rust_debuginfo_tests, rust.debuginfo_tests);
set(&mut config.codegen_tests, rust.codegen_tests);
set(&mut config.rust_rpath, rust.rpath);
set(&mut config.use_jemalloc, rust.use_jemalloc);
set(&mut config.jemalloc, rust.jemalloc);
set(&mut config.test_compare_mode, rust.test_compare_mode);
set(&mut config.backtrace, rust.backtrace);
set(&mut config.channel, rust.channel.clone());
set(&mut config.rust_dist_src, rust.dist_src);
@ -580,6 +578,8 @@ impl Config {
Some(n) => config.rust_codegen_units = Some(n),
None => {}
}
config.rust_codegen_units_std = rust.codegen_units_std;
}
if let Some(ref t) = toml.target {
@ -592,9 +592,6 @@ impl Config {
if let Some(ref s) = cfg.llvm_filecheck {
target.llvm_filecheck = Some(config.src.join(s));
}
if let Some(ref s) = cfg.jemalloc {
target.jemalloc = Some(config.src.join(s));
}
if let Some(ref s) = cfg.android_ndk {
target.ndk = Some(config.src.join(s));
}
@ -640,7 +637,6 @@ impl Config {
config.rust_debuginfo_tools = debuginfo_tools.unwrap_or(false);
let default = debug == Some(true);
config.debug_jemalloc = debug_jemalloc.unwrap_or(default);
config.rust_debuginfo = debuginfo.unwrap_or(default);
config.rust_debug_assertions = debug_assertions.unwrap_or(default);

21
src/bootstrap/configure.py
View File

@ -40,7 +40,7 @@ def v(*args):
options.append(Option(*args, value=True))
o("debug", "rust.debug", "debug mode; disables optimization unless `--enable-optimize` given")
o("debug", "rust.debug", "enables debugging environment; does not affect optimization of bootstrapped code (use `--disable-optimize` for that)")
o("docs", "build.docs", "build standard library documentation")
o("compiler-docs", "build.compiler-docs", "build compiler documentation")
o("optimize-tests", "rust.optimize-tests", "build tests with optimizations")
@ -68,6 +68,7 @@ o("cargo-native-static", "build.cargo-native-static", "static native libraries i
o("profiler", "build.profiler", "build the profiler runtime")
o("emscripten", None, "compile the emscripten backend as well as LLVM")
o("full-tools", None, "enable all tools")
o("lld", "rust.lld", "build lld")
o("lldb", "rust.lldb", "build lldb")
o("missing-tools", "dist.missing-tools", "allow failures when building tools")
@ -82,7 +83,6 @@ o("debuginfo", "rust.debuginfo", "build with debugger metadata")
o("debuginfo-lines", "rust.debuginfo-lines", "build with line number debugger metadata")
o("debuginfo-only-std", "rust.debuginfo-only-std", "build only libstd with debugging information")
o("debuginfo-tools", "rust.debuginfo-tools", "build extended tools with debugging information")
o("debug-jemalloc", "rust.debug-jemalloc", "build jemalloc with --enable-debug --enable-fill")
v("save-toolstates", "rust.save-toolstates", "save build and test status of external tools into this file")
v("prefix", "install.prefix", "set installation prefix")
@ -99,7 +99,6 @@ v("llvm-root", None, "set LLVM root")
v("llvm-config", None, "set path to llvm-config")
v("llvm-filecheck", None, "set path to LLVM's FileCheck utility")
v("python", "build.python", "set path to python")
v("jemalloc-root", None, "set directory where libjemalloc_pic.a is located")
v("android-cross-path", "target.arm-linux-androideabi.android-ndk",
"Android NDK standalone path (deprecated)")
v("i686-linux-android-ndk", "target.i686-linux-android.android-ndk",
@ -148,7 +147,6 @@ v("default-linker", "rust.default-linker", "the default linker")
# Many of these are saved below during the "writing configuration" step
# (others are conditionally saved).
o("manage-submodules", "build.submodules", "let the build manage the git submodules")
o("jemalloc", "rust.use-jemalloc", "build liballoc with jemalloc")
o("full-bootstrap", "build.full-bootstrap", "build three compilers instead of two")
o("extended", "build.extended", "build an extended rust tool set")
@ -330,8 +328,6 @@ for key in known_args:
set('target.{}.llvm-config'.format(build()), value)
elif option.name == 'llvm-filecheck':
set('target.{}.llvm-filecheck'.format(build()), value)
elif option.name == 'jemalloc-root':
set('target.{}.jemalloc'.format(build()), value + '/libjemalloc_pic.a')
elif option.name == 'tools':
set('build.tools', value.split(','))
elif option.name == 'host':
@ -393,6 +389,13 @@ for target in configured_targets:
targets[target][0] = targets[target][0].replace("x86_64-unknown-linux-gnu", target)
def is_number(value):
try:
float(value)
return True
except ValueError:
return False
# Here we walk through the constructed configuration we have from the parsed
# command line arguments. We then apply each piece of configuration by
# basically just doing a `sed` to change the various configuration line to what
@ -406,7 +409,11 @@ def to_toml(value):
elif isinstance(value, list):
return '[' + ', '.join(map(to_toml, value)) + ']'
elif isinstance(value, str):
return "'" + value + "'"
# Don't put quotes around numeric values
if is_number(value):
return value
else:
return "'" + value + "'"
else:
raise RuntimeError('no toml')

13
src/bootstrap/dist.rs
View File

@ -851,7 +851,7 @@ impl Step for Src {
t!(fs::create_dir_all(&dst_src));
let src_files = [
"src/Cargo.lock",
"Cargo.lock",
];
// This is the reduced set of paths which will become the rust-src component
// (essentially libstd and all of its path dependencies)
@ -859,8 +859,6 @@ impl Step for Src {
"src/build_helper",
"src/dlmalloc",
"src/liballoc",
"src/liballoc_jemalloc",
"src/liballoc_system",
"src/libbacktrace",
"src/libcompiler_builtins",
"src/libcore",
@ -878,13 +876,12 @@ impl Step for Src {
"src/rustc/dlmalloc_shim",
"src/libtest",
"src/libterm",
"src/jemalloc",
"src/libprofiler_builtins",
"src/stdsimd",
"src/libproc_macro",
];
let std_src_dirs_exclude = [
"src/libcompiler_builtins/compiler-rt/test",
"src/jemalloc/test/unit",
];
copy_src_dirs(builder, &std_src_dirs[..], &std_src_dirs_exclude[..], &dst_src);
@ -911,7 +908,7 @@ impl Step for Src {
}
}
const CARGO_VENDOR_VERSION: &str = "0.1.4";
const CARGO_VENDOR_VERSION: &str = "0.1.19";
#[derive(Debug, PartialOrd, Ord, Copy, Clone, Hash, PartialEq, Eq)]
pub struct PlainSourceTarball;
@ -952,6 +949,8 @@ impl Step for PlainSourceTarball {
"configure",
"x.py",
"config.toml.example",
"Cargo.toml",
"Cargo.lock",
];
let src_dirs = [
"src",
@ -995,7 +994,7 @@ impl Step for PlainSourceTarball {
// Vendor all Cargo dependencies
let mut cmd = Command::new(&builder.initial_cargo);
cmd.arg("vendor")
.current_dir(&plain_dst_src.join("src"));
.current_dir(&plain_dst_src);
builder.run(&mut cmd);
}

41
src/bootstrap/doc.rs
View File

@ -410,14 +410,15 @@ impl Step for Standalone {
cmd.arg("--html-after-content").arg(&footer)
.arg("--html-before-content").arg(&version_info)
.arg("--html-in-header").arg(&favicon)
.arg("--markdown-no-toc")
.arg("--index-page").arg(&builder.src.join("src/doc/index.md"))
.arg("--markdown-playground-url")
.arg("https://play.rust-lang.org/")
.arg("-o").arg(&out)
.arg(&path);
if filename == "not_found.md" {
cmd.arg("--markdown-no-toc")
.arg("--markdown-css")
cmd.arg("--markdown-css")
.arg("https://doc.rust-lang.org/rust.css");
} else {
cmd.arg("--markdown-css").arg("rust.css");
@ -485,23 +486,31 @@ impl Step for Std {
// will also directly handle merging.
let my_out = builder.crate_doc_out(target);
t!(symlink_dir_force(&builder.config, &my_out, &out_dir));
t!(fs::copy(builder.src.join("src/doc/rust.css"), out.join("rust.css")));
let mut cargo = builder.cargo(compiler, Mode::Std, target, "doc");
compile::std_cargo(builder, &compiler, target, &mut cargo);
let run_cargo_rustdoc_for = |package: &str| {
let mut cargo = builder.cargo(compiler, Mode::Std, target, "rustdoc");
compile::std_cargo(builder, &compiler, target, &mut cargo);
// Keep a whitelist so we do not build internal stdlib crates, these will be
// build by the rustc step later if enabled.
cargo.arg("--no-deps");
for krate in &["alloc", "core", "std"] {
cargo.arg("-p").arg(krate);
// Keep a whitelist so we do not build internal stdlib crates, these will be
// build by the rustc step later if enabled.
cargo.arg("-Z").arg("unstable-options")
.arg("-p").arg(package);
// Create all crate output directories first to make sure rustdoc uses
// relative links.
// FIXME: Cargo should probably do this itself.
t!(fs::create_dir_all(out_dir.join(krate)));
}
t!(fs::create_dir_all(out_dir.join(package)));
cargo.arg("--")
.arg("--markdown-css").arg("rust.css")
.arg("--markdown-no-toc")
.arg("--index-page").arg(&builder.src.join("src/doc/index.md"));
builder.run(&mut cargo);
builder.cp_r(&my_out, &out);
builder.run(&mut cargo);
builder.cp_r(&my_out, &out);
};
for krate in &["alloc", "core", "std"] {
run_cargo_rustdoc_for(krate);
}
}
}
@ -906,13 +915,13 @@ fn symlink_dir_force(config: &Config, src: &Path, dst: &Path) -> io::Result<()>
}
if let Ok(m) = fs::symlink_metadata(dst) {
if m.file_type().is_dir() {
try!(fs::remove_dir_all(dst));
fs::remove_dir_all(dst)?;
} else {
// handle directory junctions on windows by falling back to
// `remove_dir`.
try!(fs::remove_file(dst).or_else(|_| {
fs::remove_file(dst).or_else(|_| {
fs::remove_dir(dst)
}));
})?;
}
}

19
src/bootstrap/flags.rs
View File

@ -93,8 +93,7 @@ impl Default for Subcommand {
impl Flags {
pub fn parse(args: &[String]) -> Flags {
let mut extra_help = String::new();
let mut subcommand_help = format!(
"\
let mut subcommand_help = String::from("\
Usage: x.py <subcommand> [options] [<paths>...]
Subcommands:
@ -365,8 +364,8 @@ Arguments:
}
let cmd = match subcommand.as_str() {
"build" => Subcommand::Build { paths: paths },
"check" => Subcommand::Check { paths: paths },
"build" => Subcommand::Build { paths },
"check" => Subcommand::Check { paths },
"test" => Subcommand::Test {
paths,
bless: matches.opt_present("bless"),
@ -386,9 +385,9 @@ Arguments:
paths,
test_args: matches.opt_strs("test-args"),
},
"doc" => Subcommand::Doc { paths: paths },
"doc" => Subcommand::Doc { paths },
"clean" => {
if paths.len() > 0 {
if !paths.is_empty() {
println!("\nclean does not take a path argument\n");
usage(1, &opts, &subcommand_help, &extra_help);
}
@ -413,11 +412,11 @@ Arguments:
keep_stage: matches.opt_strs("keep-stage")
.into_iter().map(|j| j.parse().unwrap())
.collect(),
host: split(matches.opt_strs("host"))
host: split(&matches.opt_strs("host"))
.into_iter()
.map(|x| INTERNER.intern_string(x))
.collect::<Vec<_>>(),
target: split(matches.opt_strs("target"))
target: split(&matches.opt_strs("target"))
.into_iter()
.map(|x| INTERNER.intern_string(x))
.collect::<Vec<_>>(),
@ -425,7 +424,7 @@ Arguments:
jobs: matches.opt_str("jobs").map(|j| j.parse().unwrap()),
cmd,
incremental: matches.opt_present("incremental"),
exclude: split(matches.opt_strs("exclude"))
exclude: split(&matches.opt_strs("exclude"))
.into_iter()
.map(|p| p.into())
.collect::<Vec<_>>(),
@ -488,7 +487,7 @@ impl Subcommand {
}
}
fn split(s: Vec<String>) -> Vec<String> {
fn split(s: &[String]) -> Vec<String> {
s.iter()
.flat_map(|s| s.split(','))
.map(|s| s.to_string())

23
src/bootstrap/lib.rs
View File

@ -516,12 +516,6 @@ impl Build {
fn std_features(&self) -> String {
let mut features = "panic-unwind".to_string();
if self.config.debug_jemalloc {
features.push_str(" debug-jemalloc");
}
if self.config.use_jemalloc {
features.push_str(" jemalloc");
}
if self.config.backtrace {
features.push_str(" backtrace");
}
@ -537,8 +531,8 @@ impl Build {
/// Get the space-separated set of activated features for the compiler.
fn rustc_features(&self) -> String {
let mut features = String::new();
if self.config.use_jemalloc {
features.push_str(" jemalloc");
if self.config.jemalloc {
features.push_str("jemalloc");
}
features
}
@ -768,7 +762,7 @@ impl Build {
let sha = self.rust_sha().unwrap_or(channel::CFG_RELEASE_NUM);
format!("/rustc/{}", sha)
}
GitRepo::Llvm => format!("/rustc/llvm"),
GitRepo::Llvm => String::from("/rustc/llvm"),
};
Some(format!("{}={}", self.src.display(), path))
}
@ -786,12 +780,12 @@ impl Build {
let mut base = self.cc[&target].args().iter()
.map(|s| s.to_string_lossy().into_owned())
.filter(|s| !s.starts_with("-O") && !s.starts_with("/O"))
.collect::<Vec<_>>();
.collect::<Vec<String>>();
// If we're compiling on macOS then we add a few unconditional flags
// indicating that we want libc++ (more filled out than libstdc++) and
// we want to compile for 10.7. This way we can ensure that
// LLVM/jemalloc/etc are all properly compiled.
// LLVM/etc are all properly compiled.
if target.contains("apple-darwin") {
base.push("-stdlib=libc++".into());
}
@ -806,10 +800,10 @@ impl Build {
if let Some(map) = self.debuginfo_map(which) {
let cc = self.cc(target);
if cc.ends_with("clang") || cc.ends_with("gcc") {
base.push(format!("-fdebug-prefix-map={}", map).into());
base.push(format!("-fdebug-prefix-map={}", map));
} else if cc.ends_with("clang-cl.exe") {
base.push("-Xclang".into());
base.push(format!("-fdebug-prefix-map={}", map).into());
base.push(format!("-fdebug-prefix-map={}", map));
}
}
base
@ -843,7 +837,8 @@ impl Build {
} else if target != self.config.build &&
!target.contains("msvc") &&
!target.contains("emscripten") &&
!target.contains("wasm32") {
!target.contains("wasm32") &&
!target.contains("fuchsia") {
Some(self.cc(target))
} else {
None

4
src/bootstrap/native.rs
View File

@ -353,7 +353,7 @@ fn configure_cmake(builder: &Builder,
// definitely causes problems since all the env vars are pointing to
// 32-bit libraries.
//
// To hack aroudn this... again... we pass an argument that's
// To hack around this... again... we pass an argument that's
// unconditionally passed in the sccache shim. This'll get CMake to
// correctly diagnose it's doing a 32-bit compilation and LLVM will
// internally configure itself appropriately.
@ -361,7 +361,7 @@ fn configure_cmake(builder: &Builder,
cfg.env("SCCACHE_EXTRA_ARGS", "-m32");
}
// If ccache is configured we inform the build a little differently hwo
// If ccache is configured we inform the build a little differently how
// to invoke ccache while also invoking our compilers.
} else if let Some(ref ccache) = builder.config.ccache {
cfg.define("CMAKE_C_COMPILER", ccache)

8
src/bootstrap/sanity.rs
View File

@ -74,7 +74,7 @@ pub fn check(build: &mut Build) {
// one is present as part of the PATH then that can lead to the system
// being unable to identify the files properly. See
// https://github.com/rust-lang/rust/issues/34959 for more details.
if cfg!(windows) && path.to_string_lossy().contains("\"") {
if cfg!(windows) && path.to_string_lossy().contains('\"') {
panic!("PATH contains invalid character '\"'");
}
@ -152,12 +152,6 @@ pub fn check(build: &mut Build) {
if !build.config.dry_run {
cmd_finder.must_have(build.cxx(*host).unwrap());
}
// The msvc hosts don't use jemalloc, turn it off globally to
// avoid packaging the dummy liballoc_jemalloc on that platform.
if host.contains("msvc") {
build.config.use_jemalloc = false;
}
}
// Externally configured LLVM requires FileCheck to exist

50
src/bootstrap/test.rs
View File

@ -521,7 +521,7 @@ impl Step for RustdocTheme {
fn make_run(run: RunConfig) {
let compiler = run.builder.compiler(run.builder.top_stage, run.host);
run.builder.ensure(RustdocTheme { compiler: compiler });
run.builder.ensure(RustdocTheme { compiler });
}
fn run(self, builder: &Builder) {
@ -584,9 +584,9 @@ impl Step for RustdocJS {
});
builder.run(&mut command);
} else {
builder.info(&format!(
builder.info(
"No nodejs found, skipping \"src/test/rustdoc-js\" tests"
));
);
}
}
}
@ -653,7 +653,7 @@ impl Step for Tidy {
}
let _folder = builder.fold_output(|| "tidy");
builder.info(&format!("tidy check"));
builder.info("tidy check");
try_run(builder, &mut cmd);
}
@ -839,12 +839,6 @@ host_test!(CompileFailFullDeps {
suite: "compile-fail-fulldeps"
});
host_test!(IncrementalFullDeps {
path: "src/test/incremental-fulldeps",
mode: "incremental",
suite: "incremental-fulldeps"
});
host_test!(Rustdoc {
path: "src/test/rustdoc",
mode: "rustdoc",
@ -982,6 +976,11 @@ impl Step for Compiletest {
builder.ensure(compile::Std { compiler, target: compiler.host });
}
// HACK(eddyb) ensure that `libproc_macro` is available on the host.
builder.ensure(compile::Test { compiler, target: compiler.host });
// Also provide `rust_test_helpers` for the host.
builder.ensure(native::TestHelpers { target: compiler.host });
builder.ensure(native::TestHelpers { target });
builder.ensure(RemoteCopyLibs { compiler, target });
@ -1023,7 +1022,13 @@ impl Step for Compiletest {
cmd.arg("--bless");
}
let compare_mode = builder.config.cmd.compare_mode().or(self.compare_mode);
let compare_mode = builder.config.cmd.compare_mode().or_else(|| {
if builder.config.test_compare_mode {
self.compare_mode
} else {
None
}
});
if let Some(ref nodejs) = builder.config.nodejs {
cmd.arg("--nodejs").arg(nodejs);
@ -1049,7 +1054,11 @@ impl Step for Compiletest {
cmd.arg("--linker").arg(linker);
}
let hostflags = flags.clone();
let mut hostflags = flags.clone();
hostflags.push(format!(
"-Lnative={}",
builder.test_helpers_out(compiler.host).display()
));
cmd.arg("--host-rustcflags").arg(hostflags.join(" "));
let mut targetflags = flags;
@ -1168,9 +1177,9 @@ impl Step for Compiletest {
}
}
if suite == "run-make-fulldeps" && !builder.config.llvm_enabled {
builder.info(&format!(
builder.info(
"Ignoring run-make test suite as they generally don't work without LLVM"
));
);
return;
}
@ -1504,8 +1513,7 @@ impl Step for CrateNotDefault {
type Output = ();
fn should_run(run: ShouldRun) -> ShouldRun {
run.path("src/liballoc_jemalloc")
.path("src/librustc_asan")
run.path("src/librustc_asan")
.path("src/librustc_lsan")
.path("src/librustc_msan")
.path("src/librustc_tsan")
@ -1522,7 +1530,6 @@ impl Step for CrateNotDefault {
target: run.target,
test_kind,
krate: match run.path {
_ if run.path.ends_with("src/liballoc_jemalloc") => "alloc_jemalloc",
_ if run.path.ends_with("src/librustc_asan") => "rustc_asan",
_ if run.path.ends_with("src/librustc_lsan") => "rustc_lsan",
_ if run.path.ends_with("src/librustc_msan") => "rustc_msan",
@ -1561,7 +1568,6 @@ impl Step for Crate {
run = run.krate("test");
for krate in run.builder.in_tree_crates("std") {
if krate.is_local(&run.builder)
&& !krate.name.contains("jemalloc")
&& !(krate.name.starts_with("rustc_") && krate.name.ends_with("san"))
&& krate.name != "dlmalloc"
{
@ -1692,10 +1698,10 @@ impl Step for Crate {
// The javascript shim implements the syscall interface so that test
// output can be correctly reported.
if !builder.config.wasm_syscall {
builder.info(&format!(
builder.info(
"Libstd was built without `wasm_syscall` feature enabled: \
test output may not be visible."
));
);
}
// On the wasm32-unknown-unknown target we're using LTO which is
@ -1891,7 +1897,7 @@ impl Step for Distcheck {
/// Run "distcheck", a 'make check' from a tarball
fn run(self, builder: &Builder) {
builder.info(&format!("Distcheck"));
builder.info("Distcheck");
let dir = builder.out.join("tmp").join("distcheck");
let _ = fs::remove_dir_all(&dir);
t!(fs::create_dir_all(&dir));
@ -1919,7 +1925,7 @@ impl Step for Distcheck {
);
// Now make sure that rust-src has all of libstd's dependencies
builder.info(&format!("Distcheck rust-src"));
builder.info("Distcheck rust-src");
let dir = builder.out.join("tmp").join("distcheck-src");
let _ = fs::remove_dir_all(&dir);
t!(fs::create_dir_all(&dir));

16
src/bootstrap/tool.rs
View File

@ -149,7 +149,7 @@ impl Step for ToolBuild {
}
});
if is_expected && duplicates.len() != 0 {
if is_expected && !duplicates.is_empty() {
println!("duplicate artfacts found when compiling a tool, this \
typically means that something was recompiled because \
a transitive dependency has different features activated \
@ -171,7 +171,7 @@ impl Step for ToolBuild {
println!(" `{}` additionally enabled features {:?} at {:?}",
prev.0, &prev_features - &cur_features, prev.1);
}
println!("");
println!();
println!("to fix this you will probably want to edit the local \
src/tools/rustc-workspace-hack/Cargo.toml crate, as \
that will update the dependency graph to ensure that \
@ -189,7 +189,7 @@ impl Step for ToolBuild {
if !is_optional_tool {
exit(1);
} else {
return None;
None
}
} else {
let cargo_out = builder.cargo_out(compiler, self.mode, target)
@ -253,15 +253,19 @@ pub fn prepare_tool_cargo(
if let Some(date) = info.commit_date() {
cargo.env("CFG_COMMIT_DATE", date);
}
if features.len() > 0 {
if !features.is_empty() {
cargo.arg("--features").arg(&features.join(", "));
}
cargo
}
macro_rules! tool {
($($name:ident, $path:expr, $tool_name:expr, $mode:expr
$(,llvm_tools = $llvm:expr)* $(,is_external_tool = $external:expr)*;)+) => {
($(
$name:ident, $path:expr, $tool_name:expr, $mode:expr
$(,llvm_tools = $llvm:expr)*
$(,is_external_tool = $external:expr)*
;
)+) => {
#[derive(Copy, PartialEq, Eq, Clone)]
pub enum Tool {
$(

6
src/bootstrap/util.rs
View File

@ -203,11 +203,11 @@ pub fn symlink_dir(config: &Config, src: &Path, dest: &Path) -> io::Result<()> {
// We're using low-level APIs to create the junction, and these are more
// picky about paths. For example, forward slashes cannot be used as a
// path separator, so we should try to canonicalize the path first.
let target = try!(fs::canonicalize(target));
let target = fs::canonicalize(target)?;
try!(fs::create_dir(junction));
fs::create_dir(junction)?;
let path = try!(to_u16s(junction));
let path = to_u16s(junction)?;
unsafe {
let h = CreateFileW(path.as_ptr(),

10
src/ci/docker/asmjs/Dockerfile
View File

@ -20,11 +20,11 @@ COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh
ENV PATH=$PATH:/emsdk-portable
ENV PATH=$PATH:/emsdk-portable/clang/e1.37.13_64bit/
ENV PATH=$PATH:/emsdk-portable/emscripten/1.37.13/
ENV PATH=$PATH:/emsdk-portable/node/4.1.1_64bit/bin/
ENV EMSCRIPTEN=/emsdk-portable/emscripten/1.37.13/
ENV BINARYEN_ROOT=/emsdk-portable/clang/e1.37.13_64bit/binaryen/
ENV PATH=$PATH:/emsdk-portable/clang/e1.38.15_64bit/
ENV PATH=$PATH:/emsdk-portable/emscripten/1.38.15/
ENV PATH=$PATH:/emsdk-portable/node/8.9.1_64bit/bin/
ENV EMSCRIPTEN=/emsdk-portable/emscripten/1.38.15/
ENV BINARYEN_ROOT=/emsdk-portable/clang/e1.38.15_64bit/binaryen/
ENV EM_CONFIG=/emsdk-portable/.emscripten
ENV TARGETS=asmjs-unknown-emscripten

26
src/ci/docker/disabled/dist-powerpcspe-linux/Dockerfile Normal file
View File

@ -0,0 +1,26 @@
FROM ubuntu:16.04
RUN apt-get update && apt-get install -y --no-install-recommends \
g++ \
make \
file \
curl \
ca-certificates \
python2.7 \
git \
cmake \
sudo \
gdb \
xz-utils \
g++-powerpc-linux-gnuspe \
libssl-dev \
pkg-config
COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh
ENV HOSTS=powerpc-unknown-linux-gnuspe
ENV RUST_CONFIGURE_ARGS --enable-extended --disable-docs
ENV SCRIPT python2.7 ../x.py dist --host $HOSTS --target $HOSTS

10
src/ci/docker/disabled/wasm32/Dockerfile
View File

@ -21,11 +21,11 @@ COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh
ENV PATH=$PATH:/emsdk-portable
ENV PATH=$PATH:/emsdk-portable/clang/e1.37.13_64bit/
ENV PATH=$PATH:/emsdk-portable/emscripten/1.37.13/
ENV PATH=$PATH:/node-v8.0.0-linux-x64/bin/
ENV EMSCRIPTEN=/emsdk-portable/emscripten/1.37.13/
ENV BINARYEN_ROOT=/emsdk-portable/clang/e1.37.13_64bit/binaryen/
ENV PATH=$PATH:/emsdk-portable/clang/e1.38.15_64bit/
ENV PATH=$PATH:/emsdk-portable/emscripten/1.38.15/
ENV PATH=$PATH:/emsdk-portable/node/8.9.1_64bit/bin/
ENV EMSCRIPTEN=/emsdk-portable/emscripten/1.38.15/
ENV BINARYEN_ROOT=/emsdk-portable/clang/e1.38.15_64bit/binaryen/
ENV EM_CONFIG=/emsdk-portable/.emscripten
ENV TARGETS=wasm32-unknown-emscripten

5
src/ci/docker/dist-i686-linux/Dockerfile
View File

@ -67,7 +67,7 @@ RUN ./build-gcc.sh
COPY dist-x86_64-linux/build-python.sh /tmp/
RUN ./build-python.sh
# Now build LLVM+Clang 6, afterwards configuring further compilations to use the
# Now build LLVM+Clang 7, afterwards configuring further compilations to use the
# clang/clang++ compilers.
COPY dist-x86_64-linux/build-clang.sh /tmp/
RUN ./build-clang.sh
@ -98,7 +98,8 @@ ENV RUST_CONFIGURE_ARGS \
--enable-sanitizers \
--enable-profiler \
--set target.i686-unknown-linux-gnu.linker=clang \
--build=i686-unknown-linux-gnu
--build=i686-unknown-linux-gnu \
--set rust.jemalloc
ENV SCRIPT python2.7 ../x.py dist --build $HOSTS --host $HOSTS --target $HOSTS
ENV CARGO_TARGET_I686_UNKNOWN_LINUX_GNU_LINKER=clang

3
src/ci/docker/dist-various-1/Dockerfile
View File

@ -22,7 +22,8 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
libssl-dev \
pkg-config \
gcc-arm-none-eabi \
libnewlib-arm-none-eabi
libnewlib-arm-none-eabi \
qemu-system-arm
WORKDIR /build

13
src/ci/docker/dist-various-2/Dockerfile
View File

@ -47,6 +47,17 @@ ENV \
CC_x86_64_sun_solaris=x86_64-sun-solaris2.10-gcc \
CXX_x86_64_sun_solaris=x86_64-sun-solaris2.10-g++
ENV CARGO_TARGET_X86_64_FUCHSIA_AR /usr/local/bin/llvm-ar
ENV CARGO_TARGET_X86_64_FUCHSIA_RUSTFLAGS \
-C link-arg=--sysroot=/usr/local/x86_64-fuchsia \
-C link-arg=-L/usr/local/x86_64-fuchsia/lib \
-C link-arg=-L/usr/local/lib/x86_64-fuchsia/lib
ENV CARGO_TARGET_AARCH64_FUCHSIA_AR /usr/local/bin/llvm-ar
ENV CARGO_TARGET_AARCH64_FUCHSIA_RUSTFLAGS \
-C link-arg=--sysroot=/usr/local/aarch64-fuchsia \
-C link-arg=-L/usr/local/aarch64-fuchsia/lib \
-C link-arg=-L/usr/local/lib/aarch64-fuchsia/lib
ENV TARGETS=x86_64-fuchsia
ENV TARGETS=$TARGETS,aarch64-fuchsia
ENV TARGETS=$TARGETS,sparcv9-sun-solaris
@ -55,5 +66,5 @@ ENV TARGETS=$TARGETS,x86_64-sun-solaris
ENV TARGETS=$TARGETS,x86_64-unknown-linux-gnux32
ENV TARGETS=$TARGETS,x86_64-unknown-cloudabi
ENV RUST_CONFIGURE_ARGS --enable-extended --disable-docs
ENV RUST_CONFIGURE_ARGS --enable-extended --enable-lld --disable-docs
ENV SCRIPT python2.7 ../x.py dist --target $TARGETS

5
src/ci/docker/dist-x86_64-linux/Dockerfile
View File

@ -67,7 +67,7 @@ RUN ./build-gcc.sh
COPY dist-x86_64-linux/build-python.sh /tmp/
RUN ./build-python.sh
# Now build LLVM+Clang 6, afterwards configuring further compilations to use the
# Now build LLVM+Clang 7, afterwards configuring further compilations to use the
# clang/clang++ compilers.
COPY dist-x86_64-linux/build-clang.sh /tmp/
RUN ./build-clang.sh
@ -101,7 +101,8 @@ ENV RUST_CONFIGURE_ARGS \
--set target.x86_64-unknown-linux-gnu.linker=clang \
--set target.x86_64-unknown-linux-gnu.ar=/rustroot/bin/llvm-ar \
--set target.x86_64-unknown-linux-gnu.ranlib=/rustroot/bin/llvm-ranlib \
--set llvm.thin-lto=true
--set llvm.thin-lto=true \
--set rust.jemalloc
ENV SCRIPT python2.7 ../x.py dist --host $HOSTS --target $HOSTS
ENV CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_LINKER=clang

2
src/ci/docker/dist-x86_64-linux/build-clang.sh
View File

@ -13,7 +13,7 @@ set -ex
source shared.sh
LLVM=6.0.0
LLVM=7.0.0
mkdir clang
cd clang

9
src/ci/docker/scripts/emscripten.sh
View File

@ -33,8 +33,8 @@ curl -fL https://s3.amazonaws.com/mozilla-games/emscripten/releases/emsdk-portab
cd /emsdk-portable
./emsdk update
hide_output ./emsdk install sdk-1.37.13-64bit
./emsdk activate sdk-1.37.13-64bit
hide_output ./emsdk install sdk-1.38.15-64bit
./emsdk activate sdk-1.38.15-64bit
# Compile and cache libc
source ./emsdk_env.sh
@ -46,8 +46,3 @@ rm -f a.*
# Make emsdk usable by any user
cp /root/.emscripten /emsdk-portable
chmod a+rxw -R /emsdk-portable
# node 8 is required to run wasm
cd /
curl -sL https://nodejs.org/dist/v8.0.0/node-v8.0.0-linux-x64.tar.xz | \
tar -xJ

2
src/ci/docker/scripts/musl.sh
View File

@ -51,7 +51,7 @@ hide_output make clean
cd ..
LLVM=60
LLVM=70
# may have been downloaded in a previous run
if [ ! -d libunwind-release_$LLVM ]; then

2
src/ci/docker/wasm32-unknown/Dockerfile
View File

@ -1,4 +1,4 @@
FROM ubuntu:16.04
FROM ubuntu:18.04
RUN apt-get update && apt-get install -y --no-install-recommends \
g++ \

4
src/ci/docker/x86_64-gnu-nopt/Dockerfile
View File

@ -16,5 +16,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh
ENV RUST_CONFIGURE_ARGS --build=x86_64-unknown-linux-gnu --disable-optimize-tests
ENV RUST_CONFIGURE_ARGS --build=x86_64-unknown-linux-gnu \
--disable-optimize-tests \
--set rust.test-compare-mode
ENV RUST_CHECK_TARGET check

2
src/ci/docker/x86_64-gnu/Dockerfile
View File

@ -1,4 +1,4 @@
FROM ubuntu:16.04
FROM ubuntu:18.10
RUN apt-get update && apt-get install -y --no-install-recommends \
g++ \

1
src/ci/init_repo.sh
View File

@ -55,6 +55,7 @@ function fetch_submodule {
}
included="src/llvm src/llvm-emscripten src/doc/book src/doc/rust-by-example"
included="$included src/tools/lld src/tools/clang src/tools/lldb"
modules="$(git config --file .gitmodules --get-regexp '\.path$' | cut -d' ' -f2)"
modules=($modules)
use_git=""

3
src/ci/run.sh
View File

@ -40,6 +40,7 @@ RUST_CONFIGURE_ARGS="$RUST_CONFIGURE_ARGS --enable-sccache"
RUST_CONFIGURE_ARGS="$RUST_CONFIGURE_ARGS --disable-manage-submodules"
RUST_CONFIGURE_ARGS="$RUST_CONFIGURE_ARGS --enable-locked-deps"
RUST_CONFIGURE_ARGS="$RUST_CONFIGURE_ARGS --enable-cargo-native-static"
RUST_CONFIGURE_ARGS="$RUST_CONFIGURE_ARGS --set rust.codegen-units-std=1"
if [ "$DIST_SRC" = "" ]; then
RUST_CONFIGURE_ARGS="$RUST_CONFIGURE_ARGS --disable-dist-src"
@ -51,7 +52,7 @@ fi
#
# FIXME: need a scheme for changing this `nightly` value to `beta` and `stable`
# either automatically or manually.
export RUST_RELEASE_CHANNEL=stable
export RUST_RELEASE_CHANNEL=beta
if [ "$DEPLOY$DEPLOY_ALT" != "" ]; then
RUST_CONFIGURE_ARGS="$RUST_CONFIGURE_ARGS --release-channel=$RUST_RELEASE_CHANNEL"
RUST_CONFIGURE_ARGS="$RUST_CONFIGURE_ARGS --enable-llvm-static-stdcpp"

3
src/doc/index.md
View File

@ -21,6 +21,9 @@ nav {
#search-but:hover, #search-input:focus {
border-color: #55a9ff;
}
h2 {
font-size: 18px;
}
</style>
Welcome to an overview of the documentation provided by the Rust project.

1
src/doc/nomicon/src/vec-final.md
View File

@ -3,6 +3,7 @@
```rust
#![feature(ptr_internals)]
#![feature(allocator_api)]
#![feature(alloc_layout_extra)]
use std::ptr::{Unique, NonNull, self};
use std::mem;

9
src/doc/rustc-guide/.editorconfig Normal file
View File

@ -0,0 +1,9 @@
root = true
[src/*]
end_of_line = lf
insert_final_newline = true
[ci/*.sh]
indent_style = space
indent_size = 2

25
src/doc/rustc-guide/.travis.yml Normal file
View File

@ -0,0 +1,25 @@
language: rust
cache:
- cargo
before_install:
- shopt -s globstar
- MAX_LINE_LENGTH=100 bash ci/check_line_lengths.sh src/**/*.md
install:
- source ~/.cargo/env || true
- bash ci/install.sh
script:
- mdbook build
- mdbook test
notifications:
email:
on_success: never
env:
global:
secure: YQX/AWq5KsvAFYqcCK6c1DmOZX9EMrecBM5qnc4uE2HvEBS+x0l8xatI2Nv8U9eiasZYfsqmHn0ANvxu6e4oqL15m4cVsdliCzdkrPsDapxTnwwJvMQg+yHZiEd5BPlaDQt/wYvP8QBXgQsXoAJKrfAS+BFsowBFHt/LOFOunbAQrtQZqwqrnI6+xh+2TRMckws/VcTLRqwl3pyEyfacJhbbv1V3gJh7Y17hELsgsP7+7cMXT0bK6dtf7a9vne9Hsm5fw7VeMKBn1/dJ82fyEK6HHjkjdw1/OoY35YVyNZ/9ZxP2u1ClEXzCRJQ2CvKr8Tuoh/AuoL0pwrfhOTaOuWU0QZT4QBqjTimsgBLqiJicMiSndgsXinLWvlDqrMS1XfleqCKqAQy9AJTCR1LnwR90/HRxfE5YDAL/mbc0Su4jj+l5Zv3UE8vUqFE34E/jzip17JkDT5aMkl4bgW65lqJE7SLWl7gXT7eYbPEtQZoucR1hkSsBu/4YTvcxSlD98spWZ68mWwYyjLJSQDES+GefUnHJ/RbBVl9pW+sL7jXJ+kZ/NBCtCIgrkGchudEMDEvS6rcOzwCejxqL1of0jYHGopkBXSVHOPneWIdNeKXwBZA9hp0yKh0sWwrKHrA3wYhS/kF9uO19l/RnSTXAfApYR/yJUbYliuMJYCgNeKE=
deploy:
provider: pages
skip-cleanup: true
github-token: $GITHUB_TOKEN
local-dir: book/html
on:
branch: master

38
src/doc/rustc-guide/CODE_OF_CONDUCT.md Normal file
View File

@ -0,0 +1,38 @@
# The Rust Code of Conduct
A version of this document [can be found online](https://www.rust-lang.org/conduct.html).
## Conduct
**Contact**: [rust-mods@rust-lang.org](mailto:rust-mods@rust-lang.org)
* We are committed to providing a friendly, safe and welcoming environment for all, regardless of level of experience, gender identity and expression, sexual orientation, disability, personal appearance, body size, race, ethnicity, age, religion, nationality, or other similar characteristic.
* On IRC, please avoid using overtly sexual nicknames or other nicknames that might detract from a friendly, safe and welcoming environment for all.
* Please be kind and courteous. There's no need to be mean or rude.
* Respect that people have differences of opinion and that every design or implementation choice carries a trade-off and numerous costs. There is seldom a right answer.
* Please keep unstructured critique to a minimum. If you have solid ideas you want to experiment with, make a fork and see how it works.
* We will exclude you from interaction if you insult, demean or harass anyone. That is not welcome behaviour. We interpret the term "harassment" as including the definition in the <a href="http://citizencodeofconduct.org/">Citizen Code of Conduct</a>; if you have any lack of clarity about what might be included in that concept, please read their definition. In particular, we don't tolerate behavior that excludes people in socially marginalized groups.
* Private harassment is also unacceptable. No matter who you are, if you feel you have been or are being harassed or made uncomfortable by a community member, please contact one of the channel ops or any of the [Rust moderation team](/team.html#Moderation) immediately. Whether you're a regular contributor or a newcomer, we care about making this community a safe place for you and we've got your back.
* Likewise any spamming, trolling, flaming, baiting or other attention-stealing behaviour is not welcome.
## Moderation
These are the policies for upholding our community's standards of conduct. If you feel that a thread needs moderation, please contact the [Rust moderation team](/team.html#Moderation).
1. Remarks that violate the Rust standards of conduct, including hateful, hurtful, oppressive, or exclusionary remarks, are not allowed. (Cursing is allowed, but never targeting another user, and never in a hateful manner.)
2. Remarks that moderators find inappropriate, whether listed in the code of conduct or not, are also not allowed.
3. Moderators will first respond to such remarks with a warning.
4. If the warning is unheeded, the user will be "kicked," i.e. kicked out of the communication channel to cool off.
5. If the user comes back and continues to make trouble, they will be banned, i.e. indefinitely excluded.
6. Moderators may choose at their discretion to un-ban the user if it was a first offense and they offer the offended party a genuine apology.
7. If a moderator bans someone and you think it was unjustified, please take it up with that moderator, or with a different moderator, **in private**. Complaints about bans in-channel are not allowed.
8. Moderators are held to a higher standard than other community members. If a moderator creates an inappropriate situation, they should expect less leeway than others.
In the Rust community we strive to go the extra step to look out for each other. Don't just aim to be technically unimpeachable, try to be your best self. In particular, avoid flirting with offensive or sensitive issues, particularly if they're off-topic; this all too often leads to unnecessary fights, hurt feelings, and damaged trust; worse, it can drive people away from the community entirely.
And if someone takes issue with something you said or did, resist the urge to be defensive. Just stop doing what it was they complained about and apologize. Even if you feel you were misinterpreted or unfairly accused, chances are good there was something you could've communicated better — remember that it's your responsibility to make your fellow Rustaceans comfortable. Everyone wants to get along and we are all here first and foremost because we want to talk about cool technology. You will find that people will be eager to assume good intent and forgive as long as you earn their trust.
The enforcement policies listed above apply to all official Rust venues; including official IRC channels (#rust, #rust-internals, #rust-tools, #rust-libs, #rustc, #rust-beginners, #rust-docs, #rust-community, #rust-lang, and #cargo); GitHub repositories under rust-lang, rust-lang-nursery, and rust-lang-deprecated; and all forums under rust-lang.org (users.rust-lang.org, internals.rust-lang.org). For other projects adopting the Rust Code of Conduct, please contact the maintainers of those projects for enforcement. If you wish to use this code of conduct for your own project, consider explicitly mentioning your moderation policy or making a copy with your own moderation policy so as to avoid confusion.
*Adapted from the [Node.js Policy on Trolling](http://blog.izs.me/post/30036893703/policy-on-trolling) as well as the [Contributor Covenant v1.3.0](http://contributor-covenant.org/version/1/3/0/).*

0
src/vendor/arrayvec/LICENSE-APACHE → src/doc/rustc-guide/LICENSE-APACHE
View File

0
src/vendor/datafrog/LICENSE-MIT → src/doc/rustc-guide/LICENSE-MIT
View File

43
src/doc/rustc-guide/README.md Normal file
View File

@ -0,0 +1,43 @@
This is a collaborate effort to build a guide that explains how rustc
works. The aim of the guide is to help new contributors get oriented
to rustc, as well as to help more experienced folks in figuring out
some new part of the compiler that they haven't worked on before.
[You can read the latest version of the guide here.](https://rust-lang-nursery.github.io/rustc-guide/)
You may also find the rustdocs [for the compiler itself][rustdocs] useful.
[rustdocs]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/
The guide can be useful today, but it has a lot of work still go.
### Contributing to the guide
If you'd like to help improve the guide, we'd love to have you! You can find
plenty of issues on the [issue
tracker](https://github.com/rust-lang-nursery/rustc-guide/issue). Just post a
comment on the issue you would like to work on to make sure that we don't
accidentally duplicate work. If you think something is missing, please open an
issue about it!
**In general, if you don't know how the compiler works, that is not a
problem!** In that case, what we will do is to schedule a bit of time
for you to talk with someone who **does** know the code, or who wants
to pair with you and figure it out. Then you can work on writing up
what you learned.
In general, when writing about a particular part of the compiler's code, we
recommend that you link to the relevant parts of the [rustc
rustdocs][rustdocs].
To help prevent accidentally introducing broken links, we use the
`mdbook-linkcheck`. If installed on your machine `mdbook` will automatically
invoke this link checker, otherwise it will emit a warning saying it couldn't
be found.
```bash
> cargo install mdbook-linkcheck
```
You will need `mdbook` version `>= 0.2`. `linkcheck` will be run automatically
when you run `mdbook build`.

12
src/doc/rustc-guide/book.toml Normal file
View File

@ -0,0 +1,12 @@
[book]
title = "Guide to Rustc Development"
author = "Rustc developers"
description = "A guide to developing rustc"
[output.html]
[output.html.search]
[output.linkcheck]
follow-web-links = true
exclude = [ "crates\\.io", "gcc\\.godbolt\\.org" ]

43
src/doc/rustc-guide/ci/check_line_lengths.sh Executable file
View File

@ -0,0 +1,43 @@
#!/bin/bash
if [ "$1" == "--help" ]; then
echo 'Usage:'
echo ' MAX_LINE_LENGTH=100' "$0" 'src/**/*.md'
exit 1
fi
if [ "$MAX_LINE_LENGTH" == "" ]; then
echo '`MAX_LINE_LENGTH` environment variable not set. Try --help.'
exit 1
fi
if [ "$1" == "" ]; then
echo 'No files provided.'
exit 1
fi
echo "Checking line lengths in all source files <= $MAX_LINE_LENGTH chars..."
echo "Offending files and lines:"
(( bad_lines = 0 ))
(( inside_block = 0 ))
for file in "$@" ; do
echo "$file"
(( line_no = 0 ))
while IFS="" read -r line || [[ -n "$line" ]] ; do
(( line_no++ ))
if [[ "$line" =~ ^'```' ]] ; then
(( inside_block = !$inside_block ))
continue
fi
if ! (( $inside_block )) \
&& ! [[ "$line" =~ " | "|"-|-"|"://"|"]:"|\[\^[^\ ]+\]: ]] \
&& (( "${#line}" > $MAX_LINE_LENGTH )) ; then
(( bad_lines++ ))
echo -e "\t$line_no : $line"
fi
done < "$file"
done
echo "$bad_lines offending lines found."
(( $bad_lines == 0 ))

24
src/doc/rustc-guide/ci/install.sh Executable file
View File

@ -0,0 +1,24 @@
#!/bin/bash
set -ex
function cargo_install() {
local name=$1
local version=$2
if command -v $name >/dev/null 2>&1; then
local installed_version=`$name --version | sed -E 's/[a-zA-Z_-]+ v?//g'`
if [ "$installed_version" == "$version" ]; then
echo "$name $version is already installed at $(command -v $name)"
else
echo "Forcing install $name $version"
cargo install $name --version $version --force
fi
else
echo "Installing $name $version"
cargo install $name --version $version
fi
}
cargo_install mdbook 0.2.2
cargo_install mdbook-linkcheck 0.2.3

75
src/doc/rustc-guide/src/SUMMARY.md Normal file
View File

@ -0,0 +1,75 @@
# Summary
- [About this guide](./about-this-guide.md)
- [About the compiler team](./compiler-team.md)
- [How to build the compiler and run what you built](./how-to-build-and-run.md)
- [Build and Install distribution artifacts](./build-install-distribution-artifacts.md)
- [Documenting Compiler](./compiler-documenting.md)
- [Coding conventions](./conventions.md)
- [Walkthrough: a typical contribution](./walkthrough.md)
- [The compiler testing framework](./tests/intro.md)
- [Running tests](./tests/running.md)
- [Adding new tests](./tests/adding.md)
- [Using `compiletest` + commands to control test
execution](./compiletest.md)
- [Debugging the Compiler](./compiler-debugging.md)
- [Profiling the compiler](./profiling.md)
- [with the linux perf tool](./profiling/with_perf.md)
- [High-level overview of the compiler source](./high-level-overview.md)
- [The Rustc Driver](./rustc-driver.md)
- [Rustdoc](./rustdoc.md)
- [Queries: demand-driven compilation](./query.md)
- [Incremental compilation](./incremental-compilation.md)
- [Debugging and Testing](./incrcomp-debugging.md)
- [The parser](./the-parser.md)
- [`#[test]` Implementation](./test-implementation.md)
- [Macro expansion](./macro-expansion.md)
- [Name resolution](./name-resolution.md)
- [The HIR (High-level IR)](./hir.md)
- [Lowering AST to HIR](./lowering.md)
- [The `ty` module: representing types](./ty.md)
- [Type inference](./type-inference.md)
- [Trait solving (old-style)](./traits/resolution.md)
- [Higher-ranked trait bounds](./traits/hrtb.md)
- [Caching subtleties](./traits/caching.md)
- [Specialization](./traits/specialization.md)
- [Trait solving (new-style)](./traits/index.md)
- [Lowering to logic](./traits/lowering-to-logic.md)
- [Goals and clauses](./traits/goals-and-clauses.md)
- [Equality and associated types](./traits/associated-types.md)
- [Implied bounds](./traits/implied-bounds.md)
- [Region constraints](./traits/regions.md)
- [The lowering module in rustc](./traits/lowering-module.md)
- [Lowering rules](./traits/lowering-rules.md)
- [Well-formedness checking](./traits/wf.md)
- [Canonical queries](./traits/canonical-queries.md)
- [Canonicalization](./traits/canonicalization.md)
- [The SLG solver](./traits/slg.md)
- [An Overview of Chalk](./traits/chalk-overview.md)
- [Bibliography](./traits/bibliography.md)
- [Type checking](./type-checking.md)
- [Method Lookup](./method-lookup.md)
- [Variance](./variance.md)
- [Existential Types](./existential-types.md)
- [The MIR (Mid-level IR)](./mir/index.md)
- [MIR construction](./mir/construction.md)
- [MIR visitor and traversal](./mir/visitor.md)
- [MIR passes: getting the MIR for a function](./mir/passes.md)
- [MIR optimizations](./mir/optimizations.md)
- [The borrow checker](./borrow_check.md)
- [Tracking moves and initialization](./borrow_check/moves_and_initialization.md)
- [Move paths](./borrow_check/moves_and_initialization/move_paths.md)
- [MIR type checker](./borrow_check/type_check.md)
- [Region inference](./borrow_check/region_inference.md)
- [Constant evaluation](./const-eval.md)
- [miri const evaluator](./miri.md)
- [Parameter Environments](./param_env.md)
- [Code Generation](./codegen.md)
- [Emitting Diagnostics](./diag.md)
---
- [Appendix A: Stupid Stats](./appendix/stupid-stats.md)
- [Appendix B: Background material](./appendix/background.md)
- [Appendix C: Glossary](./appendix/glossary.md)
- [Appendix D: Code Index](./appendix/code-index.md)

14
src/doc/rustc-guide/src/about-this-guide.md Normal file
View File

@ -0,0 +1,14 @@
# About this guide
This guide is meant to help document how rustc – the Rust compiler –
works, as well as to help new contributors get involved in rustc
development. It is not meant to replace code documentation – each
chapter gives only high-level details – the kinds of things that
(ideally) don't change frequently.
The guide itself is of course open-source as well, and the sources can
be found at the [GitHub repository]. If you find any mistakes in the
guide, please file an issue about it, or even better, open a PR
with a correction!
[GitHub repository]: https://github.com/rust-lang-nursery/rustc-guide/

128
src/doc/rustc-guide/src/appendix/background.md Normal file
View File

@ -0,0 +1,128 @@
# Appendix B: Background topics
This section covers a numbers of common compiler terms that arise in
this guide. We try to give the general definition while providing some
Rust-specific context.
<a name="cfg"></a>
## What is a control-flow graph?
A control-flow graph is a common term from compilers. If you've ever
used a flow-chart, then the concept of a control-flow graph will be
pretty familiar to you. It's a representation of your program that
exposes the underlying control flow in a very clear way.
A control-flow graph is structured as a set of **basic blocks**
connected by edges. The key idea of a basic block is that it is a set
of statements that execute "together" – that is, whenever you branch
to a basic block, you start at the first statement and then execute
all the remainder. Only at the end of the block is there the
possibility of branching to more than one place (in MIR, we call that
final statement the **terminator**):
```mir
bb0: {
statement0;
statement1;
statement2;
...
terminator;
}
```
Many expressions that you are used to in Rust compile down to multiple
basic blocks. For example, consider an if statement:
```rust,ignore
a = 1;
if some_variable {
b = 1;
} else {
c = 1;
}
d = 1;
```
This would compile into four basic blocks:
```mir
BB0: {
a = 1;
if some_variable { goto BB1 } else { goto BB2 }
}
BB1: {
b = 1;
goto BB3;
}
BB2: {
c = 1;
goto BB3;
}
BB3: {
d = 1;
...;
}
```
When using a control-flow graph, a loop simply appears as a cycle in
the graph, and the `break` keyword translates into a path out of that
cycle.
<a name="dataflow"></a>
## What is a dataflow analysis?
[*Static Program Analysis*](https://cs.au.dk/~amoeller/spa/) by Anders Møller
and Michael I. Schwartzbach is an incredible resource!
*to be written*
<a name="quantified"></a>
## What is "universally quantified"? What about "existentially quantified"?
*to be written*
<a name="variance"></a>
## What is co- and contra-variance?
Check out the subtyping chapter from the
[Rust Nomicon](https://doc.rust-lang.org/nomicon/subtyping.html).
See the [variance](../variance.html) chapter of this guide for more info on how
the type checker handles variance.
<a name="free-vs-bound"></a>
## What is a "free region" or a "free variable"? What about "bound region"?
Let's describe the concepts of free vs bound in terms of program
variables, since that's the thing we're most familiar with.
- Consider this expression, which creates a closure: `|a,
b| a + b`. Here, the `a` and `b` in `a + b` refer to the arguments
that the closure will be given when it is called. We say that the
`a` and `b` there are **bound** to the closure, and that the closure
signature `|a, b|` is a **binder** for the names `a` and `b`
(because any references to `a` or `b` within refer to the variables
that it introduces).
- Consider this expression: `a + b`. In this expression, `a` and `b`
refer to local variables that are defined *outside* of the
expression. We say that those variables **appear free** in the
expression (i.e., they are **free**, not **bound** (tied up)).
So there you have it: a variable "appears free" in some
expression/statement/whatever if it refers to something defined
outside of that expressions/statement/whatever. Equivalently, we can
then refer to the "free variables" of an expression – which is just
the set of variables that "appear free".
So what does this have to do with regions? Well, we can apply the
analogous concept to type and regions. For example, in the type `&'a
u32`, `'a` appears free. But in the type `for<'a> fn(&'a u32)`, it
does not.

44
src/doc/rustc-guide/src/appendix/code-index.md Normal file
View File

@ -0,0 +1,44 @@
# Appendix D: Code Index
rustc has a lot of important data structures. This is an attempt to give some
guidance on where to learn more about some of the key data structures of the
compiler.
Item | Kind | Short description | Chapter | Declaration
----------------|----------|-----------------------------|--------------------|-------------------
`BodyId` | struct | One of four types of HIR node identifiers | [Identifiers in the HIR] | [src/librustc/hir/mod.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc/hir/struct.BodyId.html)
`CompileState` | struct | State that is passed to a callback at each compiler pass | [The Rustc Driver] | [src/librustc_driver/driver.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_driver/driver/struct.CompileState.html)
`ast::Crate` | struct | A syntax-level representation of a parsed crate | [The parser] | [src/librustc/hir/mod.rs](https://doc.rust-lang.org/nightly/nightly-rustc/syntax/ast/struct.Crate.html)
`hir::Crate` | struct | A more abstract, compiler-friendly form of a crate's AST | [The Hir] | [src/librustc/hir/mod.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc/hir/struct.Crate.html)
`DefId` | struct | One of four types of HIR node identifiers | [Identifiers in the HIR] | [src/librustc/hir/def_id.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc/hir/def_id/struct.DefId.html)
`DiagnosticBuilder` | struct | A struct for building up compiler diagnostics, such as errors or lints | [Emitting Diagnostics] | [src/librustc_errors/diagnostic_builder.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_errors/struct.DiagnosticBuilder.html)
`DocContext` | struct | A state container used by rustdoc when crawling through a crate to gather its documentation | [Rustdoc] | [src/librustdoc/core.rs](https://github.com/rust-lang/rust/blob/master/src/librustdoc/core.rs)
`HirId` | struct | One of four types of HIR node identifiers | [Identifiers in the HIR] | [src/librustc/hir/mod.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc/hir/struct.HirId.html)
`NodeId` | struct | One of four types of HIR node identifiers. Being phased out | [Identifiers in the HIR] | [src/libsyntax/ast.rs](https://doc.rust-lang.org/nightly/nightly-rustc/syntax/ast/struct.NodeId.html)
`ParamEnv` | struct | Information about generic parameters or `Self`, useful for working with associated or generic items | [Parameter Environment] | [src/librustc/ty/mod.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc/ty/struct.ParamEnv.html)
`ParseSess` | struct | This struct contains information about a parsing session | [The parser] | [src/libsyntax/parse/mod.rs](https://doc.rust-lang.org/nightly/nightly-rustc/syntax/parse/struct.ParseSess.html)
`Rib` | struct | Represents a single scope of names | [Name resolution] | [src/librustc_resolve/lib.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_resolve/struct.Rib.html)
`Session` | struct | The data associated with a compilation session | [The parser], [The Rustc Driver] | [src/librustc/session/mod.html](https://doc.rust-lang.org/nightly/nightly-rustc/rustc/session/struct.Session.html)
`SourceFile` | struct | Part of the `SourceMap`. Maps AST nodes to their source code for a single source file. Was previously called FileMap | [The parser] | [src/libsyntax_pos/lib.rs](https://doc.rust-lang.org/nightly/nightly-rustc/syntax/source_map/struct.SourceFile.html)
`SourceMap` | struct | Maps AST nodes to their source code. It is composed of `SourceFile`s. Was previously called CodeMap | [The parser] | [src/libsyntax/source_map.rs](https://doc.rust-lang.org/nightly/nightly-rustc/syntax/source_map/struct.SourceMap.html)
`Span` | struct | A location in the user's source code, used for error reporting primarily | [Emitting Diagnostics] | [src/libsyntax_pos/span_encoding.rs](https://doc.rust-lang.org/nightly/nightly-rustc/syntax_pos/struct.Span.html)
`StringReader` | struct | This is the lexer used during parsing. It consumes characters from the raw source code being compiled and produces a series of tokens for use by the rest of the parser | [The parser] | [src/libsyntax/parse/lexer/mod.rs](https://doc.rust-lang.org/nightly/nightly-rustc/syntax/parse/lexer/struct.StringReader.html)
`syntax::token_stream::TokenStream` | struct | An abstract sequence of tokens, organized into `TokenTree`s | [The parser], [Macro expansion] | [src/libsyntax/tokenstream.rs](https://doc.rust-lang.org/nightly/nightly-rustc/syntax/tokenstream/struct.TokenStream.html)
`TraitDef` | struct | This struct contains a trait's definition with type information | [The `ty` modules] | [src/librustc/ty/trait_def.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc/ty/trait_def/struct.TraitDef.html)
`TraitRef` | struct | The combination of a trait and its input types (e.g. `P0: Trait<P1...Pn>`) | [Trait Solving: Goals and Clauses], [Trait Solving: Lowering impls] | [src/librustc/ty/sty.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc/ty/struct.TraitRef.html)
`Ty<'tcx>` | struct | This is the internal representation of a type used for type checking | [Type checking] | [src/librustc/ty/mod.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc/ty/type.Ty.html)
`TyCtxt<'cx, 'tcx, 'tcx>` | type | The "typing context". This is the central data structure in the compiler. It is the context that you use to perform all manner of queries | [The `ty` modules] | [src/librustc/ty/context.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc/ty/struct.TyCtxt.html)
[The HIR]: ../hir.html
[Identifiers in the HIR]: ../hir.html#hir-id
[The parser]: ../the-parser.html
[The Rustc Driver]: ../rustc-driver.html
[Type checking]: ../type-checking.html
[The `ty` modules]: ../ty.html
[Rustdoc]: ../rustdoc.html
[Emitting Diagnostics]: ../diag.html
[Macro expansion]: ../macro-expansion.html
[Name resolution]: ../name-resolution.html
[Parameter Environment]: ../param_env.html
[Trait Solving: Goals and Clauses]: ../traits/goals-and-clauses.html#domain-goals
[Trait Solving: Lowering impls]: ../traits/lowering-rules.html#lowering-impls

78
src/doc/rustc-guide/src/appendix/glossary.md Normal file
View File

@ -0,0 +1,78 @@
# Appendix C: Glossary
The compiler uses a number of...idiosyncratic abbreviations and things. This
glossary attempts to list them and give you a few pointers for understanding
them better.
Term | Meaning
------------------------|--------
AST | the abstract syntax tree produced by the syntax crate; reflects user syntax very closely.
binder | a "binder" is a place where a variable or type is declared; for example, the `<T>` is a binder for the generic type parameter `T` in `fn foo<T>(..)`, and \|`a`\|` ...` is a binder for the parameter `a`. See [the background chapter for more](./background.html#free-vs-bound)
bound variable | a "bound variable" is one that is declared within an expression/term. For example, the variable `a` is bound within the closure expession \|`a`\|` a * 2`. See [the background chapter for more](./background.html#free-vs-bound)
codegen | the code to translate MIR into LLVM IR.
codegen unit | when we produce LLVM IR, we group the Rust code into a number of codegen units. Each of these units is processed by LLVM independently from one another, enabling parallelism. They are also the unit of incremental re-use.
completeness | completeness is a technical term in type theory. Completeness means that every type-safe program also type-checks. Having both soundness and completeness is very hard, and usually soundness is more important. (see "soundness").
control-flow graph | a representation of the control-flow of a program; see [the background chapter for more](./background.html#cfg)
CTFE | Compile-Time Function Evaluation. This is the ability of the compiler to evaluate `const fn`s at compile time. This is part of the compiler's constant evaluation system. ([see more](../const-eval.html))
cx | we tend to use "cx" as an abbreviation for context. See also `tcx`, `infcx`, etc.
DAG | a directed acyclic graph is used during compilation to keep track of dependencies between queries. ([see more](../incremental-compilation.html))
data-flow analysis | a static analysis that figures out what properties are true at each point in the control-flow of a program; see [the background chapter for more](./background.html#dataflow)
DefId | an index identifying a definition (see `librustc/hir/def_id.rs`). Uniquely identifies a `DefPath`.
Double pointer | a pointer with additional metadata. See "fat pointer" for more.
DST | Dynamically-Sized Type. A type for which the compiler cannot statically know the size in memory (e.g. `str` or `[u8]`). Such types don't implement `Sized` and cannot be allocated on the stack. They can only occur as the last field in a struct. They can only be used behind a pointer (e.g. `&str` or `&[u8]`).
empty type | see "uninhabited type".
Fat pointer | a two word value carrying the address of some value, along with some further information necessary to put the value to use. Rust includes two kinds of "fat pointers": references to slices, and trait objects. A reference to a slice carries the starting address of the slice and its length. A trait object carries a value's address and a pointer to the trait's implementation appropriate to that value. "Fat pointers" are also known as "wide pointers", and "double pointers".
free variable | a "free variable" is one that is not bound within an expression or term; see [the background chapter for more](./background.html#free-vs-bound)
'gcx | the lifetime of the global arena ([see more](../ty.html))
generics | the set of generic type parameters defined on a type or item
HIR | the High-level IR, created by lowering and desugaring the AST ([see more](../hir.html))
HirId | identifies a particular node in the HIR by combining a def-id with an "intra-definition offset".
HIR Map | The HIR map, accessible via tcx.hir, allows you to quickly navigate the HIR and convert between various forms of identifiers.
ICE | internal compiler error. When the compiler crashes.
ICH | incremental compilation hash. ICHs are used as fingerprints for things such as HIR and crate metadata, to check if changes have been made. This is useful in incremental compilation to see if part of a crate has changed and should be recompiled.
inference variable | when doing type or region inference, an "inference variable" is a kind of special type/region that represents what you are trying to infer. Think of X in algebra. For example, if we are trying to infer the type of a variable in a program, we create an inference variable to represent that unknown type.
infcx | the inference context (see `librustc/infer`)
IR | Intermediate Representation. A general term in compilers. During compilation, the code is transformed from raw source (ASCII text) to various IRs. In Rust, these are primarily HIR, MIR, and LLVM IR. Each IR is well-suited for some set of computations. For example, MIR is well-suited for the borrow checker, and LLVM IR is well-suited for codegen because LLVM accepts it.
local crate | the crate currently being compiled.
LTO | Link-Time Optimizations. A set of optimizations offered by LLVM that occur just before the final binary is linked. These include optimizations like removing functions that are never used in the final program, for example. _ThinLTO_ is a variant of LTO that aims to be a bit more scalable and efficient, but possibly sacrifices some optimizations. You may also read issues in the Rust repo about "FatLTO", which is the loving nickname given to non-Thin LTO. LLVM documentation: [here][lto] and [here][thinlto]
[LLVM] | (actually not an acronym :P) an open-source compiler backend. It accepts LLVM IR and outputs native binaries. Various languages (e.g. Rust) can then implement a compiler front-end that output LLVM IR and use LLVM to compile to all the platforms LLVM supports.
MIR | the Mid-level IR that is created after type-checking for use by borrowck and codegen ([see more](../mir/index.html))
miri | an interpreter for MIR used for constant evaluation ([see more](../miri.html))
normalize | a general term for converting to a more canonical form, but in the case of rustc typically refers to [associated type normalization](../traits/associated-types.html#normalize)
newtype | a "newtype" is a wrapper around some other type (e.g., `struct Foo(T)` is a "newtype" for `T`). This is commonly used in Rust to give a stronger type for indices.
NLL | [non-lexical lifetimes](../borrow_check/region_inference.html), an extension to Rust's borrowing system to make it be based on the control-flow graph.
node-id or NodeId | an index identifying a particular node in the AST or HIR; gradually being phased out and replaced with `HirId`.
obligation | something that must be proven by the trait system ([see more](../traits/resolution.html))
projection | a general term for a "relative path", e.g. `x.f` is a "field projection", and `T::Item` is an ["associated type projection"](../traits/goals-and-clauses.html#trait-ref)
promoted constants | constants extracted from a function and lifted to static scope; see [this section](../mir/index.html#promoted) for more details.
provider | the function that executes a query ([see more](../query.html))
quantified | in math or logic, existential and universal quantification are used to ask questions like "is there any type T for which is true?" or "is this true for all types T?"; see [the background chapter for more](./background.html#quantified)
query | perhaps some sub-computation during compilation ([see more](../query.html))
region | another term for "lifetime" often used in the literature and in the borrow checker.
rib | a data structure in the name resolver that keeps track of a single scope for names. ([see more](../name-resolution.html))
sess | the compiler session, which stores global data used throughout compilation
side tables | because the AST and HIR are immutable once created, we often carry extra information about them in the form of hashtables, indexed by the id of a particular node.
sigil | like a keyword but composed entirely of non-alphanumeric tokens. For example, `&` is a sigil for references.
placeholder | **NOTE: skolemization is deprecated by placeholder** a way of handling subtyping around "for-all" types (e.g., `for<'a> fn(&'a u32)`) as well as solving higher-ranked trait bounds (e.g., `for<'a> T: Trait<'a>`). See [the chapter on placeholder and universes](../borrow_check/region_inference.html#placeholder) for more details.
soundness | soundness is a technical term in type theory. Roughly, if a type system is sound, then if a program type-checks, it is type-safe; i.e. I can never (in safe rust) force a value into a variable of the wrong type. (see "completeness").
span | a location in the user's source code, used for error reporting primarily. These are like a file-name/line-number/column tuple on steroids: they carry a start/end point, and also track macro expansions and compiler desugaring. All while being packed into a few bytes (really, it's an index into a table). See the Span datatype for more.
substs | the substitutions for a given generic type or item (e.g. the `i32`, `u32` in `HashMap<i32, u32>`)
tcx | the "typing context", main data structure of the compiler ([see more](../ty.html))
'tcx | the lifetime of the currently active inference context ([see more](../ty.html))
trait reference | the name of a trait along with a suitable set of input type/lifetimes ([see more](../traits/goals-and-clauses.html#trait-ref))
token | the smallest unit of parsing. Tokens are produced after lexing ([see more](../the-parser.html)).
[TLS] | Thread-Local Storage. Variables may be defined so that each thread has its own copy (rather than all threads sharing the variable). This has some interactions with LLVM. Not all platforms support TLS.
trans | the code to translate MIR into LLVM IR. Renamed to codegen.
trait reference | a trait and values for its type parameters ([see more](../ty.html)).
ty | the internal representation of a type ([see more](../ty.html)).
UFCS | Universal Function Call Syntax. An unambiguous syntax for calling a method ([see more](../type-checking.html)).
uninhabited type | a type which has _no_ values. This is not the same as a ZST, which has exactly 1 value. An example of an uninhabited type is `enum Foo {}`, which has no variants, and so, can never be created. The compiler can treat code that deals with uninhabited types as dead code, since there is no such value to be manipulated. `!` (the never type) is an uninhabited type. Uninhabited types are also called "empty types".
upvar | a variable captured by a closure from outside the closure.
variance | variance determines how changes to a generic type/lifetime parameter affect subtyping; for example, if `T` is a subtype of `U`, then `Vec<T>` is a subtype `Vec<U>` because `Vec` is *covariant* in its generic parameter. See [the background chapter](./background.html#variance) for a more general explanation. See the [variance chapter](../variance.html) for an explanation of how type checking handles variance.
Wide pointer | a pointer with additional metadata. See "fat pointer" for more.
ZST | Zero-Sized Type. A type whose values have size 0 bytes. Since `2^0 = 1`, such types can have exactly one value. For example, `()` (unit) is a ZST. `struct Foo;` is also a ZST. The compiler can do some nice optimizations around ZSTs.
[LLVM]: https://llvm.org/
[lto]: https://llvm.org/docs/LinkTimeOptimization.html
[thinlto]: https://clang.llvm.org/docs/ThinLTO.html
[TLS]: https://llvm.org/docs/LangRef.html#thread-local-storage-models

412
src/doc/rustc-guide/src/appendix/stupid-stats.md Normal file
View File

@ -0,0 +1,412 @@
# Appendix A: A tutorial on creating a drop-in replacement for rustc
> **Note:** This is a copy of `@nrc`'s amazing [stupid-stats]. You should find
> a copy of the code on the GitHub repository although due to the compiler's
> constantly evolving nature, there is no guarantee it'll compile on the first
> go.
Many tools benefit from being a drop-in replacement for a compiler. By this, I
mean that any user of the tool can use `mytool` in all the ways they would
normally use `rustc` - whether manually compiling a single file or as part of a
complex make project or Cargo build, etc. That could be a lot of work;
rustc, like most compilers, takes a large number of command line arguments which
can affect compilation in complex and interacting ways. Emulating all of this
behaviour in your tool is annoying at best, especically if you are making many
of the same calls into librustc that the compiler is.
The kind of things I have in mind are tools like rustdoc or a future rustfmt.
These want to operate as closely as possible to real compilation, but have
totally different outputs (documentation and formatted source code,
respectively). Another use case is a customised compiler. Say you want to add a
custom code generation phase after macro expansion, then creating a new tool
should be easier than forking the compiler (and keeping it up to date as the
compiler evolves).
I have gradually been trying to improve the API of librustc to make creating a
drop-in tool easier to produce (many others have also helped improve these
interfaces over the same time frame). It is now pretty simple to make a tool
which is as close to rustc as you want it to be. In this tutorial I'll show
how.
Note/warning, everything I talk about in this tutorial is internal API for
rustc. It is all extremely unstable and likely to change often and in
unpredictable ways. Maintaining a tool which uses these APIs will be non-
trivial, although hopefully easier than maintaining one that does similar things
without using them.
This tutorial starts with a very high level view of the rustc compilation
process and of some of the code that drives compilation. Then I'll describe how
that process can be customised. In the final section of the tutorial, I'll go
through an example - stupid-stats - which shows how to build a drop-in tool.
## Overview of the compilation process
Compilation using rustc happens in several phases. We start with parsing, this
includes lexing. The output of this phase is an AST (abstract syntax tree).
There is a single AST for each crate (indeed, the entire compilation process
operates over a single crate). Parsing abstracts away details about individual
files which will all have been read in to the AST in this phase. At this stage
the AST includes all macro uses, attributes will still be present, and nothing
will have been eliminated due to `cfg`s.
The next phase is configuration and macro expansion. This can be thought of as a
function over the AST. The unexpanded AST goes in and an expanded AST comes out.
Macros and syntax extensions are expanded, and `cfg` attributes will cause some
code to disappear. The resulting AST won't have any macros or macro uses left
in.
The code for these first two phases is in [libsyntax](https://github.com/rust-lang/rust/tree/master/src/libsyntax).
After this phase, the compiler allocates ids to each node in the AST
(technically not every node, but most of them). If we are writing out
dependencies, that happens now.
The next big phase is analysis. This is the most complex phase and
uses the bulk of the code in rustc. This includes name resolution, type
checking, borrow checking, type and lifetime inference, trait selection, method
selection, linting, and so forth. Most error detection is done in this phase
(although parse errors are found during parsing). The 'output' of this phase is
a bunch of side tables containing semantic information about the source program.
The analysis code is in [librustc](https://github.com/rust-lang/rust/tree/master/src/librustc)
and a bunch of other crates with the 'librustc_' prefix.
Next is translation, this translates the AST (and all those side tables) into
LLVM IR (intermediate representation). We do this by calling into the LLVM
libraries, rather than actually writing IR directly to a file. The code for
this is in librustc_trans.
The next phase is running the LLVM backend. This runs LLVM's optimisation passes
on the generated IR and then generates machine code. The result is object files.
This phase is all done by LLVM, it is not really part of the rust compiler. The
interface between LLVM and rustc is in [librustc_llvm](https://github.com/rust-lang/rust/tree/master/src/librustc_llvm).
Finally, we link the object files into an executable. Again we outsource this to
other programs and it's not really part of the rust compiler. The interface is
in librustc_back (which also contains some things used primarily during
translation).
> NOTE: `librustc_trans` and `librustc_back` no longer exist, and we don't
> translate AST or HIR directly to LLVM IR anymore. Instead, see
> [`librustc_codegen_llvm`](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_codegen_llvm/index.html)
> and [`librustc_codegen_utils`](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_codegen_utils/index.html).
All these phases are coordinated by the driver. To see the exact sequence, look
at [the `compile_input` function in `librustc_driver`][compile-input].
The driver handles all the highest level coordination of compilation -
1. handling command-line arguments
2. maintaining compilation state (primarily in the `Session`)
3. calling the appropriate code to run each phase of compilation
4. handles high level coordination of pretty printing and testing.
To create a drop-in compiler replacement or a compiler replacement,
we leave most of compilation alone and customise the driver using its APIs.
[compile-input]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_driver/driver/fn.compile_input.html
## The driver customisation APIs
There are two primary ways to customise compilation - high level control of the
driver using `CompilerCalls` and controlling each phase of compilation using a
`CompileController`. The former lets you customise handling of command line
arguments etc., the latter lets you stop compilation early or execute code
between phases.
### `CompilerCalls`
`CompilerCalls` is a trait that you implement in your tool. It contains a fairly
ad-hoc set of methods to hook in to the process of processing command line
arguments and driving the compiler. For details, see the comments in
[librustc_driver/lib.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_driver/index.html).
I'll summarise the methods here.
`early_callback` and `late_callback` let you call arbitrary code at different
points - early is after command line arguments have been parsed, but before
anything is done with them; late is pretty much the last thing before
compilation starts, i.e., after all processing of command line arguments, etc.
is done. Currently, you get to choose whether compilation stops or continues at
each point, but you don't get to change anything the driver has done. You can
record some info for later, or perform other actions of your own.
`some_input` and `no_input` give you an opportunity to modify the primary input
to the compiler (usually the input is a file containing the top module for a
crate, but it could also be a string). You could record the input or perform
other actions of your own.
Ignore `parse_pretty`, it is unfortunate and hopefully will get improved. There
is a default implementation, so you can pretend it doesn't exist.
`build_controller` returns a `CompileController` object for more fine-grained
control of compilation, it is described next.
We might add more options in the future.
### `CompilerController`
`CompilerController` is a struct consisting of `PhaseController`s and flags.
Currently, there is only flag, `make_glob_map` which signals whether to produce
a map of glob imports (used by save-analysis and potentially other tools). There
are probably flags in the session that should be moved here.
There is a `PhaseController` for each of the phases described in the above
summary of compilation (and we could add more in the future for finer-grained
control). They are all `after_` a phase because they are checked at the end of a
phase (again, that might change), e.g., `CompilerController::after_parse`
controls what happens immediately after parsing (and before macro expansion).
Each `PhaseController` contains a flag called `stop` which indicates whether
compilation should stop or continue, and a callback to be executed at the point
indicated by the phase. The callback is called whether or not compilation
continues.
Information about the state of compilation is passed to these callbacks in a
`CompileState` object. This contains all the information the compiler has. Note
that this state information is immutable - your callback can only execute code
using the compiler state, it can't modify the state. (If there is demand, we
could change that). The state available to a callback depends on where during
compilation the callback is called. For example, after parsing there is an AST
but no semantic analysis (because the AST has not been analysed yet). After
translation, there is translation info, but no AST or analysis info (since these
have been consumed/forgotten).
## An example - stupid-stats
Our example tool is very simple, it simply collects some simple and not very
useful statistics about a program; it is called stupid-stats. You can find
the (more heavily commented) complete source for the example on [Github](https://github.com/nick29581/stupid-stats/blob/master/src).
To build, just do `cargo build`. To run on a file `foo.rs`, do `cargo run
foo.rs` (assuming you have a Rust program called `foo.rs`. You can also pass any
command line arguments that you would normally pass to rustc). When you run it
you'll see output similar to
```text
In crate: foo,
Found 12 uses of `println!`;
The most common number of arguments is 1 (67% of all functions);
25% of functions have four or more arguments.
```
To make things easier, when we talk about functions, we're excluding methods and
closures.
You can also use the executable as a drop-in replacement for rustc, because
after all, that is the whole point of this exercise. So, however you use rustc
in your makefile setup, you can use `target/stupid` (or whatever executable you
end up with) instead. That might mean setting an environment variable or it
might mean renaming your executable to `rustc` and setting your PATH. Similarly,
if you're using Cargo, you'll need to rename the executable to rustc and set the
PATH. Alternatively, you should be able to use
[multirust](https://github.com/brson/multirust) to get around all the PATH stuff
(although I haven't actually tried that).
(Note that this example prints to stdout. I'm not entirely sure what Cargo does
with stdout from rustc under different circumstances. If you don't see any
output, try inserting a `panic!` after the `println!`s to error out, then Cargo
should dump stupid-stats' stdout to Cargo's stdout).
Let's start with the `main` function for our tool, it is pretty simple:
```rust,ignore
fn main() {
let args: Vec<_> = std::env::args().collect();
rustc_driver::run_compiler(&args, &mut StupidCalls::new());
std::env::set_exit_status(0);
}
```
The first line grabs any command line arguments. The second line calls the
compiler driver with those arguments. The final line sets the exit code for the
program.
The only interesting thing is the `StupidCalls` object we pass to the driver.
This is our implementation of the `CompilerCalls` trait and is what will make
this tool different from rustc.
`StupidCalls` is a mostly empty struct:
```rust,ignore
struct StupidCalls {
default_calls: RustcDefaultCalls,
}
```
This tool is so simple that it doesn't need to store any data here, but usually
you would. We embed a `RustcDefaultCalls` object to delegate to in our impl when
we want exactly the same behaviour as the Rust compiler. Mostly you don't want
to do that (or at least don't need to) in a tool. However, Cargo calls rustc
with the `--print file-names`, so we delegate in `late_callback` and `no_input`
to keep Cargo happy.
Most of the rest of the impl of `CompilerCalls` is trivial:
```rust,ignore
impl<'a> CompilerCalls<'a> for StupidCalls {
fn early_callback(&mut self,
_: &getopts::Matches,
_: &config::Options,
_: &diagnostics::registry::Registry,
_: ErrorOutputType)
-> Compilation {
Compilation::Continue
}
fn late_callback(&mut self,
t: &TransCrate,
m: &getopts::Matches,
s: &Session,
c: &CrateStore,
i: &Input,
odir: &Option<PathBuf>,
ofile: &Option<PathBuf>)
-> Compilation {
self.default_calls.late_callback(t, m, s, c, i, odir, ofile);
Compilation::Continue
}
fn some_input(&mut self,
input: Input,
input_path: Option<Path>)
-> (Input, Option<Path>) {
(input, input_path)
}
fn no_input(&mut self,
m: &getopts::Matches,
o: &config::Options,
odir: &Option<Path>,
ofile: &Option<Path>,
r: &diagnostics::registry::Registry)
-> Option<(Input, Option<Path>)> {
self.default_calls.no_input(m, o, odir, ofile, r);
// This is not optimal error handling.
panic!("No input supplied to stupid-stats");
}
fn build_controller(&mut self, _: &Session) -> driver::CompileController<'a> {
...
}
}
```
We don't do anything for either of the callbacks, nor do we change the input if
the user supplies it. If they don't, we just `panic!`, this is the simplest way
to handle the error, but not very user-friendly, a real tool would give a
constructive message or perform a default action.
In `build_controller` we construct our `CompileController`. We only want to
parse, and we want to inspect macros before expansion, so we make compilation
stop after the first phase (parsing). The callback after that phase is where the
tool does it's actual work by walking the AST. We do that by creating an AST
visitor and making it walk the AST from the top (the crate root). Once we've
walked the crate, we print the stats we've collected:
```rust,ignore
fn build_controller(&mut self, _: &Session) -> driver::CompileController<'a> {
// We mostly want to do what rustc does, which is what basic() will return.
let mut control = driver::CompileController::basic();
// But we only need the AST, so we can stop compilation after parsing.
control.after_parse.stop = Compilation::Stop;
// And when we stop after parsing we'll call this closure.
// Note that this will give us an AST before macro expansions, which is
// not usually what you want.
control.after_parse.callback = box |state| {
// Which extracts information about the compiled crate...
let krate = state.krate.unwrap();
// ...and walks the AST, collecting stats.
let mut visitor = StupidVisitor::new();
visit::walk_crate(&mut visitor, krate);
// And finally prints out the stupid stats that we collected.
let cratename = match attr::find_crate_name(&krate.attrs[]) {
Some(name) => name.to_string(),
None => String::from_str("unknown_crate"),
};
println!("In crate: {},\n", cratename);
println!("Found {} uses of `println!`;", visitor.println_count);
let (common, common_percent, four_percent) = visitor.compute_arg_stats();
println!("The most common number of arguments is {} ({:.0}% of all functions);",
common, common_percent);
println!("{:.0}% of functions have four or more arguments.", four_percent);
};
control
}
```
That is all it takes to create your own drop-in compiler replacement or custom
compiler! For the sake of completeness I'll go over the rest of the stupid-stats
tool.
```rust
struct StupidVisitor {
println_count: usize,
arg_counts: Vec<usize>,
}
```
The `StupidVisitor` struct just keeps track of the number of `println!`s it has
seen and the count for each number of arguments. It implements
`syntax::visit::Visitor` to walk the AST. Mostly we just use the default
methods, these walk the AST taking no action. We override `visit_item` and
`visit_mac` to implement custom behaviour when we walk into items (items include
functions, modules, traits, structs, and so forth, we're only interested in
functions) and macros:
```rust,ignore
impl<'v> visit::Visitor<'v> for StupidVisitor {
fn visit_item(&mut self, i: &'v ast::Item) {
match i.node {
ast::Item_::ItemFn(ref decl, _, _, _, _) => {
// Record the number of args.
self.increment_args(decl.inputs.len());
}
_ => {}
}
// Keep walking.
visit::walk_item(self, i)
}
fn visit_mac(&mut self, mac: &'v ast::Mac) {
// Find its name and check if it is "println".
let ast::Mac_::MacInvocTT(ref path, _, _) = mac.node;
if path_to_string(path) == "println" {
self.println_count += 1;
}
// Keep walking.
visit::walk_mac(self, mac)
}
}
```
The `increment_args` method increments the correct count in
`StupidVisitor::arg_counts`. After we're done walking, `compute_arg_stats` does
some pretty basic maths to come up with the stats we want about arguments.
## What next?
These APIs are pretty new and have a long way to go until they're really good.
If there are improvements you'd like to see or things you'd like to be able to
do, let me know in a comment or [GitHub issue](https://github.com/rust-lang/rust/issues).
In particular, it's not clear to me exactly what extra flexibility is required.
If you have an existing tool that would be suited to this setup, please try it
out and let me know if you have problems.
It'd be great to see Rustdoc converted to using these APIs, if that is possible
(although long term, I'd prefer to see Rustdoc run on the output from save-
analysis, rather than doing its own analysis). Other parts of the compiler
(e.g., pretty printing, testing) could be refactored to use these APIs
internally (I already changed save-analysis to use `CompilerController`). I've
been experimenting with a prototype rustfmt which also uses these APIs.
[stupid-stats]: https://github.com/nrc/stupid-stats

63
src/doc/rustc-guide/src/borrow_check.md Normal file
View File

@ -0,0 +1,63 @@
# MIR borrow check
The borrow check is Rust's "secret sauce" – it is tasked with
enforcing a number of properties:
- That all variables are initialized before they are used.
- That you can't move the same value twice.
- That you can't move a value while it is borrowed.
- That you can't access a place while it is mutably borrowed (except through
the reference).
- That you can't mutate a place while it is shared borrowed.
- etc
At the time of this writing, the code is in a state of transition. The
"main" borrow checker still works by processing [the HIR](hir.html),
but that is being phased out in favor of the MIR-based borrow checker.
Accordingly, this documentation focuses on the new, MIR-based borrow
checker.
Doing borrow checking on MIR has several advantages:
- The MIR is *far* less complex than the HIR; the radical desugaring
helps prevent bugs in the borrow checker. (If you're curious, you
can see
[a list of bugs that the MIR-based borrow checker fixes here][47366].)
- Even more importantly, using the MIR enables ["non-lexical lifetimes"][nll],
which are regions derived from the control-flow graph.
[47366]: https://github.com/rust-lang/rust/issues/47366
[nll]: http://rust-lang.github.io/rfcs/2094-nll.html
### Major phases of the borrow checker
The borrow checker source is found in
[the `rustc_mir::borrow_check` module][b_c]. The main entry point is
the [`mir_borrowck`] query.
[b_c]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_mir/borrow_check/index.html
[`mir_borrowck`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_mir/borrow_check/fn.mir_borrowck.html
- We first create a **local copy** of the MIR. In the coming steps,
we will modify this copy in place to modify the types and things to
include references to the new regions that we are computing.
- We then invoke [`replace_regions_in_mir`] to modify our local MIR.
Among other things, this function will replace all of the [regions](./appendix/glossary.html) in
the MIR with fresh [inference variables](./appendix/glossary.html).
- Next, we perform a number of
[dataflow analyses](./appendix/background.html#dataflow) that
compute what data is moved and when.
- We then do a [second type check](borrow_check/type_check.html) across the MIR:
the purpose of this type check is to determine all of the constraints between
different regions.
- Next, we do [region inference](borrow_check/region_inference.html), which computes
the values of each region — basically, points in the control-flow graph.
- At this point, we can compute the "borrows in scope" at each point.
- Finally, we do a second walk over the MIR, looking at the actions it
does and reporting errors. For example, if we see a statement like
`*a + 1`, then we would check that the variable `a` is initialized
and that it is not mutably borrowed, as either of those would
require an error to be reported.
- Doing this check requires the results of all the previous analyses.
[`replace_regions_in_mir`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_mir/borrow_check/nll/fn.replace_regions_in_mir.html

50
src/doc/rustc-guide/src/borrow_check/moves_and_initialization.md Normal file
View File

@ -0,0 +1,50 @@
# Tracking moves and initialization
Part of the borrow checker's job is to track which variables are
"initialized" at any given point in time -- this also requires
figuring out where moves occur and tracking those.
## Initialization and moves
From a user's perspective, initialization -- giving a variable some
value -- and moves -- transfering ownership to another place -- might
seem like distinct topics. Indeed, our borrow checker error messages
often talk about them differently. But **within the borrow checker**,
they are not nearly as separate. Roughly speaking, the borrow checker
tracks the set of "initialized places" at any point in the source
code. Assigning to a previously uninitialized local variable adds it
to that set; moving from a local variable removes it from that set.
Consider this example:
```rust,ignore
fn foo() {
let a: Vec<u32>;
// a is not initialized yet
a = vec![22];
// a is initialized here
std::mem::drop(a); // a is moved here
// a is no longer initialized here
let l = a.len(); //~ ERROR
}
```
Here you can see that `a` starts off as uninitialized; once it is
assigned, it becomes initialized. But when `drop(a)` is called, that
moves `a` into the call, and hence it becomes uninitialized again.
## Subsections
To make it easier to peruse, this section is broken into a number of
subsections:
- [Move paths](./moves_and_initialization/move_paths.html) the
*move path* concept that we use to track which local variables (or parts of
local variables, in some cases) are initialized.
- TODO *Rest not yet written* =)

129
src/doc/rustc-guide/src/borrow_check/moves_and_initialization/move_paths.md Normal file
View File

@ -0,0 +1,129 @@
# Move paths
In reality, it's not enough to track initialization at the granularity
of local variables. Rust also allows us to do moves and initialization
at the field granularity:
```rust,ignore
fn foo() {
let a: (Vec<u32>, Vec<u32>) = (vec![22], vec![44]);
// a.0 and a.1 are both initialized
let b = a.0; // moves a.0
// a.0 is not initializd, but a.1 still is
let c = a.0; // ERROR
let d = a.1; // OK
}
```
To handle this, we track initialization at the granularity of a **move
path**. A [`MovePath`] represents some location that the user can
initialize, move, etc. So e.g. there is a move-path representing the
local variable `a`, and there is a move-path representing `a.0`. Move
paths roughly correspond to the concept of a [`Place`] from MIR, but
they are indexed in ways that enable us to do move analysis more
efficiently.
[`MovePath`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_mir/dataflow/move_paths/struct.MovePath.html
[`Place`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/mir/enum.Place.html
## Move path indices
Although there is a [`MovePath`] data structure, they are never
referenced directly. Instead, all the code passes around *indices* of
type
[`MovePathIndex`](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_mir/dataflow/move_paths/indexes/struct.MovePathIndex.html). If
you need to get information about a move path, you use this index with
the [`move_paths` field of the `MoveData`][move_paths]. For example,
to convert a [`MovePathIndex`] `mpi` into a MIR [`Place`], you might
access the [`MovePath::place`] field like so:
```rust,ignore
move_data.move_paths[mpi].place
```
[move_paths]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_mir/dataflow/move_paths/struct.MoveData.html#structfield.move_paths
[`MovePath::place`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_mir/dataflow/move_paths/struct.MovePath.html#structfield.place
[`MovePathIndex`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_mir/dataflow/move_paths/indexes/struct.MovePathIndex.html
## Building move paths
One of the first things we do in the MIR borrow check is to construct
the set of move paths. This is done as part of the
[`MoveData::gather_moves`] function. This function uses a MIR visitor
called [`Gatherer`] to walk the MIR and look at how each [`Place`]
within is accessed. For each such [`Place`], it constructs a
corresponding [`MovePathIndex`]. It also records when/where that
particular move path is moved/initialized, but we'll get to that in a
later section.
[`Gatherer`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_mir/dataflow/move_paths/builder/struct.Gatherer.html
[`MoveData::gather_moves`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_mir/dataflow/move_paths/struct.MoveData.html#method.gather_moves
### Illegal move paths
We don't actually create a move-path for **every** [`Place`] that gets
used. In particular, if it is illegal to move from a [`Place`], then
there is no need for a [`MovePathIndex`]. Some examples:
- You cannot move from a static variable, so we do not create a [`MovePathIndex`]
for static variables.
- You cannot move an individual element of an array, so if we have e.g. `foo: [String; 3]`,
there would be no move-path for `foo[1]`.
- You cannot move from inside of a borrowed reference, so if we have e.g. `foo: &String`,
there would be no move-path for `*foo`.
These rules are enforced by the [`move_path_for`] function, which
converts a [`Place`] into a [`MovePathIndex`] -- in error cases like
those just discussed, the function returns an `Err`. This in turn
means we don't have to bother tracking whether those places are
initialized (which lowers overhead).
[`move_path_for`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_mir/dataflow/move_paths/builder/struct.Gatherer.html#method.move_path_for
## Looking up a move-path
If you have a [`Place`] and you would like to convert it to a [`MovePathIndex`], you
can do that using the [`MovePathLookup`] structure found in the [`rev_lookup`] field
of [`MoveData`]. There are two different methods:
[`MovePathLookup`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_mir/dataflow/move_paths/struct.MovePathLookup.html
[`rev_lookup`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_mir/dataflow/move_paths/struct.MoveData.html#structfield.rev_lookup
- [`find_local`], which takes a [`mir::Local`] representing a local
variable. This is the easier method, because we **always** create a
[`MovePathIndex`] for every local variable.
- [`find`], which takes an arbitrary [`Place`]. This method is a bit
more annoying to use, precisely because we don't have a
[`MovePathIndex`] for **every** [`Place`] (as we just discussed in
the "illegal move paths" section). Therefore, [`find`] returns a
[`LookupResult`] indicating the closest path it was able to find
that exists (e.g., for `foo[1]`, it might return just the path for
`foo`).
[`find`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_mir/dataflow/move_paths/struct.MovePathLookup.html#method.find
[`find_local`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_mir/dataflow/move_paths/struct.MovePathLookup.html#method.find_local
[`mir::Local`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/mir/struct.Local.html
[`LookupResult`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_mir/dataflow/move_paths/enum.LookupResult.html
## Cross-references
As we noted above, move-paths are stored in a big vector and
referenced via their [`MovePathIndex`]. However, within this vector,
they are also structured into a tree. So for example if you have the
[`MovePathIndex`] for `a.b.c`, you can go to its parent move-path
`a.b`. You can also iterate over all children paths: so, from `a.b`,
you might iterate to find the path `a.b.c` (here you are iterating
just over the paths that are **actually referenced** in the source,
not all **possible** paths that could have been referenced). These
references are used for example in the [`has_any_child_of`] function,
which checks whether the dataflow results contain a value for the
given move-path (e.g., `a.b`) or any child of that move-path (e.g.,
`a.b.c`).
[`Place`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/mir/enum.Place.html
[`has_any_child_of`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_mir/dataflow/at_location/struct.FlowAtLocation.html#method.has_any_child_of

536
src/doc/rustc-guide/src/borrow_check/region_inference.md Normal file
View File

@ -0,0 +1,536 @@
# Region inference (NLL)
The MIR-based region checking code is located in
[the `rustc_mir::borrow_check::nll` module][nll]. (NLL, of course,
stands for "non-lexical lifetimes", a term that will hopefully be
deprecated once they become the standard kind of lifetime.)
[nll]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_mir/borrow_check/nll/index.html
The MIR-based region analysis consists of two major functions:
- `replace_regions_in_mir`, invoked first, has two jobs:
- First, it finds the set of regions that appear within the
signature of the function (e.g., `'a` in `fn foo<'a>(&'a u32) {
... }`). These are called the "universal" or "free" regions – in
particular, they are the regions that [appear free][fvb] in the
function body.
- Second, it replaces all the regions from the function body with
fresh inference variables. This is because (presently) those
regions are the results of lexical region inference and hence are
not of much interest. The intention is that – eventually – they
will be "erased regions" (i.e., no information at all), since we
won't be doing lexical region inference at all.
- `compute_regions`, invoked second: this is given as argument the
results of move analysis. It has the job of computing values for all
the inference variables that `replace_regions_in_mir` introduced.
- To do that, it first runs the [MIR type checker](#mirtypeck). This
is basically a normal type-checker but specialized to MIR, which
is much simpler than full Rust of course. Running the MIR type
checker will however create **outlives constraints** between
region variables (e.g., that one variable must outlive another
one) to reflect the subtyping relationships that arise.
- It also adds **liveness constraints** that arise from where variables
are used.
- More details to come, though the [NLL RFC] also includes fairly thorough
(and hopefully readable) coverage.
[fvb]: ../appendix/background.html#free-vs-bound
[NLL RFC]: http://rust-lang.github.io/rfcs/2094-nll.html
## Universal regions
*to be written* – explain the `UniversalRegions` type
## Region variables and constraints
*to be written* – describe the `RegionInferenceContext` and
the role of `liveness_constraints` vs other `constraints`, plus
## Closures
*to be written*
<a name="mirtypeck"></a>
## The MIR type-check
## Representing the "values" of a region variable
The value of a region can be thought of as a **set**; we call the
domain of this set a `RegionElement`. In the code, the value for all
regions is maintained in
[the `rustc_mir::borrow_check::nll::region_infer` module][ri]. For
each region we maintain a set storing what elements are present in its
value (to make this efficient, we give each kind of element an index,
the `RegionElementIndex`, and use sparse bitsets).
[ri]: https://github.com/rust-lang/rust/tree/master/src/librustc_mir/borrow_check/nll/region_infer/
The kinds of region elements are as follows:
- Each **location** in the MIR control-flow graph: a location is just
the pair of a basic block and an index. This identifies the point
**on entry** to the statement with that index (or the terminator, if
the index is equal to `statements.len()`).
- There is an element `end('a)` for each universal region `'a`,
corresponding to some portion of the caller's (or caller's caller,
etc) control-flow graph.
- Similarly, there is an element denoted `end('static)` corresponding
to the remainder of program execution after this function returns.
- There is an element `!1` for each placeholder region `!1`. This
corresponds (intuitively) to some unknown set of other elements –
for details on placeholders, see the section
[placeholders and universes](#placeholder).
## Causal tracking
*to be written* – describe how we can extend the values of a variable
with causal tracking etc
<a name="placeholder"></a>
## Placeholders and universes
(This section describes ongoing work that hasn't landed yet.)
From time to time we have to reason about regions that we can't
concretely know. For example, consider this program:
```rust,ignore
// A function that needs a static reference
fn foo(x: &'static u32) { }
fn bar(f: for<'a> fn(&'a u32)) {
// ^^^^^^^^^^^^^^^^^^^ a function that can accept **any** reference
let x = 22;
f(&x);
}
fn main() {
bar(foo);
}
```
This program ought not to type-check: `foo` needs a static reference
for its argument, and `bar` wants to be given a function that that
accepts **any** reference (so it can call it with something on its
stack, for example). But *how* do we reject it and *why*?
### Subtyping and Placeholders
When we type-check `main`, and in particular the call `bar(foo)`, we
are going to wind up with a subtyping relationship like this one:
```text
fn(&'static u32) <: for<'a> fn(&'a u32)
---------------- -------------------
the type of `foo` the type `bar` expects
```
We handle this sort of subtyping by taking the variables that are
bound in the supertype and replacing them with
[universally quantified](../appendix/background.html#quantified)
representatives, written like `!1`. We call these regions "placeholder
regions" – they represent, basically, "some unknown region".
Once we've done that replacement, we have the following relation:
```text
fn(&'static u32) <: fn(&'!1 u32)
```
The key idea here is that this unknown region `'!1` is not related to
any other regions. So if we can prove that the subtyping relationship
is true for `'!1`, then it ought to be true for any region, which is
what we wanted.
So let's work through what happens next. To check if two functions are
subtypes, we check if their arguments have the desired relationship
(fn arguments are [contravariant](../appendix/background.html#variance), so
we swap the left and right here):
```text
&'!1 u32 <: &'static u32
```
According to the basic subtyping rules for a reference, this will be
true if `'!1: 'static`. That is – if "some unknown region `!1`" lives
outlives `'static`. Now, this *might* be true – after all, `'!1`
could be `'static` – but we don't *know* that it's true. So this
should yield up an error (eventually).
### What is a universe
In the previous section, we introduced the idea of a placeholder
region, and we denoted it `!1`. We call this number `1` the **universe
index**. The idea of a "universe" is that it is a set of names that
are in scope within some type or at some point. Universes are formed
into a tree, where each child extends its parents with some new names.
So the **root universe** conceptually contains global names, such as
the the lifetime `'static` or the type `i32`. In the compiler, we also
put generic type parameters into this root universe (in this sense,
there is not just one root universe, but one per item). So consider
this function `bar`:
```rust,ignore
struct Foo { }
fn bar<'a, T>(t: &'a T) {
...
}
```
Here, the root universe would consist of the lifetimes `'static` and
`'a`. In fact, although we're focused on lifetimes here, we can apply
the same concept to types, in which case the types `Foo` and `T` would
be in the root universe (along with other global types, like `i32`).
Basically, the root universe contains all the names that
[appear free](../appendix/background.html#free-vs-bound) in the body of `bar`.
Now let's extend `bar` a bit by adding a variable `x`:
```rust,ignore
fn bar<'a, T>(t: &'a T) {
let x: for<'b> fn(&'b u32) = ...;
}
```
Here, the name `'b` is not part of the root universe. Instead, when we
"enter" into this `for<'b>` (e.g., by replacing it with a placeholder), we will create
a child universe of the root, let's call it U1:
```text
U0 (root universe)
│
└─ U1 (child universe)
```
The idea is that this child universe U1 extends the root universe U0
with a new name, which we are identifying by its universe number:
`!1`.
Now let's extend `bar` a bit by adding one more variable, `y`:
```rust,ignore
fn bar<'a, T>(t: &'a T) {
let x: for<'b> fn(&'b u32) = ...;
let y: for<'c> fn(&'b u32) = ...;
}
```
When we enter *this* type, we will again create a new universe, which
we'll call `U2`. Its parent will be the root universe, and U1 will be
its sibling:
```text
U0 (root universe)
│
├─ U1 (child universe)
│
└─ U2 (child universe)
```
This implies that, while in U2, we can name things from U0 or U2, but
not U1.
**Giving existential variables a universe.** Now that we have this
notion of universes, we can use it to extend our type-checker and
things to prevent illegal names from leaking out. The idea is that we
give each inference (existential) variable – whether it be a type or
a lifetime – a universe. That variable's value can then only
reference names visible from that universe. So for example is a
lifetime variable is created in U0, then it cannot be assigned a value
of `!1` or `!2`, because those names are not visible from the universe
U0.
**Representing universes with just a counter.** You might be surprised
to see that the compiler doesn't keep track of a full tree of
universes. Instead, it just keeps a counter – and, to determine if
one universe can see another one, it just checks if the index is
greater. For example, U2 can see U0 because 2 >= 0. But U0 cannot see
U2, because 0 >= 2 is false.
How can we get away with this? Doesn't this mean that we would allow
U2 to also see U1? The answer is that, yes, we would, **if that
question ever arose**. But because of the structure of our type
checker etc, there is no way for that to happen. In order for
something happening in the universe U1 to "communicate" with something
happening in U2, they would have to have a shared inference variable X
in common. And because everything in U1 is scoped to just U1 and its
children, that inference variable X would have to be in U0. And since
X is in U0, it cannot name anything from U1 (or U2). This is perhaps easiest
to see by using a kind of generic "logic" example:
```text
exists<X> {
forall<Y> { ... /* Y is in U1 ... */ }
forall<Z> { ... /* Z is in U2 ... */ }
}
```
Here, the only way for the two foralls to interact would be through X,
but neither Y nor Z are in scope when X is declared, so its value
cannot reference either of them.
### Universes and placeholder region elements
But where does that error come from? The way it happens is like this.
When we are constructing the region inference context, we can tell
from the type inference context how many placeholder variables exist
(the `InferCtxt` has an internal counter). For each of those, we
create a corresponding universal region variable `!n` and a "region
element" `placeholder(n)`. This corresponds to "some unknown set of other
elements". The value of `!n` is `{placeholder(n)}`.
At the same time, we also give each existential variable a
**universe** (also taken from the `InferCtxt`). This universe
determines which placeholder elements may appear in its value: For
example, a variable in universe U3 may name `placeholder(1)`, `placeholder(2)`, and
`placeholder(3)`, but not `placeholder(4)`. Note that the universe of an inference
variable controls what region elements **can** appear in its value; it
does not say region elements **will** appear.
### Placeholders and outlives constraints
In the region inference engine, outlives constraints have the form:
```text
V1: V2 @ P
```
where `V1` and `V2` are region indices, and hence map to some region
variable (which may be universally or existentially quantified). The
`P` here is a "point" in the control-flow graph; it's not important
for this section. This variable will have a universe, so let's call
those universes `U(V1)` and `U(V2)` respectively. (Actually, the only
one we are going to care about is `U(V1)`.)
When we encounter this constraint, the ordinary procedure is to start
a DFS from `P`. We keep walking so long as the nodes we are walking
are present in `value(V2)` and we add those nodes to `value(V1)`. If
we reach a return point, we add in any `end(X)` elements. That part
remains unchanged.
But then *after that* we want to iterate over the placeholder `placeholder(x)`
elements in V2 (each of those must be visible to `U(V2)`, but we
should be able to just assume that is true, we don't have to check
it). We have to ensure that `value(V1)` outlives each of those
placeholder elements.
Now there are two ways that could happen. First, if `U(V1)` can see
the universe `x` (i.e., `x <= U(V1)`), then we can just add `placeholder(x)`
to `value(V1)` and be done. But if not, then we have to approximate:
we may not know what set of elements `placeholder(x)` represents, but we
should be able to compute some sort of **upper bound** B for it –
some region B that outlives `placeholder(x)`. For now, we'll just use
`'static` for that (since it outlives everything) – in the future, we
can sometimes be smarter here (and in fact we have code for doing this
already in other contexts). Moreover, since `'static` is in the root
universe U0, we know that all variables can see it – so basically if
we find that `value(V2)` contains `placeholder(x)` for some universe `x`
that `V1` can't see, then we force `V1` to `'static`.
### Extending the "universal regions" check
After all constraints have been propagated, the NLL region inference
has one final check, where it goes over the values that wound up being
computed for each universal region and checks that they did not get
'too large'. In our case, we will go through each placeholder region
and check that it contains *only* the `placeholder(u)` element it is known to
outlive. (Later, we might be able to know that there are relationships
between two placeholder regions and take those into account, as we do
for universal regions from the fn signature.)
Put another way, the "universal regions" check can be considered to be
checking constraints like:
```text
{placeholder(1)}: V1
```
where `{placeholder(1)}` is like a constant set, and V1 is the variable we
made to represent the `!1` region.
## Back to our example
OK, so far so good. Now let's walk through what would happen with our
first example:
```text
fn(&'static u32) <: fn(&'!1 u32) @ P // this point P is not imp't here
```
The region inference engine will create a region element domain like this:
```text
{ CFG; end('static); placeholder(1) }
--- ------------ ------- from the universe `!1`
| 'static is always in scope
all points in the CFG; not especially relevant here
```
It will always create two universal variables, one representing
`'static` and one representing `'!1`. Let's call them Vs and V1. They
will have initial values like so:
```text
Vs = { CFG; end('static) } // it is in U0, so can't name anything else
V1 = { placeholder(1) }
```
From the subtyping constraint above, we would have an outlives constraint like
```text
'!1: 'static @ P
```
To process this, we would grow the value of V1 to include all of Vs:
```text
Vs = { CFG; end('static) }
V1 = { CFG; end('static), placeholder(1) }
```
At that point, constraint propagation is complete, because all the
outlives relationships are satisfied. Then we would go to the "check
universal regions" portion of the code, which would test that no
universal region grew too large.
In this case, `V1` *did* grow too large – it is not known to outlive
`end('static)`, nor any of the CFG – so we would report an error.
## Another example
What about this subtyping relationship?
```text
for<'a> fn(&'a u32, &'a u32)
<:
for<'b, 'c> fn(&'b u32, &'c u32)
```
Here we would replace the bound region in the supertype with a placeholder, as before, yielding:
```text
for<'a> fn(&'a u32, &'a u32)
<:
fn(&'!1 u32, &'!2 u32)
```
then we instantiate the variable on the left-hand side with an
existential in universe U2, yielding the following (`?n` is a notation
for an existential variable):
```text
fn(&'?3 u32, &'?3 u32)
<:
fn(&'!1 u32, &'!2 u32)
```
Then we break this down further:
```text
&'!1 u32 <: &'?3 u32
&'!2 u32 <: &'?3 u32
```
and even further, yield up our region constraints:
```text
'!1: '?3
'!2: '?3
```
Note that, in this case, both `'!1` and `'!2` have to outlive the
variable `'?3`, but the variable `'?3` is not forced to outlive
anything else. Therefore, it simply starts and ends as the empty set
of elements, and hence the type-check succeeds here.
(This should surprise you a little. It surprised me when I first realized it.
We are saying that if we are a fn that **needs both of its arguments to have
the same region**, we can accept being called with **arguments with two
distinct regions**. That seems intuitively unsound. But in fact, it's fine, as
I tried to explain in [this issue][ohdeargoditsallbroken] on the Rust issue
tracker long ago. The reason is that even if we get called with arguments of
two distinct lifetimes, those two lifetimes have some intersection (the call
itself), and that intersection can be our value of `'a` that we use as the
common lifetime of our arguments. -nmatsakis)
[ohdeargoditsallbroken]: https://github.com/rust-lang/rust/issues/32330#issuecomment-202536977
## Final example
Let's look at one last example. We'll extend the previous one to have
a return type:
```text
for<'a> fn(&'a u32, &'a u32) -> &'a u32
<:
for<'b, 'c> fn(&'b u32, &'c u32) -> &'b u32
```
Despite seeming very similar to the previous example, this case is going to get
an error. That's good: the problem is that we've gone from a fn that promises
to return one of its two arguments, to a fn that is promising to return the
first one. That is unsound. Let's see how it plays out.
First, we replace the bound region in the supertype with a placeholder:
```text
for<'a> fn(&'a u32, &'a u32) -> &'a u32
<:
fn(&'!1 u32, &'!2 u32) -> &'!1 u32
```
Then we instantiate the subtype with existentials (in U2):
```text
fn(&'?3 u32, &'?3 u32) -> &'?3 u32
<:
fn(&'!1 u32, &'!2 u32) -> &'!1 u32
```
And now we create the subtyping relationships:
```text
&'!1 u32 <: &'?3 u32 // arg 1
&'!2 u32 <: &'?3 u32 // arg 2
&'?3 u32 <: &'!1 u32 // return type
```
And finally the outlives relationships. Here, let V1, V2, and V3 be the
variables we assign to `!1`, `!2`, and `?3` respectively:
```text
V1: V3
V2: V3
V3: V1
```
Those variables will have these initial values:
```text
V1 in U1 = {placeholder(1)}
V2 in U2 = {placeholder(2)}
V3 in U2 = {}
```
Now because of the `V3: V1` constraint, we have to add `placeholder(1)` into `V3` (and
indeed it is visible from `V3`), so we get:
```text
V3 in U2 = {placeholder(1)}
```
then we have this constraint `V2: V3`, so we wind up having to enlarge
`V2` to include `placeholder(1)` (which it can also see):
```text
V2 in U2 = {placeholder(1), placeholder(2)}
```
Now constraint propagation is done, but when we check the outlives
relationships, we find that `V2` includes this new element `placeholder(1)`,
so we report an error.

10
src/doc/rustc-guide/src/borrow_check/type_check.md Normal file
View File

@ -0,0 +1,10 @@
# The MIR type-check
A key component of the borrow check is the
[MIR type-check](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_mir/borrow_check/nll/type_check/index.html).
This check walks the MIR and does a complete "type check" -- the same
kind you might find in any other language. In the process of doing
this type-check, we also uncover the region constraints that apply to
the program.
TODO -- elaborate further? Maybe? :)

29
src/doc/rustc-guide/src/build-install-distribution-artifacts.md Normal file
View File

@ -0,0 +1,29 @@
# Build distribution artifacts
You might want to build and package up the compiler for distribution.
You’ll want to run this command to do it:
```bash
./x.py dist
```
# Install distribution artifacts
If you’ve built a distribution artifact you might want to install it and
test that it works on your target system. You’ll want to run this command:
```bash
./x.py install
```
Note: If you are testing out a modification to a compiler, you
might want to use it to compile some project.
Usually, you do not want to use ./x.py install for testing.
Rather, you should create a toolchain as discussed in
[here][create-rustup-toolchain].
For example, if the toolchain you created is called foo, you
would then invoke it with `rustc +foo ...` (where ... represents
the rest of the arguments).
[create-rustup-toolchain]: ./how-to-build-and-run.md#creating-a-rustup-toolchain

54
src/doc/rustc-guide/src/codegen.md Normal file
View File

@ -0,0 +1,54 @@
# Code generation
Code generation or "codegen" is the part of the compiler that actually
generates an executable binary. rustc uses LLVM for code generation.
> NOTE: If you are looking for hints on how to debug code generation bugs,
> please see [this section of the debugging chapter][debug].
[debug]: compiler-debugging.html#debugging-llvm
## What is LLVM?
All of the preceding chapters of this guide have one thing in common: we never
generated any executable machine code at all! With this chapter, all of that
changes.
Like most compilers, rustc is composed of a "frontend" and a "backend". The
"frontend" is responsible for taking raw source code, checking it for
correctness, and getting it into a format `X` from which we can generate
executable machine code. The "backend" then takes that format `X` and produces
(possibly optimized) executable machine code for some platform. All of the
previous chapters deal with rustc's frontend.
rustc's backend is [LLVM](https://llvm.org), "a collection of modular and
reusable compiler and toolchain technologies". In particular, the LLVM project
contains a pluggable compiler backend (also called "LLVM"), which is used by
many compiler projects, including the `clang` C compiler and our beloved
`rustc`.
LLVM's "format `X`" is called LLVM IR. It is basically assembly code with
additional low-level types and annotations added. These annotations are helpful
for doing optimizations on the LLVM IR and outputted machine code. The end
result of all this is (at long last) something executable (e.g. an ELF object
or wasm).
There are a few benefits to using LLVM:
- We don't have to write a whole compiler backend. This reduces implementation
and maintenance burden.
- We benefit from the large suite of advanced optimizations that the LLVM
project has been collecting.
- We automatically can compile Rust to any of the platforms for which LLVM has
support. For example, as soon as LLVM added support for wasm, voila! rustc,
clang, and a bunch of other languages were able to compile to wasm! (Well,
there was some extra stuff to be done, but we were 90% there anyway).
- We and other compiler projects benefit from each other. For example, when the
[Spectre and Meltdown security vulnerabilities][spectre] were discovered,
only LLVM needed to be patched.
[spectre]: https://meltdownattack.com/
## Generating LLVM IR
TODO

390
src/doc/rustc-guide/src/compiler-debugging.md Normal file
View File

@ -0,0 +1,390 @@
**Note: This is copied from the
[rust-forge](https://github.com/rust-lang-nursery/rust-forge). If anything needs
updating, please open an issue or make a PR on the github repo.**
# Debugging the compiler
[debugging]: #debugging
Here are a few tips to debug the compiler:
## Getting a backtrace
[getting-a-backtrace]: #getting-a-backtrace
When you have an ICE (panic in the compiler), you can set
`RUST_BACKTRACE=1` to get the stack trace of the `panic!` like in
normal Rust programs. IIRC backtraces **don't work** on Mac and on MinGW,
sorry. If you have trouble or the backtraces are full of `unknown`,
you might want to find some way to use Linux or MSVC on Windows.
In the default configuration, you don't have line numbers enabled, so the
backtrace looks like this:
```text
stack backtrace:
0: std::sys::imp::backtrace::tracing::imp::unwind_backtrace
1: std::sys_common::backtrace::_print
2: std::panicking::default_hook::{{closure}}
3: std::panicking::default_hook
4: std::panicking::rust_panic_with_hook
5: std::panicking::begin_panic
(~~~~ LINES REMOVED BY ME FOR BREVITY ~~~~)
32: rustc_typeck::check_crate
33: <std::thread::local::LocalKey<T>>::with
34: <std::thread::local::LocalKey<T>>::with
35: rustc::ty::context::TyCtxt::create_and_enter
36: rustc_driver::driver::compile_input
37: rustc_driver::run_compiler
```
If you want line numbers for the stack trace, you can enable
`debuginfo-lines=true` or `debuginfo=true` in your config.toml and rebuild the
compiler. Then the backtrace will look like this:
```text
stack backtrace:
(~~~~ LINES REMOVED BY ME FOR BREVITY ~~~~)
at /home/user/rust/src/librustc_typeck/check/cast.rs:110
7: rustc_typeck::check::cast::CastCheck::check
at /home/user/rust/src/librustc_typeck/check/cast.rs:572
at /home/user/rust/src/librustc_typeck/check/cast.rs:460
at /home/user/rust/src/librustc_typeck/check/cast.rs:370
(~~~~ LINES REMOVED BY ME FOR BREVITY ~~~~)
33: rustc_driver::driver::compile_input
at /home/user/rust/src/librustc_driver/driver.rs:1010
at /home/user/rust/src/librustc_driver/driver.rs:212
34: rustc_driver::run_compiler
at /home/user/rust/src/librustc_driver/lib.rs:253
```
## Getting a backtrace for errors
[getting-a-backtrace-for-errors]: #getting-a-backtrace-for-errors
If you want to get a backtrace to the point where the compiler emits
an error message, you can pass the `-Z treat-err-as-bug`, which
will make the compiler panic on the first error it sees.
This can also help when debugging `delay_span_bug` calls - it will make
the first `delay_span_bug` call panic, which will give you a useful backtrace.
For example:
```bash
$ cat error.rs
fn main() {
1 + ();
}
```
```bash
$ ./build/x86_64-unknown-linux-gnu/stage1/bin/rustc error.rs
error[E0277]: the trait bound `{integer}: std::ops::Add<()>` is not satisfied
--> error.rs:2:7
|
2 | 1 + ();
| ^ no implementation for `{integer} + ()`
|
= help: the trait `std::ops::Add<()>` is not implemented for `{integer}`
error: aborting due to previous error
$ # Now, where does the error above come from?
$ RUST_BACKTRACE=1 \
./build/x86_64-unknown-linux-gnu/stage1/bin/rustc \
error.rs \
-Z treat-err-as-bug
error[E0277]: the trait bound `{integer}: std::ops::Add<()>` is not satisfied
--> error.rs:2:7
|
2 | 1 + ();
| ^ no implementation for `{integer} + ()`
|
= help: the trait `std::ops::Add<()>` is not implemented for `{integer}`
error: internal compiler error: unexpected panic
note: the compiler unexpectedly panicked. this is a bug.
note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports
note: rustc 1.24.0-dev running on x86_64-unknown-linux-gnu
note: run with `RUST_BACKTRACE=1` for a backtrace
thread 'rustc' panicked at 'encountered error with `-Z treat_err_as_bug',
/home/user/rust/src/librustc_errors/lib.rs:411:12
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose
backtrace.
stack backtrace:
(~~~ IRRELEVANT PART OF BACKTRACE REMOVED BY ME ~~~)
7: rustc::traits::error_reporting::<impl rustc::infer::InferCtxt<'a, 'gcx,
'tcx>>::report_selection_error
at /home/user/rust/src/librustc/traits/error_reporting.rs:823
8: rustc::traits::error_reporting::<impl rustc::infer::InferCtxt<'a, 'gcx,
'tcx>>::report_fulfillment_errors
at /home/user/rust/src/librustc/traits/error_reporting.rs:160
at /home/user/rust/src/librustc/traits/error_reporting.rs:112
9: rustc_typeck::check::FnCtxt::select_obligations_where_possible
at /home/user/rust/src/librustc_typeck/check/mod.rs:2192
(~~~ IRRELEVANT PART OF BACKTRACE REMOVED BY ME ~~~)
36: rustc_driver::run_compiler
at /home/user/rust/src/librustc_driver/lib.rs:253
$ # Cool, now I have a backtrace for the error
```
## Getting logging output
[getting-logging-output]: #getting-logging-output
The compiler has a lot of `debug!` calls, which print out logging information
at many points. These are very useful to at least narrow down the location of
a bug if not to find it entirely, or just to orient yourself as to why the
compiler is doing a particular thing.
To see the logs, you need to set the `RUST_LOG` environment variable to
your log filter, e.g. to get the logs for a specific module, you can run the
compiler as `RUST_LOG=module::path rustc my-file.rs`. The Rust logs are
powered by [env-logger], and you can look at the docs linked there to see
the full `RUST_LOG` syntax. All `debug!` output will then appear in
standard error.
Note that unless you use a very strict filter, the logger will emit a *lot*
of output - so it's typically a good idea to pipe standard error to a file
and look at the log output with a text editor.
So to put it together.
```bash
# This puts the output of all debug calls in `librustc/traits` into
# standard error, which might fill your console backscroll.
$ RUST_LOG=rustc::traits rustc +local my-file.rs
# This puts the output of all debug calls in `librustc/traits` in
# `traits-log`, so you can then see it with a text editor.
$ RUST_LOG=rustc::traits rustc +local my-file.rs 2>traits-log
# Not recommended. This will show the output of all `debug!` calls
# in the Rust compiler, and there are a *lot* of them, so it will be
# hard to find anything.
$ RUST_LOG=debug rustc +local my-file.rs 2>all-log
# This will show the output of all `info!` calls in `rustc_trans`.
#
# There's an `info!` statement in `trans_instance` that outputs
# every function that is translated. This is useful to find out
# which function triggers an LLVM assertion, and this is an `info!`
# log rather than a `debug!` log so it will work on the official
# compilers.
$ RUST_LOG=rustc_trans=info rustc +local my-file.rs
```
While calls to `info!` are included in every build of the compiler,
calls to `debug!` are only included in the program if the
`debug-assertions=yes` is turned on in config.toml (it is
turned off by default), so if you don't see `DEBUG` logs, especially
if you run the compiler with `RUST_LOG=rustc rustc some.rs` and only see
`INFO` logs, make sure that `debug-assertions=yes` is turned on in your
config.toml.
I also think that in some cases just setting it will not trigger a rebuild,
so if you changed it and you already have a compiler built, you might
want to call `x.py clean` to force one.
### Logging etiquette
Because calls to `debug!` are removed by default, in most cases, don't worry
about adding "unnecessary" calls to `debug!` and leaving them in code you
commit - they won't slow down the performance of what we ship, and if they
helped you pinning down a bug, they will probably help someone else with a
different one.
However, there are still a few concerns that you might care about:
### Expensive operations in logs
A note of caution: the expressions *within* the `debug!` call are run
whenever RUST_LOG is set, even if the filter would exclude the log. This means
that if in the module `rustc::foo` you have a statement
```Rust
debug!("{:?}", random_operation(tcx));
```
Then if someone runs a debug `rustc` with `RUST_LOG=rustc::bar`, then
`random_operation()` will still run - even while it's output will never be
needed!
This means that you should not put anything too expensive or likely
to crash there - that would annoy anyone who wants to use logging for their own
module. Note that if `RUST_LOG` is unset (the default), then the code will not
run - this means that if your logging code panics, then no-one will know it
until someone tries to use logging to find *another* bug.
If you *need* to do an expensive operation in a log, be aware that while log
expressions are *evaluated* even if logging is not enabled in your module,
they are not *formatted* unless it *is*. This means you can put your
expensive/crashy operations inside an `fmt::Debug` impl, and they will not be
run unless your log is enabled:
```Rust
use std::fmt;
struct ExpensiveOperationContainer<'a, 'gcx, 'tcx>
where 'tcx: 'gcx, 'a: 'tcx
{
tcx: TyCtxt<'a, 'gcx, 'tcx>
}
impl<'a, 'gcx, 'tcx> fmt::Debug for ExpensiveOperationContainer<'a, 'gcx, 'tcx> {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
let value = random_operation(tcx);
fmt::Debug::fmt(&value, fmt)
}
}
debug!("{:?}", ExpensiveOperationContainer { tcx });
```
## Formatting Graphviz output (.dot files)
[formatting-graphviz-output]: #formatting-graphviz-output
Some compiler options for debugging specific features yield graphviz graphs -
e.g. the `#[rustc_mir(borrowck_graphviz_postflow="suffix.dot")]` attribute
dumps various borrow-checker dataflow graphs.
These all produce `.dot` files. To view these files, install graphviz (e.g.
`apt-get install graphviz`) and then run the following commands:
```bash
$ dot -T pdf maybe_init_suffix.dot > maybe_init_suffix.pdf
$ firefox maybe_init_suffix.pdf # Or your favorite pdf viewer
```
## Debugging LLVM
[debugging-llvm]: #debugging-llvm
> NOTE: If you are looking for info about code generation, please see [this
> chapter][codegen] instead.
[codegen]: codegen.html
This section is about debugging compiler bugs in code generation (e.g. why the
compiler generated some piece of code or crashed in LLVM). LLVM is a big
project on its own that probably needs to have its own debugging document (not
that I could find one). But here are some tips that are important in a rustc
context:
As a general rule, compilers generate lots of information from analyzing code.
Thus, a useful first step is usually to find a minimal example. One way to do
this is to
1. create a new crate that reproduces the issue (e.g. adding whatever crate is
at fault as a dependency, and using it from there)
2. minimize the crate by removing external dependencies; that is, moving
everything relevant to the new crate
3. further minimize the issue by making the code shorter (there are tools that
help with this like `creduce`)
The official compilers (including nightlies) have LLVM assertions disabled,
which means that LLVM assertion failures can show up as compiler crashes (not
ICEs but "real" crashes) and other sorts of weird behavior. If you are
encountering these, it is a good idea to try using a compiler with LLVM
assertions enabled - either an "alt" nightly or a compiler you build yourself
by setting `[llvm] assertions=true` in your config.toml - and see whether
anything turns up.
The rustc build process builds the LLVM tools into
`./build/<host-triple>/llvm/bin`. They can be called directly.
The default rustc compilation pipeline has multiple codegen units, which is
hard to replicate manually and means that LLVM is called multiple times in
parallel. If you can get away with it (i.e. if it doesn't make your bug
disappear), passing `-C codegen-units=1` to rustc will make debugging easier.
To rustc to generate LLVM IR, you need to pass the `--emit=llvm-ir` flag. If
you are building via cargo, use the `RUSTFLAGS` environment variable (e.g.
`RUSTFLAGS='--emit=llvm-ir'`). This causes rustc to spit out LLVM IR into the
target directory.
`cargo llvm-ir [options] path` spits out the LLVM IR for a particular function
at `path`. (`cargo install cargo-asm` installs `cargo asm` and `cargo
llvm-ir`). `--build-type=debug` emits code for debug builds. There are also
other useful options. Also, debug info in LLVM IR can clutter the output a lot:
`RUSTFLAGS="-C debuginfo=0"` is really useful.
`RUSTFLAGS="-C save-temps"` outputs LLVM bitcode (not the same as IR) at
different stages during compilation, which is sometimes useful. One just needs
to convert the bitcode files to `.ll` files using `llvm-dis` which should be in
the target local compilation of rustc.
If you want to play with the optimization pipeline, you can use the `opt` tool
from `./build/<host-triple>/llvm/bin/` with the LLVM IR emitted by rustc. Note
that rustc emits different IR depending on whether `-O` is enabled, even
without LLVM's optimizations, so if you want to play with the IR rustc emits,
you should:
```bash
$ rustc +local my-file.rs --emit=llvm-ir -O -C no-prepopulate-passes \
-C codegen-units=1
$ OPT=./build/$TRIPLE/llvm/bin/opt
$ $OPT -S -O2 < my-file.ll > my
```
If you just want to get the LLVM IR during the LLVM pipeline, to e.g. see which
IR causes an optimization-time assertion to fail, or to see when LLVM performs
a particular optimization, you can pass the rustc flag `-C
llvm-args=-print-after-all`, and possibly add `-C
llvm-args='-filter-print-funcs=EXACT_FUNCTION_NAME` (e.g. `-C
llvm-args='-filter-print-funcs=_ZN11collections3str21_$LT$impl$u20$str$GT$\
7replace17hbe10ea2e7c809b0bE'`).
That produces a lot of output into standard error, so you'll want to pipe that
to some file. Also, if you are using neither `-filter-print-funcs` nor `-C
codegen-units=1`, then, because the multiple codegen units run in parallel, the
printouts will mix together and you won't be able to read anything.
If you want just the IR for a specific function (say, you want to see why it
causes an assertion or doesn't optimize correctly), you can use `llvm-extract`,
e.g.
```bash
$ ./build/$TRIPLE/llvm/bin/llvm-extract \
-func='_ZN11collections3str21_$LT$impl$u20$str$GT$7replace17hbe10ea2e7c809b0bE' \
-S \
< unextracted.ll \
> extracted.ll
```
### Filing LLVM bug reports
When filing an LLVM bug report, you will probably want some sort of minimal
working example that demonstrates the problem. The Godbolt compiler explorer is
really helpful for this.
1. Once you have some LLVM IR for the problematic code (see above), you can
create a minimal working example with Godbolt. Go to
[gcc.godbolt.org](https://gcc.godbolt.org).
2. Choose `LLVM-IR` as programming language.
3. Use `llc` to compile the IR to a particular target as is:
- There are some useful flags: `-mattr` enables target features, `-march=`
selects the target, `-mcpu=` selects the CPU, etc.
- Commands like `llc -march=help` output all architectures available, which
is useful because sometimes the Rust arch names and the LLVM names do not
match.
- If you have compiled rustc yourself somewhere, in the target directory
you have binaries for `llc`, `opt`, etc.
4. If you want to optimize the LLVM-IR, you can use `opt` to see how the LLVM
optimizations transform it.
5. Once you have a godbolt link demonstrating the issue, it is pretty easy to
fill in an LLVM bug.
[env-logger]: https://docs.rs/env_logger/0.4.3/env_logger/
## Narrowing (Bisecting) Regressions
The [cargo-bisect-rustc](https://github.com/rust-lang-nursery/cargo-bisect-rustc) tool can be used as a quick and easy way to find exactly which PR caused a change in `rustc` behavior. It automatically downloads `rustc` PR artifacts and tests them against a project you provide until it finds the regression. You can then look at the PR to get more context on *why* it was changed. See [this tutorial](https://github.com/rust-lang-nursery/cargo-bisect-rustc/blob/master/TUTORIAL.md) on how to use it.

62
src/doc/rustc-guide/src/compiler-documenting.md Normal file
View File

@ -0,0 +1,62 @@
# Documenting rustc
You might want to build documentation of the various components
available like the standard library. There’s two ways to go about this.
You can run rustdoc directly on the file to make sure the HTML is
correct, which is fast. Alternatively, you can build the documentation
as part of the build process through x.py. Both are viable methods
since documentation is more about the content.
## Document everything
```bash
./x.py doc
```
## If you want to avoid the whole Stage 2 build
```bash
./x.py doc --stage 1
```
First the compiler and rustdoc get built to make sure everything is okay
and then it documents the files.
## Document specific components
```bash
./x.py doc src/doc/book
./x.py doc src/doc/nomicon
./x.py doc src/doc/book src/libstd
```
Much like individual tests or building certain components you can build only
the documentation you want.
## Document internal rustc items
Compiler documentation is not built by default. There's a flag in
config.toml for achieving the same.
But, when enabled, compiler documentation does include internal items.
Next open up config.toml and make sure these two lines are set to true:
```bash
docs = true
compiler-docs = true
```
When you want to build the compiler docs as well run this command:
```bash
./x.py doc
```
This will see that the docs and compiler-docs options are set to true
and build the normally hidden compiler docs!
### Compiler Documentation
The documentation for the rust components are found at [rustc doc].
[rustc doc]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/

119
src/doc/rustc-guide/src/compiler-team.md Normal file
View File

@ -0,0 +1,119 @@
# About the compiler team
rustc is maintained by the
[Rust compiler team](https://www.rust-lang.org/en-US/team.html). The
people who belong to this team collectively work to track regressions
and implement new features. Members of the Rust compiler team are
people who have made significant contributions to rustc and its
design.
## Discussion
Currently the compiler team chats in a number of places. There is an
ongoing [thread] on the internals board about trying to find a permanent
home. In any case, you can find people in one of three places at the moment:
- The `#rustc` channel on mozilla's IRC (`irc.mozilla.org`)
- The `t-compiler` stream on [the Zulip instance](https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler)
- The `compiler` channel on the [rust-lang discord](https://discord.gg/rust-lang)
## Rust compiler meeting
The compiler team has a weekly meeting where we do triage and try to
generally stay on top of new bugs, regressions, and other things. This
general plan for this meeting can be found in
[the rust-compiler-meeting etherpad][etherpad]. It works roughly as
follows:
- **Review P-high bugs:** P-high bugs are those that are sufficiently
important for us to actively track progress. P-high bugs should
ideally always have an assignee.
- **Look over new regressions:** we then look for new cases where the
compiler broke previously working code in the wild. Regressions are
almost always marked as P-high; the major exception would be bug
fixes (though even there we often
[aim to give warnings first][procedure]).
- **Check I-nominated issues:** These are issues where feedback from
the team is desired.
- **Check for beta nominations:** These are nominations of things to
backport to beta.
The meeting currently takes place on Thursdays at 10am Boston time
(UTC-4 typically, but daylight savings time sometimes makes things
complicated).
The meeting is held over a "chat medium" — it used to be IRC, but we
are currently in the process of evaluating other alternatives. Check
the [etherpad] to find the current home (and see
[this internals thread][thread] for some ongoing discussion).
[etherpad]: https://public.etherpad-mozilla.org/p/rust-compiler-meeting
[thread]: https://internals.rust-lang.org/t/where-should-the-compiler-team-and-perhaps-working-groups-chat/7894
[procedure]: https://forge.rust-lang.org/rustc-bug-fix-procedure.html
## Team membership
Membership in the Rust team is typically offered when someone has been
making significant contributions to the compiler for some
time. Membership is both a recognition but also an obligation:
compiler team members are generally expected to help with upkeep as
well as doing reviews and other work.
If you are interested in becoming a compiler team member, the first
thing to do is to start fixing some bugs, or get involved in a working
group. One good way to find bugs is to look for
[open issues tagged with E-easy](https://github.com/rust-lang/rust/issues?q=is%3Aopen+is%3Aissue+label%3AE-easy)
or
[E-mentor](https://github.com/rust-lang/rust/issues?q=is%3Aopen+is%3Aissue+label%3AE-mentor).
### r+ rights
Once you have made a number of individual PRs to rustc, we will often
offer r+ privileges. This means that you have the right to instruct
"bors" (the robot that manages which PRs get landed into rustc) to
merge a PR
([here are some instructions for how to talk to bors][homu-guide]).
[homu-guide]: https://buildbot2.rust-lang.org/homu/
The guidelines for reviewers are as follows:
- You are always welcome to review any PR, regardless of who it is
assigned to. However, do not r+ PRs unless:
- You are confident in that part of the code.
- You are confident that nobody else wants to review it first.
- For example, sometimes people will express a desire to review a
PR before it lands, perhaps because it touches a particularly
sensitive part of the code.
- Always be polite when reviewing: you are a representative of the
Rust project, so it is expected that you will go above and beyond
when it comes to the [Code of Conduct].
[Code of Conduct]: https://www.rust-lang.org/en-US/conduct.html
### high-five
Once you have r+ rights, you can also be added to the high-five
rotation. high-five is the bot that assigns incoming PRs to
reviewers. If you are added, you will be randomly selected to review
PRs. If you find you are assigned a PR that you don't feel comfortable
reviewing, you can also leave a comment like `r? @so-and-so` to assign
to someone else — if you don't know who to request, just write `r?
@nikomatsakis for reassignment` and @nikomatsakis will pick someone
for you.
[hi5]: https://github.com/rust-highfive
Getting on the high-five list is much appreciated as it lowers the
review burden for all of us! However, if you don't have time to give
people timely feedback on their PRs, it may be better that you don't
get on the list.
### Full team membership
Full team membership is typically extended once someone made many
contributions to the Rust compiler over time, ideally (but not
necessarily) to multiple areas. Sometimes this might be implementing a
new feature, but it is also important — perhaps more important! — to
have time and willingness to help out with general upkeep such as
bugfixes, tracking regressions, and other less glamorous work.

226
src/doc/rustc-guide/src/compiletest.md Normal file
View File

@ -0,0 +1,226 @@
# `compiletest`
## Introduction
`compiletest` is the main test harness of the Rust test suite. It allows
test authors to organize large numbers of tests (the Rust compiler has many
thousands), efficient test execution (parallel execution is supported), and
allows the test author to configure behavior and expected results of both
individual and groups of tests.
`compiletest` tests may check test code for success, for failure or in some
cases, even failure to compile. Tests are typically organized as a Rust source
file with annotations in comments before and/or within the test code, which
serve to direct `compiletest` on if or how to run the test, what behavior to
expect, and more. If you are unfamiliar with the compiler testing framework,
see [this chapter](./tests/intro.html) for additional background.
The tests themselves are typically (but not always) organized into
"suites" – for example, `run-pass`, a folder representing tests that should
succeed, `run-fail`, a folder holding tests that should compile successfully,
but return a failure (non-zero status), `compile-fail`, a folder holding tests
that should fail to compile, and many more. The various suites are defined in
[src/tools/compiletest/src/common.rs][common] in the `pub struct Config`
declaration. And a very good introduction to the different suites of compiler
tests along with details about them can be found in [Adding new
tests](./tests/adding.html).
## Adding a new test file
Briefly, simply create your new test in the appropriate location under
[src/test][test]. No registration of test files is necessary as `compiletest`
will scan the [src/test][test] subfolder recursively, and will execute any Rust
source files it finds as tests. See [`Adding new tests`](./tests/adding.html)
for a complete guide on how to adding new tests.
## Header Commands
Source file annotations which appear in comments near the top of the source
file *before* any test code are known as header commands. These commands can
instruct `compiletest` to ignore this test, set expectations on whether it is
expected to succeed at compiling, or what the test's return code is expected to
be. Header commands (and their inline counterparts, Error Info commands) are
described more fully
[here](./tests/adding.html#header-commands-configuring-rustc).
### Adding a new header command
Header commands are defined in the `TestProps` struct in
[src/tools/compiletest/src/header.rs][header]. At a high level, there are
dozens of test properties defined here, all set to default values in the
`TestProp` struct's `impl` block. Any test can override this default value by
specifying the property in question as header command as a comment (`//`) in
the test source file, before any source code.
#### Using a header command
Here is an example, specifying the `must-compile-successfully` header command,
which takes no arguments, followed by the `failure-status` header command,
which takes a single argument (which, in this case is a value of 1).
`failure-status` is instructing `compiletest` to expect a failure status of 1
(rather than the current Rust default of 101 at the time of this writing). The
header command and the argument list (if present) are typically separated by a
colon:
```rust,ignore
// must-compile-successfully
// failure-status: 1
#![feature(termination_trait)]
use std::io::{Error, ErrorKind};
fn main() -> Result<(), Box<Error>> {
Err(Box::new(Error::new(ErrorKind::Other, "returned Box<Error> from main()")))
}
```
#### Adding a new header command property
One would add a new header command if there is a need to define some test
property or behavior on an individual, test-by-test basis. A header command
property serves as the header command's backing store (holds the command's
current value) at runtime.
To add a new header command property:
1. Look for the `pub struct TestProps` declaration in
[src/tools/compiletest/src/header.rs][header] and add the new public
property to the end of the declaration.
2. Look for the `impl TestProps` implementation block immediately following
the struct declaration and initialize the new property to its default
value.
#### Adding a new header command parser
When `compiletest` encounters a test file, it parses the file a line at a time
by calling every parser defined in the `Config` struct's implementation block,
also in [src/tools/compiletest/src/header.rs][header] (note the `Config`
struct's declaration block is found in
[src/tools/compiletest/src/common.rs][common]. `TestProps`'s `load_from()`
method will try passing the current line of text to each parser, which, in turn
typically checks to see if the line begins with a particular commented (`//`)
header command such as `// must-compile-successfully` or `// failure-status`.
Whitespace after the comment marker is optional.
Parsers will override a given header command property's default value merely by
being specified in the test file as a header command or by having a parameter
value specified in the test file, depending on the header command.
Parsers defined in `impl Config` are typically named `parse_<header_command>`
(note kebab-case `<header-command>` transformed to snake-case
`<header_command>`). `impl Config` also defines several 'low-level' parsers
which make it simple to parse common patterns like simple presence or not
(`parse_name_directive()`), header-command:parameter(s)
(`parse_name_value_directive()`), optional parsing only if a particular `cfg`
attribute is defined (`has_cfg_prefix()`) and many more. The low-level parsers
are found near the end of the `impl Config` block; be sure to look through them
and their associated parsers immediately above to see how they are used to
avoid writing additional parsing code unnecessarily.
As a concrete example, here is the implementation for the
`parse_failure_status()` parser, in
[src/tools/compiletest/src/header.rs][header]:
```diff
@@ -232,6 +232,7 @@ pub struct TestProps {
// customized normalization rules
pub normalize_stdout: Vec<(String, String)>,
pub normalize_stderr: Vec<(String, String)>,
+ pub failure_status: i32,
}
impl TestProps {
@@ -260,6 +261,7 @@ impl TestProps {
run_pass: false,
normalize_stdout: vec![],
normalize_stderr: vec![],
+ failure_status: 101,
}
}
@@ -383,6 +385,10 @@ impl TestProps {
if let Some(rule) = config.parse_custom_normalization(ln, "normalize-stderr") {
self.normalize_stderr.push(rule);
}
+
+ if let Some(code) = config.parse_failure_status(ln) {
+ self.failure_status = code;
+ }
});
for key in &["RUST_TEST_NOCAPTURE", "RUST_TEST_THREADS"] {
@@ -488,6 +494,13 @@ impl Config {
self.parse_name_directive(line, "pretty-compare-only")
}
+ fn parse_failure_status(&self, line: &str) -> Option<i32> {
+ match self.parse_name_value_directive(line, "failure-status") {
+ Some(code) => code.trim().parse::<i32>().ok(),
+ _ => None,
+ }
+ }
```
## Implementing the behavior change
When a test invokes a particular header command, it is expected that some
behavior will change as a result. What behavior, obviously, will depend on the
purpose of the header command. In the case of `failure-status`, the behavior
that changes is that `compiletest` expects the failure code defined by the
header command invoked in the test, rather than the default value.
Although specific to `failure-status` (as every header command will have a
different implementation in order to invoke behavior change) perhaps it is
helpful to see the behavior change implementation of one case, simply as an
example. To implement `failure-status`, the `check_correct_failure_status()`
function found in the `TestCx` implementation block, located in
[src/tools/compiletest/src/runtest.rs](https://github.com/rust-lang/rust/tree/master/src/tools/compiletest/src/runtest.rs),
was modified as per below:
```diff
@@ -295,11 +295,14 @@ impl<'test> TestCx<'test> {
}
fn check_correct_failure_status(&self, proc_res: &ProcRes) {
- // The value the rust runtime returns on failure
- const RUST_ERR: i32 = 101;
- if proc_res.status.code() != Some(RUST_ERR) {
+ let expected_status = Some(self.props.failure_status);
+ let received_status = proc_res.status.code();
+
+ if expected_status != received_status {
self.fatal_proc_rec(
- &format!("failure produced the wrong error: {}", proc_res.status),
+ &format!("Error: expected failure status ({:?}) but received status {:?}.",
+ expected_status,
+ received_status),
proc_res,
);
}
@@ -320,7 +323,6 @@ impl<'test> TestCx<'test> {
);
let proc_res = self.exec_compiled_test();
-
if !proc_res.status.success() {
self.fatal_proc_rec("test run failed!", &proc_res);
}
@@ -499,7 +501,6 @@ impl<'test> TestCx<'test> {
expected,
actual
);
- panic!();
}
}
```
Note the use of `self.props.failure_status` to access the header command
property. In tests which do not specify the failure status header command,
`self.props.failure_status` will evaluate to the default value of 101 at the
time of this writing. But for a test which specifies a header command of, for
example, `// failure-status: 1`, `self.props.failure_status` will evaluate to
1, as `parse_failure_status()` will have overridden the `TestProps` default
value, for that test specifically.
[test]: https://github.com/rust-lang/rust/tree/master/src/test
[header]: https://github.com/rust-lang/rust/tree/master/src/tools/compiletest/src/header.rs
[common]: https://github.com/rust-lang/rust/tree/master/src/tools/compiletest/src/common.rs

38
src/doc/rustc-guide/src/const-eval.md Normal file
View File

@ -0,0 +1,38 @@
# Constant Evaluation
Constant evaluation is the process of computing values at compile time. For a
specific item (constant/static/array length) this happens after the MIR for the
item is borrow-checked and optimized. In many cases trying to const evaluate an
item will trigger the computation of its MIR for the first time.
Prominent examples are
* The initializer of a `static`
* Array length
* needs to be known to reserve stack or heap space
* Enum variant discriminants
* needs to be known to prevent two variants from having the same
discriminant
* Patterns
* need to be known to check for overlapping patterns
Additionally constant evaluation can be used to reduce the workload or binary
size at runtime by precomputing complex operations at compiletime and only
storing the result.
Constant evaluation can be done by calling the `const_eval` query of `TyCtxt`.
The `const_eval` query takes a [`ParamEnv`](./param_env.html) of environment in
which the constant is evaluated (e.g. the function within which the constant is
used) and a `GlobalId`. The `GlobalId` is made up of an
`Instance` referring to a constant or static or of an
`Instance` of a function and an index into the function's `Promoted` table.
Constant evaluation returns a `Result` with either the error, or the simplest
representation of the constant. "simplest" meaning if it is representable as an
integer or fat pointer, it will directly yield the value (via `Value::ByVal` or
`Value::ByValPair`), instead of referring to the [`miri`](./miri.html) virtual
memory allocation (via `Value::ByRef`). This means that the `const_eval`
function cannot be used to create miri-pointers to the evaluated constant or
static. If you need that, you need to directly work with the functions in
[src/librustc_mir/const_eval.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_mir/const_eval/index.html).

134
src/doc/rustc-guide/src/conventions.md Normal file
View File

@ -0,0 +1,134 @@
This file offers some tips on the coding conventions for rustc. This
chapter covers [formatting](#formatting), [coding for correctness](#cc),
[using crates from crates.io](#cio), and some tips on
[structuring your PR for easy review](#er).
<a name="formatting"></a>
# Formatting and the tidy script
rustc is slowly moving towards the [Rust standard coding style][fmt];
at the moment, however, it follows a rather more *chaotic* style. We
do have some mandatory formatting conventions, which are automatically
enforced by a script we affectionately call the "tidy" script. The
tidy script runs automatically when you do `./x.py test` and can be run
in isolation with `./x.py test src/tools/tidy`.
[fmt]: https://github.com/rust-lang-nursery/fmt-rfcs
<a name="copyright"></a>
### Copyright notice
Some existing files begin with a copyright and license notice. Please omit this
notice for new files licensed under the standard terms (dual MIT/Apache-2.0).
For existing files, the year at the top is not meaningful: copyright
protections are in fact automatic from the moment of authorship. We do not
typically edit the years on existing files.
## Line length
Lines should be at most 100 characters. It's even better if you can
keep things to 80.
**Ignoring the line length limit.** Sometimes – in particular for
tests – it can be necessary to exempt yourself from this limit. In
that case, you can add a comment towards the top of the file (after
the copyright notice) like so:
```rust
// ignore-tidy-linelength
```
## Tabs vs spaces
Prefer 4-space indent.
<a name="cc"></a>
# Coding for correctness
Beyond formatting, there are a few other tips that are worth
following.
## Prefer exhaustive matches
Using `_` in a match is convenient, but it means that when new
variants are added to the enum, they may not get handled correctly.
Ask yourself: if a new variant were added to this enum, what's the
chance that it would want to use the `_` code, versus having some
other treatment? Unless the answer is "low", then prefer an
exhaustive match. (The same advice applies to `if let` and `while
let`, which are effectively tests for a single variant.)
## Use "TODO" comments for things you don't want to forget
As a useful tool to yourself, you can insert a `// TODO` comment
for something that you want to get back to before you land your PR:
```rust,ignore
fn do_something() {
if something_else {
unimplemented!(); // TODO write this
}
}
```
The tidy script will report an error for a `// TODO` comment, so this
code would not be able to land until the TODO is fixed (or removed).
This can also be useful in a PR as a way to signal from one commit that you are
leaving a bug that a later commit will fix:
```rust,ignore
if foo {
return true; // TODO wrong, but will be fixed in a later commit
}
```
<a name="cio"></a>
# Using crates from crates.io
It is allowed to use crates from crates.io, though external
dependencies should not be added gratuitously. All such crates must
have a suitably permissive license. There is an automatic check which
inspects the Cargo metadata to ensure this.
<a name="er"></a>
# How to structure your PR
How you prepare the commits in your PR can make a big difference for the
reviewer. Here are some tips.
**Isolate "pure refactorings" into their own commit.** For example, if
you rename a method, then put that rename into its own commit, along
with the renames of all the uses.
**More commits is usually better.** If you are doing a large change,
it's almost always better to break it up into smaller steps that can
be independently understood. The one thing to be aware of is that if
you introduce some code following one strategy, then change it
dramatically (versus adding to it) in a later commit, that
'back-and-forth' can be confusing.
**If you run rustfmt and the file was not already formatted, isolate
that into its own commit.** This is really the same as the previous
rule, but it's worth highlighting. It's ok to rustfmt files, but since
we do not currently run rustfmt all the time, that can introduce a lot
of noise into your commit. Please isolate that into its own
commit. This also makes rebases a lot less painful, since rustfmt
tends to cause a lot of merge conflicts, and having those isolated
into their own commit makes them easier to resolve.
**No merges.** We do not allow merge commits into our history, other
than those by bors. If you get a merge conflict, rebase instead via a
command like `git rebase -i rust-lang/master` (presuming you use the
name `rust-lang` for your remote).
**Individual commits do not have to build (but it's nice).** We do not
require that every intermediate commit successfully builds – we only
expect to be able to bisect at a PR level. However, if you *can* make
individual commits build, that is always helpful.

308
src/doc/rustc-guide/src/diag.md Normal file
View File

@ -0,0 +1,308 @@
# Emitting Diagnostics
A lot of effort has been put into making `rustc` have great error messages.
This chapter is about how to emit compile errors and lints from the compiler.
## `Span`
[`Span`][span] is the primary data structure in `rustc` used to represent a
location in the code being compiled. `Span`s are attached to most constructs in
HIR and MIR, allowing for more informative error reporting.
[span]: https://doc.rust-lang.org/nightly/nightly-rustc/syntax/source_map/struct.Span.html
A `Span` can be looked up in a [`SourceMap`][sourcemap] to get a "snippet"
useful for displaying errors with [`span_to_snippet`][sptosnip] and other
similar methods on the `SourceMap`.
[sourcemap]: https://doc.rust-lang.org/nightly/nightly-rustc/syntax/source_map/struct.SourceMap.html
[sptosnip]: https://doc.rust-lang.org/nightly/nightly-rustc/syntax/source_map/struct.SourceMap.html#method.span_to_snippet
## Error messages
The [`rustc_errors`][errors] crate defines most of the utilities used for
reporting errors.
[errors]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_errors/index.html
[`Session`][session] and [`ParseSess`][parsesses] have
methods (or fields with methods) that allow reporting errors. These methods
usually have names like `span_err` or `struct_span_err` or `span_warn`, etc...
There are lots of them; they emit different types of "errors", such as
warnings, errors, fatal errors, suggestions, etc.
[parsesses]: https://doc.rust-lang.org/nightly/nightly-rustc/syntax/parse/struct.ParseSess.html
[session]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/session/struct.Session.html
In general, there are two class of such methods: ones that emit an error
directly and ones that allow finer control over what to emit. For example,
[`span_err`][spanerr] emits the given error message at the given `Span`, but
[`struct_span_err`][strspanerr] instead returns a
[`DiagnosticBuilder`][diagbuild].
`DiagnosticBuilder` allows you to add related notes and suggestions to an error
before emitting it by calling the [`emit`][emit] method. (Failing to either
emit or [cancel][cancel] a `DiagnosticBuilder` will result in an ICE.) See the
[docs][diagbuild] for more info on what you can do.
[spanerr]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/session/struct.Session.html#method.span_err
[strspanerr]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/session/struct.Session.html#method.struct_span_err
[diagbuild]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_errors/diagnostic_builder/struct.DiagnosticBuilder.html
[emit]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_errors/diagnostic_builder/struct.DiagnosticBuilder.html#method.emit
[cancel]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_errors/struct.Diagnostic.html#method.cancel
```rust,ignore
// Get a DiagnosticBuilder. This does _not_ emit an error yet.
let mut err = sess.struct_span_err(sp, "oh no! this is an error!");
// In some cases, you might need to check if `sp` is generated by a macro to
// avoid printing weird errors about macro-generated code.
if let Ok(snippet) = sess.source_map().span_to_snippet(sp) {
// Use the snippet to generate a suggested fix
err.span_suggestion(suggestion_sp, "try using a qux here", format!("qux {}", snip));
} else {
// If we weren't able to generate a snippet, then emit a "help" message
// instead of a concrete "suggestion". In practice this is unlikely to be
// reached.
err.span_help(suggestion_sp, "you could use a qux here instead");
}
// emit the error
err.emit();
```
## Suggestions
In addition to telling the user exactly _why_ their code is wrong, it's
oftentimes furthermore possible to tell them how to fix it. To this end,
`DiagnosticBuilder` offers a structured suggestions API, which formats code
suggestions pleasingly in the terminal, or (when the `--error-format json` flag
is passed) as JSON for consumption by tools, most notably the [Rust Language
Server][rls] and [`rustfix`][rustfix].
[rls]: https://github.com/rust-lang-nursery/rls
[rustfix]: https://github.com/rust-lang-nursery/rustfix
Not all suggestions should be applied mechanically. Use the
[`span_suggestion_with_applicability`][sswa] method of `DiagnosticBuilder` to
make a suggestion while providing a hint to tools whether the suggestion is
mechanically applicable or not.
[sswa]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_errors/struct.DiagnosticBuilder.html#method.span_suggestion_with_applicability
For example, to make our `qux` suggestion machine-applicable, we would do:
```rust,ignore
let mut err = sess.struct_span_err(sp, "oh no! this is an error!");
if let Ok(snippet) = sess.source_map().span_to_snippet(sp) {
// Add applicability info!
err.span_suggestion_with_applicability(
suggestion_sp,
"try using a qux here",
format!("qux {}", snip),
Applicability::MachineApplicable,
);
} else {
err.span_help(suggestion_sp, "you could use a qux here instead");
}
err.emit();
```
This might emit an error like
```console
$ rustc mycode.rs
error[E0999]: oh no! this is an error!
--> mycode.rs:3:5
|
3 | sad()
| ^ help: try using a qux here: `qux sad()`
error: aborting due to previous error
For more information about this error, try `rustc --explain E0999`.
```
In some cases, like when the suggestion spans multiple lines or when there are
multiple suggestions, the suggestions are displayed on their own:
```console
error[E0999]: oh no! this is an error!
--> mycode.rs:3:5
|
3 | sad()
| ^
help: try using a qux here:
|
3 | qux sad()
| ^^^
error: aborting due to previous error
For more information about this error, try `rustc --explain E0999`.
```
There are a few other [`Applicability`][appl] possibilities:
- `MachineApplicable`: Can be applied mechanically.
- `HasPlaceholders`: Cannot be applied mechanically because it has placeholder
text in the suggestions. For example, "Try adding a type: \`let x:
\<type\>\`".
- `MaybeIncorrect`: Cannot be applied mechanically because the suggestion may
or may not be a good one.
- `Unspecified`: Cannot be applied mechanically because we don't know which
of the above cases it falls into.
[appl]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_errors/enum.Applicability.html
## Lints
The compiler linting infrastructure is defined in the [`rustc::lint`][rlint]
module.
[rlint]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/lint/index.html
### Declaring a lint
The built-in compiler lints are defined in the [`rustc_lint`][builtin]
crate.
[builtin]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_lint/index.html
Each lint is defined as a `struct` that implements the `LintPass` `trait`. The
trait implementation allows you to check certain syntactic constructs the
linter walks the source code. You can then choose to emit lints in a very
similar way to compile errors. Finally, you register the lint to actually get
it to be run by the compiler by using the `declare_lint!` macro.
For example, the following lint checks for uses
of `while true { ... }` and suggests using `loop { ... }` instead.
```rust,ignore
// Declare a lint called `WHILE_TRUE`
declare_lint! {
WHILE_TRUE,
// warn-by-default
Warn,
// This string is the lint description
"suggest using `loop { }` instead of `while true { }`"
}
// Define a struct and `impl LintPass` for it.
#[derive(Copy, Clone)]
pub struct WhileTrue;
impl LintPass for WhileTrue {
fn get_lints(&self) -> LintArray {
lint_array!(WHILE_TRUE)
}
}
// LateLintPass has lots of methods. We only override the definition of
// `check_expr` for this lint because that's all we need, but you could
// override other methods for your own lint. See the rustc docs for a full
// list of methods.
impl<'a, 'tcx> LateLintPass<'a, 'tcx> for WhileTrue {
fn check_expr(&mut self, cx: &LateContext, e: &hir::Expr) {
if let hir::ExprWhile(ref cond, ..) = e.node {
if let hir::ExprLit(ref lit) = cond.node {
if let ast::LitKind::Bool(true) = lit.node {
if lit.span.ctxt() == SyntaxContext::empty() {
let msg = "denote infinite loops with `loop { ... }`";
let condition_span = cx.tcx.sess.source_map().def_span(e.span);
let mut err = cx.struct_span_lint(WHILE_TRUE, condition_span, msg);
err.span_suggestion_short(condition_span, "use `loop`", "loop".to_owned());
err.emit();
}
}
}
}
}
}
```
### Edition-gated Lints
Sometimes we want to change the behavior of a lint in a new edition. To do this,
we just add the transition to our invocation of `declare_lint!`:
```rust,ignore
declare_lint! {
pub ANONYMOUS_PARAMETERS,
Allow,
"detects anonymous parameters",
Edition::Edition2018 => Warn,
}
```
This makes the `ANONYMOUS_PARAMETERS` lint allow-by-default in the 2015 edition
but warn-by-default in the 2018 edition.
Lints that represent an incompatibility (i.e. error) in the upcoming edition
should also be registered as `FutureIncompatibilityLint`s in
[`register_builtins`][rbuiltins] function in [`rustc_lint::lib`][builtin].
### Lint Groups
Lints can be turned on in groups. These groups are declared in the
[`register_builtins`][rbuiltins] function in [`rustc_lint::lib`][builtin]. The
`add_lint_group!` macro is used to declare a new group.
[rbuiltins]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_lint/fn.register_builtins.html
For example,
```rust,ignore
add_lint_group!(sess,
"nonstandard_style",
NON_CAMEL_CASE_TYPES,
NON_SNAKE_CASE,
NON_UPPER_CASE_GLOBALS);
```
This defines the `nonstandard_style` group which turns on the listed lints. A
user can turn on these lints with a `!#[warn(nonstandard_style)]` attribute in
the source code, or by passing `-W nonstandard-style` on the command line.
### Linting early in the compiler
On occasion, you may need to define a lint that runs before the linting system
has been initialized (e.g. during parsing or macro expansion). This is
problematic because we need to have computed lint levels to know whether we
should emit a warning or an error or nothing at all.
To solve this problem, we buffer the lints until the linting system is
processed. [`Session`][sessbl] and [`ParseSess`][parsebl] both have
`buffer_lint` methods that allow you to buffer a lint for later. The linting
system automatically takes care of handling buffered lints later.
[sessbl]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/session/struct.Session.html#method.buffer_lint
[parsebl]: https://doc.rust-lang.org/nightly/nightly-rustc/syntax/parse/struct.ParseSess.html#method.buffer_lint
Thus, to define a lint that runs early in the compilation, one defines a lint
like normal but invokes the lint with `buffer_lint`.
#### Linting even earlier in the compiler
The parser (`libsyntax`) is interesting in that it cannot have dependencies on
any of the other `librustc*` crates. In particular, it cannot depend on
`librustc::lint` or `librustc_lint`, where all of the compiler linting
infrastructure is defined. That's troublesome!
To solve this, `libsyntax` defines its own buffered lint type, which
`ParseSess::buffer_lint` uses. After macro expansion, these buffered lints are
then dumped into the `Session::buffered_lints` used by the rest of the compiler.
Usage for buffered lints in `libsyntax` is pretty much the same as the rest of
the compiler with one exception because we cannot import the `LintId`s for
lints we want to emit. Instead, the [`BufferedEarlyLintId`] type is used. If you
are defining a new lint, you will want to add an entry to this enum. Then, add
an appropriate mapping to the body of [`Lint::from_parser_lint_id`][fplid].
[`BufferedEarlyLintId`]: https://doc.rust-lang.org/nightly/nightly-rustc/syntax/early_buffered_lints/enum.BufferedEarlyLintId.html
[fplid]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/lint/struct.Lint.html#from_parser_lint_id

48
src/doc/rustc-guide/src/existential-types.md Normal file
View File

@ -0,0 +1,48 @@
# Existential Types
Existential types are essentially strong type aliases which only expose
a specific set of traits as their interface and the concrete type in the
background is inferred from a certain set of use sites of the existential
type.
In the language they are expressed via
```rust,ignore
existential type Foo: Bar;
```
This is in existential type named `Foo` which can be interacted with via
the `Bar` trait's interface.
Since there needs to be a concrete background type, you can currently
express that type by using the existential type in a "defining use site".
```rust,ignore
struct Struct;
impl Bar for Struct { /* stuff */ }
fn foo() -> Foo {
Struct
}
```
Any other "defining use site" needs to produce the exact same type.
## Defining use site(s)
Currently only the return value of a function inside can
be a defining use site of an existential type (and only if the return
type of that function contains the existential type).
The defining use of an existential type can be any code *within* the parent
of the existential type definition. This includes any siblings of the
existential type and all children of the siblings.
The initiative for *"not causing fatal brain damage to developers due to
accidentally running infinite loops in their brain while trying to
comprehend what the type system is doing"* has decided to disallow children
of existential types to be defining use sites.
### Associated existential types
Associated existential types can be defined by any other associated item
on the same trait `impl` or a child of these associated items.

142
src/doc/rustc-guide/src/high-level-overview.md Normal file
View File

@ -0,0 +1,142 @@
# High-level overview of the compiler source
## Crate structure
The main Rust repository consists of a `src` directory, under which
there live many crates. These crates contain the sources for the
standard library and the compiler. This document, of course, focuses
on the latter.
Rustc consists of a number of crates, including `syntax`,
`rustc`, `rustc_back`, `rustc_codegen`, `rustc_driver`, and
many more. The source for each crate can be found in a directory
like `src/libXXX`, where `XXX` is the crate name.
(N.B. The names and divisions of these crates are not set in
stone and may change over time. For the time being, we tend towards a
finer-grained division to help with compilation time, though as incremental
compilation improves, that may change.)
The dependency structure of these crates is roughly a diamond:
```text
rustc_driver
/ | \
/ | \
/ | \
/ v \
rustc_codegen rustc_borrowck ... rustc_metadata
\ | /
\ | /
\ | /
\ v /
rustc
|
v
syntax
/ \
/ \
syntax_pos syntax_ext
```
The `rustc_driver` crate, at the top of this lattice, is effectively
the "main" function for the rust compiler. It doesn't have much "real
code", but instead ties together all of the code defined in the other
crates and defines the overall flow of execution. (As we transition
more and more to the [query model], however, the
"flow" of compilation is becoming less centrally defined.)
At the other extreme, the `rustc` crate defines the common and
pervasive data structures that all the rest of the compiler uses
(e.g. how to represent types, traits, and the program itself). It
also contains some amount of the compiler itself, although that is
relatively limited.
Finally, all the crates in the bulge in the middle define the bulk of
the compiler – they all depend on `rustc`, so that they can make use
of the various types defined there, and they export public routines
that `rustc_driver` will invoke as needed (more and more, what these
crates export are "query definitions", but those are covered later
on).
Below `rustc` lie various crates that make up the parser and error
reporting mechanism. For historical reasons, these crates do not have
the `rustc_` prefix, but they are really just as much an internal part
of the compiler and not intended to be stable (though they do wind up
getting used by some crates in the wild; a practice we hope to
gradually phase out).
Each crate has a `README.md` file that describes, at a high-level,
what it contains, and tries to give some kind of explanation (some
better than others).
## The main stages of compilation
The Rust compiler is in a bit of transition right now. It used to be a
purely "pass-based" compiler, where we ran a number of passes over the
entire program, and each did a particular check of transformation. We
are gradually replacing this pass-based code with an alternative setup
based on on-demand **queries**. In the query-model, we work backwards,
executing a *query* that expresses our ultimate goal (e.g. "compile
this crate"). This query in turn may make other queries (e.g. "get me
a list of all modules in the crate"). Those queries make other queries
that ultimately bottom out in the base operations, like parsing the
input, running the type-checker, and so forth. This on-demand model
permits us to do exciting things like only do the minimal amount of
work needed to type-check a single function. It also helps with
incremental compilation. (For details on defining queries, check out
the [query model].)
Regardless of the general setup, the basic operations that the
compiler must perform are the same. The only thing that changes is
whether these operations are invoked front-to-back, or on demand. In
order to compile a Rust crate, these are the general steps that we
take:
1. **Parsing input**
- this processes the `.rs` files and produces the AST
("abstract syntax tree")
- the AST is defined in `src/libsyntax/ast.rs`. It is intended to match the lexical
syntax of the Rust language quite closely.
2. **Name resolution, macro expansion, and configuration**
- once parsing is complete, we process the AST recursively, resolving
paths and expanding macros. This same process also processes `#[cfg]`
nodes, and hence may strip things out of the AST as well.
3. **Lowering to HIR**
- Once name resolution completes, we convert the AST into the HIR,
or "[high-level intermediate representation]". The HIR is defined in
`src/librustc/hir/`; that module also includes the [lowering] code.
- The HIR is a lightly desugared variant of the AST. It is more processed
than the AST and more suitable for the analyses that follow.
It is **not** required to match the syntax of the Rust language.
- As a simple example, in the **AST**, we preserve the parentheses
that the user wrote, so `((1 + 2) + 3)` and `1 + 2 + 3` parse
into distinct trees, even though they are equivalent. In the
HIR, however, parentheses nodes are removed, and those two
expressions are represented in the same way.
3. **Type-checking and subsequent analyses**
- An important step in processing the HIR is to perform type
checking. This process assigns types to every HIR expression,
for example, and also is responsible for resolving some
"type-dependent" paths, such as field accesses (`x.f` – we
can't know what field `f` is being accessed until we know the
type of `x`) and associated type references (`T::Item` – we
can't know what type `Item` is until we know what `T` is).
- Type checking creates "side-tables" (`TypeckTables`) that include
the types of expressions, the way to resolve methods, and so forth.
- After type-checking, we can do other analyses, such as privacy checking.
4. **Lowering to MIR and post-processing**
- Once type-checking is done, we can lower the HIR into MIR ("middle IR"),
which is a **very** desugared version of Rust, well suited to borrowck
but also to certain high-level optimizations.
5. **Translation to LLVM and LLVM optimizations**
- From MIR, we can produce LLVM IR.
- LLVM then runs its various optimizations, which produces a number of
`.o` files (one for each "codegen unit").
6. **Linking**
- Finally, those `.o` files are linked together.
[query model]: query.html
[high-level intermediate representation]: hir.html
[lowering]: lowering.html

159
src/doc/rustc-guide/src/hir.md Normal file
View File

@ -0,0 +1,159 @@
# The HIR
The HIR – "High-Level Intermediate Representation" – is the primary IR used
in most of rustc. It is a compiler-friendly representation of the abstract
syntax tree (AST) that is generated after parsing, macro expansion, and name
resolution (see [Lowering](./lowering.html) for how the HIR is created).
Many parts of HIR resemble Rust surface syntax quite closely, with
the exception that some of Rust's expression forms have been desugared away.
For example, `for` loops are converted into a `loop` and do not appear in
the HIR. This makes HIR more amenable to analysis than a normal AST.
This chapter covers the main concepts of the HIR.
You can view the HIR representation of your code by passing the
`-Zunpretty=hir-tree` flag to rustc:
```bash
> cargo rustc -- -Zunpretty=hir-tree
```
### Out-of-band storage and the `Crate` type
The top-level data-structure in the HIR is the [`Crate`], which stores
the contents of the crate currently being compiled (we only ever
construct HIR for the current crate). Whereas in the AST the crate
data structure basically just contains the root module, the HIR
`Crate` structure contains a number of maps and other things that
serve to organize the content of the crate for easier access.
[`Crate`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/hir/struct.Crate.html
For example, the contents of individual items (e.g. modules,
functions, traits, impls, etc) in the HIR are not immediately
accessible in the parents. So, for example, if there is a module item
`foo` containing a function `bar()`:
```rust
mod foo {
fn bar() { }
}
```
then in the HIR the representation of module `foo` (the [`Mod`]
struct) would only have the **`ItemId`** `I` of `bar()`. To get the
details of the function `bar()`, we would lookup `I` in the
`items` map.
[`Mod`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/hir/struct.Mod.html
One nice result from this representation is that one can iterate
over all items in the crate by iterating over the key-value pairs
in these maps (without the need to trawl through the whole HIR).
There are similar maps for things like trait items and impl items,
as well as "bodies" (explained below).
The other reason to set up the representation this way is for better
integration with incremental compilation. This way, if you gain access
to an [`&hir::Item`] (e.g. for the mod `foo`), you do not immediately
gain access to the contents of the function `bar()`. Instead, you only
gain access to the **id** for `bar()`, and you must invoke some
function to lookup the contents of `bar()` given its id; this gives
the compiler a chance to observe that you accessed the data for
`bar()`, and then record the dependency.
[`&hir::Item`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/hir/struct.Item.html
<a name="hir-id"></a>
### Identifiers in the HIR
Most of the code that has to deal with things in HIR tends not to
carry around references into the HIR, but rather to carry around
*identifier numbers* (or just "ids"). Right now, you will find four
sorts of identifiers in active use:
- [`DefId`], which primarily names "definitions" or top-level items.
- You can think of a [`DefId`] as being shorthand for a very explicit
and complete path, like `std::collections::HashMap`. However,
these paths are able to name things that are not nameable in
normal Rust (e.g. impls), and they also include extra information
about the crate (such as its version number, as two versions of
the same crate can co-exist).
- A [`DefId`] really consists of two parts, a `CrateNum` (which
identifies the crate) and a `DefIndex` (which indexes into a list
of items that is maintained per crate).
- [`HirId`], which combines the index of a particular item with an
offset within that item.
- the key point of a [`HirId`] is that it is *relative* to some item
(which is named via a [`DefId`]).
- [`BodyId`], this is an absolute identifier that refers to a specific
body (definition of a function or constant) in the crate. It is currently
effectively a "newtype'd" [`NodeId`].
- [`NodeId`], which is an absolute id that identifies a single node in the HIR
tree.
- While these are still in common use, **they are being slowly phased out**.
- Since they are absolute within the crate, adding a new node anywhere in the
tree causes the [`NodeId`]s of all subsequent code in the crate to change.
This is terrible for incremental compilation, as you can perhaps imagine.
[`DefId`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/hir/def_id/struct.DefId.html
[`HirId`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/hir/struct.HirId.html
[`BodyId`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/hir/struct.BodyId.html
[`NodeId`]: https://doc.rust-lang.org/nightly/nightly-rustc/syntax/ast/struct.NodeId.html
### The HIR Map
Most of the time when you are working with the HIR, you will do so via
the **HIR Map**, accessible in the tcx via [`tcx.hir`] (and defined in
the [`hir::map`] module). The [HIR map] contains a [number of methods] to
convert between IDs of various kinds and to lookup data associated
with an HIR node.
[`tcx.hir`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/ty/context/struct.GlobalCtxt.html#structfield.hir
[`hir::map`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/hir/map/index.html
[HIR map]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/hir/map/struct.Map.html
[number of methods]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/hir/map/struct.Map.html#methods
For example, if you have a [`DefId`], and you would like to convert it
to a [`NodeId`], you can use
[`tcx.hir.as_local_node_id(def_id)`][as_local_node_id]. This returns
an `Option<NodeId>` – this will be `None` if the def-id refers to
something outside of the current crate (since then it has no HIR
node), but otherwise returns `Some(n)` where `n` is the node-id of the
definition.
[as_local_node_id]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/hir/map/struct.Map.html#method.as_local_node_id
Similarly, you can use [`tcx.hir.find(n)`][find] to lookup the node for a
[`NodeId`]. This returns a `Option<Node<'tcx>>`, where [`Node`] is an enum
defined in the map; by matching on this you can find out what sort of
node the node-id referred to and also get a pointer to the data
itself. Often, you know what sort of node `n` is – e.g. if you know
that `n` must be some HIR expression, you can do
[`tcx.hir.expect_expr(n)`][expect_expr], which will extract and return the
[`&hir::Expr`][Expr], panicking if `n` is not in fact an expression.
[find]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/hir/map/struct.Map.html#method.find
[`Node`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/hir/enum.Node.html
[expect_expr]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/hir/map/struct.Map.html#method.expect_expr
[Expr]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/hir/struct.Expr.html
Finally, you can use the HIR map to find the parents of nodes, via
calls like [`tcx.hir.get_parent_node(n)`][get_parent_node].
[get_parent_node]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/hir/map/struct.Map.html#method.get_parent_node
### HIR Bodies
A [`hir::Body`] represents some kind of executable code, such as the body
of a function/closure or the definition of a constant. Bodies are
associated with an **owner**, which is typically some kind of item
(e.g. an `fn()` or `const`), but could also be a closure expression
(e.g. `|x, y| x + y`). You can use the HIR map to find the body
associated with a given def-id ([`maybe_body_owned_by`]) or to find
the owner of a body ([`body_owner_def_id`]).
[`hir::Body`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/hir/struct.Body.html
[`maybe_body_owned_by`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/hir/map/struct.Map.html#method.maybe_body_owned_by
[`body_owner_def_id`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/hir/map/struct.Map.html#method.body_owner_def_id

334
src/doc/rustc-guide/src/how-to-build-and-run.md Normal file
View File

@ -0,0 +1,334 @@
# How to build the compiler and run what you built
The compiler is built using a tool called `x.py`. You will need to
have Python installed to run it. But before we get to that, if you're going to
be hacking on `rustc`, you'll want to tweak the configuration of the compiler.
The default configuration is oriented towards running the compiler as a user,
not a developer.
### Create a config.toml
To start, copy [`config.toml.example`] to `config.toml`:
[`config.toml.example`]: https://github.com/rust-lang/rust/blob/master/config.toml.example
```bash
> cd $RUST_CHECKOUT
> cp config.toml.example config.toml
```
Then you will want to open up the file and change the following
settings (and possibly others, such as `llvm.ccache`):
```toml
[llvm]
# Enables LLVM assertions, which will check that the LLVM bitcode generated
# by the compiler is internally consistent. These are particularly helpful
# if you edit `codegen`.
assertions = true
[rust]
# This enables some assertions, but more importantly it enables the `debug!`
# logging macros that are essential for debugging rustc.
debug-assertions = true
# This will make your build more parallel; it costs a bit of runtime
# performance perhaps (less inlining) but it's worth it.
codegen-units = 0
# I always enable full debuginfo, though debuginfo-lines is more important.
debuginfo = true
# Gives you line numbers for backtraces.
debuginfo-lines = true
```
### What is x.py?
x.py is the script used to orchestrate the tooling in the rustc repository.
It is the script that can build docs, run tests, and compile rustc.
It is the now preferred way to build rustc and it replaces the old makefiles
from before. Below are the different ways to utilize x.py in order to
effectively deal with the repo for various common tasks.
### Running x.py and building a stage1 compiler
One thing to keep in mind is that `rustc` is a _bootstrapping_
compiler. That is, since `rustc` is written in Rust, we need to use an
older version of the compiler to compile the newer version. In
particular, the newer version of the compiler, `libstd`, and other
tooling may use some unstable features internally. The result is that
compiling `rustc` is done in stages:
- **Stage 0:** the stage0 compiler is usually the current _beta_ compiler
(`x.py` will download it for you); you can configure `x.py` to use something
else, though.
- **Stage 1:** the code in your clone (for new version) is then
compiled with the stage0 compiler to produce the stage1 compiler.
However, it was built with an older compiler (stage0), so to
optimize the stage1 compiler we go to next stage.
- (In theory, the stage1 compiler is functionally identical to the
stage2 compiler, but in practice there are subtle differences. In
particular, the stage1 compiler itself was built by stage0 and
hence not by the source in your working directory: this means that
the symbol names used in the compiler source may not match the
symbol names that would have been made by the stage1 compiler.
This can be important when using dynamic linking (e.g., with
derives. Sometimes this means that some tests don't work when run
with stage1.)
- **Stage 2:** we rebuild our stage1 compiler with itself to produce
the stage2 compiler (i.e. it builds itself) to have all the _latest
optimizations_. (By default, we copy the stage1 libraries for use by
the stage2 compiler, since they ought to be identical.)
- _(Optional)_ **Stage 3**: to sanity check of our new compiler, we
can build the libraries with the stage2 compiler. The result ought
to be identical to before, unless something has broken.
#### Build Flags
There are other flags you can pass to the build portion of x.py that can be
beneficial to cutting down compile times or fitting other things you might
need to change. They are:
```bash
Options:
-v, --verbose use verbose output (-vv for very verbose)
-i, --incremental use incremental compilation
--config FILE TOML configuration file for build
--build BUILD build target of the stage0 compiler
--host HOST host targets to build
--target TARGET target targets to build
--on-fail CMD command to run on failure
--stage N stage to build
--keep-stage N stage to keep without recompiling
--src DIR path to the root of the rust checkout
-j, --jobs JOBS number of jobs to run in parallel
-h, --help print this help message
```
For hacking, often building the stage 1 compiler is enough, but for
final testing and release, the stage 2 compiler is used.
`./x.py check` is really fast to build the rust compiler.
It is, in particular, very useful when you're doing some kind of
"type-based refactoring", like renaming a method, or changing the
signature of some function.
<a name=command></a>
Once you've created a config.toml, you are now ready to run
`x.py`. There are a lot of options here, but let's start with what is
probably the best "go to" command for building a local rust:
```bash
> ./x.py build -i --stage 1 src/libstd
```
This may *look* like it only builds libstd, but that is not the case.
What this command does is the following:
- Build libstd using the stage0 compiler (using incremental)
- Build librustc using the stage0 compiler (using incremental)
- This produces the stage1 compiler
- Build libstd using the stage1 compiler (cannot use incremental)
This final product (stage1 compiler + libs built using that compiler)
is what you need to build other rust programs.
Note that the command includes the `-i` switch. This enables incremental
compilation. This will be used to speed up the first two steps of the process:
in particular, if you make a small change, we ought to be able to use your old
results to make producing the stage1 **compiler** faster.
Unfortunately, incremental cannot be used to speed up making the
stage1 libraries. This is because incremental only works when you run
the *same compiler* twice in a row. In this case, we are building a
*new stage1 compiler* every time. Therefore, the old incremental
results may not apply. **As a result, you will probably find that
building the stage1 libstd is a bottleneck for you** -- but fear not,
there is a (hacky) workaround. See [the section on "recommended
workflows"](#workflow) below.
Note that this whole command just gives you a subset of the full rustc
build. The **full** rustc build (what you get if you just say `./x.py
build`) has quite a few more steps:
- Build librustc and rustc with the stage1 compiler.
- The resulting compiler here is called the "stage2" compiler.
- Build libstd with stage2 compiler.
- Build librustdoc and a bunch of other things with the stage2 compiler.
<a name=toolchain></a>
### Build specific components
Build only the libcore library
```bash
> ./x.py build src/libcore
```
Build the libcore and libproc_macro library only
```bash
> ./x.py build src/libcore src/libproc_macro
```
Build only libcore up to Stage 1
```bash
> ./x.py build src/libcore --stage 1
```
Sometimes you might just want to test if the part you’re working on can
compile. Using these commands you can test that it compiles before doing
a bigger build to make sure it works with the compiler. As shown before
you can also pass flags at the end such as --stage.
### Creating a rustup toolchain
Once you have successfully built rustc, you will have created a bunch
of files in your `build` directory. In order to actually run the
resulting rustc, we recommend creating rustup toolchains. The first
one will run the stage1 compiler (which we built above). The second
will execute the stage2 compiler (which we did not build, but which
you will likely need to build at some point; for example, if you want
to run the entire test suite).
```bash
> rustup toolchain link stage1 build/<host-triple>/stage1
> rustup toolchain link stage2 build/<host-triple>/stage2
```
The `<host-triple>` would typically be one of the following:
- Linux: `x86_64-unknown-linux-gnu`
- Mac: `x86_64-apple-darwin`
- Windows: `x86_64-pc-windows-msvc`
Now you can run the rustc you built with. If you run with `-vV`, you
should see a version number ending in `-dev`, indicating a build from
your local environment:
```bash
> rustc +stage1 -vV
rustc 1.25.0-dev
binary: rustc
commit-hash: unknown
commit-date: unknown
host: x86_64-unknown-linux-gnu
release: 1.25.0-dev
LLVM version: 4.0
```
<a name=workflow></a>
### Suggested workflows for faster builds of the compiler
There are two workflows that are useful for faster builds of the
compiler.
**Check, check, and check again.** The first workflow, which is useful
when doing simple refactorings, is to run `./x.py check`
continuously. Here you are just checking that the compiler can
**build**, but often that is all you need (e.g., when renaming a
method). You can then run `./x.py build` when you actually need to
run tests.
In fact, it is sometimes useful to put off tests even when you are not
100% sure the code will work. You can then keep building up
refactoring commits and only run the tests at some later time. You can
then use `git bisect` to track down **precisely** which commit caused
the problem. A nice side-effect of this style is that you are left
with a fairly fine-grained set of commits at the end, all of which
build and pass tests. This often helps reviewing.
**Incremental builds with `--keep-stage`.** Sometimes just checking
whether the compiler builds is not enough. A common example is that
you need to add a `debug!` statement to inspect the value of some
state or better understand the problem. In that case, you really need
a full build. By leveraging incremental, though, you can often get
these builds to complete very fast (e.g., around 30 seconds): the only
catch is this requires a bit of fudging and may produce compilers that
don't work (but that is easily detected and fixed).
The sequence of commands you want is as follows:
- Initial build: `./x.py build -i --stage 1 src/libstd`
- As [documented above](#command), this will build a functional
stage1 compiler
- Subsequent builds: `./x.py build -i --stage 1 src/libstd --keep-stage 1`
- Note that we added the `--keep-stage 1` flag here
The effect of `--keep-stage 1` is that we just *assume* that the old
standard library can be re-used. If you are editing the compiler, this
is almost always true: you haven't changed the standard library, after
all. But sometimes, it's not true: for example, if you are editing
the "metadata" part of the compiler, which controls how the compiler
encodes types and other states into the `rlib` files, or if you are
editing things that wind up in the metadata (such as the definition of
the MIR).
**The TL;DR is that you might get weird behavior from a compile when
using `--keep-stage 1`** -- for example, strange
[ICEs](appendix/glossary.html) or other panics. In that case, you
should simply remove the `--keep-stage 1` from the command and
rebuild. That ought to fix the problem.
You can also use `--keep-stage 1` when running tests. Something like
this:
- Initial test run: `./x.py test -i --stage 1 src/test/ui`
- Subsequent test run: `./x.py test -i --stage 1 src/test/ui --keep-stage 1`
### Other x.py commands
Here are a few other useful x.py commands. We'll cover some of them in detail
in other sections:
- Building things:
- `./x.py clean` – clean up the build directory (`rm -rf build` works too,
but then you have to rebuild LLVM)
- `./x.py build --stage 1` – builds everything using the stage 1 compiler,
not just up to libstd
- `./x.py build` – builds the stage2 compiler
- Running tests (see the [section on running tests](./tests/running.html) for
more details):
- `./x.py test --stage 1 src/libstd` – runs the `#[test]` tests from libstd
- `./x.py test --stage 1 src/test/run-pass` – runs the `run-pass` test suite
### ctags
One of the challenges with rustc is that the RLS can't handle it, making code
navigation difficult. One solution is to use `ctags`. The following script can
be used to set it up: [https://github.com/nikomatsakis/rust-etags][etags].
CTAGS integrates into emacs and vim quite easily. The following can then be
used to build and generate tags:
```console
$ rust-ctags src/lib* && ./x.py build <something>
```
This allows you to do "jump-to-def" with whatever functions were around when
you last built, which is ridiculously useful.
[etags]: https://github.com/nikomatsakis/rust-etags
### Cleaning out build directories
Sometimes you need to start fresh, but this is normally not the case.
If you need to run this then rustbuild is most likely not acting right and
you should file a bug as to what is going wrong. If you do need to clean
everything up then you only need to run one command!
```bash
> ./x.py clean
```
### Compiler Documentation
The documentation for the rust components are found at [rustc doc].
[rustc doc]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/

115
src/doc/rustc-guide/src/incrcomp-debugging.md Normal file
View File

@ -0,0 +1,115 @@
# Debugging and Testing Dependencies
## Testing the dependency graph
There are various ways to write tests against the dependency graph.
The simplest mechanisms are the `#[rustc_if_this_changed]` and
`#[rustc_then_this_would_need]` annotations. These are used in compile-fail
tests to test whether the expected set of paths exist in the dependency graph.
As an example, see `src/test/compile-fail/dep-graph-caller-callee.rs`.
The idea is that you can annotate a test like:
```rust,ignore
#[rustc_if_this_changed]
fn foo() { }
#[rustc_then_this_would_need(TypeckTables)] //~ ERROR OK
fn bar() { foo(); }
#[rustc_then_this_would_need(TypeckTables)] //~ ERROR no path
fn baz() { }
```
This will check whether there is a path in the dependency graph from `Hir(foo)`
to `TypeckTables(bar)`. An error is reported for each
`#[rustc_then_this_would_need]` annotation that indicates whether a path
exists. `//~ ERROR` annotations can then be used to test if a path is found (as
demonstrated above).
## Debugging the dependency graph
### Dumping the graph
The compiler is also capable of dumping the dependency graph for your
debugging pleasure. To do so, pass the `-Z dump-dep-graph` flag. The
graph will be dumped to `dep_graph.{txt,dot}` in the current
directory. You can override the filename with the `RUST_DEP_GRAPH`
environment variable.
Frequently, though, the full dep graph is quite overwhelming and not
particularly helpful. Therefore, the compiler also allows you to filter
the graph. You can filter in three ways:
1. All edges originating in a particular set of nodes (usually a single node).
2. All edges reaching a particular set of nodes.
3. All edges that lie between given start and end nodes.
To filter, use the `RUST_DEP_GRAPH_FILTER` environment variable, which should
look like one of the following:
```text
source_filter // nodes originating from source_filter
-> target_filter // nodes that can reach target_filter
source_filter -> target_filter // nodes in between source_filter and target_filter
```
`source_filter` and `target_filter` are a `&`-separated list of strings.
A node is considered to match a filter if all of those strings appear in its
label. So, for example:
```text
RUST_DEP_GRAPH_FILTER='-> TypeckTables'
```
would select the predecessors of all `TypeckTables` nodes. Usually though you
want the `TypeckTables` node for some particular fn, so you might write:
```text
RUST_DEP_GRAPH_FILTER='-> TypeckTables & bar'
```
This will select only the predecessors of `TypeckTables` nodes for functions
with `bar` in their name.
Perhaps you are finding that when you change `foo` you need to re-type-check
`bar`, but you don't think you should have to. In that case, you might do:
```text
RUST_DEP_GRAPH_FILTER='Hir & foo -> TypeckTables & bar'
```
This will dump out all the nodes that lead from `Hir(foo)` to
`TypeckTables(bar)`, from which you can (hopefully) see the source
of the erroneous edge.
### Tracking down incorrect edges
Sometimes, after you dump the dependency graph, you will find some
path that should not exist, but you will not be quite sure how it came
to be. **When the compiler is built with debug assertions,** it can
help you track that down. Simply set the `RUST_FORBID_DEP_GRAPH_EDGE`
environment variable to a filter. Every edge created in the dep-graph
will be tested against that filter – if it matches, a `bug!` is
reported, so you can easily see the backtrace (`RUST_BACKTRACE=1`).
The syntax for these filters is the same as described in the previous
section. However, note that this filter is applied to every **edge**
and doesn't handle longer paths in the graph, unlike the previous
section.
Example:
You find that there is a path from the `Hir` of `foo` to the type
check of `bar` and you don't think there should be. You dump the
dep-graph as described in the previous section and open `dep-graph.txt`
to see something like:
```text
Hir(foo) -> Collect(bar)
Collect(bar) -> TypeckTables(bar)
```
That first edge looks suspicious to you. So you set
`RUST_FORBID_DEP_GRAPH_EDGE` to `Hir&foo -> Collect&bar`, re-run, and
then observe the backtrace. Voila, bug fixed!

141
src/doc/rustc-guide/src/incremental-compilation.md Normal file
View File

@ -0,0 +1,141 @@
# Incremental compilation
The incremental compilation scheme is, in essence, a surprisingly
simple extension to the overall query system. We'll start by describing
a slightly simplified variant of the real thing – the "basic algorithm" –
and then describe some possible improvements.
## The basic algorithm
The basic algorithm is
called the **red-green** algorithm[^salsa]. The high-level idea is
that, after each run of the compiler, we will save the results of all
the queries that we do, as well as the **query DAG**. The
**query DAG** is a [DAG] that indexes which queries executed which
other queries. So, for example, there would be an edge from a query Q1
to another query Q2 if computing Q1 required computing Q2 (note that
because queries cannot depend on themselves, this results in a DAG and
not a general graph).
[DAG]: https://en.wikipedia.org/wiki/Directed_acyclic_graph
On the next run of the compiler, then, we can sometimes reuse these
query results to avoid re-executing a query. We do this by assigning
every query a **color**:
- If a query is colored **red**, that means that its result during
this compilation has **changed** from the previous compilation.
- If a query is colored **green**, that means that its result is
the **same** as the previous compilation.
There are two key insights here:
- First, if all the inputs to query Q are colored green, then the
query Q **must** result in the same value as last time and hence
need not be re-executed (or else the compiler is not deterministic).
- Second, even if some inputs to a query changes, it may be that it
**still** produces the same result as the previous compilation. In
particular, the query may only use part of its input.
- Therefore, after executing a query, we always check whether it
produced the same result as the previous time. **If it did,** we
can still mark the query as green, and hence avoid re-executing
dependent queries.
### The try-mark-green algorithm
At the core of incremental compilation is an algorithm called
"try-mark-green". It has the job of determining the color of a given
query Q (which must not have yet been executed). In cases where Q has
red inputs, determining Q's color may involve re-executing Q so that
we can compare its output, but if all of Q's inputs are green, then we
can conclude that Q must be green without re-executing it or inspecting
its value at all. In the compiler, this allows us to avoid
deserializing the result from disk when we don't need it, and in fact
enables us to sometimes skip *serializing* the result as well
(see the refinements section below).
Try-mark-green works as follows:
- First check if the query Q was executed during the previous compilation.
- If not, we can just re-execute the query as normal, and assign it the
color of red.
- If yes, then load the 'dependent queries' of Q.
- If there is a saved result, then we load the `reads(Q)` vector from the
query DAG. The "reads" is the set of queries that Q executed during
its execution.
- For each query R in `reads(Q)`, we recursively demand the color
of R using try-mark-green.
- Note: it is important that we visit each node in `reads(Q)` in same order
as they occurred in the original compilation. See [the section on the
query DAG below](#dag).
- If **any** of the nodes in `reads(Q)` wind up colored **red**, then Q is
dirty.
- We re-execute Q and compare the hash of its result to the hash of the
result from the previous compilation.
- If the hash has not changed, we can mark Q as **green** and return.
- Otherwise, **all** of the nodes in `reads(Q)` must be **green**. In that
case, we can color Q as **green** and return.
<a name="dag"></a>
### The query DAG
The query DAG code is stored in
[`src/librustc/dep_graph`][dep_graph]. Construction of the DAG is done
by instrumenting the query execution.
One key point is that the query DAG also tracks ordering; that is, for
each query Q, we not only track the queries that Q reads, we track the
**order** in which they were read. This allows try-mark-green to walk
those queries back in the same order. This is important because once a
subquery comes back as red, we can no longer be sure that Q will continue
along the same path as before. That is, imagine a query like this:
```rust,ignore
fn main_query(tcx) {
if tcx.subquery1() {
tcx.subquery2()
} else {
tcx.subquery3()
}
}
```
Now imagine that in the first compilation, `main_query` starts by
executing `subquery1`, and this returns true. In that case, the next
query `main_query` executes will be `subquery2`, and `subquery3` will
not be executed at all.
But now imagine that in the **next** compilation, the input has
changed such that `subquery1` returns **false**. In this case, `subquery2`
would never execute. If try-mark-green were to visit `reads(main_query)` out
of order, however, it might visit `subquery2` before `subquery1`, and hence
execute it.
This can lead to ICEs and other problems in the compiler.
[dep_graph]: https://github.com/rust-lang/rust/tree/master/src/librustc/dep_graph
## Improvements to the basic algorithm
In the description of the basic algorithm, we said that at the end of
compilation we would save the results of all the queries that were
performed. In practice, this can be quite wasteful – many of those
results are very cheap to recompute, and serializing and deserializing
them is not a particular win. In practice, what we would do is to save
**the hashes** of all the subqueries that we performed. Then, in select cases,
we **also** save the results.
This is why the incremental algorithm separates computing the
**color** of a node, which often does not require its value, from
computing the **result** of a node. Computing the result is done via a simple
algorithm like so:
- Check if a saved result for Q is available. If so, compute the color of Q.
If Q is green, deserialize and return the saved result.
- Otherwise, execute Q.
- We can then compare the hash of the result and color Q as green if
it did not change.
# Footnotes
[^salsa]: I have long wanted to rename it to the Salsa algorithm, but it never caught on. -@nikomatsakis

48
src/doc/rustc-guide/src/lowering.md Normal file
View File

@ -0,0 +1,48 @@
# Lowering
The lowering step converts AST to [HIR](hir.html).
This means many structures are removed if they are irrelevant
for type analysis or similar syntax agnostic analyses. Examples
of such structures include but are not limited to
* Parenthesis
* Removed without replacement, the tree structure makes order explicit
* `for` loops and `while (let)` loops
* Converted to `loop` + `match` and some `let` bindings
* `if let`
* Converted to `match`
* Universal `impl Trait`
* Converted to generic arguments
(but with some flags, to know that the user didn't write them)
* Existential `impl Trait`
* Converted to a virtual `existential type` declaration
Lowering needs to uphold several invariants in order to not trigger the
sanity checks in `src/librustc/hir/map/hir_id_validator.rs`:
1. A `HirId` must be used if created. So if you use the `lower_node_id`,
you *must* use the resulting `NodeId` or `HirId` (either is fine, since
any `NodeId`s in the `HIR` are checked for existing `HirId`s)
2. Lowering a `HirId` must be done in the scope of the *owning* item.
This means you need to use `with_hir_id_owner` if you are creating parts
of an item other than the one being currently lowered. This happens for
example during the lowering of existential `impl Trait`
3. A `NodeId` that will be placed into a HIR structure must be lowered,
even if its `HirId` is unused. Calling
`let _ = self.lower_node_id(node_id);` is perfectly legitimate.
4. If you are creating new nodes that didn't exist in the `AST`, you *must*
create new ids for them. This is done by calling the `next_id` method,
which produces both a new `NodeId` as well as automatically lowering it
for you so you also get the `HirId`.
If you are creating new `DefId`s, since each `DefId` needs to have a
corresponding `NodeId`, it is advisable to add these `NodeId`s to the
`AST` so you don't have to generate new ones during lowering. This has
the advantage of creating a way to find the `DefId` of something via its
`NodeId`. If lowering needs this `DefId` in multiple places, you can't
generate a new `NodeId` in all those places because you'd also get a new
`DefId` then. With a `NodeId` from the `AST` this is not an issue.
Having the `NodeId` also allows the `DefCollector` to generate the `DefId`s
instead of lowering having to do it on the fly. Centralizing the `DefId`
generation in one place makes it easier to refactor and reason about.

212
src/doc/rustc-guide/src/macro-expansion.md Normal file
View File

@ -0,0 +1,212 @@
# Macro expansion
Macro expansion happens during parsing. `rustc` has two parsers, in fact: the
normal Rust parser, and the macro parser. During the parsing phase, the normal
Rust parser will set aside the contents of macros and their invocations. Later,
before name resolution, macros are expanded using these portions of the code.
The macro parser, in turn, may call the normal Rust parser when it needs to
bind a metavariable (e.g. `$my_expr`) while parsing the contents of a macro
invocation. The code for macro expansion is in
[`src/libsyntax/ext/tt/`][code_dir]. This chapter aims to explain how macro
expansion works.
### Example
It's helpful to have an example to refer to. For the remainder of this chapter,
whenever we refer to the "example _definition_", we mean the following:
```rust,ignore
macro_rules! printer {
(print $mvar:ident) => {
println!("{}", $mvar);
}
(print twice $mvar:ident) => {
println!("{}", $mvar);
println!("{}", $mvar);
}
}
```
`$mvar` is called a _metavariable_. Unlike normal variables, rather than
binding to a value in a computation, a metavariable binds _at compile time_ to
a tree of _tokens_. A _token_ is a single "unit" of the grammar, such as an
identifier (e.g. `foo`) or punctuation (e.g. `=>`). There are also other
special tokens, such as `EOF`, which indicates that there are no more tokens.
Token trees resulting from paired parentheses-like characters (`(`...`)`,
`[`...`]`, and `{`...`}`) – they include the open and close and all the tokens
in between (we do require that parentheses-like characters be balanced). Having
macro expansion operate on token streams rather than the raw bytes of a source
file abstracts away a lot of complexity. The macro expander (and much of the
rest of the compiler) doesn't really care that much about the exact line and
column of some syntactic construct in the code; it cares about what constructs
are used in the code. Using tokens allows us to care about _what_ without
worrying about _where_. For more information about tokens, see the
[Parsing][parsing] chapter of this book.
Whenever we refer to the "example _invocation_", we mean the following snippet:
```rust,ignore
printer!(print foo); // Assume `foo` is a variable defined somewhere else...
```
The process of expanding the macro invocation into the syntax tree
`println!("{}", foo)` and then expanding that into a call to `Display::fmt` is
called _macro expansion_, and it is the topic of this chapter.
### The macro parser
There are two parts to macro expansion: parsing the definition and parsing the
invocations. Interestingly, both are done by the macro parser.
Basically, the macro parser is like an NFA-based regex parser. It uses an
algorithm similar in spirit to the [Earley parsing
algorithm](https://en.wikipedia.org/wiki/Earley_parser). The macro parser is
defined in [`src/libsyntax/ext/tt/macro_parser.rs`][code_mp].
The interface of the macro parser is as follows (this is slightly simplified):
```rust,ignore
fn parse(
sess: ParserSession,
tts: TokenStream,
ms: &[TokenTree]
) -> NamedParseResult
```
In this interface:
- `sess` is a "parsing session", which keeps track of some metadata. Most
notably, this is used to keep track of errors that are generated so they can
be reported to the user.
- `tts` is a stream of tokens. The macro parser's job is to consume the raw
stream of tokens and output a binding of metavariables to corresponding token
trees.
- `ms` a _matcher_. This is a sequence of token trees that we want to match
`tts` against.
In the analogy of a regex parser, `tts` is the input and we are matching it
against the pattern `ms`. Using our examples, `tts` could be the stream of
tokens containing the inside of the example invocation `print foo`, while `ms`
might be the sequence of token (trees) `print $mvar:ident`.
The output of the parser is a `NamedParseResult`, which indicates which of
three cases has occurred:
- Success: `tts` matches the given matcher `ms`, and we have produced a binding
from metavariables to the corresponding token trees.
- Failure: `tts` does not match `ms`. This results in an error message such as
"No rule expected token _blah_".
- Error: some fatal error has occurred _in the parser_. For example, this
happens if there are more than one pattern match, since that indicates
the macro is ambiguous.
The full interface is defined [here][code_parse_int].
The macro parser does pretty much exactly the same as a normal regex parser with
one exception: in order to parse different types of metavariables, such as
`ident`, `block`, `expr`, etc., the macro parser must sometimes call back to the
normal Rust parser.
As mentioned above, both definitions and invocations of macros are parsed using
the macro parser. This is extremely non-intuitive and self-referential. The code
to parse macro _definitions_ is in
[`src/libsyntax/ext/tt/macro_rules.rs`][code_mr]. It defines the pattern for
matching for a macro definition as `$( $lhs:tt => $rhs:tt );+`. In other words,
a `macro_rules` definition should have in its body at least one occurrence of a
token tree followed by `=>` followed by another token tree. When the compiler
comes to a `macro_rules` definition, it uses this pattern to match the two token
trees per rule in the definition of the macro _using the macro parser itself_.
In our example definition, the metavariable `$lhs` would match the patterns of
both arms: `(print $mvar:ident)` and `(print twice $mvar:ident)`. And `$rhs`
would match the bodies of both arms: `{ println!("{}", $mvar); }` and `{
println!("{}", $mvar); println!("{}", $mvar); }`. The parser would keep this
knowledge around for when it needs to expand a macro invocation.
When the compiler comes to a macro invocation, it parses that invocation using
the same NFA-based macro parser that is described above. However, the matcher
used is the first token tree (`$lhs`) extracted from the arms of the macro
_definition_. Using our example, we would try to match the token stream `print
foo` from the invocation against the matchers `print $mvar:ident` and `print
twice $mvar:ident` that we previously extracted from the definition. The
algorithm is exactly the same, but when the macro parser comes to a place in the
current matcher where it needs to match a _non-terminal_ (e.g. `$mvar:ident`),
it calls back to the normal Rust parser to get the contents of that
non-terminal. In this case, the Rust parser would look for an `ident` token,
which it finds (`foo`) and returns to the macro parser. Then, the macro parser
proceeds in parsing as normal. Also, note that exactly one of the matchers from
the various arms should match the invocation; if there is more than one match,
the parse is ambiguous, while if there are no matches at all, there is a syntax
error.
For more information about the macro parser's implementation, see the comments
in [`src/libsyntax/ext/tt/macro_parser.rs`][code_mp].
### Hygiene
If you have ever used C/C++ preprocessor macros, you know that there are some
annoying and hard-to-debug gotchas! For example, consider the following C code:
```c
#define DEFINE_FOO struct Bar {int x;}; struct Foo {Bar bar;};
// Then, somewhere else
struct Bar {
...
};
DEFINE_FOO
```
Most people avoid writing C like this – and for good reason: it doesn't
compile. The `struct Bar` defined by the macro clashes names with the `struct
Bar` defined in the code. Consider also the following example:
```c
#define DO_FOO(x) {\
int y = 0;\
foo(x, y);\
}
// Then elsewhere
int y = 22;
DO_FOO(y);
```
Do you see the problem? We wanted to generate a call `foo(22, 0)`, but instead
we got `foo(0, 0)` because the macro defined its own `y`!
These are both examples of _macro hygiene_ issues. _Hygiene_ relates to how to
handle names defined _within a macro_. In particular, a hygienic macro system
prevents errors due to names introduced within a macro. Rust macros are hygienic
in that they do not allow one to write the sorts of bugs above.
At a high level, hygiene within the rust compiler is accomplished by keeping
track of the context where a name is introduced and used. We can then
disambiguate names based on that context. Future iterations of the macro system
will allow greater control to the macro author to use that context. For example,
a macro author may want to introduce a new name to the context where the macro
was called. Alternately, the macro author may be defining a variable for use
only within the macro (i.e. it should not be visible outside the macro).
In rustc, this "context" is tracked via `Span`s.
TODO: what is call-site hygiene? what is def-site hygiene?
TODO
### Procedural Macros
TODO
### Custom Derive
TODO
TODO: maybe something about macros 2.0?
[code_dir]: https://github.com/rust-lang/rust/tree/master/src/libsyntax/ext/tt
[code_mp]: https://doc.rust-lang.org/nightly/nightly-rustc/syntax/ext/tt/macro_parser/
[code_mr]: https://doc.rust-lang.org/nightly/nightly-rustc/syntax/ext/tt/macro_rules/
[code_parse_int]: https://doc.rust-lang.org/nightly/nightly-rustc/syntax/ext/tt/macro_parser/fn.parse.html
[parsing]: ./the-parser.html

121
src/doc/rustc-guide/src/method-lookup.md Normal file
View File

@ -0,0 +1,121 @@
# Method lookup
Method lookup can be rather complex due to the interaction of a number
of factors, such as self types, autoderef, trait lookup, etc. This
file provides an overview of the process. More detailed notes are in
the code itself, naturally.
One way to think of method lookup is that we convert an expression of
the form:
```rust,ignore
receiver.method(...)
```
into a more explicit [UFCS] form:
```rust,ignore
Trait::method(ADJ(receiver), ...) // for a trait call
ReceiverType::method(ADJ(receiver), ...) // for an inherent method call
```
Here `ADJ` is some kind of adjustment, which is typically a series of
autoderefs and then possibly an autoref (e.g., `&**receiver`). However
we sometimes do other adjustments and coercions along the way, in
particular unsizing (e.g., converting from `[T; n]` to `[T]`).
Method lookup is divided into two major phases:
1. Probing ([`probe.rs`][probe]). The probe phase is when we decide what method
to call and how to adjust the receiver.
2. Confirmation ([`confirm.rs`][confirm]). The confirmation phase "applies"
this selection, updating the side-tables, unifying type variables, and
otherwise doing side-effectful things.
One reason for this division is to be more amenable to caching. The
probe phase produces a "pick" (`probe::Pick`), which is designed to be
cacheable across method-call sites. Therefore, it does not include
inference variables or other information.
[UFCS]: https://github.com/rust-lang/rfcs/blob/master/text/0132-ufcs.md
[probe]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_typeck/check/method/probe/
[confirm]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_typeck/check/method/confirm/
## The Probe phase
### Steps
The first thing that the probe phase does is to create a series of
*steps*. This is done by progressively dereferencing the receiver type
until it cannot be deref'd anymore, as well as applying an optional
"unsize" step. So if the receiver has type `Rc<Box<[T; 3]>>`, this
might yield:
```rust,ignore
Rc<Box<[T; 3]>>
Box<[T; 3]>
[T; 3]
[T]
```
### Candidate assembly
We then search along those steps to create a list of *candidates*. A
`Candidate` is a method item that might plausibly be the method being
invoked. For each candidate, we'll derive a "transformed self type"
that takes into account explicit self.
Candidates are grouped into two kinds, inherent and extension.
**Inherent candidates** are those that are derived from the
type of the receiver itself. So, if you have a receiver of some
nominal type `Foo` (e.g., a struct), any methods defined within an
impl like `impl Foo` are inherent methods. Nothing needs to be
imported to use an inherent method, they are associated with the type
itself (note that inherent impls can only be defined in the same
module as the type itself).
FIXME: Inherent candidates are not always derived from impls. If you
have a trait object, such as a value of type `Box<ToString>`, then the
trait methods (`to_string()`, in this case) are inherently associated
with it. Another case is type parameters, in which case the methods of
their bounds are inherent. However, this part of the rules is subject
to change: when DST's "impl Trait for Trait" is complete, trait object
dispatch could be subsumed into trait matching, and the type parameter
behavior should be reconsidered in light of where clauses.
TODO: Is this FIXME still accurate?
**Extension candidates** are derived from imported traits. If I have
the trait `ToString` imported, and I call `to_string()` on a value of
type `T`, then we will go off to find out whether there is an impl of
`ToString` for `T`. These kinds of method calls are called "extension
methods". They can be defined in any module, not only the one that
defined `T`. Furthermore, you must import the trait to call such a
method.
So, let's continue our example. Imagine that we were calling a method
`foo` with the receiver `Rc<Box<[T; 3]>>` and there is a trait `Foo`
that defines it with `&self` for the type `Rc<U>` as well as a method
on the type `Box` that defines `Foo` but with `&mut self`. Then we
might have two candidates:
```text
&Rc<Box<[T; 3]>> from the impl of `Foo` for `Rc<U>` where `U=Box<T; 3]>
&mut Box<[T; 3]>> from the inherent impl on `Box<U>` where `U=[T; 3]`
```
### Candidate search
Finally, to actually pick the method, we will search down the steps,
trying to match the receiver type against the candidate types. At
each step, we also consider an auto-ref and auto-mut-ref to see whether
that makes any of the candidates match. We pick the first step where
we find a match.
In the case of our example, the first step is `Rc<Box<[T; 3]>>`,
which does not itself match any candidate. But when we autoref it, we
get the type `&Rc<Box<[T; 3]>>` which does match. We would then
recursively consider all where-clauses that appear on the impl: if
those match (or we cannot rule out that they do), then this is the
method we would pick. Otherwise, we would continue down the series of
steps.

150
src/doc/rustc-guide/src/mir/construction.md Normal file
View File

@ -0,0 +1,150 @@
# MIR construction
The lowering of [HIR] to [MIR] occurs for the following (probably incomplete)
list of items:
* Function and Closure bodies
* Initializers of `static` and `const` items
* Initializers of enum discriminants
* Glue and Shims of any kind
* Tuple struct initializer functions
* Drop code (the `Drop::drop` function is not called directly)
* Drop implementations of types without an explicit `Drop` implementation
The lowering is triggered by calling the [`mir_built`] query.
There is an intermediate representation
between [HIR] and [MIR] called the [HAIR] that is only used during the lowering.
The [HAIR]'s most important feature is that the various adjustments (which happen
without explicit syntax) like coercions, autoderef, autoref and overloaded method
calls have become explicit casts, deref operations, reference expressions or
concrete function calls.
The [HAIR] has datatypes that mirror the [HIR] datatypes, but instead of e.g. `-x`
being a `hair::ExprKind::Neg(hair::Expr)` it is a `hair::ExprKind::Neg(hir::Expr)`.
This shallowness enables the `HAIR` to represent all datatypes that [HIR] has, but
without having to create an in-memory copy of the entire [HIR].
[MIR] lowering will first convert the topmost expression from
[HIR] to [HAIR] (in
[https://doc.rust-lang.org/nightly/nightly-rustc/rustc_mir/hair/cx/expr/index.html])
and then process the [HAIR] expressions recursively.
The lowering creates local variables for every argument as specified in the signature.
Next it creates local variables for every binding specified (e.g. `(a, b): (i32, String)`)
produces 3 bindings, one for the argument, and two for the bindings. Next it generates
field accesses that read the fields from the argument and writes the value to the binding
variable.
With this initialization out of the way, the lowering triggers a recursive call
to a function that generates the MIR for the body (a `Block` expression) and
writes the result into the `RETURN_PLACE`.
## `unpack!` all the things
Functions that generate MIR tend to fall into one of two patterns.
First, if the function generates only statements, then it will take a
basic block as argument onto which those statements should be appended.
It can then return a result as normal:
```rust,ignore
fn generate_some_mir(&mut self, block: BasicBlock) -> ResultType {
...
}
```
But there are other functions that may generate new basic blocks as well.
For example, lowering an expression like `if foo { 22 } else { 44 }`
requires generating a small "diamond-shaped graph".
In this case, the functions take a basic block where their code starts
and return a (potentially) new basic block where the code generation ends.
The `BlockAnd` type is used to represent this:
```rust,ignore
fn generate_more_mir(&mut self, block: BasicBlock) -> BlockAnd<ResultType> {
...
}
```
When you invoke these functions, it is common to have a local variable `block`
that is effectively a "cursor". It represents the point at which we are adding new MIR.
When you invoke `generate_more_mir`, you want to update this cursor.
You can do this manually, but it's tedious:
```rust,ignore
let mut block;
let v = match self.generate_more_mir(..) {
BlockAnd { block: new_block, value: v } => {
block = new_block;
v
}
};
```
For this reason, we offer a macro that lets you write
`let v = unpack!(block = self.generate_more_mir(...))`.
It simply extracts the new block and overwrites the
variable `block` that you named in the `unpack!`.
## Lowering expressions into the desired MIR
There are essentially four kinds of representations one might want of an expression:
* `Place` refers to a (or part of a) preexisting memory location (local, static, promoted)
* `Rvalue` is something that can be assigned to a `Place`
* `Operand` is an argument to e.g. a `+` operation or a function call
* a temporary variable containing a copy of the value
We start out with lowering the function body to an `Rvalue` so we can create an
assignment to `RETURN_PLACE`, This `Rvalue` lowering will in turn trigger lowering to
`Operand` for its arguments (if any). `Operand` lowering either produces a `const`
operand, or moves/copies out of a `Place`, thus triggering a `Place` lowering. An
expression being lowered to a `Place` can in turn trigger a temporary to be created
if the expression being lowered contains operations. This is where the snake bites its
own tail and we need to trigger an `Rvalue` lowering for the expression to be written
into the local.
## Operator lowering
Operators on builtin types are not lowered to function calls (which would end up being
infinite recursion calls, because the trait impls just contain the operation itself
again). Instead there are `Rvalue`s for binary and unary operators and index operations.
These `Rvalue`s later get codegened to llvm primitive operations or llvm intrinsics.
Operators on all other types get lowered to a function call to their `impl` of the
operator's corresponding trait.
Regardless of the lowering kind, the arguments to the operator are lowered to `Operand`s.
This means all arguments are either constants, or refer to an already existing value
somewhere in a local or static.
## Method call lowering
Method calls are lowered to the same `TerminatorKind` that function calls are.
In [MIR] there is no difference between method calls and function calls anymore.
## Conditions
`if` conditions and `match` statements for `enum`s without variants with fields are
lowered to `TerminatorKind::SwitchInt`. Each possible value (so `0` and `1` for `if`
conditions) has a corresponding `BasicBlock` to which the code continues.
The argument being branched on is (again) an `Operand` representing the value of
the if condition.
### Pattern matching
`match` statements for `enum`s with variants that have fields are lowered to
`TerminatorKind::SwitchInt`, too, but the `Operand` refers to a `Place` where the
discriminant of the value can be found. This often involves reading the discriminant
to a new temporary variable.
## Aggregate construction
Aggregate values of any kind (e.g. structs or tuples) are built via `Rvalue::Aggregate`.
All fields are
lowered to `Operator`s. This is essentially equivalent to one assignment
statement per aggregate field plus an assignment to the discriminant in the
case of `enum`s.
[MIR]: ./index.html
[HIR]: ../hir.html
[HAIR]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_mir/hair/index.html
[`mir_built`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_mir/transform/fn.mir_built.html

247
src/doc/rustc-guide/src/mir/index.md Normal file
View File

@ -0,0 +1,247 @@
# The MIR (Mid-level IR)
MIR is Rust's _Mid-level Intermediate Representation_. It is
constructed from [HIR](../hir.html). MIR was introduced in
[RFC 1211]. It is a radically simplified form of Rust that is used for
certain flow-sensitive safety checks – notably the borrow checker! –
and also for optimization and code generation.
If you'd like a very high-level introduction to MIR, as well as some
of the compiler concepts that it relies on (such as control-flow
graphs and desugaring), you may enjoy the
[rust-lang blog post that introduced MIR][blog].
[blog]: https://blog.rust-lang.org/2016/04/19/MIR.html
## Introduction to MIR
MIR is defined in the [`src/librustc/mir/`][mir] module, but much of the code
that manipulates it is found in [`src/librustc_mir`][mirmanip].
[RFC 1211]: http://rust-lang.github.io/rfcs/1211-mir.html
Some of the key characteristics of MIR are:
- It is based on a [control-flow graph][cfg].
- It does not have nested expressions.
- All types in MIR are fully explicit.
[cfg]: ../appendix/background.html#cfg
## Key MIR vocabulary
This section introduces the key concepts of MIR, summarized here:
- **Basic blocks**: units of the control-flow graph, consisting of:
- **statements:** actions with one successor
- **terminators:** actions with potentially multiple successors; always at
the end of a block
- (if you're not familiar with the term *basic block*, see the [background
chapter][cfg])
- **Locals:** Memory locations allocated on the stack (conceptually, at
least), such as function arguments, local variables, and
temporaries. These are identified by an index, written with a
leading underscore, like `_1`. There is also a special "local"
(`_0`) allocated to store the return value.
- **Places:** expressions that identify a location in memory, like `_1` or
`_1.f`.
- **Rvalues:** expressions that produce a value. The "R" stands for
the fact that these are the "right-hand side" of an assignment.
- **Operands:** the arguments to an rvalue, which can either be a
constant (like `22`) or a place (like `_1`).
You can get a feeling for how MIR is structed by translating simple
programs into MIR and reading the pretty printed output. In fact, the
playground makes this easy, since it supplies a MIR button that will
show you the MIR for your program. Try putting this program into play
(or [clicking on this link][sample-play]), and then clicking the "MIR"
button on the top:
[sample-play]: https://play.rust-lang.org/?gist=30074856e62e74e91f06abd19bd72ece&version=stable
```rust
fn main() {
let mut vec = Vec::new();
vec.push(1);
vec.push(2);
}
```
You should see something like:
```mir
// WARNING: This output format is intended for human consumers only
// and is subject to change without notice. Knock yourself out.
fn main() -> () {
...
}
```
This is the MIR format for the `main` function.
**Variable declarations.** If we drill in a bit, we'll see it begins
with a bunch of variable declarations. They look like this:
```mir
let mut _0: (); // return place
scope 1 {
let mut _1: std::vec::Vec<i32>; // "vec" in scope 1 at src/main.rs:2:9: 2:16
}
scope 2 {
}
let mut _2: ();
let mut _3: &mut std::vec::Vec<i32>;
let mut _4: ();
let mut _5: &mut std::vec::Vec<i32>;
```
You can see that variables in MIR don't have names, they have indices,
like `_0` or `_1`. We also intermingle the user's variables (e.g.,
`_1`) with temporary values (e.g., `_2` or `_3`). You can tell the
difference between user-defined variables have a comment that gives
you their original name (`// "vec" in scope 1...`). The "scope" blocks
(e.g., `scope 1 { .. }`) describe the lexical structure of the source
program (which names were in scope when).
**Basic blocks.** Reading further, we see our first **basic block** (naturally
it may look slightly different when you view it, and I am ignoring some of the
comments):
```mir
bb0: {
StorageLive(_1);
_1 = const <std::vec::Vec<T>>::new() -> bb2;
}
```
A basic block is defined by a series of **statements** and a final
**terminator**. In this case, there is one statement:
```mir
StorageLive(_1);
```
This statement indicates that the variable `_1` is "live", meaning
that it may be used later – this will persist until we encounter a
`StorageDead(_1)` statement, which indicates that the variable `_1` is
done being used. These "storage statements" are used by LLVM to
allocate stack space.
The **terminator** of the block `bb0` is the call to `Vec::new`:
```mir
_1 = const <std::vec::Vec<T>>::new() -> bb2;
```
Terminators are different from statements because they can have more
than one successor – that is, control may flow to different
places. Function calls like the call to `Vec::new` are always
terminators because of the possibility of unwinding, although in the
case of `Vec::new` we are able to see that indeed unwinding is not
possible, and hence we list only one succssor block, `bb2`.
If we look ahead to `bb2`, we will see it looks like this:
```mir
bb2: {
StorageLive(_3);
_3 = &mut _1;
_2 = const <std::vec::Vec<T>>::push(move _3, const 1i32) -> [return: bb3, unwind: bb4];
}
```
Here there are two statements: another `StorageLive`, introducing the `_3`
temporary, and then an assignment:
```mir
_3 = &mut _1;
```
Assignments in general have the form:
```text
<Place> = <Rvalue>
```
A place is an expression like `_3`, `_3.f` or `*_3` – it denotes a
location in memory. An **Rvalue** is an expression that creates a
value: in this case, the rvalue is a mutable borrow expression, which
looks like `&mut <Place>`. So we can kind of define a grammar for
rvalues like so:
```text
<Rvalue> = & (mut)? <Place>
| <Operand> + <Operand>
| <Operand> - <Operand>
| ...
<Operand> = Constant
| copy Place
| move Place
```
As you can see from this grammar, rvalues cannot be nested – they can
only reference places and constants. Moreover, when you use a place,
we indicate whether we are **copying it** (which requires that the
place have a type `T` where `T: Copy`) or **moving it** (which works
for a place of any type). So, for example, if we had the expression `x
= a + b + c` in Rust, that would get compile to two statements and a
temporary:
```mir
TMP1 = a + b
x = TMP1 + c
```
([Try it and see][play-abc], though you may want to do release mode to skip
over the overflow checks.)
[play-abc]: https://play.rust-lang.org/?gist=1751196d63b2a71f8208119e59d8a5b6&version=stable
## MIR data types
The MIR data types are defined in the [`src/librustc/mir/`][mir]
module. Each of the key concepts mentioned in the previous section
maps in a fairly straightforward way to a Rust type.
The main MIR data type is `Mir`. It contains the data for a single
function (along with sub-instances of Mir for "promoted constants",
but [you can read about those below](#promoted)).
- **Basic blocks**: The basic blocks are stored in the field
`basic_blocks`; this is a vector of `BasicBlockData`
structures. Nobody ever references a basic block directly: instead,
we pass around `BasicBlock` values, which are
[newtype'd] indices into this vector.
- **Statements** are represented by the type `Statement`.
- **Terminators** are represented by the `Terminator`.
- **Locals** are represented by a [newtype'd] index type `Local`. The
data for a local variable is found in the `Mir` (the `local_decls`
vector). There is also a special constant `RETURN_PLACE` identifying
the special "local" representing the return value.
- **Places** are identified by the enum `Place`. There are a few variants:
- Local variables like `_1`
- Static variables `FOO`
- **Projections**, which are fields or other things that "project
out" from a base place. So e.g. the place `_1.f` is a projection,
with `f` being the "projection element and `_1` being the base
path. `*_1` is also a projection, with the `*` being represented
by the `ProjectionElem::Deref` element.
- **Rvalues** are represented by the enum `Rvalue`.
- **Operands** are represented by the enum `Operand`.
## Representing constants
*to be written*
<a name="promoted"></a>
### Promoted constants
*to be written*
[mir]: https://github.com/rust-lang/rust/tree/master/src/librustc/mir
[mirmanip]: https://github.com/rust-lang/rust/tree/master/src/librustc_mir
[mir]: https://github.com/rust-lang/rust/tree/master/src/librustc/mir
[newtype'd]: ../appendix/glossary.html

1
src/doc/rustc-guide/src/mir/optimizations.md Normal file
View File

@ -0,0 +1 @@
# MIR optimizations

177
src/doc/rustc-guide/src/mir/passes.md Normal file
View File

@ -0,0 +1,177 @@
# MIR passes
If you would like to get the MIR for a function (or constant, etc),
you can use the `optimized_mir(def_id)` query. This will give you back
the final, optimized MIR. For foreign def-ids, we simply read the MIR
from the other crate's metadata. But for local def-ids, the query will
construct the MIR and then iteratively optimize it by applying a
series of passes. This section describes how those passes work and how
you can extend them.
To produce the `optimized_mir(D)` for a given def-id `D`, the MIR
passes through several suites of optimizations, each represented by a
query. Each suite consists of multiple optimizations and
transformations. These suites represent useful intermediate points
where we want to access the MIR for type checking or other purposes:
- `mir_build(D)` – not a query, but this constructs the initial MIR
- `mir_const(D)` – applies some simple transformations to make MIR ready for
constant evaluation;
- `mir_validated(D)` – applies some more transformations, making MIR ready for
borrow checking;
- `optimized_mir(D)` – the final state, after all optimizations have been
performed.
### Seeing how the MIR changes as the compiler executes
`-Zdump-mir=F` is a handy compiler options that will let you view the MIR for
each function at each stage of compilation. `-Zdump-mir` takes a **filter** `F`
which allows you to control which functions and which passes you are
interesting in. For example:
```bash
> rustc -Zdump-mir=foo ...
```
This will dump the MIR for any function whose name contains `foo`; it
will dump the MIR both before and after every pass. Those files will
be created in the `mir_dump` directory. There will likely be quite a
lot of them!
```bash
> cat > foo.rs
fn main() {
println!("Hello, world!");
}
^D
> rustc -Zdump-mir=main foo.rs
> ls mir_dump/* | wc -l
161
```
The files have names like `rustc.main.000-000.CleanEndRegions.after.mir`. These
names have a number of parts:
```text
rustc.main.000-000.CleanEndRegions.after.mir
---- --- --- --------------- ----- either before or after
| | | name of the pass
| | index of dump within the pass (usually 0, but some passes dump intermediate states)
| index of the pass
def-path to the function etc being dumped
```
You can also make more selective filters. For example, `main & CleanEndRegions`
will select for things that reference *both* `main` and the pass
`CleanEndRegions`:
```bash
> rustc -Zdump-mir='main & CleanEndRegions' foo.rs
> ls mir_dump
rustc.main.000-000.CleanEndRegions.after.mir rustc.main.000-000.CleanEndRegions.before.mir
```
Filters can also have `|` parts to combine multiple sets of
`&`-filters. For example `main & CleanEndRegions | main &
NoLandingPads` will select *either* `main` and `CleanEndRegions` *or*
`main` and `NoLandingPads`:
```bash
> rustc -Zdump-mir='main & CleanEndRegions | main & NoLandingPads' foo.rs
> ls mir_dump
rustc.main-promoted[0].002-000.NoLandingPads.after.mir
rustc.main-promoted[0].002-000.NoLandingPads.before.mir
rustc.main-promoted[0].002-006.NoLandingPads.after.mir
rustc.main-promoted[0].002-006.NoLandingPads.before.mir
rustc.main-promoted[1].002-000.NoLandingPads.after.mir
rustc.main-promoted[1].002-000.NoLandingPads.before.mir
rustc.main-promoted[1].002-006.NoLandingPads.after.mir
rustc.main-promoted[1].002-006.NoLandingPads.before.mir
rustc.main.000-000.CleanEndRegions.after.mir
rustc.main.000-000.CleanEndRegions.before.mir
rustc.main.002-000.NoLandingPads.after.mir
rustc.main.002-000.NoLandingPads.before.mir
rustc.main.002-006.NoLandingPads.after.mir
rustc.main.002-006.NoLandingPads.before.mir
```
(Here, the `main-promoted[0]` files refer to the MIR for "promoted constants"
that appeared within the `main` function.)
### Implementing and registering a pass
A `MirPass` is some bit of code that processes the MIR, typically –
but not always – transforming it along the way somehow. For example,
it might perform an optimization. The `MirPass` trait itself is found
in in [the `rustc_mir::transform` module][mirtransform], and it
basically consists of one method, `run_pass`, that simply gets an
`&mut Mir` (along with the tcx and some information about where it
came from). The MIR is therefore modified in place (which helps to
keep things efficient).
A good example of a basic MIR pass is [`NoLandingPads`], which walks
the MIR and removes all edges that are due to unwinding – this is
used when configured with `panic=abort`, which never unwinds. As you
can see from its source, a MIR pass is defined by first defining a
dummy type, a struct with no fields, something like:
```rust
struct MyPass;
```
for which you then implement the `MirPass` trait. You can then insert
this pass into the appropriate list of passes found in a query like
`optimized_mir`, `mir_validated`, etc. (If this is an optimization, it
should go into the `optimized_mir` list.)
If you are writing a pass, there's a good chance that you are going to
want to use a [MIR visitor]. MIR visitors are a handy way to walk all
the parts of the MIR, either to search for something or to make small
edits.
### Stealing
The intermediate queries `mir_const()` and `mir_validated()` yield up
a `&'tcx Steal<Mir<'tcx>>`, allocated using
`tcx.alloc_steal_mir()`. This indicates that the result may be
**stolen** by the next suite of optimizations – this is an
optimization to avoid cloning the MIR. Attempting to use a stolen
result will cause a panic in the compiler. Therefore, it is important
that you do not read directly from these intermediate queries except as
part of the MIR processing pipeline.
Because of this stealing mechanism, some care must also be taken to
ensure that, before the MIR at a particular phase in the processing
pipeline is stolen, anyone who may want to read from it has already
done so. Concretely, this means that if you have some query `foo(D)`
that wants to access the result of `mir_const(D)` or
`mir_validated(D)`, you need to have the successor pass "force"
`foo(D)` using `ty::queries::foo::force(...)`. This will force a query
to execute even though you don't directly require its result.
As an example, consider MIR const qualification. It wants to read the
result produced by the `mir_const()` suite. However, that result will
be **stolen** by the `mir_validated()` suite. If nothing was done,
then `mir_const_qualif(D)` would succeed if it came before
`mir_validated(D)`, but fail otherwise. Therefore, `mir_validated(D)`
will **force** `mir_const_qualif` before it actually steals, thus
ensuring that the reads have already happened (remember that
[queries are memoized](../query.html), so executing a query twice
simply loads from a cache the second time):
```text
mir_const(D) --read-by--> mir_const_qualif(D)
| ^
stolen-by |
| (forces)
v |
mir_validated(D) ------------+
```
This mechanism is a bit dodgy. There is a discussion of more elegant
alternatives in [rust-lang/rust#41710].
[rust-lang/rust#41710]: https://github.com/rust-lang/rust/issues/41710
[mirtransform]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_mir/transform/
[`NoLandingPads`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_mir/transform/no_landing_pads/struct.NoLandingPads.html
[MIR visitor]: ./visitor.html

55
src/doc/rustc-guide/src/mir/visitor.md Normal file
View File

@ -0,0 +1,55 @@
# MIR visitor
The MIR visitor is a convenient tool for traversing the MIR and either
looking for things or making changes to it. The visitor traits are
defined in [the `rustc::mir::visit` module][m-v] – there are two of
them, generated via a single macro: `Visitor` (which operates on a
`&Mir` and gives back shared references) and `MutVisitor` (which
operates on a `&mut Mir` and gives back mutable references).
[m-v]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/mir/visit/index.html
To implement a visitor, you have to create a type that represents
your visitor. Typically, this type wants to "hang on" to whatever
state you will need while processing MIR:
```rust,ignore
struct MyVisitor<...> {
tcx: TyCtxt<'cx, 'tcx, 'tcx>,
...
}
```
and you then implement the `Visitor` or `MutVisitor` trait for that type:
```rust,ignore
impl<'tcx> MutVisitor<'tcx> for NoLandingPads {
fn visit_foo(&mut self, ...) {
...
self.super_foo(...);
}
}
```
As shown above, within the impl, you can override any of the
`visit_foo` methods (e.g., `visit_terminator`) in order to write some
code that will execute whenever a `foo` is found. If you want to
recursively walk the contents of the `foo`, you then invoke the
`super_foo` method. (NB. You never want to override `super_foo`.)
A very simple example of a visitor can be found in [`NoLandingPads`].
That visitor doesn't even require any state: it just visits all
terminators and removes their `unwind` successors.
[`NoLandingPads`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_mir/transform/no_landing_pads/struct.NoLandingPads.html
## Traversal
In addition the visitor, [the `rustc::mir::traversal` module][t]
contains useful functions for walking the MIR CFG in
[different standard orders][traversal] (e.g. pre-order, reverse
post-order, and so forth).
[t]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/mir/traversal/index.html
[traversal]: https://en.wikipedia.org/wiki/Tree_traversal

142
src/doc/rustc-guide/src/miri.md Normal file
View File

@ -0,0 +1,142 @@
# Miri
Miri (**MIR** **I**nterpreter) is a virtual machine for executing MIR without
compiling to machine code. It is usually invoked via `tcx.const_eval`.
If you start out with a constant
```rust
const FOO: usize = 1 << 12;
```
rustc doesn't actually invoke anything until the constant is either used or
placed into metadata.
Once you have a use-site like
```rust,ignore
type Foo = [u8; FOO - 42];
```
The compiler needs to figure out the length of the array before being able to
create items that use the type (locals, constants, function arguments, ...).
To obtain the (in this case empty) parameter environment, one can call
`let param_env = tcx.param_env(length_def_id);`. The `GlobalId` needed is
```rust,ignore
let gid = GlobalId {
promoted: None,
instance: Instance::mono(length_def_id),
};
```
Invoking `tcx.const_eval(param_env.and(gid))` will now trigger the creation of
the MIR of the array length expression. The MIR will look something like this:
```mir
const Foo::{{initializer}}: usize = {
let mut _0: usize; // return pointer
let mut _1: (usize, bool);
bb0: {
_1 = CheckedSub(const Unevaluated(FOO, Slice([])), const 42usize);
assert(!(_1.1: bool), "attempt to subtract with overflow") -> bb1;
}
bb1: {
_0 = (_1.0: usize);
return;
}
}
```
Before the evaluation, a virtual memory location (in this case essentially a
`vec![u8; 4]` or `vec![u8; 8]`) is created for storing the evaluation result.
At the start of the evaluation, `_0` and `_1` are
`Value::ByVal(PrimVal::Undef)`. When the initialization of `_1` is invoked, the
value of the `FOO` constant is required, and triggers another call to
`tcx.const_eval`, which will not be shown here. If the evaluation of FOO is
successful, 42 will be subtracted by its value `4096` and the result stored in
`_1` as `Value::ByValPair(PrimVal::Bytes(4054), PrimVal::Bytes(0))`. The first
part of the pair is the computed value, the second part is a bool that's true if
an overflow happened.
The next statement asserts that said boolean is `0`. In case the assertion
fails, its error message is used for reporting a compile-time error.
Since it does not fail, `Value::ByVal(PrimVal::Bytes(4054))` is stored in the
virtual memory was allocated before the evaluation. `_0` always refers to that
location directly.
After the evaluation is done, the virtual memory allocation is interned into the
`TyCtxt`. Future evaluations of the same constants will not actually invoke
miri, but just extract the value from the interned allocation.
The `tcx.const_eval` function has one additional feature: it will not return a
`ByRef(interned_allocation_id)`, but a `ByVal(computed_value)` if possible. This
makes using the result much more convenient, as no further queries need to be
executed in order to get at something as simple as a `usize`.
## Datastructures
Miri's core datastructures can be found in
[librustc/mir/interpret](https://github.com/rust-lang/rust/blob/master/src/librustc/mir/interpret).
This is mainly the error enum and the `Value` and `PrimVal` types. A `Value` can
be either `ByVal` (a single `PrimVal`), `ByValPair` (two `PrimVal`s, usually fat
pointers or two element tuples) or `ByRef`, which is used for anything else and
refers to a virtual allocation. These allocations can be accessed via the
methods on `tcx.interpret_interner`.
If you are expecting a numeric result, you can use `unwrap_u64` (panics on
anything that can't be representad as a `u64`) or `to_raw_bits` which results
in an `Option<u128>` yielding the `ByVal` if possible.
## Allocations
A miri allocation is either a byte sequence of the memory or an `Instance` in
the case of function pointers. Byte sequences can additionally contain
relocations that mark a group of bytes as a pointer to another allocation. The
actual bytes at the relocation refer to the offset inside the other allocation.
These allocations exist so that references and raw pointers have something to
point to. There is no global linear heap in which things are allocated, but each
allocation (be it for a local variable, a static or a (future) heap allocation)
gets its own little memory with exactly the required size. So if you have a
pointer to an allocation for a local variable `a`, there is no possible (no
matter how unsafe) operation that you can do that would ever change said pointer
to a pointer to `b`.
## Interpretation
Although the main entry point to constant evaluation is the `tcx.const_eval`
query, there are additional functions in
[librustc_mir/const_eval.rs](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_mir/const_eval/index.html)
that allow accessing the fields of a `Value` (`ByRef` or otherwise). You should
never have to access an `Allocation` directly except for translating it to the
compilation target (at the moment just LLVM).
Miri starts by creating a virtual stack frame for the current constant that is
being evaluated. There's essentially no difference between a constant and a
function with no arguments, except that constants do not allow local (named)
variables at the time of writing this guide.
A stack frame is defined by the `Frame` type in
[librustc_mir/interpret/eval_context.rs](https://github.com/rust-lang/rust/blob/master/src/librustc_mir/interpret/eval_context.rs)
and contains all the local
variables memory (`None` at the start of evaluation). Each frame refers to the
evaluation of either the root constant or subsequent calls to `const fn`. The
evaluation of another constant simply calls `tcx.const_eval`, which produces an
entirely new and independent stack frame.
The frames are just a `Vec<Frame>`, there's no way to actually refer to a
`Frame`'s memory even if horrible shenigans are done via unsafe code. The only
memory that can be referred to are `Allocation`s.
Miri now calls the `step` method (in
[librustc_mir/interpret/step.rs](https://github.com/rust-lang/rust/blob/master/src/librustc_mir/interpret/step.rs)
) until it either returns an error or has no further statements to execute. Each
statement will now initialize or modify the locals or the virtual memory
referred to by a local. This might require evaluating other constants or
statics, which just recursively invokes `tcx.const_eval`.

119
src/doc/rustc-guide/src/name-resolution.md Normal file
View File

@ -0,0 +1,119 @@
# Name resolution
The name resolution is a two-phase process. In the first phase, which runs
during macro expansion, we build a tree of modules and resolve imports. Macro
expansion and name resolution communicate with each other via the `Resolver`
trait, defined in `libsyntax`.
The input to the second phase is the syntax tree, produced by parsing input
files and expanding macros. This phase produces links from all the names in the
source to relevant places where the name was introduced. It also generates
helpful error messages, like typo suggestions, traits to import or lints about
unused items.
A successful run of the second phase (`Resolver::resolve_crate`) creates kind
of an index the rest of the compilation may use to ask about the present names
(through the `hir::lowering::Resolver` interface).
The name resolution lives in the `librustc_resolve` crate, with the meat in
`lib.rs` and some helpers or symbol-type specific logic in the other modules.
## Namespaces
Different kind of symbols live in different namespaces ‒ eg. types don't
clash with variables. This usually doesn't happen, because variables start with
lower-case letter while types with upper case one, but this is only a
convention. This is legal Rust code that'll compile (with warnings):
```rust
type x = u32;
let x: x = 1;
let y: x = 2; // See? x is still a type here.
```
To cope with this, and with slightly different scoping rules for these
namespaces, the resolver keeps them separated and builds separate structures for
them.
In other words, when the code talks about namespaces, it doesn't mean the module
hierarchy, it's types vs. values vs. macros.
## Scopes and ribs
A name is visible only in certain area in the source code. This forms a
hierarchical structure, but not necessarily a simple one ‒ if one scope is
part of another, it doesn't mean the name visible in the outer one is also
visible in the inner one, or that it refers to the same thing.
To cope with that, the compiler introduces the concept of Ribs. This is
abstraction of a scope. Every time the set of visible names potentially changes,
a new rib is pushed onto a stack. The places where this can happen includes for
example:
* The obvious places ‒ curly braces enclosing a block, function boundaries,
modules.
* Introducing a let binding ‒ this can shadow another binding with the same
name.
* Macro expansion border ‒ to cope with macro hygiene.
When searching for a name, the stack of ribs is traversed from the innermost
outwards. This helps to find the closest meaning of the name (the one not
shadowed by anything else). The transition to outer rib may also change the
rules what names are usable ‒ if there are nested functions (not closures),
the inner one can't access parameters and local bindings of the outer one,
even though they should be visible by ordinary scoping rules. An example:
```rust
fn do_something<T: Default>(val: T) { // <- New rib in both types and values (1)
// `val` is accessible, as is the helper function
// `T` is accessible
let helper = || { // New rib on `helper` (2) and another on the block (3)
// `val` is accessible here
}; // End of (3)
// `val` is accessible, `helper` variable shadows `helper` function
fn helper() { // <- New rib in both types and values (4)
// `val` is not accessible here, (4) is not transparent for locals)
// `T` is not accessible here
} // End of (4)
let val = T::default(); // New rib (5)
// `val` is the variable, not the parameter here
} // End of (5), (2) and (1)
```
Because the rules for different namespaces are a bit different, each namespace
has its own independent rib stack that is constructed in parallel to the others.
In addition, there's also a rib stack for local labels (eg. names of loops or
blocks), which isn't a full namespace in its own right.
## Overall strategy
To perform the name resolution of the whole crate, the syntax tree is traversed
top-down and every encountered name is resolved. This works for most kinds of
names, because at the point of use of a name it is already introduced in the Rib
hierarchy.
There are some exceptions to this. Items are bit tricky, because they can be
used even before encountered ‒ therefore every block needs to be first scanned
for items to fill in its Rib.
Other, even more problematic ones, are imports which need recursive fixed-point
resolution and macros, that need to be resolved and expanded before the rest of
the code can be processed.
Therefore, the resolution is performed in multiple stages.
## TODO:
This is a result of the first pass of learning the code. It is definitely
incomplete and not detailed enough. It also might be inaccurate in places.
Still, it probably provides useful first guidepost to what happens in there.
* What exactly does it link to and how is that published and consumed by
following stages of compilation?
* Who calls it and how it is actually used.
* Is it a pass and then the result is only used, or can it be computed
incrementally (eg. for RLS)?
* The overall strategy description is a bit vague.
* Where does the name `Rib` come from?
* Does this thing have its own tests, or is it tested only as part of some e2e
testing?

30
src/doc/rustc-guide/src/param_env.md Normal file
View File

@ -0,0 +1,30 @@
# Parameter Environment
When working with associated and/or or generic items (types, constants,
functions/methods) it is often relevant to have more information about the
`Self` or generic parameters. Trait bounds and similar information is encoded in
the `ParamEnv`. Often this is not enough information to obtain things like the
type's `Layout`, but you can do all kinds of other checks on it (e.g. whether a
type implements `Copy`) or you can evaluate an associated constant whose value
does not depend on anything from the parameter environment.
For example if you have a function
```rust
fn foo<T: Copy>(t: T) {
}
```
the parameter environment for that function is `[T: Copy]`. This means any
evaluation within this function will, when accessing the type `T`, know about
its `Copy` bound via the parameter environment.
Although you can obtain a valid `ParamEnv` for any item via
`tcx.param_env(def_id)`, this `ParamEnv` can be too generic for your use case.
Using the `ParamEnv` from the surrounding context can allow you to evaluate more
things.
Another great thing about `ParamEnv` is that you can use it to bundle the thing
depending on generic parameters (e.g. a `Ty`) by calling `param_env.and(ty)`.
This will produce a `ParamEnvAnd<Ty>`, making clear that you should probably not
be using the inner value without taking care to also use the `ParamEnv`.

9
src/doc/rustc-guide/src/profiling.md Normal file
View File

@ -0,0 +1,9 @@
# Profiling the compiler
This discussion talks about how profile the compiler and find out
where it spends its time. If you just want to get a general overview,
it is often a good idea to just add `-Zself-profile` option to the
rustc command line. This will break down time spent into various
categories. But if you want a more detailed look, you probably want
to break out a custom profiler.

331
src/doc/rustc-guide/src/profiling/with_perf.md Normal file
View File

@ -0,0 +1,331 @@
# Profiling with perf
This is a guide for how to profile rustc with [perf](https://perf.wiki.kernel.org/index.php/Main_Page).
## Initial steps
- Get a clean checkout of rust-lang/master, or whatever it is you want
to profile.
- Set the following settings in your `config.toml`:
- `debuginfo-lines = true`
- `use-jemalloc = false` — lets you do memory use profiling with valgrind
- leave everything else the defaults
- Run `./x.py build` to get a full build
- Make a rustup toolchain pointing to that result
- see [the "build and run" section for instructions][b-a-r]
[b-a-r]: ../how-to-build-and-run.html#toolchain
## Gathering a perf profile
perf is an excellent tool on linux that can be used to gather and
analyze all kinds of information. Mostly it is used to figure out
where a program spends its time. It can also be used for other sorts
of events, though, like cache misses and so forth.
### The basics
The basic `perf` command is this:
```bash
> perf record -F99 --call-graph dwarf XXX
```
The `-F99` tells perf to sample at 99 Hz, which avoids generating too
much data for longer runs (why 99 Hz you ask? It is often chosen
because it is unlikely to be in lockstep with other periodic
activity). The `--call-graph dwarf` tells perf to get call-graph
information from debuginfo, which is accurate. The `XXX` is the
command you want to profile. So, for example, you might do:
```bash
> perf record -F99 --call-graph dwarf cargo +<toolchain> rustc
```
to run `cargo` -- here `<toolchain>` should be the name of the toolchain
you made in the beginning. But there are some things to be aware of:
- You probably don't want to profile the time spend building
dependencies. So something like `cargo build; cargo clean -p $C` may
be helpful (where `$C` is the crate name)
- Though usually I just do `touch src/lib.rs` and rebuild instead. =)
- You probably don't want incremental messing about with your
profile. So something like `CARGO_INCREMENTAL=0` can be helpful.
### Gathering a perf profile from a `perf.rust-lang.org` test
Often we want to analyze a specific test from `perf.rust-lang.org`. To
do that, the first step is to clone
[the rustc-perf repository][rustc-perf-gh]:
```bash
> git clone https://github.com/rust-lang-nursery/rustc-perf
```
[rustc-perf-gh]: https://github.com/rust-lang-nursery/rustc-perf
#### Doing it the easy way
Once you've cloned the repo, you can use the `collector` executable to
do profiling for you! You can find
[instructions in the rustc-perf readme][rustc-perf-readme].
[rustc-perf-readme]: https://github.com/rust-lang-nursery/rustc-perf/blob/master/collector/README.md#profiling
For example, to measure the clap-rs test, you might do:
```bash
> ./target/release/collector
--output-repo /path/to/place/output
profile perf-record
--rustc /path/to/rustc/executable/from/your/build/directory
--cargo `which cargo`
--filter clap-rs
--builds Check
```
You can also use that same command to use cachegrind or other profiling tools.
#### Doing it the hard way
If you prefer to run things manually, that is also possible. You first
need to find the source for the test you want. Sources for the tests
are found in [the `collector/benchmarks` directory][dir]. So let's go
into the directory of a specific test; we'll use `clap-rs` as an
example:
[dir]: https://github.com/rust-lang-nursery/rustc-perf/tree/master/collector/benchmarks
```bash
> cd collector/benchmarks/clap-rs
```
In this case, let's say we want to profile the `cargo check`
performance. In that case, I would first run some basic commands to
build the dependencies:
```bash
# Setup: first clean out any old results and build the dependencies:
> cargo +<toolchain> clean
> CARGO_INCREMENTAL=0 cargo +<toolchain> check
```
(Again, `<toolchain>` should be replaced with the name of the
toolchain we made in the first step.)
Next: we want record the execution time for *just* the clap-rs crate,
running cargo check. I tend to use `cargo rustc` for this, since it
also allows me to add explicit flags, which we'll do later on.
```bash
> touch src/lib.rs
> CARGO_INCREMENTAL=0 perf record -F99 --call-graph dwarf cargo rustc --profile check --lib
```
Note that final command: it's a doozy! It uses the `cargo rustc`
command, which executes rustc with (potentially) additional options;
the `--profile check` and `--lib` options specify that we are doing a
`cargo check` execution, and that this is a library (not a binary).
At this point, we can use `perf` tooling to analyze the results. For example:
```bash
> perf report
```
will open up an interactive TUI program. In simple cases, that can be
helpful. For more detailed examination, the [`perf-focus` tool][pf]
can be helpful; it is covered below.
**A note of caution.** Each of the rustc-perf tests is its own special
snowflake. In particular, some of them are not libraries, in which
case you would want to do `touch src/main.rs` and avoid passing
`--lib`. I'm not sure how best to tell which test is which to be
honest.
### Gathering NLL data
If you want to profile an NLL run, you can just pass extra options to
the `cargo rustc` command, like so:
```bash
> touch src/lib.rs
> CARGO_INCREMENTAL=0 perf record -F99 --call-graph dwarf cargo rustc --profile check --lib -- -Zborrowck=mir
```
[pf]: https://github.com/nikomatsakis/perf-focus
## Analyzing a perf profile with `perf focus`
Once you've gathered a perf profile, we want to get some information
about it. For this, I personally use [perf focus][pf]. It's a kind of
simple but useful tool that lets you answer queries like:
- "how much time was spent in function F" (no matter where it was called from)
- "how much time was spent in function F when it was called from G"
- "how much time was spent in function F *excluding* time spent in G"
- "what functions does F call and how much time does it spend in them"
To understand how it works, you have to know just a bit about
perf. Basically, perf works by *sampling* your process on a regular
basis (or whenever some event occurs). For each sample, perf gathers a
backtrace. `perf focus` lets you write a regular expression that tests
which functions appear in that backtrace, and then tells you which
percentage of samples had a backtrace that met the regular
expression. It's probably easiest to explain by walking through how I
would analyze NLL performance.
### Installing `perf-focus`
You can install perf-focus using `cargo install`:
```bash
> cargo install perf-focus
```
### Example: How much time is spent in MIR borrowck?
Let's say we've gathered the NLL data for a test. We'd like to know
how much time it is spending in the MIR borrow-checker. The "main"
function of the MIR borrowck is called `do_mir_borrowck`, so we can do
this command:
```bash
> perf focus '{do_mir_borrowck}'
Matcher : {do_mir_borrowck}
Matches : 228
Not Matches: 542
Percentage : 29%
```
The `'{do_mir_borrowck}'` argument is called the **matcher**. It
specifies the test to be applied on the backtrace. In this case, the
`{X}` indicates that there must be *some* function on the backtrace
that meets the regular expression `X`. In this case, that regex is
just the name of the function we want (in fact, it's a subset of the name;
the full name includes a bunch of other stuff, like the module
path). In this mode, perf-focus just prints out the percentage of
samples where `do_mir_borrowck` was on the stack: in this case, 29%.
**A note about c++filt.** To get the data from `perf`, `perf focus`
currently executes `perf script` (perhaps there is a better
way...). I've sometimes found that `perf script` outputs C++ mangled
names. This is annoying. You can tell by running `perf script |
head` yourself — if you see names like `5rustc6middle` instead of
`rustc::middle`, then you have the same problem. You can solve this
by doing:
```bash
> perf script | c++filt | perf focus --from-stdin ...
```
This will pipe the output from `perf script` through `c++filt` and
should mostly convert those names into a more friendly format. The
`--from-stdin` flag to `perf focus` tells it to get its data from
stdin, rather than executing `perf focus`. We should make this more
convenient (at worst, maybe add a `c++filt` option to `perf focus`, or
just always use it — it's pretty harmless).
### Example: How much time does MIR borrowck spend solving traits?
Perhaps we'd like to know how much time MIR borrowck spends in the
trait checker. We can ask this using a more complex regex:
```bash
> perf focus '{do_mir_borrowck}..{^rustc::traits}'
Matcher : {do_mir_borrowck},..{^rustc::traits}
Matches : 12
Not Matches: 1311
Percentage : 0%
```
Here we used the `..` operator to ask "how often do we have
`do_mir_borrowck` on the stack and then, later, some function whose
name begins with `rusc::traits`?" (basically, code in that module). It
turns out the answer is "almost never" — only 12 samples fit that
description (if you ever see *no* samples, that often indicates your
query is messed up).
If you're curious, you can find out exactly which samples by using the
`--print-match` option. This will print out the full backtrace for
each sample. The `|` at the front of the line indicates the part that
the regular expression matched.
### Example: Where does MIR borrowck spend its time?
Often we want to do a more "explorational" queries. Like, we know that
MIR borrowck is 29% of the time, but where does that time get spent?
For that, the `--tree-callees` option is often the best tool. You
usually also want to give `--tree-min-percent` or
`--tree-max-depth`. The result looks like this:
```bash
> perf focus '{do_mir_borrowck}' --tree-callees --tree-min-percent 3
Matcher : {do_mir_borrowck}
Matches : 577
Not Matches: 746
Percentage : 43%
Tree
| matched `{do_mir_borrowck}` (43% total, 0% self)
: | rustc_mir::borrow_check::nll::compute_regions (20% total, 0% self)
: : | rustc_mir::borrow_check::nll::type_check::type_check_internal (13% total, 0% self)
: : : | core::ops::function::FnOnce::call_once (5% total, 0% self)
: : : : | rustc_mir::borrow_check::nll::type_check::liveness::generate (5% total, 3% self)
: : : | <rustc_mir::borrow_check::nll::type_check::TypeVerifier<'a, 'b, 'gcx, 'tcx> as rustc::mir::visit::Visitor<'tcx>>::visit_mir (3% total, 0% self)
: | rustc::mir::visit::Visitor::visit_mir (8% total, 6% self)
: | <rustc_mir::borrow_check::MirBorrowckCtxt<'cx, 'gcx, 'tcx> as rustc_mir::dataflow::DataflowResultsConsumer<'cx, 'tcx>>::visit_statement_entry (5% total, 0% self)
: | rustc_mir::dataflow::do_dataflow (3% total, 0% self)
```
What happens with `--tree-callees` is that
- we find each sample matching the regular expression
- we look at the code that is occurs *after* the regex match and try
to build up a call tree
The `--tree-min-percent 3` option says "only show me things that take
more than 3% of the time. Without this, the tree often gets really
noisy and includes random stuff like the innards of
malloc. `--tree-max-depth` can be useful too, it just limits how many
levels we print.
For each line, we display the percent of time in that function
altogether ("total") and the percent of time spent in **just that
function and not some callee of that function** (self). Usually
"total" is the more interesting number, but not always.
### Relative percentages
By default, all in perf-focus are relative to the **total program
execution**. This is useful to help you keep perspective — often as
we drill down to find hot spots, we can lose sight of the fact that,
in terms of overall program execution, this "hot spot" is actually not
important. It also ensures that percentages between different queries
are easily compared against one another.
That said, sometimes it's useful to get relative percentages, so `perf
focus` offers a `--relative` option. In this case, the percentages are
listed only for samples that match (vs all samples). So for example we
could get our percentages relative to the borrowck itself
like so:
```bash
> perf focus '{do_mir_borrowck}' --tree-callees --relative --tree-max-depth 1 --tree-min-percent 5
Matcher : {do_mir_borrowck}
Matches : 577
Not Matches: 746
Percentage : 100%
Tree
| matched `{do_mir_borrowck}` (100% total, 0% self)
: | rustc_mir::borrow_check::nll::compute_regions (47% total, 0% self) [...]
: | rustc::mir::visit::Visitor::visit_mir (19% total, 15% self) [...]
: | <rustc_mir::borrow_check::MirBorrowckCtxt<'cx, 'gcx, 'tcx> as rustc_mir::dataflow::DataflowResultsConsumer<'cx, 'tcx>>::visit_statement_entry (13% total, 0% self) [...]
: | rustc_mir::dataflow::do_dataflow (8% total, 1% self) [...]
```
Here you see that `compute_regions` came up as "47% total" — that
means that 47% of `do_mir_borrowck` is spent in that function. Before,
we saw 20% — that's because `do_mir_borrowck` itself is only 43% of
the total time (and `.47 * .43 = .20`).

317
src/doc/rustc-guide/src/query.md Normal file
View File

@ -0,0 +1,317 @@
# Queries: demand-driven compilation
As described in [the high-level overview of the compiler][hl], the
Rust compiler is current transitioning from a traditional "pass-based"
setup to a "demand-driven" system. **The Compiler Query System is the
key to our new demand-driven organization.** The idea is pretty
simple. You have various queries that compute things about the input
– for example, there is a query called `type_of(def_id)` that, given
the def-id of some item, will compute the type of that item and return
it to you.
[hl]: high-level-overview.html
Query execution is **memoized** – so the first time you invoke a
query, it will go do the computation, but the next time, the result is
returned from a hashtable. Moreover, query execution fits nicely into
**incremental computation**; the idea is roughly that, when you do a
query, the result **may** be returned to you by loading stored data
from disk (but that's a separate topic we won't discuss further here).
The overall vision is that, eventually, the entire compiler
control-flow will be query driven. There will effectively be one
top-level query ("compile") that will run compilation on a crate; this
will in turn demand information about that crate, starting from the
*end*. For example:
- This "compile" query might demand to get a list of codegen-units
(i.e. modules that need to be compiled by LLVM).
- But computing the list of codegen-units would invoke some subquery
that returns the list of all modules defined in the Rust source.
- That query in turn would invoke something asking for the HIR.
- This keeps going further and further back until we wind up doing the
actual parsing.
However, that vision is not fully realized. Still, big chunks of the
compiler (for example, generating MIR) work exactly like this.
### Invoking queries
To invoke a query is simple. The tcx ("type context") offers a method
for each defined query. So, for example, to invoke the `type_of`
query, you would just do this:
```rust,ignore
let ty = tcx.type_of(some_def_id);
```
### Cycles between queries
A cycle is when a query becomes stuck in a loop e.g. query A generates query B
which generates query A again.
Currently, cycles during query execution should always result in a
compilation error. Typically, they arise because of illegal programs
that contain cyclic references they shouldn't (though sometimes they
arise because of compiler bugs, in which case we need to factor our
queries in a more fine-grained fashion to avoid them).
However, it is nonetheless often useful to *recover* from a cycle
(after reporting an error, say) and try to soldier on, so as to give a
better user experience. In order to recover from a cycle, you don't
get to use the nice method-call-style syntax. Instead, you invoke
using the `try_get` method, which looks roughly like this:
```rust,ignore
use ty::queries;
...
match queries::type_of::try_get(tcx, DUMMY_SP, self.did) {
Ok(result) => {
// no cycle occurred! You can use `result`
}
Err(err) => {
// A cycle occurred! The error value `err` is a `DiagnosticBuilder`,
// meaning essentially an "in-progress", not-yet-reported error message.
// See below for more details on what to do here.
}
}
```
So, if you get back an `Err` from `try_get`, then a cycle *did* occur. This
means that you must ensure that a compiler error message is reported. You can
do that in two ways:
The simplest is to invoke `err.emit()`. This will emit the cycle error to the
user.
However, often cycles happen because of an illegal program, and you
know at that point that an error either already has been reported or
will be reported due to this cycle by some other bit of code. In that
case, you can invoke `err.cancel()` to not emit any error. It is
traditional to then invoke:
```rust,ignore
tcx.sess.delay_span_bug(some_span, "some message")
```
`delay_span_bug()` is a helper that says: we expect a compilation
error to have happened or to happen in the future; so, if compilation
ultimately succeeds, make an ICE with the message `"some
message"`. This is basically just a precaution in case you are wrong.
### How the compiler executes a query
So you may be wondering what happens when you invoke a query
method. The answer is that, for each query, the compiler maintains a
cache – if your query has already been executed, then, the answer is
simple: we clone the return value out of the cache and return it
(therefore, you should try to ensure that the return types of queries
are cheaply cloneable; insert a `Rc` if necessary).
#### Providers
If, however, the query is *not* in the cache, then the compiler will
try to find a suitable **provider**. A provider is a function that has
been defined and linked into the compiler somewhere that contains the
code to compute the result of the query.
**Providers are defined per-crate.** The compiler maintains,
internally, a table of providers for every crate, at least
conceptually. Right now, there are really two sets: the providers for
queries about the **local crate** (that is, the one being compiled)
and providers for queries about **external crates** (that is,
dependencies of the local crate). Note that what determines the crate
that a query is targeting is not the *kind* of query, but the *key*.
For example, when you invoke `tcx.type_of(def_id)`, that could be a
local query or an external query, depending on what crate the `def_id`
is referring to (see the `self::keys::Key` trait for more information
on how that works).
Providers always have the same signature:
```rust,ignore
fn provider<'cx, 'tcx>(tcx: TyCtxt<'cx, 'tcx, 'tcx>,
key: QUERY_KEY)
-> QUERY_RESULT
{
...
}
```
Providers take two arguments: the `tcx` and the query key. Note also
that they take the *global* tcx (i.e. they use the `'tcx` lifetime
twice), rather than taking a tcx with some active inference context.
They return the result of the query.
#### How providers are setup
When the tcx is created, it is given the providers by its creator using
the `Providers` struct. This struct is generated by the macros here, but it
is basically a big list of function pointers:
```rust,ignore
struct Providers {
type_of: for<'cx, 'tcx> fn(TyCtxt<'cx, 'tcx, 'tcx>, DefId) -> Ty<'tcx>,
...
}
```
At present, we have one copy of the struct for local crates, and one
for external crates, though the plan is that we may eventually have
one per crate.
These `Provider` structs are ultimately created and populated by
`librustc_driver`, but it does this by distributing the work
throughout the other `rustc_*` crates. This is done by invoking
various `provide` functions. These functions tend to look something
like this:
```rust,ignore
pub fn provide(providers: &mut Providers) {
*providers = Providers {
type_of,
..*providers
};
}
```
That is, they take an `&mut Providers` and mutate it in place. Usually
we use the formulation above just because it looks nice, but you could
as well do `providers.type_of = type_of`, which would be equivalent.
(Here, `type_of` would be a top-level function, defined as we saw
before.) So, if we want to add a provider for some other query,
let's call it `fubar`, into the crate above, we might modify the `provide()`
function like so:
```rust,ignore
pub fn provide(providers: &mut Providers) {
*providers = Providers {
type_of,
fubar,
..*providers
};
}
fn fubar<'cx, 'tcx>(tcx: TyCtxt<'cx, 'tcx>, key: DefId) -> Fubar<'tcx> { ... }
```
N.B. Most of the `rustc_*` crates only provide **local
providers**. Almost all **extern providers** wind up going through the
[`rustc_metadata` crate][rustc_metadata], which loads the information from the
crate metadata. But in some cases there are crates that provide queries for
*both* local and external crates, in which case they define both a
`provide` and a `provide_extern` function that `rustc_driver` can
invoke.
[rustc_metadata]: https://github.com/rust-lang/rust/tree/master/src/librustc_metadata
### Adding a new kind of query
So suppose you want to add a new kind of query, how do you do so?
Well, defining a query takes place in two steps:
1. first, you have to specify the query name and arguments; and then,
2. you have to supply query providers where needed.
To specify the query name and arguments, you simply add an entry to
the big macro invocation in
[`src/librustc/ty/query/mod.rs`][query-mod], which looks something like:
[query-mod]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/ty/query/index.html
```rust,ignore
define_queries! { <'tcx>
/// Records the type of every item.
[] fn type_of: TypeOfItem(DefId) -> Ty<'tcx>,
...
}
```
Each line of the macro defines one query. The name is broken up like this:
```rust,ignore
[] fn type_of: TypeOfItem(DefId) -> Ty<'tcx>,
^^ ^^^^^^^ ^^^^^^^^^^ ^^^^^ ^^^^^^^^
| | | | |
| | | | result type of query
| | | query key type
| | dep-node constructor
| name of query
query flags
```
Let's go over them one by one:
- **Query flags:** these are largely unused right now, but the intention
is that we'll be able to customize various aspects of how the query is
processed.
- **Name of query:** the name of the query method
(`tcx.type_of(..)`). Also used as the name of a struct
(`ty::queries::type_of`) that will be generated to represent
this query.
- **Dep-node constructor:** indicates the constructor function that
connects this query to incremental compilation. Typically, this is a
`DepNode` variant, which can be added by modifying the
`define_dep_nodes!` macro invocation in
[`librustc/dep_graph/dep_node.rs`][dep-node].
- However, sometimes we use a custom function, in which case the
name will be in snake case and the function will be defined at the
bottom of the file. This is typically used when the query key is
not a def-id, or just not the type that the dep-node expects.
- **Query key type:** the type of the argument to this query.
This type must implement the `ty::query::keys::Key` trait, which
defines (for example) how to map it to a crate, and so forth.
- **Result type of query:** the type produced by this query. This type
should (a) not use `RefCell` or other interior mutability and (b) be
cheaply cloneable. Interning or using `Rc` or `Arc` is recommended for
non-trivial data types.
- The one exception to those rules is the `ty::steal::Steal` type,
which is used to cheaply modify MIR in place. See the definition
of `Steal` for more details. New uses of `Steal` should **not** be
added without alerting `@rust-lang/compiler`.
[dep-node]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/dep_graph/struct.DepNode.html
So, to add a query:
- Add an entry to `define_queries!` using the format above.
- Possibly add a corresponding entry to the dep-node macro.
- Link the provider by modifying the appropriate `provide` method;
or add a new one if needed and ensure that `rustc_driver` is invoking it.
#### Query structs and descriptions
For each kind, the `define_queries` macro will generate a "query struct"
named after the query. This struct is a kind of a place-holder
describing the query. Each such struct implements the
`self::config::QueryConfig` trait, which has associated types for the
key/value of that particular query. Basically the code generated looks something
like this:
```rust,ignore
// Dummy struct representing a particular kind of query:
pub struct type_of<'tcx> { phantom: PhantomData<&'tcx ()> }
impl<'tcx> QueryConfig for type_of<'tcx> {
type Key = DefId;
type Value = Ty<'tcx>;
}
```
There is an additional trait that you may wish to implement called
`self::config::QueryDescription`. This trait is used during cycle
errors to give a "human readable" name for the query, so that we can
summarize what was happening when the cycle occurred. Implementing
this trait is optional if the query key is `DefId`, but if you *don't*
implement it, you get a pretty generic error ("processing `foo`...").
You can put new impls into the `config` module. They look something like this:
```rust,ignore
impl<'tcx> QueryDescription for queries::type_of<'tcx> {
fn describe(tcx: TyCtxt, key: DefId) -> String {
format!("computing the type of `{}`", tcx.item_path_str(key))
}
}
```

76
src/doc/rustc-guide/src/rustc-driver.md Normal file
View File

@ -0,0 +1,76 @@
# The Rustc Driver
The [`rustc_driver`] is essentially `rustc`'s `main()` function. It acts as
the glue for running the various phases of the compiler in the correct order,
managing state such as the [`SourceMap`] \(maps AST nodes to source code),
[`Session`] \(general build context and error messaging) and the [`TyCtxt`]
\(the "typing context", allowing you to query the type system and other cool
stuff). The `rustc_driver` crate also provides external users with a method
for running code at particular times during the compilation process, allowing
third parties to effectively use `rustc`'s internals as a library for
analysing a crate or emulating the compiler in-process (e.g. the RLS).
For those using `rustc` as a library, the `run_compiler()` function is the main
entrypoint to the compiler. Its main parameters are a list of command-line
arguments and a reference to something which implements the `CompilerCalls`
trait. A `CompilerCalls` creates the overall `CompileController`, letting it
govern which compiler passes are run and attach callbacks to be fired at the end
of each phase.
From `rustc_driver`'s perspective, the main phases of the compiler are:
1. *Parse Input:* Initial crate parsing
2. *Configure and Expand:* Resolve `#[cfg]` attributes, name resolution, and
expand macros
3. *Run Analysis Passes:* Run trait resolution, typechecking, region checking
and other miscellaneous analysis passes on the crate
4. *Translate to LLVM:* Translate to the in-memory form of LLVM IR and turn it
into an executable/object files
The `CompileController` then gives users the ability to inspect the ongoing
compilation process
- after parsing
- after AST expansion
- after HIR lowering
- after analysis, and
- when compilation is done
The `CompileState`'s various `state_after_*()` constructors can be inspected to
determine what bits of information are available to which callback.
For a more detailed explanation on using `rustc_driver`, check out the
[stupid-stats] guide by `@nrc` (attached as [Appendix A]).
> **Warning:** By its very nature, the internal compiler APIs are always going
> to be unstable. That said, we do try not to break things unnecessarily.
## A Note On Lifetimes
The Rust compiler is a fairly large program containing lots of big data
structures (e.g. the AST, HIR, and the type system) and as such, arenas and
references are heavily relied upon to minimize unnecessary memory use. This
manifests itself in the way people can plug into the compiler, preferring a
"push"-style API (callbacks) instead of the more Rust-ic "pull" style (think
the `Iterator` trait).
For example the [`CompileState`], the state passed to callbacks after each
phase, is essentially just a box of optional references to pieces inside the
compiler. The lifetime bound on the `CompilerCalls` trait then helps to ensure
compiler internals don't "escape" the compiler (e.g. if you tried to keep a
reference to the AST after the compiler is finished), while still letting users
record *some* state for use after the `run_compiler()` function finishes.
Thread-local storage and interning are used a lot through the compiler to reduce
duplication while also preventing a lot of the ergonomic issues due to many
pervasive lifetimes. The `rustc::ty::tls` module is used to access these
thread-locals, although you should rarely need to touch it.
[`rustc_driver`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_driver/
[`CompileState`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_driver/driver/struct.CompileState.html
[`Session`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/session/struct.Session.html
[`TyCtxt`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/ty/struct.TyCtxt.html
[`SourceMap`]: https://doc.rust-lang.org/nightly/nightly-rustc/syntax/source_map/struct.SourceMap.html
[stupid-stats]: https://github.com/nrc/stupid-stats
[Appendix A]: appendix/stupid-stats.html

242
src/doc/rustc-guide/src/rustdoc.md Normal file
View File

@ -0,0 +1,242 @@
# The walking tour of rustdoc
Rustdoc actually uses the rustc internals directly. It lives in-tree with the
compiler and standard library. This chapter is about how it works.
Rustdoc is implemented entirely within the crate [`librustdoc`][rd]. It runs
the compiler up to the point where we have an internal representation of a
crate (HIR) and the ability to run some queries about the types of items. [HIR]
and [queries] are discussed in the linked chapters.
[HIR]: ./hir.html
[queries]: ./query.html
[rd]: https://github.com/rust-lang/rust/tree/master/src/librustdoc
`librustdoc` performs two major steps after that to render a set of
documentation:
* "Clean" the AST into a form that's more suited to creating documentation (and
slightly more resistant to churn in the compiler).
* Use this cleaned AST to render a crate's documentation, one page at a time.
Naturally, there's more than just this, and those descriptions simplify out
lots of details, but that's the high-level overview.
(Side note: `librustdoc` is a library crate! The `rustdoc` binary is created
using the project in [`src/tools/rustdoc`][bin]. Note that literally all that
does is call the `main()` that's in this crate's `lib.rs`, though.)
[bin]: https://github.com/rust-lang/rust/tree/master/src/tools/rustdoc
## Cheat sheet
* Use `./x.py build --stage 1 src/libstd src/tools/rustdoc` to make a usable
rustdoc you can run on other projects.
* Add `src/libtest` to be able to use `rustdoc --test`.
* If you've used `rustup toolchain link local /path/to/build/$TARGET/stage1`
previously, then after the previous build command, `cargo +local doc` will
Just Work.
* Use `./x.py doc --stage 1 src/libstd` to use this rustdoc to generate the
standard library docs.
* The completed docs will be available in `build/$TARGET/doc/std`, though the
bundle is meant to be used as though you would copy out the `doc` folder to
a web server, since that's where the CSS/JS and landing page are.
* Most of the HTML printing code is in `html/format.rs` and `html/render.rs`.
It's in a bunch of `fmt::Display` implementations and supplementary
functions.
* The types that got `Display` impls above are defined in `clean/mod.rs`, right
next to the custom `Clean` trait used to process them out of the rustc HIR.
* The bits specific to using rustdoc as a test harness are in `test.rs`.
* The Markdown renderer is loaded up in `html/markdown.rs`, including functions
for extracting doctests from a given block of Markdown.
* The tests on rustdoc *output* are located in `src/test/rustdoc`, where
they're handled by the test runner of rustbuild and the supplementary script
`src/etc/htmldocck.py`.
* Tests on search index generation are located in `src/test/rustdoc-js`, as a
series of JavaScript files that encode queries on the standard library search
index and expected results.
## From crate to clean
In `core.rs` are two central items: the `DocContext` struct, and the `run_core`
function. The latter is where rustdoc calls out to rustc to compile a crate to
the point where rustdoc can take over. The former is a state container used
when crawling through a crate to gather its documentation.
The main process of crate crawling is done in `clean/mod.rs` through several
implementations of the `Clean` trait defined within. This is a conversion
trait, which defines one method:
```rust,ignore
pub trait Clean<T> {
fn clean(&self, cx: &DocContext) -> T;
}
```
`clean/mod.rs` also defines the types for the "cleaned" AST used later on to
render documentation pages. Each usually accompanies an implementation of
`Clean` that takes some AST or HIR type from rustc and converts it into the
appropriate "cleaned" type. "Big" items like modules or associated items may
have some extra processing in its `Clean` implementation, but for the most part
these impls are straightforward conversions. The "entry point" to this module
is the `impl Clean<Crate> for visit_ast::RustdocVisitor`, which is called by
`run_core` above.
You see, I actually lied a little earlier: There's another AST transformation
that happens before the events in `clean/mod.rs`. In `visit_ast.rs` is the
type `RustdocVisitor`, which *actually* crawls a `hir::Crate` to get the first
intermediate representation, defined in `doctree.rs`. This pass is mainly to
get a few intermediate wrappers around the HIR types and to process visibility
and inlining. This is where `#[doc(inline)]`, `#[doc(no_inline)]`, and
`#[doc(hidden)]` are processed, as well as the logic for whether a `pub use`
should get the full page or a "Reexport" line in the module page.
The other major thing that happens in `clean/mod.rs` is the collection of doc
comments and `#[doc=""]` attributes into a separate field of the Attributes
struct, present on anything that gets hand-written documentation. This makes it
easier to collect this documentation later in the process.
The primary output of this process is a `clean::Crate` with a tree of Items
which describe the publicly-documentable items in the target crate.
### Hot potato
Before moving on to the next major step, a few important "passes" occur over
the documentation. These do things like combine the separate "attributes" into
a single string and strip leading whitespace to make the document easier on the
markdown parser, or drop items that are not public or deliberately hidden with
`#[doc(hidden)]`. These are all implemented in the `passes/` directory, one
file per pass. By default, all of these passes are run on a crate, but the ones
regarding dropping private/hidden items can be bypassed by passing
`--document-private-items` to rustdoc. Note that unlike the previous set of AST
transformations, the passes happen on the _cleaned_ crate.
(Strictly speaking, you can fine-tune the passes run and even add your own, but
[we're trying to deprecate that][44136]. If you need finer-grain control over
these passes, please let us know!)
[44136]: https://github.com/rust-lang/rust/issues/44136
Here is current (as of this writing) list of passes:
- `propagate-doc-cfg` - propagates `#[doc(cfg(...))]` to child items.
- `collapse-docs` concatenates all document attributes into one document
attribute. This is necessary because each line of a doc comment is given as a
separate doc attribute, and this will combine them into a single string with
line breaks between each attribute.
- `unindent-comments` removes excess indentation on comments in order for
markdown to like it. This is necessary because the convention for writing
documentation is to provide a space between the `///` or `//!` marker and the
text, and stripping that leading space will make the text easier to parse by
the Markdown parser. (In the past, the markdown parser used was not
Commonmark- compliant, which caused annoyances with extra whitespace but this
seems to be less of an issue today.)
- `strip-priv-imports` strips all private import statements (`use`, `extern
crate`) from a crate. This is necessary because rustdoc will handle *public*
imports by either inlining the item's documentation to the module or creating
a "Reexports" section with the import in it. The pass ensures that all of
these imports are actually relevant to documentation.
- `strip-hidden` and `strip-private` strip all `doc(hidden)` and private items
from the output. `strip-private` implies `strip-priv-imports`. Basically, the
goal is to remove items that are not relevant for public documentation.
## From clean to crate
This is where the "second phase" in rustdoc begins. This phase primarily lives
in the `html/` folder, and it all starts with `run()` in `html/render.rs`. This
code is responsible for setting up the `Context`, `SharedContext`, and `Cache`
which are used during rendering, copying out the static files which live in
every rendered set of documentation (things like the fonts, CSS, and JavaScript
that live in `html/static/`), creating the search index, and printing out the
source code rendering, before beginning the process of rendering all the
documentation for the crate.
Several functions implemented directly on `Context` take the `clean::Crate` and
set up some state between rendering items or recursing on a module's child
items. From here the "page rendering" begins, via an enormous `write!()` call
in `html/layout.rs`. The parts that actually generate HTML from the items and
documentation occurs within a series of `std::fmt::Display` implementations and
functions that pass around a `&mut std::fmt::Formatter`. The top-level
implementation that writes out the page body is the `impl<'a> fmt::Display for
Item<'a>` in `html/render.rs`, which switches out to one of several `item_*`
functions based on the kind of `Item` being rendered.
Depending on what kind of rendering code you're looking for, you'll probably
find it either in `html/render.rs` for major items like "what sections should I
print for a struct page" or `html/format.rs` for smaller component pieces like
"how should I print a where clause as part of some other item".
Whenever rustdoc comes across an item that should print hand-written
documentation alongside, it calls out to `html/markdown.rs` which interfaces
with the Markdown parser. This is exposed as a series of types that wrap a
string of Markdown, and implement `fmt::Display` to emit HTML text. It takes
special care to enable certain features like footnotes and tables and add
syntax highlighting to Rust code blocks (via `html/highlight.rs`) before
running the Markdown parser. There's also a function in here
(`find_testable_code`) that specifically scans for Rust code blocks so the
test-runner code can find all the doctests in the crate.
### From soup to nuts
(alternate title: ["An unbroken thread that stretches from those first `Cell`s
to us"][video])
[video]: https://www.youtube.com/watch?v=hOLAGYmUQV0
It's important to note that the AST cleaning can ask the compiler for
information (crucially, `DocContext` contains a `TyCtxt`), but page rendering
cannot. The `clean::Crate` created within `run_core` is passed outside the
compiler context before being handed to `html::render::run`. This means that a
lot of the "supplementary data" that isn't immediately available inside an
item's definition, like which trait is the `Deref` trait used by the language,
needs to be collected during cleaning, stored in the `DocContext`, and passed
along to the `SharedContext` during HTML rendering. This manifests as a bunch
of shared state, context variables, and `RefCell`s.
Also of note is that some items that come from "asking the compiler" don't go
directly into the `DocContext` - for example, when loading items from a foreign
crate, rustdoc will ask about trait implementations and generate new `Item`s
for the impls based on that information. This goes directly into the returned
`Crate` rather than roundabout through the `DocContext`. This way, these
implementations can be collected alongside the others, right before rendering
the HTML.
## Other tricks up its sleeve
All this describes the process for generating HTML documentation from a Rust
crate, but there are couple other major modes that rustdoc runs in. It can also
be run on a standalone Markdown file, or it can run doctests on Rust code or
standalone Markdown files. For the former, it shortcuts straight to
`html/markdown.rs`, optionally including a mode which inserts a Table of
Contents to the output HTML.
For the latter, rustdoc runs a similar partial-compilation to get relevant
documentation in `test.rs`, but instead of going through the full clean and
render process, it runs a much simpler crate walk to grab *just* the
hand-written documentation. Combined with the aforementioned
"`find_testable_code`" in `html/markdown.rs`, it builds up a collection of
tests to run before handing them off to the libtest test runner. One notable
location in `test.rs` is the function `make_test`, which is where hand-written
doctests get transformed into something that can be executed.
Some extra reading about `make_test` can be found
[here](https://quietmisdreavus.net/code/2018/02/23/how-the-doctests-get-made/).
## Dotting i's and crossing t's
So that's rustdoc's code in a nutshell, but there's more things in the repo
that deal with it. Since we have the full `compiletest` suite at hand, there's
a set of tests in `src/test/rustdoc` that make sure the final HTML is what we
expect in various situations. These tests also use a supplementary script,
`src/etc/htmldocck.py`, that allows it to look through the final HTML using
XPath notation to get a precise look at the output. The full description of all
the commands available to rustdoc tests is in `htmldocck.py`.
In addition, there are separate tests for the search index and rustdoc's
ability to query it. The files in `src/test/rustdoc-js` each contain a
different search query and the expected results, broken out by search tab.
These files are processed by a script in `src/tools/rustdoc-js` and the Node.js
runtime. These tests don't have as thorough of a writeup, but a broad example
that features results in all tabs can be found in `basic.js`. The basic idea is
that you match a given `QUERY` with a set of `EXPECTED` results, complete with
the full item path of each item.

154
src/doc/rustc-guide/src/test-implementation.md Normal file
View File

@ -0,0 +1,154 @@
### The `#[test]` attribute
Today, rust programmers rely on a built in attribute called `#[test]`. All
you have to do is mark a function as a test and include some asserts like so:
```rust,ignore
#[test]
fn my_test() {
assert!(2+2 == 4);
}
```
When this program is compiled using `rustc --test` or `cargo test`, it will
produce an executable that can run this, and any other test function. This
method of testing allows tests to live alongside code in an organic way. You
can even put tests inside private modules:
```rust,ignore
mod my_priv_mod {
fn my_priv_func() -> bool {}
#[test]
fn test_priv_func() {
assert!(my_priv_func());
}
}
```
Private items can thus be easily tested without worrying about how to expose
the them to any sort of external testing apparatus. This is key to the
ergonomics of testing in Rust. Semantically, however, it's rather odd.
How does any sort of `main` function invoke these tests if they're not visible?
What exactly is `rustc --test` doing?
`#[test]` is implemented as a syntactic transformation inside the compiler's
[`libsyntax` crate][libsyntax]. Essentially, it's a fancy macro, that
rewrites the crate in 3 steps:
#### Step 1: Re-Exporting
As mentioned earlier, tests can exist inside private modules, so we need a
way of exposing them to the main function, without breaking any existing
code. To that end, `libsyntax` will create local modules called
`__test_reexports` that recursively reexport tests. This expansion translates
the above example into:
```rust,ignore
mod my_priv_mod {
fn my_priv_func() -> bool {}
pub fn test_priv_func() {
assert!(my_priv_func());
}
pub mod __test_reexports {
pub use super::test_priv_func;
}
}
```
Now, our test can be accessed as
`my_priv_mod::__test_reexports::test_priv_func`. For deeper module
structures, `__test_reexports` will reexport modules that contain tests, so a
test at `a::b::my_test` becomes
`a::__test_reexports::b::__test_reexports::my_test`. While this process seems
pretty safe, what happens if there is an existing `__test_reexports` module?
The answer: nothing.
To explain, we need to understand [how the AST represents
identifiers][Ident]. The name of every function, variable, module, etc. is
not stored as a string, but rather as an opaque [Symbol][Symbol] which is
essentially an ID number for each identifier. The compiler keeps a separate
hashtable that allows us to recover the human-readable name of a Symbol when
necessary (such as when printing a syntax error). When the compiler generates
the `__test_reexports` module, it generates a new Symbol for the identifier,
so while the compiler-generated `__test_reexports` may share a name with your
hand-written one, it will not share a Symbol. This technique prevents name
collision during code generation and is the foundation of Rust's macro
hygiene.
#### Step 2: Harness Generation
Now that our tests are accessible from the root of our crate, we need to do
something with them. `libsyntax` generates a module like so:
```rust,ignore
pub mod __test {
extern crate test;
const TESTS: &'static [self::test::TestDescAndFn] = &[/*...*/];
#[main]
pub fn main() {
self::test::test_static_main(TESTS);
}
}
```
While this transformation is simple, it gives us a lot of insight into how
tests are actually run. The tests are aggregated into an array and passed to
a test runner called `test_static_main`. We'll come back to exactly what
`TestDescAndFn` is, but for now, the key takeaway is that there is a crate
called [`test`][test] that is part of Rust core, that implements all of the
runtime for testing. `test`'s interface is unstable, so the only stable way
to interact with it is through the `#[test]` macro.
#### Step 3: Test Object Generation
If you've written tests in Rust before, you may be familiar with some of the
optional attributes available on test functions. For example, a test can be
annotated with `#[should_panic]` if we expect the test to cause a panic. It
looks something like this:
```rust,ignore
#[test]
#[should_panic]
fn foo() {
panic!("intentional");
}
```
This means our tests are more than just simple functions, they have
configuration information as well. `test` encodes this configuration data
into a struct called [`TestDesc`][TestDesc]. For each test function in a
crate, `libsyntax` will parse its attributes and generate a `TestDesc`
instance. It then combines the `TestDesc` and test function into the
predictably named `TestDescAndFn` struct, that `test_static_main` operates
on. For a given test, the generated `TestDescAndFn` instance looks like so:
```rust,ignore
self::test::TestDescAndFn{
desc: self::test::TestDesc{
name: self::test::StaticTestName("foo"),
ignore: false,
should_panic: self::test::ShouldPanic::Yes,
allow_fail: false,
},
testfn: self::test::StaticTestFn(||
self::test::assert_test_result(::crate::__test_reexports::foo())),
}
```
Once we've constructed an array of these test objects, they're passed to the
test runner via the harness generated in step 2.
### Inspecting the generated code
On nightly rust, there's an unstable flag called `unpretty` that you can use
to print out the module source after macro expansion:
```bash
$ rustc my_mod.rs -Z unpretty=hir
```
[test]: https://doc.rust-lang.org/test/index.html
[TestDesc]: https://doc.rust-lang.org/test/struct.TestDesc.html
[Symbol]: https://doc.rust-lang.org/nightly/nightly-rustc/syntax/ast/struct.Ident.html
[Ident]: https://doc.rust-lang.org/nightly/nightly-rustc/syntax/ast/struct.Ident.html
[eRFC]: https://github.com/rust-lang/rfcs/blob/master/text/2318-custom-test-frameworks.md
[libsyntax]: https://github.com/rust-lang/rust/tree/master/src/libsyntax

346
src/doc/rustc-guide/src/tests/adding.md Normal file
View File

@ -0,0 +1,346 @@
# Adding new tests
**In general, we expect every PR that fixes a bug in rustc to come
accompanied by a regression test of some kind.** This test should fail
in master but pass after the PR. These tests are really useful for
preventing us from repeating the mistakes of the past.
To add a new test, the first thing you generally do is to create a
file, typically a Rust source file. Test files have a particular
structure:
- They should have some kind of
[comment explaining what the test is about](#explanatory_comment);
- next, they can have one or more [header commands](#header_commands), which
are special comments that the test interpreter knows how to interpret.
- finally, they have the Rust source. This may have various [error
annotations](#error_annotations) which indicate expected compilation errors or
warnings.
Depending on the test suite, there may be some other details to be aware of:
- For [the `ui` test suite](#ui), you need to generate reference output files.
## What kind of test should I add?
It can be difficult to know what kind of test to use. Here are some
rough heuristics:
- Some tests have specialized needs:
- need to run gdb or lldb? use the `debuginfo` test suite
- need to inspect LLVM IR or MIR IR? use the `codegen` or `mir-opt` test
suites
- need to run rustdoc? Prefer a `rustdoc` test
- need to inspect the resulting binary in some way? Then use `run-make`
- For most other things, [a `ui` (or `ui-fulldeps`) test](#ui) is to be
preferred:
- `ui` tests subsume both run-pass, compile-fail, and parse-fail tests
- in the case of warnings or errors, `ui` tests capture the full output,
which makes it easier to review but also helps prevent "hidden" regressions
in the output
## Naming your test
We have not traditionally had a lot of structure in the names of
tests. Moreover, for a long time, the rustc test runner did not
support subdirectories (it now does), so test suites like
[`src/test/run-pass`] have a huge mess of files in them. This is not
considered an ideal setup.
[`src/test/run-pass`]: https://github.com/rust-lang/rust/tree/master/src/test/run-pass/
For regression tests – basically, some random snippet of code that
came in from the internet – we often just name the test after the
issue. For example, `src/test/run-pass/issue-12345.rs`. If possible,
though, it is better if you can put the test into a directory that
helps identify what piece of code is being tested here (e.g.,
`borrowck/issue-12345.rs` is much better), or perhaps give it a more
meaningful name. Still, **do include the issue number somewhere**.
When writing a new feature, **create a subdirectory to store your
tests**. For example, if you are implementing RFC 1234 ("Widgets"),
then it might make sense to put the tests in directories like:
- `src/test/ui/rfc1234-widgets/`
- `src/test/run-pass/rfc1234-widgets/`
- etc
In other cases, there may already be a suitable directory. (The proper
directory structure to use is actually an area of active debate.)
<a name="explanatory_comment"></a>
## Comment explaining what the test is about
When you create a test file, **include a comment summarizing the point
of the test at the start of the file**. This should highlight which
parts of the test are more important, and what the bug was that the
test is fixing. Citing an issue number is often very helpful.
This comment doesn't have to be super extensive. Just something like
"Regression test for #18060: match arms were matching in the wrong
order." might already be enough.
These comments are very useful to others later on when your test
breaks, since they often can highlight what the problem is. They are
also useful if for some reason the tests need to be refactored, since
they let others know which parts of the test were important (often a
test must be rewritten because it no longer tests what is was meant to
test, and then it's useful to know what it *was* meant to test
exactly).
<a name="header_commands"></a>
## Header commands: configuring rustc
Header commands are special comments that the test runner knows how to
interpret. They must appear before the Rust source in the test. They
are normally put after the short comment that explains the point of
this test. For example, this test uses the `// compile-flags` command
to specify a custom flag to give to rustc when the test is compiled:
```rust,ignore
// Test the behavior of `0 - 1` when overflow checks are disabled.
// compile-flags: -Coverflow-checks=off
fn main() {
let x = 0 - 1;
...
}
```
### Ignoring tests
These are used to ignore the test in some situations, which means the test won't
be compiled or run.
* `ignore-X` where `X` is a target detail or stage will ignore the
test accordingly (see below)
* `only-X` is like `ignore-X`, but will *only* run the test on that
target or stage
* `ignore-pretty` will not compile the pretty-printed test (this is
done to test the pretty-printer, but might not always work)
* `ignore-test` always ignores the test
* `ignore-lldb` and `ignore-gdb` will skip a debuginfo test on that
debugger.
* `ignore-gdb-version` can be used to ignore the test when certain gdb
versions are used
Some examples of `X` in `ignore-X`:
* Architecture: `aarch64`, `arm`, `asmjs`, `mips`, `wasm32`, `x86_64`,
`x86`, ...
* OS: `android`, `emscripten`, `freebsd`, `ios`, `linux`, `macos`,
`windows`, ...
* Environment (fourth word of the target triple): `gnu`, `msvc`,
`musl`.
* Pointer width: `32bit`, `64bit`.
* Stage: `stage0`, `stage1`, `stage2`.
### Other Header Commands
Here is a list of other header commands. This list is not
exhaustive. Header commands can generally be found by browsing the
`TestProps` structure found in [`header.rs`] from the compiletest
source.
* `run-rustfix` for UI tests, indicates that the test produces
structured suggestions. The test writer should create a `.fixed`
file, which contains the source with the suggestions applied.
When the test is run, compiletest first checks that the correct
lint/warning is generated. Then, it applies the suggestion and
compares against `.fixed` (they must match). Finally, the fixed
source is compiled, and this compilation is required to succeed.
The `.fixed` file can also be generated automatically with the
`--bless` option, discussed [below](#bless).
* `min-gdb-version` specifies the minimum gdb version required for
this test; see also `ignore-gdb-version`
* `min-lldb-version` specifies the minimum lldb version required for
this test
* `rust-lldb` causes the lldb part of the test to only be run if the
lldb in use contains the Rust plugin
* `no-system-llvm` causes the test to be ignored if the system llvm is used
* `min-llvm-version` specifies the minimum llvm version required for
this test
* `min-system-llvm-version` specifies the minimum system llvm version
required for this test; the test is ignored if the system llvm is in
use and it doesn't meet the minimum version. This is useful when an
llvm feature has been backported to rust-llvm
* `ignore-llvm-version` can be used to skip the test when certain LLVM
versions are used. This takes one or two arguments; the first
argument is the first version to ignore. If no second argument is
given, all subsequent versions are ignored; otherwise, the second
argument is the last version to ignore.
* `compile-pass` for UI tests, indicates that the test is
supposed to compile, as opposed to the default where the test is
supposed to error out.
* `compile-flags` passes extra command-line args to the compiler,
e.g. `compile-flags -g` which forces debuginfo to be enabled.
* `should-fail` indicates that the test should fail; used for "meta
testing", where we test the compiletest program itself to check that
it will generate errors in appropriate scenarios. This header is
ignored for pretty-printer tests.
* `gate-test-X` where `X` is a feature marks the test as "gate test"
for feature X. Such tests are supposed to ensure that the compiler
errors when usage of a gated feature is attempted without the proper
`#![feature(X)]` tag. Each unstable lang feature is required to
have a gate test.
[`header.rs`]: https://github.com/rust-lang/rust/tree/master/src/tools/compiletest/src/header.rs
<a name="error_annotations"></a>
## Error annotations
Error annotations specify the errors that the compiler is expected to
emit. They are "attached" to the line in source where the error is
located.
* `~`: Associates the following error level and message with the
current line
* `~|`: Associates the following error level and message with the same
line as the previous comment
* `~^`: Associates the following error level and message with the
previous line. Each caret (`^`) that you add adds a line to this, so
`~^^^^^^^` is seven lines up.
The error levels that you can have are:
1. `ERROR`
2. `WARNING`
3. `NOTE`
4. `HELP` and `SUGGESTION`*
\* **Note**: `SUGGESTION` must follow immediately after `HELP`.
## Revisions
Certain classes of tests support "revisions" (as of the time of this
writing, this includes run-pass, compile-fail, run-fail, and
incremental, though incremental tests are somewhat
different). Revisions allow a single test file to be used for multiple
tests. This is done by adding a special header at the top of the file:
```rust
// revisions: foo bar baz
```
This will result in the test being compiled (and tested) three times,
once with `--cfg foo`, once with `--cfg bar`, and once with `--cfg
baz`. You can therefore use `#[cfg(foo)]` etc within the test to tweak
each of these results.
You can also customize headers and expected error messages to a particular
revision. To do this, add `[foo]` (or `bar`, `baz`, etc) after the `//`
comment, like so:
```rust
// A flag to pass in only for cfg `foo`:
//[foo]compile-flags: -Z verbose
#[cfg(foo)]
fn test_foo() {
let x: usize = 32_u32; //[foo]~ ERROR mismatched types
}
```
Note that not all headers have meaning when customized to a revision.
For example, the `ignore-test` header (and all "ignore" headers)
currently only apply to the test as a whole, not to particular
revisions. The only headers that are intended to really work when
customized to a revision are error patterns and compiler flags.
<a name="ui"></a>
## Guide to the UI tests
The UI tests are intended to capture the compiler's complete output,
so that we can test all aspects of the presentation. They work by
compiling a file (e.g., [`ui/hello_world/main.rs`][hw-main]),
capturing the output, and then applying some normalization (see
below). This normalized result is then compared against reference
files named `ui/hello_world/main.stderr` and
`ui/hello_world/main.stdout`. If either of those files doesn't exist,
the output must be empty (that is actually the case for
[this particular test][hw]). If the test run fails, we will print out
the current output, but it is also saved in
`build/<target-triple>/test/ui/hello_world/main.stdout` (this path is
printed as part of the test failure message), so you can run `diff`
and so forth.
[hw-main]: https://github.com/rust-lang/rust/blob/master/src/test/ui/hello_world/main.rs
[hw]: https://github.com/rust-lang/rust/blob/master/src/test/ui/hello_world/
### Tests that do not result in compile errors
By default, a UI test is expected **not to compile** (in which case,
it should contain at least one `//~ ERROR` annotation). However, you
can also make UI tests where compilation is expected to succeed, and
you can even run the resulting program. Just add one of the following
[header commands](#header_commands):
- `// compile-pass` – compilation should succeed but do
not run the resulting binary
- `// run-pass` – compilation should succeed and we should run the
resulting binary
<a name="bless"></a>
### Editing and updating the reference files
If you have changed the compiler's output intentionally, or you are
making a new test, you can pass `--bless` to the test subcommand. E.g.
if some tests in `src/test/ui` are failing, you can run
```text
./x.py test --stage 1 src/test/ui --bless
```
to automatically adjust the `.stderr`, `.stdout` or `.fixed` files of
all tests. Of course you can also target just specific tests with the
`--test-args your_test_name` flag, just like when running the tests.
### Normalization
The normalization applied is aimed at eliminating output difference
between platforms, mainly about filenames:
- the test directory is replaced with `$DIR`
- all backslashes (`\`) are converted to forward slashes (`/`) (for Windows)
- all CR LF newlines are converted to LF
Sometimes these built-in normalizations are not enough. In such cases, you
may provide custom normalization rules using the header commands, e.g.
```rust
// normalize-stdout-test: "foo" -> "bar"
// normalize-stderr-32bit: "fn\(\) \(32 bits\)" -> "fn\(\) \($$PTR bits\)"
// normalize-stderr-64bit: "fn\(\) \(64 bits\)" -> "fn\(\) \($$PTR bits\)"
```
This tells the test, on 32-bit platforms, whenever the compiler writes
`fn() (32 bits)` to stderr, it should be normalized to read `fn() ($PTR bits)`
instead. Similar for 64-bit. The replacement is performed by regexes using
default regex flavor provided by `regex` crate.
The corresponding reference file will use the normalized output to test both
32-bit and 64-bit platforms:
```text
...
|
= note: source type: fn() ($PTR bits)
= note: target type: u16 (16 bits)
...
```
Please see [`ui/transmute/main.rs`][mrs] and [`main.stderr`][] for a
concrete usage example.
[mrs]: https://github.com/rust-lang/rust/blob/master/src/test/ui/transmute/main.rs
[`main.stderr`]: https://github.com/rust-lang/rust/blob/master/src/test/ui/transmute/main.stderr
Besides `normalize-stderr-32bit` and `-64bit`, one may use any target
information or stage supported by `ignore-X` here as well (e.g.
`normalize-stderr-windows` or simply `normalize-stderr-test` for unconditional
replacement).

237
src/doc/rustc-guide/src/tests/intro.md Normal file
View File

@ -0,0 +1,237 @@
# The compiler testing framework
The Rust project runs a wide variety of different tests, orchestrated
by the build system (`x.py test`). The main test harness for testing
the compiler itself is a tool called compiletest (sources in the
[`src/tools/compiletest`]). This section gives a brief overview of how
the testing framework is setup, and then gets into some of the details
on [how to run tests](./running.html#ui) as well as
[how to add new tests](./adding.html).
[`src/tools/compiletest`]: https://github.com/rust-lang/rust/tree/master/src/tools/compiletest
## Compiletest test suites
The compiletest tests are located in the tree in the [`src/test`]
directory. Immediately within you will see a series of subdirectories
(e.g. `ui`, `run-make`, and so forth). Each of those directories is
called a **test suite** – they house a group of tests that are run in
a distinct mode.
[`src/test`]: https://github.com/rust-lang/rust/tree/master/src/test
Here is a brief summary of the test suites as of this writing and what
they mean. In some cases, the test suites are linked to parts of the manual
that give more details.
- [`ui`](./adding.html#ui) – tests that check the exact
stdout/stderr from compilation and/or running the test
- `run-pass` – tests that are expected to compile and execute
successfully (no panics)
- `run-pass-valgrind` – tests that ought to run with valgrind
- `run-fail` – tests that are expected to compile but then panic
during execution
- `compile-fail` – tests that are expected to fail compilation.
- `parse-fail` – tests that are expected to fail to parse
- `pretty` – tests targeting the Rust "pretty printer", which
generates valid Rust code from the AST
- `debuginfo` – tests that run in gdb or lldb and query the debug info
- `codegen` – tests that compile and then test the generated LLVM
code to make sure that the optimizations we want are taking effect.
- `mir-opt` – tests that check parts of the generated MIR to make
sure we are building things correctly or doing the optimizations we
expect.
- `incremental` – tests for incremental compilation, checking that
when certain modifications are performed, we are able to reuse the
results from previous compilations.
- `run-make` – tests that basically just execute a `Makefile`; the
ultimate in flexibility but quite annoying to write.
- `rustdoc` – tests for rustdoc, making sure that the generated files
contain the expected documentation.
- `*-fulldeps` – same as above, but indicates that the test depends
on things other than `libstd` (and hence those things must be built)
## Other Tests
The Rust build system handles running tests for various other things,
including:
- **Tidy** – This is a custom tool used for validating source code
style and formatting conventions, such as rejecting long lines.
There is more information in the
[section on coding conventions](../conventions.html#formatting).
Example: `./x.py test src/tools/tidy`
- **Unit tests** – The Rust standard library and many of the Rust packages
include typical Rust `#[test]` unittests. Under the hood, `x.py` will run
`cargo test` on each package to run all the tests.
Example: `./x.py test src/libstd`
- **Doc tests** – Example code embedded within Rust documentation is executed
via `rustdoc --test`. Examples:
`./x.py test src/doc` – Runs `rustdoc --test` for all documentation in
`src/doc`.
`./x.py test --doc src/libstd` – Runs `rustdoc --test` on the standard
library.
- **Link checker** – A small tool for verifying `href` links within
documentation.
Example: `./x.py test src/tools/linkchecker`
- **Dist check** – This verifies that the source distribution tarball created
by the build system will unpack, build, and run all tests.
Example: `./x.py test distcheck`
- **Tool tests** – Packages that are included with Rust have all of their
tests run as well (typically by running `cargo test` within their
directory). This includes things such as cargo, clippy, rustfmt, rls, miri,
bootstrap (testing the Rust build system itself), etc.
- **Cargo test** – This is a small tool which runs `cargo test` on a few
significant projects (such as `servo`, `ripgrep`, `tokei`, etc.) just to
ensure there aren't any significant regressions.
Example: `./x.py test src/tools/cargotest`
## Testing infrastructure
When a Pull Request is opened on Github, [Travis] will automatically launch a
build that will run all tests on a single configuration (x86-64 linux). In
essence, it runs `./x.py test` after building.
The integration bot [bors] is used for coordinating merges to the master
branch. When a PR is approved, it goes into a [queue] where merges are tested
one at a time on a wide set of platforms using Travis and [Appveyor]
(currently over 50 different configurations). Most platforms only run the
build steps, some run a restricted set of tests, only a subset run the full
suite of tests (see Rust's [platform tiers]).
[Travis]: https://travis-ci.org/rust-lang/rust
[bors]: https://github.com/servo/homu
[queue]: https://buildbot2.rust-lang.org/homu/queue/rust
[Appveyor]: https://ci.appveyor.com/project/rust-lang/rust
[platform tiers]: https://forge.rust-lang.org/platform-support.html
## Testing with Docker images
The Rust tree includes [Docker] image definitions for the platforms used on
Travis in [src/ci/docker]. The script [src/ci/docker/run.sh] is used to build
the Docker image, run it, build Rust within the image, and run the tests.
> TODO: What is a typical workflow for testing/debugging on a platform that
> you don't have easy access to? Do people build Docker images and enter them
> to test things out?
[Docker]: https://www.docker.com/
[src/ci/docker]: https://github.com/rust-lang/rust/tree/master/src/ci/docker
[src/ci/docker/run.sh]: https://github.com/rust-lang/rust/blob/master/src/ci/docker/run.sh
## Testing on emulators
Some platforms are tested via an emulator for architectures that aren't
readily available. There is a set of tools for orchestrating running the
tests within the emulator. Platforms such as `arm-android` and
`arm-unknown-linux-gnueabihf` are set up to automatically run the tests under
emulation on Travis. The following will take a look at how a target's tests
are run under emulation.
The Docker image for [armhf-gnu] includes [QEMU] to emulate the ARM CPU
architecture. Included in the Rust tree are the tools [remote-test-client]
and [remote-test-server] which are programs for sending test programs and
libraries to the emulator, and running the tests within the emulator, and
reading the results. The Docker image is set up to launch
`remote-test-server` and the build tools use `remote-test-client` to
communicate with the server to coordinate running tests (see
[src/bootstrap/test.rs]).
> TODO: What are the steps for manually running tests within an emulator?
> `./src/ci/docker/run.sh armhf-gnu` will do everything, but takes hours to
> run and doesn't offer much help with interacting within the emulator.
>
> Is there any support for emulating other (non-Android) platforms, such as
> running on an iOS emulator?
>
> Is there anything else interesting that can be said here about running tests
> remotely on real hardware?
>
> It's also unclear to me how the wasm or asm.js tests are run.
[armhf-gnu]: https://github.com/rust-lang/rust/tree/master/src/ci/docker/armhf-gnu
[QEMU]: https://www.qemu.org/
[remote-test-client]: https://github.com/rust-lang/rust/tree/master/src/tools/remote-test-client
[remote-test-server]: https://github.com/rust-lang/rust/tree/master/src/tools/remote-test-server
[src/bootstrap/test.rs]: https://github.com/rust-lang/rust/tree/master/src/bootstrap/test.rs
## Crater
[Crater](https://github.com/rust-lang-nursery/crater) is a tool for compiling
and running tests for _every_ crate on [crates.io](https://crates.io) (and a
few on GitHub). It is mainly used for checking for extent of breakage when
implementing potentially breaking changes and ensuring lack of breakage by
running beta vs stable compiler versions.
### When to run Crater
You should request a crater run if your PR makes large changes to the compiler
or could cause breakage. If you are unsure, feel free to ask your PR's reviewer.
### Requesting Crater Runs
The rust team maintains a few machines that can be used for running crater runs
on the changes introduced by a PR. If your PR needs a crater run, leave a
comment for the triage team in the PR thread. Please inform the team whether
you require a "check-only" crater run, a "build only" crater run, or a
"build-and-test" crater run. The difference is primarily in time; the
conservative (if you're not sure) option is to go for the build-and-test run.
If making changes that will only have an effect at compile-time (e.g.,
implementing a new trait) then you only need a check run.
Your PR will be enqueued by the triage team and the results will be posted when
they are ready. Check runs will take around ~3-4 days, with the other two
taking 5-6 days on average.
While crater is really useful, it is also important to be aware of a few
caveats:
- Not all code is on crates.io! There is a lot of code in repos on GitHub and
elsewhere. Also, companies may not wish to publish their code. Thus, a
successful crater run is not a magically green light that there will be no
breakage; you still need to be careful.
- Crater only runs Linux builds on x86_64. Thus, other architectures and
platforms are not tested. Critically, this includes Windows.
- Many crates are not tested. This could be for a lot of reasons, including
that the crate doesn't compile any more (e.g. used old nightly features),
has broken or flaky tests, requires network access, or other reasons.
- Before crater can be run, `@bors try` needs to succeed in building artifacts.
This means that if your code doesn't compile, you cannot run crater.
## Perf runs
A lot of work is put into improving the performance of the compiler and
preventing performance regressions. A "perf run" is used to compare the
performance of the compiler in different configurations for a large collection
of popular crates. Different configurations include "fresh builds", builds
with incremental compilation, etc.
The result of a perf run is a comparison between two versions of the
compiler (by their commit hashes).
You should request a perf run if your PR may affect performance, especially
if it can affect performance adversely.
## Further reading
The following blog posts may also be of interest:
- brson's classic ["How Rust is tested"][howtest]
[howtest]: https://brson.github.io/2017/07/10/how-rust-is-tested

121
src/doc/rustc-guide/src/tests/running.md Normal file
View File

@ -0,0 +1,121 @@
# Running tests
You can run the tests using `x.py`. The most basic command – which
you will almost never want to use! – is as follows:
```bash
> ./x.py test
```
This will build the full stage 2 compiler and then run the whole test
suite. You probably don't want to do this very often, because it takes
a very long time, and anyway bors / travis will do it for you. (Often,
I will run this command in the background after opening a PR that I
think is done, but rarely otherwise. -nmatsakis)
The test results are cached and previously successful tests are
`ignored` during testing. The stdout/stderr contents as well as a
timestamp file for every test can be found under `build/ARCH/test/`.
To force-rerun a test (e.g. in case the test runner fails to notice
a change) you can simply remove the timestamp file.
## Running a subset of the test suites
When working on a specific PR, you will usually want to run a smaller
set of tests, and with a stage 1 build. For example, a good "smoke
test" that can be used after modifying rustc to see if things are
generally working correctly would be the following:
```bash
> ./x.py test --stage 1 src/test/{ui,compile-fail,run-pass}
```
This will run the `ui`, `compile-fail`, and `run-pass` test suites,
and only with the stage 1 build. Of course, the choice of test suites
is somewhat arbitrary, and may not suit the task you are doing. For
example, if you are hacking on debuginfo, you may be better off with
the debuginfo test suite:
```bash
> ./x.py test --stage 1 src/test/debuginfo
```
### Run only the tidy script
```bash
> ./x.py test src/tools/tidy
```
### Run tests on the standard library
```bash
> ./x.py test src/libstd
```
### Run tests on the standard library and run the tidy script
```bash
> ./x.py test src/libstd src/tools/tidy
```
### Run tests on the standard library using a stage 1 compiler
```bash
> ./x.py test src/libstd --stage 1
```
By listing which test suites you want to run you avoid having to run
tests for components you did not change at all.
**Warning:** Note that bors only runs the tests with the full stage 2
build; therefore, while the tests **usually** work fine with stage 1,
there are some limitations. In particular, the stage1 compiler doesn't
work well with procedural macros or custom derive tests.
## Running an individual test
Another common thing that people want to do is to run an **individual
test**, often the test they are trying to fix. One way to do this is
to invoke `x.py` with the `--test-args` option:
```bash
> ./x.py test --stage 1 src/test/ui --test-args issue-1234
```
Under the hood, the test runner invokes the standard rust test runner
(the same one you get with `#[test]`), so this command would wind up
filtering for tests that include "issue-1234" in the name.
## Using incremental compilation
You can further enable the `--incremental` flag to save additional
time in subsequent rebuilds:
```bash
> ./x.py test --stage 1 src/test/ui --incremental --test-args issue-1234
```
If you don't want to include the flag with every command, you can
enable it in the `config.toml`, too:
```toml
# Whether to always use incremental compilation when building rustc
incremental = true
```
Note that incremental compilation will use more disk space than usual.
If disk space is a concern for you, you might want to check the size
of the `build` directory from time to time.
## Running tests manually
Sometimes it's easier and faster to just run the test by hand. Most tests are
just `rs` files, so you can do something like
```bash
> rustc +stage1 src/test/ui/issue-1234.rs
```
This is much faster, but doesn't always work. For example, some tests
include directives that specify specific compiler flags, or which rely
on other crates, and they may not run the same without those options.

44
src/doc/rustc-guide/src/the-parser.md Normal file
View File

@ -0,0 +1,44 @@
# The Parser
The parser is responsible for converting raw Rust source code into a structured
form which is easier for the compiler to work with, usually called an [*Abstract
Syntax Tree*][ast]. An AST mirrors the structure of a Rust program in memory,
using a `Span` to link a particular AST node back to its source text.
The bulk of the parser lives in the [libsyntax] crate.
Like most parsers, the parsing process is composed of two main steps,
- lexical analysis – turn a stream of characters into a stream of token trees
- parsing – turn the token trees into an AST
The `syntax` crate contains several main players,
- a [`SourceMap`] for mapping AST nodes to their source code
- the [ast module] contains types corresponding to each AST node
- a [`StringReader`] for lexing source code into tokens
- the [parser module] and [`Parser`] struct are in charge of actually parsing
tokens into AST nodes,
- and a [visit module] for walking the AST and inspecting or mutating the AST
nodes.
The main entrypoint to the parser is via the various `parse_*` functions in the
[parser module]. They let you do things like turn a [`SourceFile`][sourcefile]
(e.g. the source in a single file) into a token stream, create a parser from
the token stream, and then execute the parser to get a `Crate` (the root AST
node).
To minimise the amount of copying that is done, both the `StringReader` and
`Parser` have lifetimes which bind them to the parent `ParseSess`. This contains
all the information needed while parsing, as well as the `SourceMap` itself.
[libsyntax]: https://doc.rust-lang.org/nightly/nightly-rustc/syntax/index.html
[rustc_errors]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_errors/index.html
[ast]: https://en.wikipedia.org/wiki/Abstract_syntax_tree
[`SourceMap`]: https://doc.rust-lang.org/nightly/nightly-rustc/syntax/source_map/struct.SourceMap.html
[ast module]: https://doc.rust-lang.org/nightly/nightly-rustc/syntax/ast/index.html
[parser module]: https://doc.rust-lang.org/nightly/nightly-rustc/syntax/parse/index.html
[`Parser`]: https://doc.rust-lang.org/nightly/nightly-rustc/syntax/parse/parser/struct.Parser.html
[`StringReader`]: https://doc.rust-lang.org/nightly/nightly-rustc/syntax/parse/lexer/struct.StringReader.html
[visit module]: https://doc.rust-lang.org/nightly/nightly-rustc/syntax/visit/index.html
[sourcefile]: https://doc.rust-lang.org/nightly/nightly-rustc/syntax/source_map/struct.SourceFile.html

168
src/doc/rustc-guide/src/traits/associated-types.md Normal file
View File

@ -0,0 +1,168 @@
# Equality and associated types
This section covers how the trait system handles equality between
associated types. The full system consists of several moving parts,
which we will introduce one by one:
- Projection and the `Normalize` predicate
- Placeholder associated type projections
- The `ProjectionEq` predicate
- Integration with unification
## Associated type projection and normalization
When a trait defines an associated type (e.g.,
[the `Item` type in the `IntoIterator` trait][intoiter-item]), that
type can be referenced by the user using an **associated type
projection** like `<Option<u32> as IntoIterator>::Item`.
> Often, people will use the shorthand syntax `T::Item`. Presently, that
> syntax is expanded during ["type collection"](../type-checking.html) into the
> explicit form, though that is something we may want to change in the future.
[intoiter-item]: https://doc.rust-lang.org/nightly/core/iter/trait.IntoIterator.html#associatedtype.Item
<a name="normalize"></a>
In some cases, associated type projections can be **normalized** –
that is, simplified – based on the types given in an impl. So, to
continue with our example, the impl of `IntoIterator` for `Option<T>`
declares (among other things) that `Item = T`:
```rust,ignore
impl<T> IntoIterator for Option<T> {
type Item = T;
...
}
```
This means we can normalize the projection `<Option<u32> as
IntoIterator>::Item` to just `u32`.
In this case, the projection was a "monomorphic" one – that is, it
did not have any type parameters. Monomorphic projections are special
because they can **always** be fully normalized.
Often, we can normalize other associated type projections as well. For
example, `<Option<?T> as IntoIterator>::Item`, where `?T` is an inference
variable, can be normalized to just `?T`.
In our logic, normalization is defined by a predicate
`Normalize`. The `Normalize` clauses arise only from
impls. For example, the `impl` of `IntoIterator` for `Option<T>` that
we saw above would be lowered to a program clause like so:
```text
forall<T> {
Normalize(<Option<T> as IntoIterator>::Item -> T) :-
Implemented(Option<T>: IntoIterator)
}
```
where in this case, the one `Implemented` condition is always true.
> Since we do not permit quantification over traits, this is really more like
> a family of program clauses, one for each associated type.
We could apply that rule to normalize either of the examples that
we've seen so far.
## Placeholder associated types
Sometimes however we want to work with associated types that cannot be
normalized. For example, consider this function:
```rust,ignore
fn foo<T: IntoIterator>(...) { ... }
```
In this context, how would we normalize the type `T::Item`?
Without knowing what `T` is, we can't really do so. To represent this case,
we introduce a type called a **placeholder associated type projection**. This
is written like so: `(IntoIterator::Item)<T>`.
You may note that it looks a lot like a regular type (e.g., `Option<T>`),
except that the "name" of the type is `(IntoIterator::Item)`. This is not an
accident: placeholder associated type projections work just like ordinary
types like `Vec<T>` when it comes to unification. That is, they are only
considered equal if (a) they are both references to the same associated type,
like `IntoIterator::Item` and (b) their type arguments are equal.
Placeholder associated types are never written directly by the user.
They are used internally by the trait system only, as we will see
shortly.
In rustc, they correspond to the `TyKind::UnnormalizedProjectionTy` enum
variant, declared in [`librustc/ty/sty.rs`][sty]. In chalk, we use an
`ApplicationTy` with a name living in a special namespace dedicated to
placeholder associated types (see the `TypeName` enum declared in
[`chalk-ir/src/lib.rs`][chalk_type_name]).
[sty]: https://github.com/rust-lang/rust/blob/master/src/librustc/ty/sty.rs
[chalk_type_name]: https://github.com/rust-lang-nursery/chalk/blob/master/chalk-ir/src/lib.rs
## Projection equality
So far we have seen two ways to answer the question of "When can we
consider an associated type projection equal to another type?":
- the `Normalize` predicate could be used to transform projections when we
knew which impl applied;
- **placeholder** associated types can be used when we don't. This is also
known as **lazy normalization**.
We now introduce the `ProjectionEq` predicate to bring those two cases
together. The `ProjectionEq` predicate looks like so:
```text
ProjectionEq(<T as IntoIterator>::Item = U)
```
and we will see that it can be proven *either* via normalization or
via the placeholder type. As part of lowering an associated type declaration from
some trait, we create two program clauses for `ProjectionEq`:
```text
forall<T, U> {
ProjectionEq(<T as IntoIterator>::Item = U) :-
Normalize(<T as IntoIterator>::Item -> U)
}
forall<T> {
ProjectionEq(<T as IntoIterator>::Item = (IntoIterator::Item)<T>)
}
```
These are the only two `ProjectionEq` program clauses we ever make for
any given associated item.
## Integration with unification
Now we are ready to discuss how associated type equality integrates
with unification. As described in the
[type inference](../type-inference.html) section, unification is
basically a procedure with a signature like this:
```text
Unify(A, B) = Result<(Subgoals, RegionConstraints), NoSolution>
```
In other words, we try to unify two things A and B. That procedure
might just fail, in which case we get back `Err(NoSolution)`. This
would happen, for example, if we tried to unify `u32` and `i32`.
The key point is that, on success, unification can also give back to
us a set of subgoals that still remain to be proven. (It can also give
back region constraints, but those are not relevant here).
Whenever unification encounters a non-placeholder associated type
projection P being equated with some other type T, it always succeeds,
but it produces a subgoal `ProjectionEq(P = T)` that is propagated
back up. Thus it falls to the ordinary workings of the trait system
to process that constraint.
> If we unify two projections P1 and P2, then unification produces a
> variable X and asks us to prove that `ProjectionEq(P1 = X)` and
> `ProjectionEq(P2 = X)`. (That used to be needed in an older system to
> prevent cycles; I rather doubt it still is. -nmatsakis)

29
src/doc/rustc-guide/src/traits/bibliography.md Normal file
View File

@ -0,0 +1,29 @@
# Bibliography
If you'd like to read more background material, here are some
recommended texts and papers:
[Programming with Higher-order Logic][phl], by Dale Miller and Gopalan
Nadathur, covers the key concepts of Lambda prolog. Although it's a
slim little volume, it's the kind of book where you learn something
new every time you open it.
[phl]: https://www.amazon.com/Programming-Higher-Order-Logic-Dale-Miller/dp/052187940X
<a name="pphhf"></a>
["A proof procedure for the logic of Hereditary Harrop formulas"][pphhf],
by Gopalan Nadathur. This paper covers the basics of universes,
environments, and Lambda Prolog-style proof search. Quite readable.
[pphhf]: https://dl.acm.org/citation.cfm?id=868380
<a name="slg"></a>
["A new formulation of tabled resolution with delay"][nftrd], by
[Theresa Swift]. This paper gives a kind of abstract treatment of the
SLG formulation that is the basis for our on-demand solver.
[nftrd]: https://dl.acm.org/citation.cfm?id=651202
[ts]: http://www3.cs.stonybrook.edu/~tswift/
[Theresa Swift]: http://www3.cs.stonybrook.edu/~tswift/

67
src/doc/rustc-guide/src/traits/caching.md Normal file
View File

@ -0,0 +1,67 @@
# Caching and subtle considerations therewith
In general, we attempt to cache the results of trait selection. This
is a somewhat complex process. Part of the reason for this is that we
want to be able to cache results even when all the types in the trait
reference are not fully known. In that case, it may happen that the
trait selection process is also influencing type variables, so we have
to be able to not only cache the *result* of the selection process,
but *replay* its effects on the type variables.
## An example
The high-level idea of how the cache works is that we first replace
all unbound inference variables with placeholder versions. Therefore,
if we had a trait reference `usize : Foo<$t>`, where `$t` is an unbound
inference variable, we might replace it with `usize : Foo<$0>`, where
`$0` is a placeholder type. We would then look this up in the cache.
If we found a hit, the hit would tell us the immediate next step to
take in the selection process (e.g. apply impl #22, or apply where
clause `X : Foo<Y>`).
On the other hand, if there is no hit, we need to go through the [selection
process] from scratch. Suppose, we come to the conclusion that the only
possible impl is this one, with def-id 22:
[selection process]: ./resolution.html#selection
```rust,ignore
impl Foo<isize> for usize { ... } // Impl #22
```
We would then record in the cache `usize : Foo<$0> => ImplCandidate(22)`. Next
we would [confirm] `ImplCandidate(22)`, which would (as a side-effect) unify
`$t` with `isize`.
[confirm]: ./resolution.html#confirmation
Now, at some later time, we might come along and see a `usize :
Foo<$u>`. When replaced with a placeholder, this would yield `usize : Foo<$0>`, just as
before, and hence the cache lookup would succeed, yielding
`ImplCandidate(22)`. We would confirm `ImplCandidate(22)` which would
(as a side-effect) unify `$u` with `isize`.
## Where clauses and the local vs global cache
One subtle interaction is that the results of trait lookup will vary
depending on what where clauses are in scope. Therefore, we actually
have *two* caches, a local and a global cache. The local cache is
attached to the [`ParamEnv`], and the global cache attached to the
[`tcx`]. We use the local cache whenever the result might depend on the
where clauses that are in scope. The determination of which cache to
use is done by the method `pick_candidate_cache` in `select.rs`. At
the moment, we use a very simple, conservative rule: if there are any
where-clauses in scope, then we use the local cache. We used to try
and draw finer-grained distinctions, but that led to a serious of
annoying and weird bugs like [#22019] and [#18290]. This simple rule seems
to be pretty clearly safe and also still retains a very high hit rate
(~95% when compiling rustc).
**TODO**: it looks like `pick_candidate_cache` no longer exists. In
general, is this section still accurate at all?
[`ParamEnv`]: ../param_env.html
[`tcx`]: ../ty.html
[#18290]: https://github.com/rust-lang/rust/issues/18290
[#22019]: https://github.com/rust-lang/rust/issues/22019

Some files were not shown because too many files have changed in this diff Show More