* Start renaming preview1 to p1 and preview2 to p2
This is an initial start at renaming the "preview" terminology in WASI
targets to "pX". For example the `wasm32-wasi` target should transition
to `wasm32-wasip1`, `wasm32-wasi-preview2` should transition to
`wasm32-wasip2`, and `wasm32-wasi-threads` should transition to
`wasm32-wasip1-threads`. This commit applies a few renames in the
`Makefile` such as:
* `WASI_SNAPSHOT` is now either "p1" or "p2"
* The default p2 target triple is now `wasm32-wasip2` instead of
`wasm32-wasi-preview2` (in the hopes that it's early enough to change
the default).
* Bindings for WASIp2 were renamed from "preview2" terminology to "wasip2".
* The expected-defines files are renamed and the logic of which
expectation was used has been updated slightly.
With this commit the intention is that non-preview2 defaults do not
change. For example the default build still produces a `wasm32-wasi`
sysroot. If `TARGET_TRIPLE=wasm32-wasip1` is passed, however, then that
sysroot is produced instead. Similarly a `THREAD_MODEL=posix` build
produces a `wasm32-wasi-threads` sysroot target but you can now also
pass `TARGET_TRIPLE=wasm32-wasip1-threads` to rename the sysroot.
My hope is to integrate this into the wasi-sdk repository and build a
dual sysroot for these new targets for a release or two so both are
supported and then in the future the defaults can be switched away from
`wasm32-wasi` to `wasm32-wasip1` as built-by-default.
* Update builds in CI
* Update test workflow
* Fix test for wasm32-wasip1-threads
* Make github actions rules a bit more readable
* make the Makefiles a bit more robust
- Escape "." character in `sed` regex
- Ensure that %.wasm target fails cleanly (i.e. without generating the target file) if `wasm-tools` fails
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
* split `component new` rule out of link rule for Preview 2
We now explicitly distinquish between core module files (%.core.wasm) and
component files (%.wasm), which helps avoid the trickery in my previous commit.
In order to test this properly, I needed to update the Wasmtime URL to point to
v17.0.0 instead of dev (which we needed to do anyway), and that in turn required
updating the bindings to use the final WASI 0.2.0 release.
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
---------
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
* add WASI Preview 2 bindings
This adds C bindings generated from the `wasi:cli/imports@0.2.0-rc-2023-12-05`
world, plus a makefile target to regenerate them from the WIT source files.
We'll use these bindings to call Preview 2 host functions when building for the
`wasm32-wasi-preview2` target.
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
* update to pre-release of `wit-bindgen` 0.17.0
This includes https://github.com/bytecodealliance/wit-bindgen/pull/804 (fix
broken indentation in generated code) and
https://github.com/bytecodealliance/wit-bindgen/pull/805 (support overriding
world name and adding a suffix to the component type custom section).
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
* test all targets; update preview2 expected output files
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
* build for `wasm32-wasi-threads` before testing it
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
* move generated bindings per review feedback
Since these files aren't part of cloudlibc, no reason to put them under the
cloudlibc directory.
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
* move preview2.h to wasi directory
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
---------
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
* Convert preopen initialization to be lazy.
Insteead of eagerly initializing the preopens in a static constructor,
perform preopen initialization the first time it's needed, or before a
close or a renumber which might disrupt the file descriptor space.
And, use a weak symbol with a stub function for use by `close` or
`fd_renumber`, we that they can trigger preopen initialization only
if it's actually needed.
This way, if a program doesn't contain any calls to any function that
needs preopens, it can avoid linking in the preopen initialization code.
And if it contains calls but doesn't execute them at runtime, it can
avoid executing the preopen intiailization code.
A downside here is that this may cause problems for users that call
`__wasi_fd_close` or `__wasi_fd_renumber` directly and close over
overwrite some preopens before libc has a chance to scan them. To
partially address this, this PR does add a declaration for
`__wasilibc_populate_preopens` to <wasi/libc.h> so that users can call
it manually if they need to.
* Fix calling `internal_register_preopened_fd` with the lock held.
Factor out the lock acquisition from the implementation of
`internal_register_preopened_fd` so that we can call it from
`__wasilibc_populate_preopens` with the lock held.
as __wasi_errno_t is uint16_t, with the current coding,
__pthread_create will never see negative return values from
wasi:thread_spawn.
eg. (int)(uint16_t)-1 == 65535.
* threads: implement `pthread_create`
As described in the [`wasi-threads`] proposal, this change implements
`pthread_create` using the new `wasi_thread_spawn(void *arg)` API. As
described there, `wasi-libc` exports the thread entry point with the
expected name, `wasi_thread_start`, and then unwraps the passed argument
`struct` to invoke the user function with the user argument `struct`.
[`wasi-threads`]: https://github.com/WebAssembly/wasi-threads/pull/5
Previously, the TID was only passed to the child thread entry point; the
parent thread was forced to wait until the child thread set the TID in
the pthread structure. With this change, the TID will be passed not only
to the child thread but also returned to the parent thread, so that
either side can make progress. The `i32.store` becomes an
`i32.atomic.store` to avoid concurrent writes.
This change extracts the `weak*`-related parts of #303 as a separate PR.
Note that this is slightly strange in that it uses some top-half MUSL
headers in the bottom-half code, but discussion around this led me to
believe that the advantages of, e.g., `LOCK` made this worthwhile.
Beyond just changing uses of `weak` to `__weak__`, we also MUSL's `weak`
and `weak_alias` macros in a few more places.
The WASI signal constants and proc_raise function were removed in the
latest [ephemeral], which had been scheduled to be in the next snapshot,
however WASI itself is now transitioning away from the snapshot system.
WASI libc will also be transitioning to updated wit specs once they're
ready, however until that time, we can make the simple change of
removing these signal constants to avoid confusion.
Fixes#271.
Fixes#272.
[ephemeral]: https://github.com/WebAssembly/WASI/tree/main/phases/ephemeral/witx
Add a `__wasilibc_deinit_environ` function which clears the current
environment variable state to the state where next time the environment
variable functions are called, they'll reinitialize the environment.
And add a `__wasilibc_maybe_reinitialize_environ_eagerly` function to
reinitialize the environment variable state if `environ` or `_environ`
are needed.
These functions are needed by wizer to be able to suspend and resume
a program and have it read new environment variables from the host
environment; see bytecodealliance/wizer#8 for background.
* Update to the next version of the `witx` crate
* Generate adapter functions instead of simply a header file to have a
place where adapter code can go.
* Implement adapters in terms of the instructions that the `witx` crate
tells us about.
* Update the interface of functions to what `witx` expects, notably
string arguments are now only taken as `char*` and `strlen` happens in
the adapter function.
* Update defined/predefined/undefined symbol lists for types that have
been updated.
Some precise generated code has changed but the actual APIs should all
be the same except for the change to not take the length of the string
in the raw WASI call, since idiomatically C doesn't pass the length of
strings around.
Eventually it's expected that the shim functions, while sometimes not
necessary today, will implement more checks and more conversions as
necessary for new APIs.
* Actually update witx submodule
* Comment how to regenerate files
* Tweak name of actual function imports
Make it a bit clearer that they're the ones that we're importing and
calling.
* Update submodule to point to WebAssembly
AT_FDCWD is a special constant in POSIX that can be passed to *at
functions to indicate the current working directory. Since the
current working directory is emulated in wasi libc, add emulated
AT_FDCWD support as well.
Fixes#42.
* Add basic emulation of getcwd/chdir
This commit adds basic emulation of a current working directory to
wasi-libc. The `getcwd` and `chdir` symbols are now implemented and
available for use. The `getcwd` implementation is pretty simple in that
it just copies out of a new global, `__wasilibc_cwd`, which defaults to
`"/"`. The `chdir` implementation is much more involved and has more
ramification, however.
A new function, `make_absolute`, was added to the preopens object. Paths
stored in the preopen table are now always stored as absolute paths
instead of relative paths, and initial relative paths are interpreted as
being relative to `/`. Looking up a path to preopen now always turns it
into an absolute path, relative to the current working directory, and an
appropriate path is then returned.
The signature of `__wasilibc_find_relpath` has changed as well. It now
returns two path components, one for the absolute part and one for the
relative part. Additionally the relative part is always dynamically
allocated since it may no longer be a substring of the original input
path.
This has been tested lightly against the Rust standard library so far,
but I'm not a regular C developer so there's likely a few things to
improve!
* Amortize mallocs made in syscalls
* Avoid size bloat on programs that don't use `chdir`
* Add threading compat
* Collect `link`/`renameat` second path lookup
* Update comments about chdir.c in makefile
* Move definition of `__wasilibc_find_relpath_alloc` to header
* Expand comments
* Document the format of strings a bit more
* Fixup a few issues in path logic
* Fix GitHub Actions
* Rewrite the preopen functionality.
Rewrite the preopen functionality to be simpler, better organized,
and better integrated into WASI libc. Preopen support has diverged so
much from libpreopen that it no longer makes sense to track libpreopen
as an explicit upstream. And add more documentation.
* Fix missing #include.
* Fix a compilation error.
* Lazy-initialize the environment variables.
This is the first in a series of PRs to make it easier to use WASI libc
in Wasm modules that don't have a `main` function. By initializing the
environment on demand, we avoid depending on having `__wasm_call_ctors`
run.
This uses weak symbols strategically to ensure that if `environ` is
used, it is initialized eagerly, but if only `getenv` and friends
are used, the environment is initialized lazily.
Eventually, I expect we'll have a convention for wasm modules without
main functions which will allow the `__wasm_call_ctors` function to be
called automatically, but this helps in simple cases for now.
Fixes#180.
* Add comments explaining the libc-environ-compat.h header usage.
* use pch/layout branch for witx; generate assertions of layout
* address review comments, add asserts for handle
* change wasm32 support comment to a preprocessor error
* expose `to_c_header` in wasi-headers crate for use in external test harness
* main.rs: inputs and output arguments are optional
so that generate-libc command works
* regen header
The header api.h was update using the following command:
$ cd tools/wasi-headers
$ cargo run -- WASI/phases/snapshot/witx/typenames.witx \
WASI/phases/snapshot/witx/wasi_snapshot_preview1.witx \
--output ../../libc-bottom-half/headers/public/wasi/api.h
* Add the WASI repo as a submodule.
Also, add the witx filenames to the generated output, and just have
`cargo run` auto-generate the api.h header, rather than using clap.
* Switch witx to a path dependency.
* Add a test.
* Add a test that the generated file is in sync with the generator.
* Enable CI testing with Github Actions.
* Fix the name of the wasi-headers directory.
* Enable submodules.
* Add a diff mechanism to help explain failures.
* Sort the inputs for display.
* More debugging.
* More debugging.
* Add a .gitattributes file forcing text files to be eol=lf.
Most editors these days can deal with eof=lf files, even on Windows, and
this avoids trouble with headers and other generated files differing in
line endings.
* Make __wasi_linkcount_t a uint64_t (#134)
Refs: https://github.com/WebAssembly/WASI/pull/127
* Generate the WASI interface from witx.
This replaces the hand-maintained <wasi/core.h> header with a
<wasi/api.h> generated from witx.
Most of the churn here is caused by upstream WASI renamings; hopefully
in the future ABI updates will be less noisy.
Don't ignore paths which don't have the required rights. This means
that if the lookup finds a path that doesn't have the required
rights, it'll just proceed to the actual operation which will fail
with `ENOTCAPABLE`.
Intuitively, use cases which would depend on having multiple
overlapping matching paths for a given lookup and intelligently
picking the one with the required rights seems like they should
be uncommon.
This is simpler overall, and requires less code.
* Link `populate_environ` only if we actually need environment variables.
This avoids linking in the environment variable initialization code,
and the __wasi_environ_sizes_get and __wasi_environ_get imports, in
programs that don't use environment variables.
This also removes the "___environ" (three underscores) alias symbol,
which is only in musl for backwards compatibility.
* Switch to //-style comments.
* If malloc fails, don't leave `__environ` pointing to an uninitialized buffer.
* Fix a memory leak if one malloc succeeds and the other fails.
* Use calloc to handle multiplication overflow.
This also handles the NULL terminator.
* Don't initialize __environ until everything has succeeded.
* Avoid leaking in case __wasi_environ_get fails.
* Handle overflow in the add too.
* Add #include <stdlib.h> for malloc etc.
* If the environment is empty, don't allocate any memory.
`lseek(x, 0, SEEK_CUR)` has no effect other than to return the current
file offset. The patch here uses a macro with `__builtin_constant_p` to
recognize this case and rewrite it to a library call that uses `fd_tell`
rather than `fd_seek`, so that programs that don't need actual seeking
don't end up importing `fd_seek`.
This is also the first usage of `__wasi_fd_tell` in WASI libc, so this
adds it to undefined-symbols.txt.
* Provide a public interface to preopened directory lookups.
For users of especially non-C compilers, provide an API for looking up
preopened directories. This is used internally in WASI libc to translate
from eg. `open` to `openat`, but it can now also be used by user code.
