The WASI signal constants and proc_raise function were removed in the
latest [ephemeral], which had been scheduled to be in the next snapshot,
however WASI itself is now transitioning away from the snapshot system.
WASI libc will also be transitioning to updated wit specs once they're
ready, however until that time, we can make the simple change of
removing these signal constants to avoid confusion.
Fixes#271.
Fixes#272.
[ephemeral]: https://github.com/WebAssembly/WASI/tree/main/phases/ephemeral/witx
Add a `__wasilibc_deinit_environ` function which clears the current
environment variable state to the state where next time the environment
variable functions are called, they'll reinitialize the environment.
And add a `__wasilibc_maybe_reinitialize_environ_eagerly` function to
reinitialize the environment variable state if `environ` or `_environ`
are needed.
These functions are needed by wizer to be able to suspend and resume
a program and have it read new environment variables from the host
environment; see bytecodealliance/wizer#8 for background.
Upcoming WASI snapshots omit the `PROCESS_CPUTIME` clock, since WASI has
no inherent concept of processes, and since implementations which don't
use a process for each instance don't have a way to implement it
efficiently.
However, `clock`, `times`, and `getrusage` are useful functions, so
provide optional emulated version of them, using the `MONOTONIC` clock.
This means these implementations will measure not just the program's
own CPU time, but also time spent suspended while other programs are
running.
Due to this difference in behavior, put these implementations behind
a flag. Users must pass `-D_WASI_EMULATED_PROCESS_CLOCK` and link with
`-lwasi-emulated-process-clocks` to enable them.
* Update to the next version of the `witx` crate
* Generate adapter functions instead of simply a header file to have a
place where adapter code can go.
* Implement adapters in terms of the instructions that the `witx` crate
tells us about.
* Update the interface of functions to what `witx` expects, notably
string arguments are now only taken as `char*` and `strlen` happens in
the adapter function.
* Update defined/predefined/undefined symbol lists for types that have
been updated.
Some precise generated code has changed but the actual APIs should all
be the same except for the change to not take the length of the string
in the raw WASI call, since idiomatically C doesn't pass the length of
strings around.
Eventually it's expected that the shim functions, while sometimes not
necessary today, will implement more checks and more conversions as
necessary for new APIs.
* Actually update witx submodule
* Comment how to regenerate files
* Tweak name of actual function imports
Make it a bit clearer that they're the ones that we're importing and
calling.
* Update submodule to point to WebAssembly
AT_FDCWD is a special constant in POSIX that can be passed to *at
functions to indicate the current working directory. Since the
current working directory is emulated in wasi libc, add emulated
AT_FDCWD support as well.
Fixes#42.
We've already started removing this; this just removes all remaining
ones under the libc-bottom-half directory.
These markers were originally intended to help track upstream changes,
however in practice they created a lot of clutter and weren't that
helpful. And now, upstream cloudlibc is no longer active.
* Add basic emulation of getcwd/chdir
This commit adds basic emulation of a current working directory to
wasi-libc. The `getcwd` and `chdir` symbols are now implemented and
available for use. The `getcwd` implementation is pretty simple in that
it just copies out of a new global, `__wasilibc_cwd`, which defaults to
`"/"`. The `chdir` implementation is much more involved and has more
ramification, however.
A new function, `make_absolute`, was added to the preopens object. Paths
stored in the preopen table are now always stored as absolute paths
instead of relative paths, and initial relative paths are interpreted as
being relative to `/`. Looking up a path to preopen now always turns it
into an absolute path, relative to the current working directory, and an
appropriate path is then returned.
The signature of `__wasilibc_find_relpath` has changed as well. It now
returns two path components, one for the absolute part and one for the
relative part. Additionally the relative part is always dynamically
allocated since it may no longer be a substring of the original input
path.
This has been tested lightly against the Rust standard library so far,
but I'm not a regular C developer so there's likely a few things to
improve!
* Amortize mallocs made in syscalls
* Avoid size bloat on programs that don't use `chdir`
* Add threading compat
* Collect `link`/`renameat` second path lookup
* Update comments about chdir.c in makefile
* Move definition of `__wasilibc_find_relpath_alloc` to header
* Expand comments
* Document the format of strings a bit more
* Fixup a few issues in path logic
* Fix GitHub Actions
This adds a new crt1-command.c startup file, which uses
[new-style command support]. Instead of calling `__wasm_call_ctors`
and `__wasm_call_dtors` directly, this lets wasm-ld automatically call
them.
This preserves the existing crt1.c, so that the same wasi-libc build
can support old-style and new-style commands, for compatibility during
the transition.
[new-style command support]: https://reviews.llvm.org/D81689
Co-authored-by: Dan Gohman <sunfish@mozilla.com>
Normally bits like this would be considered implementation details, but
in this case, `X_OK`, `W_OK`, and `R_OK` line up with `S_IXOTH`,
`S_IWOTH`, and `S_IROTH` on other systems, and those bits do have
well-known values.
This emphasizes the relationship with `__wasm_call_ctors`. Note however
that while `__wasm_call_ctors` is synthesized by the linker,
`__wasm_call_dtors` is still defined by libc.
Static constructors are registered statically, but static destructors
need to be registered dynamically so that they only run if their
corresponding static constructors have run, and so that they're
ordered with respect to interleaved `atexit` calls.
* Avoid calling `poll_oneoff` with zero subscriptions.
With https://github.com/WebAssembly/WASI/pull/193 merged, WASI is moving
to make `poll_oneoff` with no arguments an error. Even though that's in
ephemeral and not yet in a snapshot, we can start to anticipate it in
libc:
- Remove the `pause` function, since WASI has no signals and thus no
way to ever wake it up short of having the host terminate it.
- Make `poll` and `pselect` return `ENOTSUP` in the case of having no
events to wait for.
* Remove `pause` from the defined-symbols.txt list.
* Fix __wasilibc_unmodified_upstream markers.
* Check for zero subscriptions, rather than zero events.
Make `poll` and `pselect` return `ENOTSUP` when asked to poll on zero
subscriptions, rather than when the systerm returns zero events.
While here, drop the `__wasilibc_unmodified_upstream` markers, which
were already pretty noisy here, and would be significantly worse with
this change.
* Add comments about the subtle relationship between nfds and nsubscriptions.
* Rewrite the comment.
* Fix code quotes.
* Rewrite the preopen functionality.
Rewrite the preopen functionality to be simpler, better organized,
and better integrated into WASI libc. Preopen support has diverged so
much from libpreopen that it no longer makes sense to track libpreopen
as an explicit upstream. And add more documentation.
* Fix missing #include.
* Fix a compilation error.
* Add support for the Reactor model.
* Mark _activate and _start as wasm exports.
* Rename _activate to _initialize.
* Don't define `_fini`.
* Rename reactor-crt1.c to crt1-reactor.c.
Before this commit, he header of a mapped area, `struct map`, was
defined as follows:
struct map {
int prot;
int flags;
off_t offset;
size_t length;
char body[];
};
Because the size and alignment of an `off_t` is 8 bytes, the entire
structure was padded to 24 bytes. However, the offset of `body` into
`struct map` was only 20 bytes. Therefore the code in mmap() and
munmap() did not agree on the offset from header to body.
This commit changes mmap() to skip the entire header, which is what
munmap() expects and what the size calculation uses.
* Added utime.h
* Changes after code review
* Auto-generated expected files instead of manually editing
* Fix libpreopoen stat and utime not following symlinks correctly
* Lazy-initialize the environment variables.
This is the first in a series of PRs to make it easier to use WASI libc
in Wasm modules that don't have a `main` function. By initializing the
environment on demand, we avoid depending on having `__wasm_call_ctors`
run.
This uses weak symbols strategically to ensure that if `environ` is
used, it is initialized eagerly, but if only `getenv` and friends
are used, the environment is initialized lazily.
Eventually, I expect we'll have a convention for wasm modules without
main functions which will allow the `__wasm_call_ctors` function to be
called automatically, but this helps in simple cases for now.
Fixes#180.
* Add comments explaining the libc-environ-compat.h header usage.
This adds support for the `__main_argc_argv` change, while preserving
compatibility with `__original_main`. This is needed by the LTO build
because the `__original_main` hack works in LLVM codegen, which is after
LTO. The `__main_argc_argv` change is implemented in clang, which makes
it properly visible to LTO.
See the WHATSNEW file for details; this doesn't have any major changes
for wasi-libc; in particular, the new catgets and GLOB_TILDE features
are disabled.
we decided to abandon the upstream code guarded by
#ifdef __wasilibc_unmodified_upstream // non-anonymous unions
because these changes are sprawling and those guards are of diminishing
importance
* use pch/layout branch for witx; generate assertions of layout
* address review comments, add asserts for handle
* change wasm32 support comment to a preprocessor error
* expose `to_c_header` in wasi-headers crate for use in external test harness
* main.rs: inputs and output arguments are optional
so that generate-libc command works
* regen header
The header api.h was update using the following command:
$ cd tools/wasi-headers
$ cargo run -- WASI/phases/snapshot/witx/typenames.witx \
WASI/phases/snapshot/witx/wasi_snapshot_preview1.witx \
--output ../../libc-bottom-half/headers/public/wasi/api.h
This should probably have been part of #140 but we don't actually
support signals so this doesn't get much testing I imagine.
The old names like `__WASI_SIGBUS` no longer exist and the new names
look like `__WASI_SIGNAL_BUS`.
* Use constructor functions for optional init routines.
Instead of using weak symbols, use constructor function attributes for the
environment and preopen initialization routines. This is simpler, uses
less code, and is more LTO-friendly.
* Change the constructor priorities to start at 50.
We don't currently have specific plans for other levels in the reserved
range (0-100), so leave room for both lower and higher priorities.
* Add the WASI repo as a submodule.
Also, add the witx filenames to the generated output, and just have
`cargo run` auto-generate the api.h header, rather than using clap.
* Switch witx to a path dependency.
* Add a test.
* Add a test that the generated file is in sync with the generator.
* Enable CI testing with Github Actions.
* Fix the name of the wasi-headers directory.
* Enable submodules.
* Add a diff mechanism to help explain failures.
* Sort the inputs for display.
* More debugging.
* More debugging.
* Add a .gitattributes file forcing text files to be eol=lf.
Most editors these days can deal with eof=lf files, even on Windows, and
this avoids trouble with headers and other generated files differing in
line endings.
* Make __wasi_linkcount_t a uint64_t (#134)
Refs: https://github.com/WebAssembly/WASI/pull/127
* Generate the WASI interface from witx.
This replaces the hand-maintained <wasi/core.h> header with a
<wasi/api.h> generated from witx.
Most of the churn here is caused by upstream WASI renamings; hopefully
in the future ABI updates will be less noisy.
Don't ignore paths which don't have the required rights. This means
that if the lookup finds a path that doesn't have the required
rights, it'll just proceed to the actual operation which will fail
with `ENOTCAPABLE`.
Intuitively, use cases which would depend on having multiple
overlapping matching paths for a given lookup and intelligently
picking the one with the required rights seems like they should
be uncommon.
This is simpler overall, and requires less code.
Rely on the WASI implementation to check capabilities flags, rather
than also checking them in the userspace `openat` implementation.
This code is admittedly getting fairly dense with `#ifdef`s, so it
may soon make sense to move this file out of the `cloudlibc`
directory and removing the upstream change markers.
For now, this means using `//`-style comments in .c source files (though
not public header files), and spaces rather than tabs. No strong opinion
here; this is just what the majority of the current code is using.
This also synchronizes basics/crt/crt1.c with libc-bottom-half's
version, though this is just a cleanup as the former isn't currently used
by the main wasi-libc build.
* Avoid a `strdup` call in `__wasilibc_populate_libpreopen`.
Optimize `__wasilibc_populate_libpreopen` to avoid calling `strdup` in
the common case where it's called from `__wasilibc_populate_libpreopen`.
* Convert an if into a ?:.
wasi-libc's copy of libpreopen has evolved so many local changes that
it's no longer worth keeping the upstream code structure and marking
changes with __wasilibc_unmodified_upstream.
This PR merges the source files into a single file, removes all
__wasilibc_unmodified_upstream code, eliminates the ability to
allocate multiple preopen lists, eliminates the need for
__wasilibc_init_preopen, eliminates the non-standard eaccess, and
makes several other cleanups. It also enables NDEBUG so that internal
assertions are disabled in release builds.
POSIX requires `environ` to be a pointer to a NULL-terminated array of
pointers, so it itself can't be NULL.
This fixes a regression in src/functional/env.c in wasi-libc-test.
* Link `populate_args` only if we actually need command-line arguments.
This avoids linking in the argv/argc initialization code,
and the __wasi_args_sizes_get and __wasi_args_get imports, in
programs that don't use command-line arguments. The way this works is,
if the user writes `int main(int argc, char *argv[])`, the argument
initialization code is loaded, and if they write `int main(void)`,
it's not loaded.
This promotes the `__original_main` mechanism into an effective contract
between the compiler and libc, which wasn't its original purpose,
however it seems to fit this purpose quite well.
* Document that `__original_main` may be the user's zero-arg `main`.
* Link `populate_environ` only if we actually need environment variables.
This avoids linking in the environment variable initialization code,
and the __wasi_environ_sizes_get and __wasi_environ_get imports, in
programs that don't use environment variables.
This also removes the "___environ" (three underscores) alias symbol,
which is only in musl for backwards compatibility.
* Switch to //-style comments.
* If malloc fails, don't leave `__environ` pointing to an uninitialized buffer.
* Fix a memory leak if one malloc succeeds and the other fails.
* Use calloc to handle multiplication overflow.
This also handles the NULL terminator.
* Don't initialize __environ until everything has succeeded.
* Avoid leaking in case __wasi_environ_get fails.
* Handle overflow in the add too.
* Add #include <stdlib.h> for malloc etc.
* If the environment is empty, don't allocate any memory.
This replaces our custom `unlink` wrapper with an upstream one. We still
end up replacing the entire body with local changes, but this makes it
easier to see what those changes are.
The other change here is a fix to ignore repeated '/'s in paths.
POSIX requires fcntl.h to define the SEEK_* macros, so this satisfies
that requirement. Also, this allows <stdio.h> to avoid including as much
unnecessary content.
This fixes one issue with src/api/fcntl.c.
These functions aren't specific to the underlying system call interface,
so they don't need to be in the "bottom half".
This also fixes src/functional/inet_pton.c and
src/regression/inet_pton-empty-last-field.c in musl's libc-test.
Initialize `environ` even if there are no environment variables, so that
it alwasy points to a NULL-terminated array even if that array just
contains the NULL. This fixes src/functional/env.c.
`lseek(x, 0, SEEK_CUR)` has no effect other than to return the current
file offset. The patch here uses a macro with `__builtin_constant_p` to
recognize this case and rewrite it to a library call that uses `fd_tell`
rather than `fd_seek`, so that programs that don't need actual seeking
don't end up importing `fd_seek`.
This is also the first usage of `__wasi_fd_tell` in WASI libc, so this
adds it to undefined-symbols.txt.
Some systems, such as Darwin, only declare getentropy in <sys/random.h>,
so declare it there on WASI too for compatibility.
Also, give getentropy the underscore-prefix/weak-symbol treatment, as
it's not a standard-reserved identifier.
Previously, FD_SET and friends were missing their actual definitions.
This provides definitions, entirely within the system headers in a
way that doesn't need instantiated out-of-line definitions.
* Provide a public interface to preopened directory lookups.
For users of especially non-C compilers, provide an API for looking up
preopened directories. This is used internally in WASI libc to translate
from eg. `open` to `openat`, but it can now also be used by user code.
