Start to port Wasmtime to the new wasi-io API with resources. (#7029)

* Rename `Host*` things to avoid name conflicts with bindings.

* Update to the latest resource-enabled wit files.

* Adapting the code to the new bindings.

* Update wasi-http to the resource-enabled wit deps.

* Start adapting the wasi-http code to the new bindings.

* Make `get_directories` always return new owned handles.

* Simplify the `poll_one` implementation.

* Update the wasi-preview1-component-adapter.

FIXME: temporarily disable wasi-http tests.

Add logging to the cli world, since stderr is now a reseource that
can only be claimed once.

* Work around a bug hit by poll-list, fix a bug in poll-one.

* Comment out `test_fd_readwrite_invalid_fd`, which panics now.

* Fix a few FIXMEs.

* Use `.as_ref().trapping_unwrap()` instead of `TrappingUnwrapRef`.

* Use `drop_in_place`.

* Remove `State::with_mut`.

* Remove the `RefCell` around the `State`.

* Update to wit-bindgen 0.12.

* Update wasi-http to use resources for poll and I/O.

This required making incoming-body and outgoing-body resourrces too, to
work with `push_input_stream_child` and `push_output_stream_child`.

* Re-enable disabled tests, remove logging from the worlds.

* Remove the `poll_list` workarounds that are no longer needed.

* Remove logging from the adapter.

That said, there is no replacement yet, so add a FIXME comment.

* Reenable a test that now passes.

* Remove `.descriptors_mut` and use `with_descriptors_mut` instead.

Replace `.descriptors()` and `.descriptors_mut()` with functions
that take closures, which limits their scope, to prevent them from
invalid aliasing.

* Implement dynamic borrow checking for descriptors.

* Add a cargo-vet audit for wasmtime-wmemcheck.

* Update cargo vet for wit-bindgen 0.12.

* Cut down on duplicate sync/async resource types (#1)

* Allow calling `get-directories` more than once (#2)

For now `Clone` the directories into new descriptor slots as needed.

* Start to lift restriction of stdio only once  (#3)

* Start to lift restriction of stdio only once

This commit adds new `{Stdin,Stdout}Stream` traits which take over the
job of the stdio streams in `WasiCtxBuilder` and `WasiCtx`. These traits
bake in the ability to create a stream at any time to satisfy the API
of `wasi:cli`. The TTY functionality is folded into them as while I was
at it.

The implementation for stdin is relatively trivial since the stdin
implementation already handles multiple streams reading it. Built-in
impls of the `StdinStream` trait are also provided for helper types in
`preview2::pipe` which resulted in the implementation of
`MemoryInputPipe` being updated to support `Clone` where all clones read
the same original data.

* Get tests building

* Un-ignore now-passing test

* Remove unneeded argument from `WasiCtxBuilder::build`

* Fix tests

* Remove some workarounds

Stdio functions can now be called multiple times.

* If `poll_oneoff` fails part-way through, clean up properly.

Fix the `Drop` implementation for pollables to only drop the pollables
that have been successfully added to the list.

This fixes the poll_oneoff_files failure and removes a FIXME.

---------

Co-authored-by: Alex Crichton <[email protected]>

Author: sunfishcode

Cargo.lock

@@ -213,7 +213,7 @@ io-extras = "0.18.0"
 rustix = "0.38.8"
 is-terminal = "0.4.0"
 # wit-bindgen:
-wit-bindgen = { version = "0.11.0", default-features = false }
+wit-bindgen = { version = "0.12.0", default-features = false }
 
 # wasm-tools family:
 wasmparser = "0.113.2"

Cargo.toml

@@ -1,24 +1,24 @@
 use command_tests::wasi::cli::environment;
 use command_tests::wasi::cli::stdin;
+use command_tests::wasi::io::poll;
 use command_tests::wasi::io::streams;
-use command_tests::wasi::poll::poll;
 
 fn main() {
     let args = environment::get_arguments();
 
     if args == &["correct"] {
         let stdin: streams::InputStream = stdin::get_stdin();
-        let stdin_pollable = streams::subscribe_to_input_stream(stdin);
-        let ready = poll::poll_oneoff(&[stdin_pollable]);
-        assert_eq!(ready, &[true]);
-        poll::drop_pollable(stdin_pollable);
-        streams::drop_input_stream(stdin);
+        let stdin_pollable = stdin.subscribe();
+        let ready = poll::poll_list(&[&stdin_pollable]);
+        assert_eq!(ready, &[0]);
+        drop(stdin_pollable);
+        drop(stdin);
     } else if args == &["trap"] {
         let stdin: streams::InputStream = stdin::get_stdin();
-        let stdin_pollable = streams::subscribe_to_input_stream(stdin);
-        let ready = poll::poll_oneoff(&[stdin_pollable]);
-        assert_eq!(ready, &[true]);
-        streams::drop_input_stream(stdin);
+        let stdin_pollable = stdin.subscribe();
+        let ready = poll::poll_list(&[&stdin_pollable]);
+        assert_eq!(ready, &[0]);
+        drop(stdin);
         unreachable!(
             "execution should have trapped in line above when stream dropped before pollable"
         );

crates/test-programs/command-tests/src/bin/stream_pollable_lifetimes.rs

@@ -7,21 +7,10 @@ wit_bindgen::generate!({
 });
 
 struct T;
-use wasi::io::streams;
-use wasi::poll::poll;
+use wasi::io::poll;
 
 static mut STATE: Vec<String> = Vec::new();
 
-struct DropPollable {
-    pollable: poll::Pollable,
-}
-
-impl Drop for DropPollable {
-    fn drop(&mut self) {
-        poll::drop_pollable(self.pollable);
-    }
-}
-
 impl Guest for T {
     fn add_strings(ss: Vec<String>) -> u32 {
         for s in ss {
@@ -40,34 +29,32 @@ impl Guest for T {
     }
 
     fn write_strings_to(o: OutputStream) -> Result<(), ()> {
-        let sub = DropPollable {
-            pollable: streams::subscribe_to_output_stream(o),
-        };
+        let pollable = o.subscribe();
         unsafe {
             for s in STATE.iter() {
                 let mut out = s.as_bytes();
                 while !out.is_empty() {
-                    poll::poll_oneoff(&[sub.pollable]);
-                    let n = match streams::check_write(o) {
+                    poll::poll_list(&[&pollable]);
+                    let n = match o.check_write() {
                         Ok(n) => n,
                         Err(_) => return Err(()),
                     };
 
                     let len = (n as usize).min(out.len());
-                    match streams::write(o, &out[..len]) {
+                    match o.write(&out[..len]) {
                         Ok(_) => out = &out[len..],
                         Err(_) => return Err(()),
                     }
                 }
             }
 
-            match streams::flush(o) {
+            match o.flush() {
                 Ok(_) => {}
                 Err(_) => return Err(()),
             }
 
-            poll::poll_oneoff(&[sub.pollable]);
-            match streams::check_write(o) {
+            poll::poll_list(&[&pollable]);
+            match o.check_write() {
                 Ok(_) => {}
                 Err(_) => return Err(()),
             }