D guest bindings generator

[QuantumSegfault]

Implements a d subcrate to support generating bindings for the D programming language.

[QuantumSegfault]

This PR is not yet finished. Just opening a draft early to help interested parties keep track.

[alexcrichton]

Thanks! One thing I'd also recommend as you're implementing things is to add D to this enum as well to hook this into the testing infrastructure. You can then add d to this matrix as appropriate with CI-specific steps to setup the compiler and environment. That'll get codegen/runtime tests working and you'll be able to test out your work by dropping files into the tests/runtime/* folder to implement various components. You'll need to write some orchestration for the test runner to know how to compile D programs (you can draw inspiration from how C is organized) too.

Happy to help answer questions about anything in specific if you have them, and if you have any questions about Rust/idioms/etc feel free to leave a comment here and I can dig in. Otherwise I'll leave this be until you're ready, in which case feel free to ping me and I can take a closer look.

[QuantumSegfault]

@alexcrichton

Things are coming along nicely, but there's one question I have so far.

The codegen tests are excellent for verifying the output of generated imports and types bindings.

Problems arise with the testing of the export wrappers. The way I've implemented exports is using D templates, which are not and cannot be semantically analyzed at all until they have been instantiated. Until an implementation is provided, the wrappers can't be tested.

Is this a problem? This gap in the testing? Or are the runtime tests that will be written enough to compensate?

[alexcrichton]

A good question! If I understand the problem right I believe you're looking for this. The --stubs argument to the Rust generator generates a stub implementation of all exports to ensure that they're codegen'd well. That can also be useful from a "getting started with the generated code" perspective. Would something like that make sense for D, and would that solve the problem too?

[QuantumSegfault]

Yeah! That's pretty much the answer I'm looking for. Something that can create a very basic stub (no impl; just declarations of functions and types) that I can pump into the exports wrapper soley for tests. Thanks!

[QuantumSegfault]

@alexcrichton

I'm still working on getting full test coverage, but I'm ready for this to start getting reviewed.

Support for the new async stuff will be tackled in a future PR. Same with map and implements. This PR brings D support up to the feature set needed for WASIp2.

[alexcrichton]

Integration points all look good to me 👍. I can't speak much to D idioms and I'm not looking too closesly at crates/d/src/lib.rs, but that's what'd be in your purview

[alexcrichton]

For the CI failure, can you publish a wit-bindgen-d crate to crates.io and then invite the wasmtime-publish user? I'll configure the crate through that and remove you as an owner afterwards

[QuantumSegfault]

For the CI failure, can you publish a wit-bindgen-d crate to crates.io and then invite the wasmtime-publish user? I'll configure the crate through that and remove you as an owner afterwards

I'll do that closer to when this is all ready. I might find more small things needing tweaking as I flesh out the tests. No point making more releases than necessary.

[QuantumSegfault]

@alexcrichton

Perhaps you have some opinion or suggestions... (e.g. based on the choices made by other bindings)?

---

Up to now I've been hammering this out trying to make the API as symmetric as possible, and deferring certain aspects of memory management to the user.

Import parameters are "borrowing" (caller takes a constant reference; callee retains ownership of memory), and returns are owning, where the user has to free when they are done. D's scope(exit) (similar to Go's defer) makes it not terrible. But you still have to keep that in mind.

Same applies to exports, where for parameters the API provides you a const "borrow" and frees the memory itself. However you have to move any returned memory to the C heap (witClone is provided to handle a deep copy easily).

To keep signatures consistent for parameters vs. arguments, a thin WitList wrapper has to be used instead of D's native slices (due to ABI difference in ptr & length order). This is a very minor inconvenience, but it's not as idiomatic as it could be.

But once resources come into play, this starts getting more difficult. Right now you have to drop own handles yourself when you are done, and there are no guardrails to prevent use-after-free. And I just realized that borrow handles provided to you via parameters must ALSO be dropped. I've been allowing quietly coercing owns into borrows, but now one has to keep track of where the MyResource.Borrow comes from to know whether or not to drop it. Including now iterating over lists deeply to drop all the handles (so now on top of witFree that does both memory free and handle drops, now you need a specialized one just for parameters).

---

So I'm considering maybe bringing in some RAII (and some other D features) to help make this easier to deal with. Specifically, I'm thinking of making resource handles drop themselves on destroy, and disabling copying, making moves explicit. But moving an own handle out of a parameter requires said parameter to be mutable.

Lists, etc. also have to be aware of this.

So I'm thinking maybe switch things up. Split the types used for parameters from the ones use for returns (which requires duplicating all the record types; making two variants). Parameters can be idiomatic D slices (since lifting lists of lists requires extra copies anyway because of all_bits_valid not being true for pointers), and when there are resources handles in play, I can make them take mutable refs instead.

And returns can be wrapped in their own RAII type that handles freeing all the memory and handles contained (if you don't explicitly move particular buffers or handles out).

---

Switching it now would be setting this back, as I have to rework... most of it. Increases the convenience of the bindings, but increases the complexity of the generator.

But better to do it before merging, rather than breaking things later?

[QuantumSegfault]

After some more thought, I think I'll just keep going with the current model, since I'm so close to having something working available.

I might switch things up later. Need to get more feedback first.