What's the correct way to use Steel's own dylibs?

[tesujimath]

I am interested in using steel-regex in my own application. This is not installed by cargo xtask install.

I was able to use it by doing this:

aya> cd libs/steel-regex
aya> cargo steel-lib
    Finished `release` profile [optimized + debuginfo] target(s) in 1m 18s
Found a cdylib!
Copying /home/sjg/vc/third-party/steel/target/release/libsteel_regex.so to /home/sjg/.steel/native/libsteel_regex.so
Done!

and successfully used it like this:

λ > (#%require-dylib "libsteel_regex" (only-in regex regex/match?))
λ > (define abc-re (regex "abc"))
λ > (regex/match? abc-re "xyz")
=> #false
λ > (regex/match? abc-re "xyzabcz")
=> #true

My question is about how to package this up for use in my application so that my own users don't have to grapple with such things themselves.

For several of Steel's dylibs there is an associated Scheme file which contains the #%require-dylib such as I used above, e.g. steel-rustls has one. This is currently missing for steel-regex (steel-regex/regex.scm is an empty file). This Scheme wrapper hides from the user that the library is a dylib rather than a simple Scheme cog, so such wrappers are clearly required.

I think what is needed is addition of the missing Scheme wrappers, than a cargo xtask rule to build and install them.

Is this right?

(I also see quite a bit of infrastructure for package management, but I am unclear on the status of this, and whether it is intended for Steel's own native dylibs or just for third party packages. There are a couple of old issues touching on this, but I think things have moved on since then.)

[mattwparas]

Long story short - there is a package manager forge which should make all of this easier, but I'm currently working on a PR that should make things better #354

Ideal state - there is a cog.scm file associated with your project, and running forge install will do all of the things.

Generally, I've been trying to set up dylibs to have a cog.scm file, and also an associated scheme wrapper that is generated during the dylib build process, but this isn't complete for every dylib that I've set up yet. For example, this is generally how I'd like things to be set up: https://github.com/mattwparas/steel-editor-config/blob/master/src/main.rs

I'm debating whether the main should happen in the build file vs just as a thing to run.

Does this make sense sort of as an update of the status quo? In general I'm open to feed back and would also love if you'd be willing to dog food the forge tool to see if it fits your use case.

[mattwparas]

There is even a registry so that the forge tool knows how to fetch and install any libraries registered in the "blessed" registry, as well as anything via git url - https://github.com/mattwparas/steel-packages - so I've been cooking on this one for a while 😄

[tesujimath]

Thanks, forge sounds great, I'll give it a go and report back!

[mattwparas]

Fair bit of warning - there aren't any docs really, I'm trying to finish off this latest PR before getting those up. Happy to answer any questions you may have if you feel like spelunking