[RFC] Rust Support

[tqchen]

There are two parts of the things that can be done in rust:

• Rust "frontend" as in java/js, that wraps the current C API and expose a rust programming model
  • This allows most users to use tvm and its runtime(gpu/vulkan support etc)
  • Need to link runtime(by c++, or "backend" in next part)
• Rust "backend" which serves as an alternative to current C++ runtime.
  • It gives a standalone WASM module that even cpp does not yet offer(due to stdc++)
  • Security stuffs e.g. SGX

Both have values and I think we should have a good discussion on how can we organize them.

• Add rust runtime #1597 is mainly about "backend"
• @ehsanmok has implemented a version of "frontend"

cc @jroesch @nhynes @ehsanmok

[nhynes]

From an organizational perspective, we can have

tvm/rust/
    Cargo.toml  # a virtual manifest
    frontend/  # what @ehsanmok has been developing
        Cargo.toml
        src/
    runtime/  # what's currently in /tvm/rust/runtime
         Cargo.toml

the runtime/ would continue to be the tvm package on crates.io and the frontend could be named elsewise.

[ehsanmok]

I agree @nhynes and I'd love to hear your feedbacks of my frontend when is completed as this is my first major Rust project and still learning a lot. The same to @jroesch as well :)

[tqchen]

Please also comment on official code-styles that we should generally follow in rust package

[nhynes]

Please also comment on official code-styles that we should generally follow in rust package

The .rustfmt.toml currently used in #1597 is the default produced by rustup --dump-config for rustfmt version 0.6.1. We should just use the default unless there's a good reason not to.

[jroesch]

I would be happy to jump in and do some Rust hacking in the future with you guys, but am pretty busy with some new TVM stuff @tqchen and I have been working on. I'm happy to provide feedback and code review though, until I have a few more cycles available. Current incremental plan to getting started with Rust sounds good to me.

[tqchen]

rust runtime is merged in #1597

[tqchen]

Here are a few next steps:

• Add document generation to the ci flow
  • see https://github.com/dmlc/tvm/blob/master/tests/scripts/task_python_docs.sh#L10 which includes jsdoc and c++ docs
  • see results in https://docs.tvm.ai/api_links.html
• Add a README.md to rust with a few get started examples
• Rust fronend if @ehsanmok is interested in bring that in as well

[nhynes]

regarding docs, the conventional approach is to simply upload each release of the tvm crate to crates.io which will automatically build the docs and post them on docs.rs

[tqchen]

OK, we can do that then if that is the conventional approach of rust

[tqchen]

Rust CI is up and running

[ehsanmok]

@tqchen Sorry, I was quite detached for some time and trying to finish it as soon as I can in my free time. I'm working on ergonimic registration of callbacks in Rust, when that's done I'd probably need help for the RPC as looking at the python and java RPC it's seems they have been sort of done from scratch!

[nhynes]

RPC [...] seems they have been sort of done from scratch!

Now that we're on the topic, might it be a good time to start experimenting with a more established framework like grpc? There's even a nice crate for it!

I may be completely incorrect in saying this, but I think that many of the roll-your-own components are a result of C++ having a terrible package ecosystem. For instance, the dmlc-core json parser, the memory stream reader used for params serdes, the thread-local store. In Rust, these become serde, nom, and thread_local!, respectively.

In some version of reality, the TVM compiler is rewritten in Rust (with clear separation from the python components!) and is much more maintainable because of it.

[tqchen]

We roll out our own RPC because the requirement is somewhat different from grpc. We need a lightweight stateful session that persists over multiple function calls, it also helps porting into devices like android.

So there is a reason for rolling out the rpc module. I agree that json parser and thread local are somewhat unnecessary(thread_local is already supported in c++11 and it is there only for old compilers)