todo: enable CI to run torchdynamo/tensorrt tests

[stas00]

@ydshieh, let's talk about instrumenting one of the jobs to run tests for torchdynamo/tensorrt

It's quite a handful of things to build, the instructions are in the OP:

#17765

I installed the environment locally, the tests work. I didn't want to get in the way of the PR, so doing it in a separate task.

thank you.

[ydshieh]

Hi, @stas00

Sure! It sounds that you have already some tests written, and we just want to run them with CI, right? Let me know how you would like to proceed (discussion or PR review etc.).

[ydshieh]

Hi @stas00 Let me know if we should proceed :-)

[stas00]

Yes, please. Please let me know how I can help.

Thank you, @ydshieh!

[ydshieh]

Hi! Basically I don't know what the task is. I didn't go through #17765 in detail, and only knows that PR merged.

Is the (new) goal to install some libraries in docker files, write some specific test methods, or something else.

You mentioned I installed the environment locally, the tests work.. It would be nice if you can let me know which test you mean here, and if you want me to installed the environment for the scheduled CI(s) to run those tests 🙏

Thank you, @stas00 !

[stas00]

1. The instructions of what needs to be installed are verbatim in the OP of Enable torchdynamo with torch_tensorrt(fx path) #17765

2. To test:

pytest tests/trainer/test_trainer.py -k torchdynamo

[ydshieh]

OK, I get it better now. So basically I need to

• install what mentioned in To reproduce and set up the environment section in Enable torchdynamo with torch_tensorrt(fx path) #17765 inside some docker files
• have a job to run torchdynamo tests

Do you think it's better to have a new docker image for this (just like deepspeed), or we can just put it in the transformers-all-latest-gpu (the one used for mode/tokenizer/generation tests)? I can try the later first in any case.

[stas00]

I'd say one docker image for all the extensions.

[stas00]

At some point it will become stable and will work with just the official released version.

[ydshieh]

@stas00 I have tried to build the image and run the tests.

Here is the run page

It failed with

>           from torchdynamo.optimizations.training import aot_autograd_speedup_strategy
E           ImportError: cannot import name 'aot_autograd_speedup_strategy' from 'torchdynamo.optimizations.training'

I can't find aot_autograd_speedup_strategy in the latest torchdynamo repo.

cc @frank-wei

[stas00]

Thank you for trying, @ydshieh. It's too bad that the API appears to be unstable :(

Let's wait for @frank-wei to reply

[frank-wei]

torchdynamo repo

Thanks for setting up the testing @stas00 and @ydshieh
Looks like @Chillee or @anijain2305 had made some updates there for AOTAutoGrad. Could you elaborate here?

[anijain2305]

We have cleaned up some Aot Autograd related things in TorchDynamo repo.

transformers/src/transformers/trainer.py

Line 652 in 4eed2be

return torchdynamo.optimize(aot_autograd_speedup_strategy)

The above line could be replaced with

return torchdynamo.optimize("aot_nvfuser")

Therefore, we do not need the import anymore on line 645

[stas00]

We have cleaned up some Aot Autograd related things in TorchDynamo repo.

transformers/src/transformers/trainer.py

Line 652 in 4eed2be

return torchdynamo.optimize(aot_autograd_speedup_strategy)

The above line could be replaced with

return torchdynamo.optimize("aot_nvfuser")

1. could you please make a PR that fixes things.

2. could you please include the relevant transformers tests in your CI, so that if you break things in the future you'd instantly know and then update the transformers side? Thank you.

e.g. you can see how Deepspeed runs transformers/deepspeed integration tests on their CI https://github.com/microsoft/DeepSpeed/blob/master/.github/workflows/nv-transformers-v100.yml

In your case it'd be cloning the latest transformers repo and running:

pytest tests/trainer/test_trainer.py -k torchdynamo