You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docsrc/ts/getting_started_with_cpp_api.rst
+16-14Lines changed: 16 additions & 14 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -100,7 +100,7 @@ As you can see it is pretty similar to the Python API. When you call the ``forwa
100
100
101
101
Compiling with Torch-TensorRT in C++
102
102
-------------------------------------
103
-
We are also at the point were we can compile and optimize our module with Torch-TensorRT, but instead of in a JIT fashion we must do it ahead-of-time (AOT) i.e. before we start doing actual inference work
103
+
We are also at the point where we can compile and optimize our module with Torch-TensorRT, but instead of in a JIT fashion we must do it ahead-of-time (AOT) i.e. before we start doing actual inference work
104
104
since it takes a bit of time to optimize the module, it would not make sense to do this every time you run the module or even the first time you run it.
105
105
106
106
With our module loaded, we can feed it into the Torch-TensorRT compiler. When we do so we must provide some information on the expected input size and also configure any additional settings.
@@ -113,9 +113,10 @@ With our module loaded, we can feed it into the Torch-TensorRT compiler. When we
113
113
114
114
mod.to(at::kCUDA);
115
115
mod.eval();
116
-
117
-
auto in = torch::randn({1, 1, 32, 32}, {torch::kCUDA});
118
-
auto trt_mod = torch_tensorrt::CompileGraph(mod, std::vector<torch_tensorrt::CompileSpec::InputRange>{{in.sizes()}});
0 commit comments