You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[Doc] Add I/O binding example using onnx data type in python API summary (microsoft#22695)
### Description
Add I/O binding example using onnx data type in python API summary. The
API is available since 1.20 release.
### Motivation and Context
Follow up of microsoft#22306 to add
some documentation.
Copy file name to clipboardExpand all lines: docs/python/api_summary.rst
+28-1Lines changed: 28 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -244,9 +244,36 @@ You can also bind inputs and outputs directly to a PyTorch tensor.
244
244
)
245
245
246
246
session.run_with_iobinding(binding)
247
-
247
+
248
248
You can also see code examples of this API in in the `ONNX Runtime inferences examples <https://github.com/microsoft/onnxruntime-inference-examples/blob/main/python/api/onnxruntime-python-api.py>`_.
249
249
250
+
Some onnx data type (like TensorProto.BFLOAT16, TensorProto.FLOAT8E4M3FN and TensorProto.FLOAT8E5M2) are not supported by Numpy. You can directly bind input or output with Torch tensor of corresponding data type
251
+
(like torch.bfloat16, torch.float8_e4m3fn and torch.float8_e5m2) in GPU memory.
252
+
253
+
.. code-block:: python
254
+
255
+
x = torch.ones([3], dtype=torch.float8_e5m2, device='cuda:0')
256
+
y = torch.empty([3], dtype=torch.bfloat16, device='cuda:0')
0 commit comments