feat: Lazy engine initialization

[narendasan]

Description

Allows engines to not be setup immediately after compilation but all at once before the module is returned back to the user.

Fixes #2673

Type of change

Please delete options that are not relevant and/or add your own.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

Checklist:

• My code follows the style guidelines of this project (You can use the linters)
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas and hacks
• I have made corresponding changes to the documentation
• I have added tests to verify my fix or my feature
• New and existing unit tests pass locally with my changes
• I have added the relevant labels to my PR in so that relevant reviewers are notified

[narendasan]

Remove

[peri044]

LGTM. Could you add a testcase with a model that has fallback ops ?

[zewenli98]

LGTM

[github-actions]

There are some changes that do not conform to Python style guidelines:

--- /home/runner/work/TensorRT/TensorRT/tests/py/dynamo/conversion/harness.py	2024-07-31 15:42:55.193957+00:00
+++ /home/runner/work/TensorRT/TensorRT/tests/py/dynamo/conversion/harness.py	2024-07-31 15:44:51.304037+00:00
@@ -62,11 +62,11 @@
        interpreter,
        rtol,
        atol,
        check_dtype=True,
        pyt_inputs=None,
-        rt_cls=PythonTorchTensorRTModule
+        rt_cls=PythonTorchTensorRTModule,
    ):
        with torch.no_grad():
            cuda_inputs = []
            for i in inputs:
                cuda_inputs.append(i.cuda())
@@ -132,11 +132,11 @@
        inputs,
        expected_ops,
        interpreter,
        comparators: List[Tuple[Callable, List]],
        fp16_mode=False,
-        rt_cls=PythonTorchTensorRTModule
+        rt_cls=PythonTorchTensorRTModule,
    ):
        """
        Runs the test and compares the result using the provided comparators.
        The size of comparators must be equal to the number of outputs from 'mod'.

[zewenli98]

LGTM, just a few minor comments