Skip to content

Conversation

@oandreeva-nv
Copy link
Contributor

@oandreeva-nv oandreeva-nv commented Nov 1, 2023

To set model transaction policy from Python backend's autocomple function, I need to add this field on core-side config, when TritonModel::UpdateModelConfig is called. TritonModel::UpdateModelConfig seem to be called only for auto_complete tasks.

@tanmayv25
Copy link
Contributor

@oandreeva-nv You might want to update the title of the PR to remove the Oandreeva.
Also, try to make the the title of the PRs more generic.. Such as in this case: Allow backends to auto-complete model transaction policy.
In the description, you can call out that this is needed for vLLM auto-complete support.

@oandreeva-nv oandreeva-nv changed the title Oandreeva auto complete for vllm Allow backends to auto-complete model transaction policy Nov 2, 2023
@oandreeva-nv oandreeva-nv force-pushed the oandreeva_auto_complete_for_vllm branch from 24be4e9 to 50d8910 Compare November 2, 2023 19:31
@oandreeva-nv oandreeva-nv merged commit 90309ef into main Nov 2, 2023
@oandreeva-nv oandreeva-nv deleted the oandreeva_auto_complete_for_vllm branch November 2, 2023 21:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

5 participants