chore: add trtllm-serve json schema example into doc. #6418
Conversation
📝 Walkthrough""" WalkthroughA new example script demonstrating the use of the OpenAI Python client with a JSON schema response format was added. Additionally, a pytest fixture for an asynchronous client was removed from a test file, and a test function was updated to include the new example script in its parameterization. Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~8 minutes Suggested labels
Suggested reviewers
📜 Recent review detailsConfiguration used: .coderabbit.yaml 📒 Files selected for processing (3)
💤 Files with no reviewable changes (1)
✅ Files skipped from review due to trivial changes (1)
🚧 Files skipped from review as they are similar to previous changes (1)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
✨ Finishing Touches
🧪 Generate unit tests
🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
coderabbitai
bot
added
the
Doc
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
examples/serve/openai_completion_client_json_schema.py (2)
16-17: Fix unnecessary line break in role assignment.The line break between
"role":and"user"is unnecessary and affects readability.- "role": - "user", + "role": "user",
19-19: Remove unnecessary f-string formatting.The f-string is unnecessary since no variables are being formatted into the string.
- f"Give me the information of the biggest city of China in the JSON format.", + "Give me the information of the biggest city of China in the JSON format.",
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
examples/serve/openai_completion_client_json_schema.py(1 hunks)tests/unittest/llmapi/apps/_test_openai_chat_json.py(0 hunks)
💤 Files with no reviewable changes (1)
- tests/unittest/llmapi/apps/_test_openai_chat_json.py
🧰 Additional context used
📓 Path-based instructions (2)
**/*.py
📄 CodeRabbit Inference Engine (CODING_GUIDELINES.md)
**/*.py: Python code should conform to Python 3.8+.
Indent Python code with 4 spaces. Do not use tabs.
Always maintain the namespace when importing in Python, even if only one class or function from a module is used.
Python filenames should use snake_case (e.g., some_file.py).
Python classes should use PascalCase (e.g., class SomeClass).
Python functions and methods should use snake_case (e.g., def my_awesome_function():).
Python local variables should use snake_case, and prefix k for variable names that start with a number (e.g., k_99th_percentile).
Python global variables should use upper snake_case and prefix G (e.g., G_MY_GLOBAL).
Python constants should use upper snake_case (e.g., MY_CONSTANT).
Avoid shadowing variables declared in an outer scope in Python.
Initialize all externally visible members of a Python class in the constructor.
For interfaces that may be used outside a Python file, prefer docstrings over comments.
Comments in Python should be reserved for code within a function, or interfaces that are local to a file.
Use Google style docstrings for Python classes and functions, which can be parsed by Sphinx.
Attributes and variables in Python can be documented inline; attribute docstrings will be rendered under the docstring for the class.
Avoid using reflection in Python when functionality can be easily achieved without it.
When using try-except blocks in Python, limit the except to the smallest set of errors possible.
When using try-except blocks to handle multiple possible variable types in Python, keep the body of the try as small as possible, using the else block to implement the logic.
Files:
examples/serve/openai_completion_client_json_schema.py
**/*.{cpp,h,hpp,cc,cxx,cu,py}
📄 CodeRabbit Inference Engine (CODING_GUIDELINES.md)
All TensorRT-LLM Open Source Software code should contain an NVIDIA copyright header that includes the current year. This includes .cpp, .h, .cu, .py, and any other source files which are compiled or interpreted.
Files:
examples/serve/openai_completion_client_json_schema.py
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Pre-commit Check
🔇 Additional comments (3)
examples/serve/openai_completion_client_json_schema.py (3)
3-8: LGTM! Proper import and client setup for example.The import follows namespace preservation guidelines, and the client initialization uses appropriate example values for demonstration purposes.
23-40: Well-structured JSON schema demonstration.The JSON schema configuration effectively demonstrates the feature with a clear object structure, required fields, and TensorRT-LLM specific chat template options.
42-42: LGTM! Appropriate response handling for example.The response output follows the standard OpenAI client pattern and is suitable for this demonstration script.
|
/bot run |
|
PR_Github #13221 [ run ] triggered by Bot |
|
PR_Github #13221 [ run ] completed with state |
nv-guomingz
force-pushed
the
user/guomingz/update_trtllm-serve_doc
branch
from
a8c6819 to
1861014
Compare
|
/bot run |
|
PR_Github #13270 [ run ] triggered by Bot |
|
PR_Github #13270 [ run ] completed with state |
nv-guomingz
force-pushed
the
user/guomingz/update_trtllm-serve_doc
branch
from
1861014 to
2bd193b
Compare
|
/bot run |
|
PR_Github #13286 [ run ] triggered by Bot |
Sure, let me update the code. |
nv-guomingz
force-pushed
the
user/guomingz/update_trtllm-serve_doc
branch
from
2bd193b to
ecc6650
Compare
|
PR_Github #13286 [ run ] completed with state |
nv-guomingz
force-pushed
the
user/guomingz/update_trtllm-serve_doc
branch
from
ecc6650 to
a873f32
Compare
nv-guomingz
force-pushed
the
user/guomingz/update_trtllm-serve_doc
branch
from
a873f32 to
1dabd0a
Compare
|
/bot run |
|
PR_Github #13384 [ run ] triggered by Bot |
|
PR_Github #13384 [ run ] completed with state |
nv-guomingz
force-pushed
the
user/guomingz/update_trtllm-serve_doc
branch
from
1dabd0a to
6bed9d2
Compare
|
/bot run |
|
PR_Github #13483 [ run ] triggered by Bot |
|
PR_Github #13483 [ run ] completed with state |
Signed-off-by: nv-guomingz <[email protected]>
nv-guomingz
force-pushed
the
user/guomingz/update_trtllm-serve_doc
branch
from
6bed9d2 to
8c50fe7
Compare
|
/bot reuse-pipeline |
|
PR_Github #13512 [ reuse-pipeline ] triggered by Bot |
|
PR_Github #13512 [ reuse-pipeline ] completed with state |
Signed-off-by: nv-guomingz <[email protected]> Signed-off-by: Lanyu Liao <[email protected]>
Signed-off-by: nv-guomingz <[email protected]>
5 participants
It's a supplement PR of #6321.
Summary by CodeRabbit
New Features
Tests