[LLM] fix doc test for Working with LLMs guide #55917
Conversation
- fix rst/python syntax issues causing crashes - fix pytest and datasets compatibility issues[LLM] - patch VLM example (dependency sisue) and OpenAI API example - conditional execution based on OPENAI_API_KEY / demo mode Signed-off-by: Nikhil Ghosh <[email protected]>
Signed-off-by: Nikhil Ghosh <[email protected]>
nrghosh
changed the title
| # Now handle Ray's compatibility issue by patching the dynamic modules function | ||
| import datasets.load | ||
|
|
||
| # Create a compatibility wrapper for the removed init_dynamic_modules function |
There was a problem hiding this comment.
does some hacky stuff for package version mismatch issues - otherwise tough to have all these examples in the same place.
|
|
||
| vision_processed_ds = vision_processor(vision_dataset).materialize() | ||
| vision_processed_ds.show(3) | ||
| # For doctest, we'll just set up the processor without running the full dataset |
There was a problem hiding this comment.
was running into resource exhaustion when running this example (after debugging format / syntax / package versions)
There was a problem hiding this comment.
Does this snippet need GPUs?
There was a problem hiding this comment.
yes, expects L4 - but this doesn't actually load/run the model (to avoid that issue). Is there a different way we want to handle this?
nrghosh
added
go
|
Actually @nrghosh let's do a similar thing to what @eicherseiji did in this PR. The idea being, create the full working doc-test code outside of the doc rst files. In the rst files use includeliterals and separately run proper tests on those scripts however you'd like without having to rely on any particular infrastructure parsing the code outside of docs. This approach makes our code examples clean hiding the infra code that is needed for running and setting up the test envs. e.g. it will remove the need for |
ack @kouroshHakha will take a second pass with those changes |
nrghosh
force-pushed
the
nrghosh/llms-doctest
branch
from
ad64964 to
156d469
Compare
- Move infrastructure code from RST to standalone Python files with literalinclude - Add run_test() functions and fix transformers version mismatch for RoPE config - Replace complex module imports with direct configuration validation in testcode - Prevent GPU memory errors by avoiding inference execution during tests - Maintain clean separation: RST shows code, testcode validates configs only - Keep 1:1 functionality with the original RST code example file Signed-off-by: Nikhil Ghosh <[email protected]>
nrghosh
force-pushed
the
nrghosh/llms-doctest
branch
from
156d469 to
70db35f
Compare
Signed-off-by: Nikhil Ghosh <[email protected]>
Signed-off-by: Nikhil Ghosh <[email protected]>
Signed-off-by: Nikhil Ghosh <[email protected]>
nrghosh
force-pushed
the
nrghosh/llms-doctest
branch
3 times, most recently
from
d4d6b58 to
cb7fc7b
Compare
- check that bazel pytest section actually does run the python test files - remote testcode blocks from .rst and replace with literalinclude / code-block references Signed-off-by: Nikhil Ghosh <[email protected]>
Signed-off-by: Nikhil Ghosh <[email protected]>
nrghosh
force-pushed
the
nrghosh/llms-doctest
branch
from
5cd2198 to
f71704a
Compare
Signed-off-by: Nikhil Ghosh <[email protected]>
Signed-off-by: Nikhil Ghosh <[email protected]>
Signed-off-by: Nikhil Ghosh <[email protected]>
Signed-off-by: Nikhil Ghosh <[email protected]>
Signed-off-by: Nikhil Ghosh <[email protected]>
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Signed-off-by: Nikhil Ghosh <[email protected]>
Signed-off-by: Nikhil Ghosh <[email protected]>
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Signed-off-by: Nikhil Ghosh <[email protected]>
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Signed-off-by: Nikhil Ghosh <[email protected]>
Signed-off-by: Nikhil Ghosh <[email protected]>
Signed-off-by: Nikhil Ghosh <[email protected]>
Signed-off-by: Nikhil Ghosh <[email protected]>
Signed-off-by: Nikhil Ghosh <[email protected]>
There was a problem hiding this comment.
- addressed all detailed comments
- fixed version / code example issues
- refactored code examples out of the main doc
- unblocked tests from waiting for GPUs in CI
Working With LLMs Doc rendering - https://anyscale-ray--55917.com.readthedocs.build/en/55917/data/working-with-llms.html
Signed-off-by: Nikhil Ghosh <[email protected]> Signed-off-by: Marco Stephan <[email protected]>
Signed-off-by: Nikhil Ghosh <[email protected]> Signed-off-by: elliot-barn <[email protected]>
Signed-off-by: Nikhil Ghosh <[email protected]> Signed-off-by: Douglas Strodtman <[email protected]>
Signed-off-by: Nikhil Ghosh <[email protected]>
Signed-off-by: Nikhil Ghosh <[email protected]>
7 participants
LLM Doctest Fix
working-with-llms.rstthat were previously brokenliteralincludereferences for detailed implementationBUILD.bazel, documentation focuses on user guidance and code snippetsNotes
doc/source/dataanddoc/source/ray-overview/examplesdirectories, which is whyquick-start.rst(indoc/source/serve/llm/) can usecode-block:: pythonbut our .rst file cannotdata: doc gpu tests(premerge)Log
Why are these changes needed?
In order to enable Working with LLMs docs / example code
Related issue number
addresses #55796
Checks
git commit -s) in this PR.scripts/format.shto lint the changes in this PR.method in Tune, I've added it in
doc/source/tune/api/under thecorresponding
.rstfile.Unit Test
pytest -vs --doctest-modules doc/source/data/working-with-llms.rstOutput: