[V1] Support LLM.apply_model

[DarkLight1337]

This enables a bunch of tests to be run in V1

[DarkLight1337]

@youkaichao can you review this?

[DarkLight1337]

This part has been moved inside WorkerBase

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[DarkLight1337]

And also @mgoin since this touches the quantization tests

[youkaichao]

LGTM in general

[DarkLight1337]

@njhill I'm unable to get tests/models/multimodal/generation/test_qwen2_vl.py::test_qwen2_vl_multiple_image_embeddings_input[10-128-half-size_factors1-Qwen/Qwen2-VL-2B-Instruct] to pass - the output of apply_model only has the tensor's dtype and shape rather than the tensor data. I think this is related to the msgspec encoding/decoding logic. Can you help take a look?

[mgoin]

This looks reasonable to me. A section in the docs or example script would be useful to demonstrate the interface

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @DarkLight1337.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[patrick-toulme]

Why has this not been merged? vLLM has no way right now to easily access the underlying model. That is a rather basic feature.

[DarkLight1337]

There is an issue with the msgspec serialization that needs to be fixed by @njhill before this PR can be merged.

[DarkLight1337]

cc @WoosukKwon heads-up that we need to merge this PR before V1 can be removed entirely in order to keep the quantization tests

[njhill]

@DarkLight1337 OK I finally got to this! #25294

[fxmarty-amd]

@DarkLight1337 are you familiar with errors as: tests/kernels/moe/test_ocp_mx_moe.py::test_mxfp4_loading_and_execution_moe[model_case0] - Exception: Call to collective_rpc method failed: Can't get local object 'test_mxfp4_loading_and_execution_moe.<locals>.check_model'? How should we fix?

[fxmarty-amd]

e.g. in

vllm/tests/kernels/moe/test_mxfp4_moe.py

Lines 63 to 79 in d83f3f7

	with vllm_runner(model_case.model_id,
	tensor_parallel_size=model_case.tp,
	load_format="dummy") as llm:
	def check_model(model):
	layer = model.model.layers[0]
	qkv_proj = layer.self_attn.qkv_proj
	assert isinstance(qkv_proj.quant_method, QuarkLinearMethod)
	assert isinstance(qkv_proj.scheme, QuarkW4A4MXFP4)
	assert isinstance(layer.mlp.experts.quant_method,
	QuarkW4A4MXFp4MoEMethod)
	if model_case.model_id == "fxmarty/qwen_1.5-moe-a2.7b-mxfp4":
	llm.apply_model(check_model)

[DarkLight1337]

Try moving the imports inside the inner function

[fxmarty-amd]

Do you mean QuarkOCP_MX_MoEMethod, QuarkLinearMethod, QuarkOCP_MX imports? It does not seem to work, I'll disable for now

[DarkLight1337]

Can you show the full stack trace of the error?