Skip to content

TypeError: 'builtins.safe_open' object is not iterable #1733

@jpeig

Description

@jpeig
/vllm/vllm/model_executor/weight_utils.py", line 243, in hf_model_weights_iterator
    for name in f:
TypeError: 'builtins.safe_open' object is not iterable

I am using the main branch build.
Triggers when using /openai/api_server.py and loading the model weights for the engine object (AsyncLLMEngine)

Traceback:

Traceback (most recent call last):
  File "runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "api_server.py", line 644, in <module>
    engine = AsyncLLMEngine.from_engine_args(engine_args)
  File "async_llm_engine.py", line 486, in from_engine_args
    engine = cls(parallel_config.worker_use_ray,
  File "async_llm_engine.py", line 269, in __init__
    self.engine = self._init_engine(*args, **kwargs)
  File "async_llm_engine.py", line 305, in _init_engine
    return engine_class(*args, **kwargs)
  File "llm_engine.py", line 110, in __init__
    self._init_workers(distributed_init_method)
  File "llm_engine.py", line 142, in _init_workers
    self._run_workers(
  File "llm_engine.py", line 700, in _run_workers
    output = executor(*args, **kwargs)
  File "worker.py", line 70, in init_model
    self.model = get_model(self.model_config)
  File "model_loader.py", line 98, in get_model
    model.load_weights(model_config.model, model_config.download_dir,
  File "mistral.py", line 307, in load_weights
    for name, loaded_weight in hf_model_weights_iterator(
  File "weight_utils.py", line 243, in hf_model_weights_iterator
    for name in f:
TypeError: 'builtins.safe_open' object is not iterable

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions