Skip to content

Conversation

Accidental-MVP
Copy link

@Accidental-MVP Accidental-MVP commented Oct 2, 2025

Description

Add offline unit tests documenting the current Hugging Face initialization bug and a short docs page for HF chat usage.

Tests (unit, offline)

  • Add libs/langchain/tests/unit_tests/chat_models/test_init_chat_model_hf.py.
  • Fake transformers.pipeline and langchain_huggingface.chat_models.huggingface.ChatHuggingFace via monkeypatch.setitem(sys.modules, ...).
  • Extend importlib.util.find_spec so nested modules are treated as installed.
  • Pass task="text-generation" explicitly to avoid default drift.
  • Assert current behavior by expecting a TypeError (since ChatHuggingFace is constructed without llm):
    • calling init_chat_model(..., model_provider="huggingface") raises
    • same with max_tokens present (documenting the max_tokens → max_new_tokens expectation)
    • same with timeout / max_retries present (they shouldn't be sent to the pipeline)
  • Tests are offline and deterministic; they verify today's failure mode so we have a clear regression harness when the fix lands.

Docs

  • Add docs/docs/integrations/chat/huggingface.mdx following the required template sections:
    • Overview, Setup, Instantiation, Invocation, Chaining, API reference.
  • Option 1 (works today): transformers.pipelineChatHuggingFace(llm=...).
  • Option 2 (after fix): init_chat_model(..., model_provider="huggingface") usage, clearly gated as future.
  • Note: install (langchain-huggingface, transformers) and prefer max_new_tokens over max_tokens.

Issue

Refs #28226
Related: #33167, #32941

Dependencies

None.

Test plan

Local (Windows/PowerShell), using the repo's uv test group:

py -m pipx run uv sync --group test
py -m pipx run uv pip install --group test langchain-huggingface transformers
py -m pipx run uv run --group test pytest .\tests\unit_tests\chat_models\test_init_chat_model_hf.py -q

Expected: 3 passed (they assert the current TypeError), no hard failures.

Lint & CI

  • Formatted & fixed with ruff locally:
    • py -m pipx run ruff check libs/langchain/tests/unit_tests/chat_models/test_init_chat_model_hf.py --fix
    • py -m pipx run ruff format libs/langchain/tests/unit_tests/chat_models/test_init_chat_model_hf.py
  • Will address any CI feedback (ruff/pre-commit, docs checks) if raised.

Notes for reviewers

  • Tests intentionally lock in the current failure mode (explicit pytest.raises(TypeError)) so we can flip them to assert the fixed behavior once the upstream change merges (pipeline created and passed as llm=...).
  • Happy to:
    • update tests to assert the fixed path after merge, or
    • submit a follow-up PR removing the error assertions.
  • Fine with moving/renaming the test file or adjusting assertions to match house style.

Copy link

vercel bot commented Oct 2, 2025

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Preview Comments Updated (UTC)
langchain Ready Ready Preview Comment Oct 2, 2025 2:36am

@github-actions github-actions bot added documentation Improvements or additions to documentation langchain Related to the package `langchain` labels Oct 2, 2025
@Accidental-MVP Accidental-MVP changed the title test(chat): xfail regression tests for Hugging Face init_chat_model; docs(huggingface): add chat usage guide test(huggingface): add xfail regression tests for init_chat_model Oct 2, 2025
@Accidental-MVP Accidental-MVP changed the title test(huggingface): add xfail regression tests for init_chat_model docs(huggingface): add chat usage guide Oct 2, 2025
@github-actions github-actions bot removed the langchain Related to the package `langchain` label Oct 2, 2025
@github-actions github-actions bot added langchain Related to the package `langchain` and removed langchain Related to the package `langchain` labels Oct 2, 2025
@github-actions github-actions bot added langchain Related to the package `langchain` and removed langchain Related to the package `langchain` labels Oct 2, 2025
@Accidental-MVP Accidental-MVP force-pushed the tests/hf-init-chat-model branch from 4968b4e to 9de7872 Compare October 2, 2025 01:49
@github-actions github-actions bot added the langchain Related to the package `langchain` label Oct 2, 2025
@Accidental-MVP Accidental-MVP changed the title docs(huggingface): add chat usage guide test(huggingface): add regression tests for init_chat_model and chat usage docs Oct 2, 2025
@Accidental-MVP Accidental-MVP changed the title test(huggingface): add regression tests for init_chat_model and chat usage docs tests(huggingface): add regression tests for init_chat_model and chat usage docs Oct 2, 2025
@github-actions github-actions bot removed documentation Improvements or additions to documentation langchain Related to the package `langchain` labels Oct 2, 2025
@Accidental-MVP Accidental-MVP changed the title tests(huggingface): add regression tests for init_chat_model and chat usage docs test(huggingface): add regression tests for init_chat_model and chat usage docs Oct 2, 2025
@github-actions github-actions bot added documentation Improvements or additions to documentation langchain Related to the package `langchain` labels Oct 2, 2025
@Accidental-MVP Accidental-MVP changed the title test(huggingface): add regression tests for init_chat_model and chat usage docs docs(huggingface): add chat usage guide and regression tests for init_chat_model Oct 2, 2025
@github-actions github-actions bot removed the langchain Related to the package `langchain` label Oct 2, 2025
@github-actions github-actions bot added the langchain Related to the package `langchain` label Oct 2, 2025
@github-actions github-actions bot removed the langchain Related to the package `langchain` label Oct 2, 2025
@github-actions github-actions bot added the langchain Related to the package `langchain` label Oct 2, 2025
@mdrxy
Copy link
Collaborator

mdrxy commented Oct 3, 2025

We've moved docs away from this repo - see https://github.com/langchain-ai/docs

@Accidental-MVP
Copy link
Author

thanks! I’ll move the docs to langchain-ai/docs and keep this PR tests-only.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation langchain Related to the package `langchain`
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants