Skip to content

Conversation

@mattt
Copy link
Collaborator

@mattt mattt commented Sep 24, 2025

Earlier this month, I onboarded to help the team get swift-tranformers to a 1.0 release. As part of that, I've been talking to folks on Twitter about what they'd like to see. Something that a few folks mentioned was improving chat templates to make them faster, compatible with more models, and more resilient to errors.

The original implementation had some architectural constraints that were hard to work around incrementally, namely:

  • Protocol-based AST nodes causing dispatch overhead
  • Reference semantics in value types leading to unnecessary allocations

This PR introduces the following changes:

  • AST Redesign: Migrated from protocol-based AST nodes to enum-based Node and Expression types, eliminating dynamic dispatch overhead and enabling value semantics throughout the evaluation pipeline.
  • Value System: Introduced a unified Value enum that handles all runtime types (strings, numbers, arrays, objects, functions, macros)
  • Interpreter Optimization: New interpreter implementation with improved error handling, scope management, and evaluation performance.
  • Modular Organization: Separated concerns into focused modules (Filters.swift, Tests.swift, Globals.swift) for better maintainability and testing.

Based on initial benchmarks, this new implementation shows 4–10x performance improvement for end-to-end chat template processing, including several order-of-magnitude improvement in tokenization performance.

The public API of this rewrite is very close to the current version, so despite introducing major breaking changes, upgrading requires minimal changes at call sites.

To verify the behavior of this new implementation, this PR ported the original tests and incorporated new test cases from huggingface.js's Jinja package.

Copy link
Member

@pcuenca pcuenca left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing work! 🔥

@mattt mattt merged commit 80d3567 into main Sep 24, 2025
2 checks passed
@mattt mattt deleted the v2 branch September 25, 2025 11:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants