Import v2 sources and tests #26
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Earlier this month, I onboarded to help the team get swift-tranformers to a 1.0 release. As part of that, I've been talking to folks on Twitter about what they'd like to see. Something that a few folks mentioned was improving chat templates to make them faster, compatible with more models, and more resilient to errors.
The original implementation had some architectural constraints that were hard to work around incrementally, namely:
This PR introduces the following changes:
NodeandExpressiontypes, eliminating dynamic dispatch overhead and enabling value semantics throughout the evaluation pipeline.Valueenum that handles all runtime types (strings, numbers, arrays, objects, functions, macros)Filters.swift,Tests.swift,Globals.swift) for better maintainability and testing.Based on initial benchmarks, this new implementation shows 4–10x performance improvement for end-to-end chat template processing, including several order-of-magnitude improvement in tokenization performance.
The public API of this rewrite is very close to the current version, so despite introducing major breaking changes, upgrading requires minimal changes at call sites.
To verify the behavior of this new implementation, this PR ported the original tests and incorporated new test cases from huggingface.js's Jinja package.