Skip to content

Conversation

mattt
Copy link
Collaborator

@mattt mattt commented Sep 10, 2025

This PR migrates most of our tests from XCTest to the new Swift Testing framework, which has better ergonomics and performance.

@mattt mattt mentioned this pull request Sep 11, 2025
@pcuenca pcuenca added the v1.0 label Sep 17, 2025
@mattt mattt force-pushed the mattt/swift-testing branch from 3135ab1 to d448cfa Compare September 17, 2025 16:25
@mattt mattt marked this pull request as ready for review September 17, 2025 19:30
@mattt mattt force-pushed the mattt/swift-testing branch 2 times, most recently from 80245da to a1f455d Compare September 23, 2025 10:51
@mattt mattt force-pushed the mattt/swift-testing branch from a1f455d to 2ff616f Compare September 23, 2025 11:20
@mattt mattt force-pushed the mattt/swift-testing branch from 3c181ca to 6b95a5d Compare September 23, 2025 12:03
@mattt mattt requested a review from pcuenca September 23, 2025 13:16
Copy link
Member

@pcuenca pcuenca left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great to me from a quick look – I did visit the individual commits and the history makes sense to me.

Are parallel tokenizer tests still revealing race conditions?

XCTFail("Unexpected error: \(error)", file: file, line: line)
}
/// Check if two floating-point arrays are equal within a given accuracy
func isClose<T: FloatingPoint>(_ lhs: [T], _ rhs: [T], accuracy: T) -> Bool {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice. This could become a general utility function later, like Pytorch or MLX allClose.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that would be nice. Official guidance from the Swift team for replacing the XCTest assertion with accuracy is to pull in swift-numerics to use their implementation of isApproximatelyEqual. Pulling that in as a separate dependency just for tests is overkill, but we could certainly vendor it.

@pcuenca
Copy link
Member

pcuenca commented Sep 23, 2025

Are parallel tokenizer tests still revealing race conditions?

Ah, I see they work now!

@mattt
Copy link
Collaborator Author

mattt commented Sep 23, 2025

Are parallel tokenizer tests still revealing race conditions?

Ah, I see they work now!

Yes, that's the race conditions I was mentioning before. There's a more comprehensive change we can do to make that reentrancy safe, but for now it's much easier to ensure that we don't have more than one test calling AutoTokenizer.from(pretrained:) with the same model.

@mattt
Copy link
Collaborator Author

mattt commented Sep 23, 2025

Ooh, nice — after making everything parallel again, we're now down to 3 minutes 40 seconds 🚀

@mattt mattt merged commit d14f43f into main Sep 23, 2025
2 checks passed
@mattt mattt deleted the mattt/swift-testing branch September 23, 2025 13:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants