Add EnzymeTestUtils package for testing Enzyme rules #782
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov ReportPatch coverage:
❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more. Additional details and impacted files@@ Coverage Diff @@
## main #782 +/- ##
==========================================
+ Coverage 77.41% 77.91% +0.49%
==========================================
Files 19 27 +8
Lines 8897 9137 +240
==========================================
+ Hits 6888 7119 +231
- Misses 2009 2018 +9
☔ View full report in Codecov by Sentry. |
sethaxen
commented
May 18, 2023
|
|
|
@wsmoses @vchuravy the tests now pass on v1.7 and v1.9. I had to add a lot of exceptions (tests marked as skip or broken) to work around version-specific errors and failures that occur when Enzyme differentiates Julia code. Not all are captured by issues, but I don't really have time right now to open issues with MWEs and errors for them all. Reverse-mode tests could use more tests for batch mode, but these could be added in a later PR. I suggest that this is ready for merge and initial package registration. |
wsmoses
approved these changes
Aug 31, 2023
|
Thanks @wsmoses! Looks like it stilled needs a registration with |
|
Error while trying to register: Register Failed |
michel2323
pushed a commit
that referenced
this pull request
Nov 7, 2023
* Add initial forward-mode tester * Rework auto_forward_activity * Refactor to pass activities to _make_jvp_call * Generalize rand_tangent * Update test_approx * Add zero_tangent * Don't return tuple if return argument is not a tuple * Support batched activities * Update test_forward * Update comments and docstring * Add all_or_no_batch utility * Return NamedTuple * Return zero tangent instead of nothing * Add zero_tangent and rand_tangent for (Named)Tuples * Add test_forward tests * Test multi-argument function * Don't call autodiff with kwargs * Add meta-testing functions * Test kwargs correctly forwarded * Test different types of errors are caught * Deactivate failing test * Reformat * Use MetaTesting code * Test complex cases * Remove redundant test * Remove redundant comment * Update docstring * Add ConstructionBase as a test dependency * Fix construction * Rename function * Rename function * Add test_reverse * Add test_reverse tests * Avoid trig functions * Only pass adjoint for Active * Resolve comment * Add docstring * Test multi-arg mutating function * Avoid exp * Test multi-arg function separately * Generalize test_reverse * Remove limitation * Fix map * Test mutated callable * Add broken test for mutated callable * Run formatter * Update f_kwargs rules * Test that testers catch incorrect reverse-mode rules * Add missing import * Add and update docstrings * Update Project.toml * Test tester tests * Rename Foo * Clarify limitations * Add more clarification * Add EnzymeTestUtils package * Remove module file * Move testers to module file * Make EnzymeTestUtils module * Add package deps and extras * Create runtests.jl * Add Random to Project.toml * Remove duplicated code * Add missing include * Remove unneeded import * Duplicate code * Remove redundant code * Duplicate code * Remove redundant code * Add more useful function name * Rename file * Add EnzymeTestUtils to the API docs * Add docs example for EnzymeTestUtils * Turn docstrings for private functions into comments * Dev EnzymeTestUtils for docs build * Uncomment test * Reformat * Limit what's imported to namespace * Copy tests * Remove redundant code * Add EnzymeTestUtils workflow stage * Set project globally * Run in Julia shell * Remove test deps * Handle nonnumeric types in test_approx * Add tests for test_approx * Add tests for compatible activities * Add tests for tangent generation * Test auto_activity * Reuse test helper * Adapt output_control from ChainRulesTestUtils * Add messages to test_approx * Make test failures more informative * Improve test messages * Bump EnzymeCore version bound * Rename functions * Define _any_batch utility * Remove use of invoke * Mark broken tests as broken * Avoid constructor with types * Skip ComplexF32 for mutated callable tests * Remove broken annotation * Simplify broken annotation * Simplify CI code * Fix string generation * Avoid more complicated elementary functions * Fix tests for v1.7 * Fix tests for v1.9 * Fixes for batch in reverse-mode * Expand examples in docstrings * Add batch size computation * Batchify zero tangent vectors * Make batch cotangents if necessary * Add batch array test * Avoid broadcast * Define f_mut_nothing! * Avoid error caused by runtime activity * Test batch mutating cases * Test mutated callable in batch mode * Update documented constraints * Copy code * Split testers into two files * Split testers tests across files * Support Active combined with batch activities * Test more compatible activities * Update function call * Remove NamedTuple wrapping * Run formatter * Update function name * Add exception for v1.7 --------- Co-authored-by: Valentin Churavy <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds functions
test_forwardandtest_reverse, which support testing results ofautodiffagainst FiniteDifferences. Fixes #780Edit:
test_forwardandtest_reverseare patterned off oftest_fruleandtest_rrulein ChainRulesTestUtils.