Introduce (reflective) Arb testing with Kotest #10880
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
plugins/package-managers/composer/src/funTest/kotlin/ComposerFunTest.kt
Dismissed
Show dismissed
Hide dismissed
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #10880 +/- ##
=========================================
Coverage 57.53% 57.53%
+ Complexity 1700 1698 -2
=========================================
Files 346 346
Lines 12823 12823
Branches 1212 1212
=========================================
Hits 7378 7378
Misses 4978 4978
Partials 467 467
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
sschuberth
force-pushed
the
arb-file-testing
branch
from
ab21e3a to
8fe3cbe
Compare
| implementation(project(":utils:test-utils")) | ||
|
|
||
| implementation(libs.kotest.assertions.core) | ||
| implementation(libs.kotest.property) |
There was a problem hiding this comment.
just curious: What is the experiment after? What is the unkown and how can we make use of outcome?
I've a bit doubts that generating "meaningful" test data, is valuable for parameter values which do not matter. E.g. the need to emphasize (by using arbs) could also be a code smell hinting at suboptimal clarity.
There was a problem hiding this comment.
I've a bit doubts that generating "meaningful" test data, is valuable for parameter values which do not matter. E.g. the need to emphasize (by using arbs) could also be a code smell hinting at suboptimal clarity.
I see it pretty much as the opposite: Arb (which is short for "arbitrary") deliberately creates "unmeaningful" test data to emphasize that the test should work for any value, and not just for the hard-coded one. This also creates a chance to uncover edge-case bugs in tests where a test only works for an existing hard-coded value, whereas it should generally work. It's a little bit like fuzzing.
So I see it as clarity being added by deliberately using arbitrary / random values.
What is the experiment after? What is the unkown and how can we make use of outcome?
Eventually, I hope to get rid of e.g. some EMPTY constants that we have for many data classes. That's because these constants are very often only used in tests, and in places where the data should not actually be empty, but where we do not care about concrete values, so actually the data should be arbitrary.
There was a problem hiding this comment.
LGTM, but let's get a second approval.
See [1]. Going forward, as an experiment some tests will be converted to use arbitrary (reflective) values [2] where no fixed empty / dummy values are strictly needed, to emphasize the invariance on those inputs. [1]: https://kotest.io/docs/proptest/property-based-testing.html [2]: https://kotest.io/docs/proptest/reflective-arbs.html Signed-off-by: Sebastian Schuberth <[email protected]>
Emphasize that the `File` argument does not matter for the test by using an arbitrary value instead. Signed-off-by: Sebastian Schuberth <[email protected]>
sschuberth
force-pushed
the
arb-file-testing
branch
from
8fe3cbe to
96c0cb3
Compare
| "Project files from vendor directories are ignored" { | ||
| val projectFiles = ComposerFactory.create().mapDefinitionFiles( | ||
| File("."), | ||
| Arb.file().single(), |
There was a problem hiding this comment.
@mnonnenmacher, I found a little bit of a nicer syntax here.
There was a problem hiding this comment.
It's better, but if Kotest would provide a function like arbitraryFile() (and arbitraryFiles() for the not single case) it would make the tests more readable.
Please have a look at the individual commit messages for the details.