Skip to content

Introduce (reflective) Arb testing with Kotest #10880

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Sep 23, 2025
Merged

Introduce (reflective) Arb testing with Kotest #10880

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e4a1ece
build: Introduce the Kotest property testing dependency
sschuberth Sep 22, 2025
96c0cb3
test(composer): Avoid relying on the current test directory
sschuberth Sep 22, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions buildSrc/src/main/kotlin/ort-kotlin-conventions.gradle.kts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@ testing {
implementation(project(":utils:test-utils"))

implementation(libs.kotest.assertions.core)
implementation(libs.kotest.property)
Copy link
Member

@fviernau fviernau Sep 22, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just curious: What is the experiment after? What is the unkown and how can we make use of outcome?

I've a bit doubts that generating "meaningful" test data, is valuable for parameter values which do not matter. E.g. the need to emphasize (by using arbs) could also be a code smell hinting at suboptimal clarity.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've a bit doubts that generating "meaningful" test data, is valuable for parameter values which do not matter. E.g. the need to emphasize (by using arbs) could also be a code smell hinting at suboptimal clarity.

I see it pretty much as the opposite: Arb (which is short for "arbitrary") deliberately creates "unmeaningful" test data to emphasize that the test should work for any value, and not just for the hard-coded one. This also creates a chance to uncover edge-case bugs in tests where a test only works for an existing hard-coded value, whereas it should generally work. It's a little bit like fuzzing.

So I see it as clarity being added by deliberately using arbitrary / random values.

What is the experiment after? What is the unkown and how can we make use of outcome?

Eventually, I hope to get rid of e.g. some EMPTY constants that we have for many data classes. That's because these constants are very often only used in tests, and in places where the data should not actually be empty, but where we do not care about concrete values, so actually the data should be arbitrary.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but let's get a second approval.

implementation(libs.kotest.runner.junit5)
}
}
Expand Down
1 change: 1 addition & 0 deletions gradle/libs.versions.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,7 @@ kotest-extensions = { module = "io.kotest:kotest-extensions", version.ref = "kot
kotest-extensions-junitXml = { module = "io.kotest:kotest-extensions-junitxml", version.ref = "kotest" }
kotest-framework-datatest = { module = "io.kotest:kotest-framework-datatest", version.ref = "kotest" }
kotest-framework-engine = { module = "io.kotest:kotest-framework-engine", version.ref = "kotest" }
kotest-property = { module = "io.kotest:kotest-property", version.ref = "kotest" }
kotest-runner-junit5 = { module = "io.kotest:kotest-runner-junit5", version.ref = "kotest" }
kotlinpoet = { module = "com.squareup:kotlinpoet", version.ref = "kotlinPoet"}
kotlinpoet-ksp = { module = "com.squareup:kotlinpoet-ksp", version.ref = "kotlinPoet"}
Expand Down
5 changes: 4 additions & 1 deletion plugins/package-managers/composer/src/funTest/kotlin/ComposerFunTest.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,9 @@
import io.kotest.matchers.should
import io.kotest.matchers.shouldBe
import io.kotest.matchers.string.haveSubstring
import io.kotest.property.Arb
import io.kotest.property.arbitrary.file
import io.kotest.property.arbitrary.single

Check warning on line 31 in plugins/package-managers/composer/src/funTest/kotlin/ComposerFunTest.kt

View workflow job for this annotation

GitHub Actions / qodana-scan

Unused import directive 

Unused import directive

import java.io.File

Expand All @@ -39,7 +42,7 @@
class ComposerFunTest : StringSpec({
"Project files from vendor directories are ignored" {
val projectFiles = ComposerFactory.create().mapDefinitionFiles(
File("."),
Arb.file().single(),
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mnonnenmacher, I found a little bit of a nicer syntax here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's better, but if Kotest would provide a function like arbitraryFile() (and arbitraryFiles() for the not single case) it would make the tests more readable.

listOf(
"projectA/composer.json",
"projectA/vendor/dependency1/composer.json",
Expand Down
Loading