Skip to content

Conversation

jmid
Copy link
Member

@jmid jmid commented Sep 18, 2025

On a number of PRs I've observed tezt failures to run with-test, e.g., on #28437
https://opam.ci.ocaml.org/github/ocaml/opam-repository/commit/c56e58d25e43ce7dd35f9942ad32b42e22d608c9

tezt.3.1.0 (failed: The compilation of tezt.3.1.0 failed at "dune runtest -p tezt -j 255".)
tezt.3.1.1 (failed: The compilation of tezt.3.1.1 failed at "dune runtest -p tezt -j 255".)
tezt.4.0.0 (failed: The compilation of tezt.4.0.0 failed at "dune runtest -p tezt -j 255".)
tezt.4.1.0 (failed: The compilation of tezt.4.1.0 failed at "dune runtest -p tezt -j 255".)
tezt.4.2.0 (failed: The compilation of tezt.4.2.0 failed at "dune runtest -p tezt -j 71".)

and now again today on #28544
https://opam.ci.ocaml.org/github/ocaml/opam-repository/commit/8165ebc987d9de35b2fdba552fd80e37f5bd842d

The issue turns out to be caused by dune.3.19.0 changing the observed value of Sys.argv.(0) - which happens to be printed in the test suite (in cram tests) of tezt 3.1.0-4.2.0 ocaml/dune#11881

I've confirmed locally that opam install tezt --with-test installs cleanly with dune.3.18.2 and below.
According to the above issue the change of behaviour should be rolled back with dune.3.19.1, however
locally with ocaml.4.14.2, dune.3.19.1, and tezt.4.2.0 I still observe test failures:

#=== ERROR while compiling tezt.4.2.0 =========================================#
# context     2.3.0 | linux/x86_64 | ocaml.4.14.2 | https://opam.ocaml.org#bcac5d17315c7dd28b93a43f52e43d98094b72ec
# path        ~/.opam/4.14.2~opam-repo/.opam-switch/build/tezt.4.2.0
# command     ~/.opam/opam-init/hooks/sandbox.sh build dune runtest -p tezt -j 7
# exit-code   1
# env-file    ~/.opam/log/tezt-223492-ba7ffd.env
# output-file ~/.opam/log/tezt-223492-ba7ffd.out
### output ###
# [...]
# index 35cfd4b..46f7fdc 100644
# --- a/_build/default/test/cram/test-timeout.t
# +++ b/_build/default/test/cram/test-timeout.t.corrected
# @@ -49,7 +49,7 @@ But one can force it to die:
#    [error] Sending SIGKILL to test which is taking too long to stop: "sleep one second without cooperating"
#    [error] worker was killed by SIGKILL
#    [FAILURE] (1/1, 1 failed) sleep one second without cooperating
# -  Try again with: _build/default/main.exe --verbose --file test/cram/main.ml --title 'sleep one second without cooperating'
# +  Try again with: ./_build/default/main.exe --verbose --file test/cram/main.ml --title 'sleep one second without cooperating'

I've therefore gone with a hard < 3.19.0 bound for all 5 tezt version where we've observed the failure.
Note: this is not a hard bound - only a bound for where we run the tezt testsuite.

@jmid
Copy link
Member Author

jmid commented Sep 19, 2025

Adding these bounds reveal additional run-test failures that I need to look into.
I've therefore added the do not merge label.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant