Unit / Integration Tests are Broken

[sgp]

Expected Behavior

I should be able to run make test on a system with either Docker Compose or Podman (and its equivalent) and have all the tests pass in master.

Current Behavior

See test-output.txt from a run on my system.

I'm willing to believe that this could be a compatibility problem with using Podman over Docker Compose, but it's hard to see how.

I do note that there appears to be a setup that works enough for Github CI, but ... I don't think it's using this containerized environment at all.

Possible Solution

1. The --save "" directive in docker-compose.yml seems to have the double-quotes escaped by the shell for three of the five environments fails; using --save \"\" seems to fix that so the Redis VMs start up.
2. Fixing the above still causes some tests to not be able to connect; uncommenting the commented out ports: and removing the network: "host" line seemed better, but still broken.

Here's the diff of what I did above proposed-change-diff.txt,
as well as output from running make test after those changes: test-output-better.txt.

Steps to Reproduce

Just run make test as outlined in CONTRIBUTING.md. 🤷

On my system, I'm using Podman. Output from podman version:

❯ podman version
Client:        Podman Engine
Version:       5.6.0
API Version:   5.6.0
Go Version:    go1.25.0
Built:         Fri Aug 15 09:42:35 2025
Build Origin:  brew
OS/Arch:       darwin/arm64

Server:       Podman Engine
Version:      5.4.2
API Version:  5.4.2
Go Version:   go1.23.7
Git Commit:   be85287fcf4590961614ee37be65eeb315e5d9ff
Built:        Tue Apr  1 20:00:00 2025
OS/Arch:      linux/arm64

Context (Environment)

I'm just trying to develop patches and contribute to the project, and not having an test environment makes this riskier as I am unable to test my changes. This impacts all contributors to this project.

Detailed Description

Make the tests run clean.

Possible Implementation

I mean, fix it? (Or direct me to what is going on here so I can help?)

[ndyakov]

@sgp thank you for reporting this. I personally am using docker on osx host and all the tests on master are green. I do use either make docker.start and make test.ci or directly make test. Does your setup support the host network for the docker containers?

p.s. there are some env variables that contribute to the test execution, I am looking at them right now.

[ndyakov]

@sgp feel free to test with #3491 where the Makefile includes the default env variables and will use the latest build for the testing container.

[ndyakov]

@sgp any updates on your side? After rereading your issue, we do need the host network for some of the sentinel tests to work properly at the moment. Do you think we can close this issue and reopen it with more details related to why we need the host network and see if we can find a workaround for it? If I remember correctly, the main reason is that the sentinel will return you an address it observes, which may be a private docker address and if the host (on which the tests are running) is not part of that network it won't be able to connect.

[sgp]

One thing that wasn't addressed in this issue was the fact that the --save options were malformed due to the fact that that the shell seems to eat the unescaped "" argument. I wonder if this is a podman bug, though, since you are using genuine Docker, and we are not.

[ndyakov]

@sgp I do think the issue you are experiencing is related to podman, we do not have any issues with running the tests with Docker. If you are able to find a way to run the tests with podman (and you will need the host network for them to work) feel free to open a PR and we can verify if it continues to work with docker.