Fix flaky test TestEthClient

[rjl493456442]

Flaky test TestEthClient

FAIL github.com/ethereum/go-ethereum/ethclient 600.327s

panic: test timed out after 10m0s
	running tests:
		TestEthClient (10m0s)
		TestEthClient/AtFunctions (10m0s)

---

--- FAIL: TestTransactionRollbackBehavior (10.15s)
    rollback_test.go:53: failed to send transaction: already known
FAIL

Guidance for reproducing the issues: #29830

[arnabnandikgp]

I would like to try this! thanks.

[arnabnandikgp]

hey @rjl493456442 actually am new to geth, and having a hard time trying to reproduce the flakiness of the above tests like running the tests multiple times or even in parallel are not helping...can you help me in this regard.

[jwasinger]

@arnabnandikgp The stress tool can help here.

[kashitaka]

@rjl493456442 Hi, Took a quick look and it seemed like a simple fix, so I went ahead and opened a PR: #32280
Let me know if the direction looks good to you.

I noticed that someone else is interested in working on this, so I tried to be mindful not to step on anyone's toes. Hope that's okay — just wanted to help move things forward. Thanks!

[rjl493456442]

@arnabnandikgp checkout #29830 for more information!

[kashitaka]

@rjl493456442 Thank you for merging #32280 and we can close here.

[rjl493456442]

@kashitaka We still have this one

panic: test timed out after 10m0s
	running tests:
		TestEthClient (10m0s)
		TestEthClient/AtFunctions (10m0s)

want to pick it up?

[kashitaka]

@rjl493456442 Ah, I missed that. Ok I'll take a look at it.

[kashitaka]

@rjl493456442 I’ve identified the cause of the failing test, but it appears to stem from a deeper issue. I would appreciate if you have any advice.

The cause is in the loop here:

https://github.com/ethereum/go-ethereum/blob/master/ethclient/ethclient_test.go#L513-L523

PendingTransactionCount() never returns 1, causing the test to hang and eventually timeout after 10 minutes. I could reproduce this issue locally and confirmed.

Even in failure cases, the preceding sendTransaction() appears to successfully send the transaction, but it never reaches the pending state.

A suspicious behavior is that the nonce used for the transaction is retrieved using ec.NonceAt(), and it returns 2 in both successful and failing cases. However, when I inspected the result of ec.PendingNonceAt(context.Background(), testAddr) just before sending, I saw that it returned 2 in the success case, but 0 in the failure case. This might indicate that the two transactions added here:

https://github.com/ethereum/go-ethereum/blob/master/ethclient/ethclient_test.go#L134-L138

were never actually included in the failure case, meaning there might be deeper instability in how test-backend is initialized.

I thought getting nonce using PendingNonceAt() might help, as it returns nonce 0 which matches initial state of backend, but PendingTransactionCount keeps returning 0.

One workaround would be to retry this test several times and pass it if it succeeds once… though I know that’s not an excellent solution.

I’d appreciate hearing your thoughts on this.