"commons-pool-evictor" randomly throws "Unexpected end of stream.".

[nripendra]

Expected behavior

Make it possible to handle these type of error, may be even retry with exponential back-off.

Actual behavior

Thread name: "commons-pool-evictor"

redis.clients.jedis.exceptions.JedisConnectionException: Unexpected end of stream.
	at redis.clients.jedis.util.RedisInputStream.ensureFill(RedisInputStream.java:202)
	at redis.clients.jedis.util.RedisInputStream.readByte(RedisInputStream.java:43)
	at redis.clients.jedis.Protocol.process(Protocol.java:165)
	at redis.clients.jedis.Protocol.read(Protocol.java:230)
	at redis.clients.jedis.Connection.readProtocolWithCheckingBroken(Connection.java:352)
	at redis.clients.jedis.Connection.getStatusCodeReply(Connection.java:270)
	at redis.clients.jedis.BinaryJedis.ping(BinaryJedis.java:384)
	at redis.clients.jedis.JedisFactory.validateObject(JedisFactory.java:214)
	at org.apache.commons.pool2.impl.GenericObjectPool.evict(GenericObjectPool.java:745)
	at org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor.run(BaseGenericObjectPool.java:160)
	at org.apache.commons.pool2.impl.EvictionTimer$WeakRunner.run(EvictionTimer.java:113)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

Steps to reproduce:

No steps to reproduce, we are seeing these exception randomly in our error logs, and the thread name is "commons-pool-evictor"

Redis / Jedis Configuration

• RedisStandaloneConfiguration
  • standard configuration with hostname, port, password and database set to 0.
• JedisClientConfiguration
  • connectTimeout: 30s
  • readTimeout: 10s
  • useSsl
  • usePooling
• PoolConfig
  • maxTotal: 500
  • maxIdle: 100
  • minIdle: 50
  • testWhileIdle: true
  • minEvictableIdleTime: 1 minute
  • timeBetweenEvictionRuns: 20 seconds
  • numTestsPerEvictionRun -1
  • testOnBorrow: true

Jedis version:

• 3.8.0

Redis version:

• Azure enterprise redis

Java version:

• zulujdk11 unprivileged, 11.0.13_3

[sazzad16]

Make it possible to handle these type of error, may be even retry with exponential back-off.

Jedis just provides an implementation of validateObject in JedisFactory. But the actual handling is done by commons-pool (version 2) library. At current state, we can't entertain this request.

@nripendra You may ask commons-pool for your feature.

[ranjoy001]

@nripendra Did you fine any workaround to get rid of this issue i am facing something similar here - #4028 let me know if you found any solution if would be so much of help thanks!!

[nripendra]

@ranjoy001 this issue isn't resolved for me. It's been quite sometime since this issue. If I remember correctly, the workaround I implemented was to apply exponential backoff retry using spring boot's retrytemplate. The number of failures logged for this error has significantly reduced for us.

Hope this helps.