Fix: Bulk processor retries indefinitely on failure #1661
Conversation
When all retries are exhausted the worker internal requests buffer needs to be cleared in failure scenarios. That is required because the commitFunc (and consequently the underlying BulkService.Do call) doesn't reset it when some error happens. Without clearing the internal buffer the worker will continue sending the same requests on the following rounds of execution. Kudos for this solution goes to @rwynn and @raiRaiyan . Resolves olivere#1278
|
Hey @olivere , as this repository is deprecated, is it not accepting patches anymore? |
|
Yes. This repository is deprecated and basically dormant. You are strongly encouraged to switch to the official client. |
|
I suspect merging this PR would solve this problem in Jaeger: jaegertracing/jaeger#2192. |
|
@jstasiak I understand your frustration. But please see it through my eyes. My decision to stop this project was to send a clear signal to the users of this library to move to the official client supported by Elastic, the company. They took over years ago, now. Sure, I could merge the PR. But that might open up a new stream of changes, or have side-effects that I can't foresee and don't have the energy to work on any more. Another choice for jaeger would be to fork this repository and keep their own version of the library. I'm of course perfectly OK with this. Thanks for your understanding. |
|
Hey @olivere! I totally understand this repository is not maintained and I didn't want to appear as if I was putting any pressure on anyone or expecting anything - I'm not. No need to explain anything. I left links with notes here and in #1278 because I thought they could be useful to someone else investigating this. Not much frustration here, just sharing what I found. Thank you for all your work on this and have a great day. |
3 participants
When all retries are exhausted the worker internal requests buffer needs to be cleared in failure scenarios. That is required because the
commitFunc(and consequently the underlyingBulkService.Docall) doesn't reset it when some error happens. Without clearing the internal buffer the worker will continue sending the same requests on the following rounds of execution.Kudos for this solution goes to @rwynn and @raiRaiyan .
Resolves #1278