* storage/remote: disable resharding during active retry backoffs
Today, remote_write reshards based on pure throughput. This is
problematic if throughput has been diminished because of HTTP 429s;
increasing the number of shards due to backpressure will only exacerbate
the problem.
This commit disables resharding for twice the retry backoff, ensuring
that resharding will never occur during an active backoff, and that
resharding does not become enabled again until enough time has elapsed
to allow any pending requests to be retried.
Signed-off-by: Robert Fratto <robertfratto@gmail.com>
* storage/remote: test that resharding is disabled on retry
Signed-off-by: Robert Fratto <robertfratto@gmail.com>
* storage/remote: address review feedback
Signed-off-by: Robert Fratto <robertfratto@gmail.com>
* storage/remote: track time where resharding initially got disabled
This change introduces a second atomic int64 to roughly track when
resharding got disabled. This int64 is only updated after updating the
disabled timestamp if resharding was previously enabled.
Signed-off-by: Robert Fratto <robertfratto@gmail.com>
---------
Signed-off-by: Robert Fratto <robertfratto@gmail.com>
a09465baee