prometheus/tsdb/chunks
Peter Štibraný ffc60d8397
Reduce chunk write queue memory usage 2 (#10874)
* Job queue

This PR reimplements chan chunkWriteJob with custom buffered queue that should use less memory, because it doesn't preallocate entire buffer for maximum queue size at once. Instead it allocates individual "segments" with smaller size.

As elements are added to the queue, they fill individual segments. When elements are removed from the queue (and segments), empty segments can be thrown away. This doesn't change memory usage of the queue when it's full, but should decrease its memory footprint when it's empty (queue will keep max 1 segment in such case).

Signed-off-by: Peter Štibraný <pstibrany@gmail.com>

* Modify test to work with low resolution timer.

Signed-off-by: Peter Štibraný <pstibrany@gmail.com>

* Improve comments.

Signed-off-by: Peter Štibraný <pstibrany@gmail.com>
2022-06-29 17:51:27 +05:30
..
chunk_write_queue_test.go Fix panic on query when m-map replay fails with snapshot enabled (#10348) 2022-02-25 08:53:40 -07:00
chunk_write_queue.go Reduce chunk write queue memory usage 2 (#10874) 2022-06-29 17:51:27 +05:30
chunks_test.go Testify: move to require (#8122) 2020-10-29 09:43:23 +00:00
chunks.go tsdb: faster CRC check by avoiding allocations (#10789) 2022-06-08 08:00:59 +05:30
head_chunks_other.go tsdb/chunks: preallocate at least some space on non-Windows systems (#9581) 2021-11-09 13:47:00 +05:30
head_chunks_test.go refactor: move from io/ioutil to io and os packages (#10528) 2022-04-27 11:24:36 +02:00
head_chunks_windows.go Format Go source files using 'gofumpt -w -s -extra' 2021-11-02 19:52:34 +01:00
head_chunks.go Correctly format error message (#10669) 2022-05-06 00:42:31 +02:00
queue_test.go Reduce chunk write queue memory usage 2 (#10874) 2022-06-29 17:51:27 +05:30
queue.go Reduce chunk write queue memory usage 2 (#10874) 2022-06-29 17:51:27 +05:30