RepoMirrors/prometheus
1
0
Fork 0
mirror of https://github.com/prometheus/prometheus synced 2024-12-28 09:42:22 +00:00
Code Issues Packages Projects Releases Wiki Activity
ffc60d8397
prometheus/tsdb
History
Peter Štibraný ffc60d8397
Reduce chunk write queue memory usage 2 (#10874) 
* Job queue

This PR reimplements chan chunkWriteJob with custom buffered queue that should use less memory, because it doesn't preallocate entire buffer for maximum queue size at once. Instead it allocates individual "segments" with smaller size.

As elements are added to the queue, they fill individual segments. When elements are removed from the queue (and segments), empty segments can be thrown away. This doesn't change memory usage of the queue when it's full, but should decrease its memory footprint when it's empty (queue will keep max 1 segment in such case).

Signed-off-by: Peter Štibraný <pstibrany@gmail.com>

* Modify test to work with low resolution timer.

Signed-off-by: Peter Štibraný <pstibrany@gmail.com>

* Improve comments.

Signed-off-by: Peter Štibraný <pstibrany@gmail.com>
2022-06-29 17:51:27 +05:30
..
agent unit tests: make all Labels sorted alphabetically (#10532) 2022-05-04 23:41:36 +02:00
chunkenc Add SyncForState Implementation for Ruler HA (#10070) 2022-03-29 02:16:46 +02:00
chunks Reduce chunk write queue memory usage 2 (#10874) 2022-06-29 17:51:27 +05:30
docs Update godoc links and remove note about TSDB versioning (#10754) 2022-05-26 18:34:43 +10:00
encoding Format Go source files using 'gofumpt -w -s -extra' 2021-11-02 19:52:34 +01:00
errors Address golint failures from revive 2021-10-23 00:53:11 +02:00
fileutil Make necessary change to compile promql parser to wasm (#10683) 2022-05-12 09:12:05 +02:00
goversion Add new Go build tags. 2021-08-27 10:24:14 +02:00
index Fix typo in symbol table size exceeded error message (#10746) 2022-05-25 10:40:36 +02:00
record Add basic initial developer docs for TSDB (#9451) 2021-11-17 15:51:27 +05:30
test Move packages out of deprecated pkg directory 2021-11-09 08:03:10 +01:00
testdata
tombstones refactor: move from io/ioutil to io and os packages (#10528) 2022-04-27 11:24:36 +02:00
tsdbutil refactor: move from io/ioutil to io and os packages (#10528) 2022-04-27 11:24:36 +02:00
wal Remote Write: Rename confusing walDir parameter to dir (#10464) 2022-05-30 21:45:30 -07:00
.gitignore
block_test.go unit tests: make all Labels sorted alphabetically (#10532) 2022-05-04 23:41:36 +02:00
block.go fix typo (#10696) 2022-05-25 18:01:45 +02:00
blockwriter_test.go refactor: use T.TempDir() and B.TempDir to create temporary directory 2022-01-22 18:57:30 +08:00
blockwriter.go refactor: move from io/ioutil to io and os packages (#10528) 2022-04-27 11:24:36 +02:00
CHANGELOG.md Rename default branch to main 2021-02-22 20:28:02 +01:00
compact_test.go refactor: use T.TempDir() and B.TempDir to create temporary directory 2022-01-22 18:57:30 +08:00
compact.go CompactBlockMetas should produce correct mint/maxt for overlapping blocks. (#10108) 2022-01-05 15:10:00 +05:30
db_test.go test(tsdb): break up repeated test to avoid timeout (#10671) 2022-05-09 00:39:26 +02:00
db.go Don't increment prometheus_tsdb_compactions_failed_total when context is canceled (#10772) 2022-06-17 11:21:43 +05:30
example_test.go refactor: move from io/ioutil to io and os packages (#10528) 2022-04-27 11:24:36 +02:00
exemplar_test.go unit tests: make all Labels sorted alphabetically (#10532) 2022-05-04 23:41:36 +02:00
exemplar.go tsdb: use simpler map key to improve exemplar ingest performance (#10111) 2022-01-06 15:58:58 +05:30
head_append.go Fix chunk overflow appending samples at a variable rate (#10607) 2022-04-20 14:54:20 +02:00
head_bench_test.go refactor: use T.TempDir() and B.TempDir to create temporary directory 2022-01-22 18:57:30 +08:00
head_read.go Label values with matchers by intersecting postings (#9907) 2021-12-28 15:59:03 +01:00
head_test.go unit tests: make all Labels sorted alphabetically (#10532) 2022-05-04 23:41:36 +02:00
head_wal.go tsdb: reduce sleep time when reading WAL (#10859) 2022-06-12 11:54:11 +05:30
head.go fix(tsdb): inc mmap corruption counter on mmap out of sequence error (#10406) 2022-06-22 14:03:12 +05:30
isolation_test.go Add the ability to disable tsdb isolation (#9270) 2021-11-19 15:41:32 +05:30
isolation.go Add the ability to disable tsdb isolation (#9270) 2021-11-19 15:41:32 +05:30
mocks_test.go Move packages out of deprecated pkg directory 2021-11-09 08:03:10 +01:00
querier_bench_test.go refactor: use T.TempDir() and B.TempDir to create temporary directory 2022-01-22 18:57:30 +08:00
querier_test.go refactor: use T.TempDir() and B.TempDir to create temporary directory 2022-01-22 18:57:30 +08:00
querier.go Label values with matchers by intersecting postings (#9907) 2021-12-28 15:59:03 +01:00
README.md Update godoc links and remove note about TSDB versioning (#10754) 2022-05-26 18:34:43 +10:00
repair_test.go refactor: use T.TempDir() and B.TempDir to create temporary directory 2022-01-22 18:57:30 +08:00
repair.go refactor: move from io/ioutil to io and os packages (#10528) 2022-04-27 11:24:36 +02:00
tsdbblockutil.go tsdb: commit data periodically in CreateBlock (#10788) 2022-06-17 11:26:19 +05:30
wal_test.go refactor: move from io/ioutil to io and os packages (#10528) 2022-04-27 11:24:36 +02:00
wal.go fix potential goroutine leaks 2021-12-17 18:35:30 -05:00

README.md

TSDB

GoPkg

This directory contains the Prometheus TSDB (Time Series DataBase) library, which handles storage and querying of all Prometheus v2 data.

Documentation

External resources

A series of blog posts explaining different components of TSDB: