prometheus/tsdb
Marco Pracucci 501bc6419e
Add ShardedPostings() support to TSDB (#10421)
This PR is a reference implementation of the proposal described in #10420.

In addition to what described in #10420, in this PR I've introduced labels.StableHash(). The idea is to offer an hashing function which doesn't change over time, and that's used by query sharding in order to get a stable behaviour over time. The implementation of labels.StableHash() is the hashing function used by Prometheus before stringlabels, and what's used by Grafana Mimir for query sharding (because built before stringlabels was a thing).

Follow up work
As mentioned in #10420, if this PR is accepted I'm also open to upload another foundamental piece used by Grafana Mimir query sharding to accelerate the query execution: an optional, configurable and fast in-memory cache for the series hashes.

Signed-off-by: Marco Pracucci <marco@pracucci.com>
2024-01-29 11:57:27 +00:00
..
agent agent: add BenchmarkCreateSeries 2023-12-31 10:23:43 +00:00
chunkenc Optimize histogram iterators (#13340) 2024-01-23 17:02:14 +01:00
chunks Optimize histogram iterators (#13340) 2024-01-23 17:02:14 +01:00
docs
encoding tsdb/encoding: use Go standard errors package 2023-11-11 19:01:11 +01:00
errors Enable default revive rules (#13068) 2023-11-29 17:23:34 +00:00
fileutil remove obsolete build tag 2024-01-17 22:26:32 +08:00
goversion remove obsolete build tag 2024-01-17 22:26:32 +08:00
index Add ShardedPostings() support to TSDB (#10421) 2024-01-29 11:57:27 +00:00
record Move metric type definitions to common/model 2023-12-19 18:56:54 +00:00
testdata
tombstones Update tombstones.go 2023-11-11 19:22:06 +01:00
tsdbutil Revert "Adding small test update for temp dir using t.TempDir (#13293)" 2023-12-30 19:17:30 +00:00
wlog tsdb: use Go standard errors 2023-12-11 12:18:54 +00:00
.gitignore
block_test.go TestHeadLabelValuesWithMatchers: Add test case (#13414) 2024-01-24 10:47:56 +01:00
block.go Add ShardedPostings() support to TSDB (#10421) 2024-01-29 11:57:27 +00:00
blockwriter_test.go Add a chunk size limit in bytes (#12054) 2023-08-24 15:21:17 +02:00
blockwriter.go tsdb: use Go standard errors 2023-12-11 12:18:54 +00:00
CHANGELOG.md
compact_test.go Optimize histogram iterators (#13340) 2024-01-23 17:02:14 +01:00
compact.go Fix regressions introduced by #13242 2024-01-16 12:00:53 +01:00
db_test.go Optimize histogram iterators (#13340) 2024-01-23 17:02:14 +01:00
db.go Add ShardedPostings() support to TSDB (#10421) 2024-01-29 11:57:27 +00:00
example_test.go Add context argument to Querier.Select (#12660) 2023-09-12 12:37:38 +02:00
exemplar_test.go golangci-lint: enable testifylint linter (#13254) 2023-12-07 11:35:01 +00:00
exemplar.go tsdb: use Go standard errors 2023-12-11 12:18:54 +00:00
head_append.go Move metric type definitions to common/model 2023-12-19 18:56:54 +00:00
head_bench_test.go tsdb: use Go standard errors 2023-12-11 12:18:54 +00:00
head_read_test.go Add ShardedPostings() support to TSDB (#10421) 2024-01-29 11:57:27 +00:00
head_read.go Add ShardedPostings() support to TSDB (#10421) 2024-01-29 11:57:27 +00:00
head_test.go Add ShardedPostings() support to TSDB (#10421) 2024-01-29 11:57:27 +00:00
head_wal.go Move metric type definitions to common/model 2023-12-19 18:56:54 +00:00
head.go Add ShardedPostings() support to TSDB (#10421) 2024-01-29 11:57:27 +00:00
isolation_test.go
isolation.go tsdb: create isolation transaction slice on demand 2023-10-21 13:45:47 +00:00
mocks_test.go tsdb: use Go standard errors 2023-12-11 12:18:54 +00:00
ooo_head_read_test.go Optimize label values with matchers by taking shortcuts (#13426) 2024-01-23 11:40:21 +01:00
ooo_head_read.go Add ShardedPostings() support to TSDB (#10421) 2024-01-29 11:57:27 +00:00
ooo_head_test.go ci(lint): enable godot; append dot at the end of comments 2023-10-31 19:53:38 +02:00
ooo_head.go Fix issue where queries can fail or omit OOO samples if OOO head compaction occurs between creating a querier and reading chunks (#13115) 2023-11-24 12:38:38 +01:00
ooo_isolation_test.go Fix issue where queries can fail or omit OOO samples if OOO head compaction occurs between creating a querier and reading chunks (#13115) 2023-11-24 12:38:38 +01:00
ooo_isolation.go Fix issue where queries can fail or omit OOO samples if OOO head compaction occurs between creating a querier and reading chunks (#13115) 2023-11-24 12:38:38 +01:00
querier_bench_test.go Optimize label values with matchers by taking shortcuts (#13426) 2024-01-23 11:40:21 +01:00
querier_test.go Add ShardedPostings() support to TSDB (#10421) 2024-01-29 11:57:27 +00:00
querier.go Add ShardedPostings() support to TSDB (#10421) 2024-01-29 11:57:27 +00:00
README.md
repair_test.go Add context argument to IndexReader.Postings (#12667) 2023-09-13 17:45:06 +02:00
repair.go tsdb: use Go standard errors 2023-12-11 12:18:54 +00:00
tsdbblockutil.go Optimize histogram iterators (#13340) 2024-01-23 17:02:14 +01:00
wal_test.go remove obsolete build tag 2024-01-17 22:26:32 +08:00
wal.go tsdb: use Go standard errors 2023-12-11 12:18:54 +00:00

TSDB

GoPkg

This directory contains the Prometheus TSDB (Time Series DataBase) library, which handles storage and querying of all Prometheus v2 data.

Documentation

External resources

A series of blog posts explaining different components of TSDB: