prometheus/tsdb
Filip Petkovski 583f3e587c
Optimize histogram iterators (#13340)
Optimize histogram iterators

Histogram iterators allocate new objects in the AtHistogram and
AtFloatHistogram methods, which makes calculating rates over long
ranges expensive.

In #13215 we allowed an existing object to be reused
when converting an integer histogram to a float histogram. This commit follows
the same idea and allows injecting an existing object in the AtHistogram and
AtFloatHistogram methods. When the injected value is nil, iterators allocate
new histograms, otherwise they populate and return the injected object.

The commit also adds a CopyTo method to Histogram and FloatHistogram which
is used in the BufferedIterator to overwrite items in the ring instead of making
new copies.

Note that a specialized HPoint pool is needed for all of this to work 
(`matrixSelectorHPool`).

---------

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
2024-01-23 17:02:14 +01:00
..
agent agent: add BenchmarkCreateSeries 2023-12-31 10:23:43 +00:00
chunkenc Optimize histogram iterators (#13340) 2024-01-23 17:02:14 +01:00
chunks Optimize histogram iterators (#13340) 2024-01-23 17:02:14 +01:00
docs Merge branch 'main' into sparsehistogram 2022-08-10 17:54:37 +02:00
encoding tsdb/encoding: use Go standard errors package 2023-11-11 19:01:11 +01:00
errors Enable default revive rules (#13068) 2023-11-29 17:23:34 +00:00
fileutil remove obsolete build tag 2024-01-17 22:26:32 +08:00
goversion remove obsolete build tag 2024-01-17 22:26:32 +08:00
index tsdb/{index,compact}: allow using custom postings encoding format (#13242) 2024-01-08 09:48:27 +00:00
record Move metric type definitions to common/model 2023-12-19 18:56:54 +00:00
testdata
tombstones Update tombstones.go 2023-11-11 19:22:06 +01:00
tsdbutil Revert "Adding small test update for temp dir using t.TempDir (#13293)" 2023-12-30 19:17:30 +00:00
wlog tsdb: use Go standard errors 2023-12-11 12:18:54 +00:00
.gitignore
block_test.go Optimize histogram iterators (#13340) 2024-01-23 17:02:14 +01:00
block.go tsdb: use Go standard errors 2023-12-11 12:18:54 +00:00
blockwriter_test.go Add a chunk size limit in bytes (#12054) 2023-08-24 15:21:17 +02:00
blockwriter.go tsdb: use Go standard errors 2023-12-11 12:18:54 +00:00
CHANGELOG.md
compact_test.go Optimize histogram iterators (#13340) 2024-01-23 17:02:14 +01:00
compact.go Fix regressions introduced by #13242 2024-01-16 12:00:53 +01:00
db_test.go Optimize histogram iterators (#13340) 2024-01-23 17:02:14 +01:00
db.go tsdb/main: wire "EnableOverlappingCompaction" to tsdb.Options (#13398) 2024-01-15 16:42:40 +01:00
example_test.go Add context argument to Querier.Select (#12660) 2023-09-12 12:37:38 +02:00
exemplar_test.go golangci-lint: enable testifylint linter (#13254) 2023-12-07 11:35:01 +00:00
exemplar.go tsdb: use Go standard errors 2023-12-11 12:18:54 +00:00
head_append.go Move metric type definitions to common/model 2023-12-19 18:56:54 +00:00
head_bench_test.go tsdb: use Go standard errors 2023-12-11 12:18:54 +00:00
head_read_test.go golangci-lint: enable testifylint linter (#13254) 2023-12-07 11:35:01 +00:00
head_read.go tsdb: use Go standard errors 2023-12-11 12:18:54 +00:00
head_test.go Optimize histogram iterators (#13340) 2024-01-23 17:02:14 +01:00
head_wal.go Move metric type definitions to common/model 2023-12-19 18:56:54 +00:00
head.go tsdb: use Go standard errors 2023-12-11 12:18:54 +00:00
isolation_test.go tsdb: turn off transaction isolation for head compaction (#11317) 2022-09-27 19:31:23 +05:30
isolation.go ci(lint): enable predeclared linter 2023-05-21 07:33:54 +00:00
mocks_test.go tsdb: use Go standard errors 2023-12-11 12:18:54 +00:00
ooo_head_read_test.go Optimize label values with matchers by taking shortcuts (#13426) 2024-01-23 11:40:21 +01:00
ooo_head_read.go Change ChunkReader.Chunk() to ChunkOrIterable() 2023-11-28 11:14:29 +01:00
ooo_head_test.go ci(lint): enable godot; append dot at the end of comments 2023-10-31 19:53:38 +02:00
ooo_head.go Fix issue where queries can fail or omit OOO samples if OOO head compaction occurs between creating a querier and reading chunks (#13115) 2023-11-24 12:38:38 +01:00
ooo_isolation_test.go Fix issue where queries can fail or omit OOO samples if OOO head compaction occurs between creating a querier and reading chunks (#13115) 2023-11-24 12:38:38 +01:00
ooo_isolation.go Fix issue where queries can fail or omit OOO samples if OOO head compaction occurs between creating a querier and reading chunks (#13115) 2023-11-24 12:38:38 +01:00
querier_bench_test.go Optimize label values with matchers by taking shortcuts (#13426) 2024-01-23 11:40:21 +01:00
querier_test.go Optimize histogram iterators (#13340) 2024-01-23 17:02:14 +01:00
querier.go Optimize histogram iterators (#13340) 2024-01-23 17:02:14 +01:00
README.md Fixed broken link in tsdb README.md 2022-10-07 16:20:20 +00:00
repair_test.go Add context argument to IndexReader.Postings (#12667) 2023-09-13 17:45:06 +02:00
repair.go tsdb: use Go standard errors 2023-12-11 12:18:54 +00:00
tsdbblockutil.go Optimize histogram iterators (#13340) 2024-01-23 17:02:14 +01:00
wal_test.go remove obsolete build tag 2024-01-17 22:26:32 +08:00
wal.go tsdb: use Go standard errors 2023-12-11 12:18:54 +00:00

TSDB

GoPkg

This directory contains the Prometheus TSDB (Time Series DataBase) library, which handles storage and querying of all Prometheus v2 data.

Documentation

External resources

A series of blog posts explaining different components of TSDB: