RepoMirrors/prometheus
1
0
Fork 0
mirror of https://github.com/prometheus/prometheus synced 2024-12-27 17:13:22 +00:00
Code Issues Packages Projects Releases Wiki Activity
cebe36c7d5
prometheus/tsdb
History
Brian Brazil cebe36c7d5 Make head Postings only return series in time range 
Series() will fetch all the metadata for a series,
even if it's going to be filtered later due to time ranges.

For 1M series we save ~1.1s if you only needed some of the data, but take an
extra ~.2s if you did want everything.

benchmark                                  old ns/op      new ns/op      delta
BenchmarkHeadSeries/1of1000000-4           1443715987     131553480      -90.89%
BenchmarkHeadSeries/10of1000000-4          1433394040     130730596      -90.88%
BenchmarkHeadSeries/100of1000000-4         1437444672     131360813      -90.86%
BenchmarkHeadSeries/1000of1000000-4        1438958659     132573137      -90.79%
BenchmarkHeadSeries/10000of1000000-4       1438061766     145742377      -89.87%
BenchmarkHeadSeries/100000of1000000-4      1455060948     281659416      -80.64%
BenchmarkHeadSeries/1000000of1000000-4     1633524504     1803550153     +10.41%

benchmark                                  old allocs     new allocs     delta
BenchmarkHeadSeries/1of1000000-4           4000055        28             -100.00%
BenchmarkHeadSeries/10of1000000-4          4000073        87             -100.00%
BenchmarkHeadSeries/100of1000000-4         4000253        630            -99.98%
BenchmarkHeadSeries/1000of1000000-4        4002053        6036           -99.85%
BenchmarkHeadSeries/10000of1000000-4       4020053        60054          -98.51%
BenchmarkHeadSeries/100000of1000000-4      4200053        600074         -85.71%
BenchmarkHeadSeries/1000000of1000000-4     6000053        6000094        +0.00%

benchmark                                  old bytes     new bytes     delta
BenchmarkHeadSeries/1of1000000-4           229192184     2488          -100.00%
BenchmarkHeadSeries/10of1000000-4          229193336     5568          -100.00%
BenchmarkHeadSeries/100of1000000-4         229204856     35536         -99.98%
BenchmarkHeadSeries/1000of1000000-4        229320056     345104        -99.85%
BenchmarkHeadSeries/10000of1000000-4       230472056     3894673       -98.31%
BenchmarkHeadSeries/100000of1000000-4      241992056     40511632      -83.26%
BenchmarkHeadSeries/1000000of1000000-4     357192056     402380440     +12.65%

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2020-02-20 22:18:42 +01:00
..
chunkenc Addressed Brian's comments, moved metrics to main.go 2020-02-17 18:03:57 +00:00
chunks Refactor tsdb/chunks/chunks.go for future PRs (#6754) 2020-02-05 19:09:40 +05:30
cmd/tsdb Addressed comments. 2020-02-17 18:03:57 +00:00
docs/format Stream symbols during compaction. (#6468) 2019-12-17 19:49:54 +00:00
encoding Avoid WriteAt for Postings. 2019-12-16 15:30:49 +00:00
errors Moving tsdb into its own subdirectory 2019-08-13 13:58:49 +05:30
fileutil Add OpenMmapFileWithSize method (#6753) 2020-02-05 19:08:30 +05:30
goversion Fix some typo in comments (#6730) 2020-01-31 12:11:52 +05:30
index tsdb: writePostingsToTmpFiles returns nil instead of err (#6618) 2020-01-13 22:40:12 +00:00
record Spelling (#6517) 2020-01-02 15:54:09 +01:00
test Port tsdb to use pkg/labels. (#6326) 2019-11-18 11:53:33 -08:00
testdata Handle V1 indexes, some of which have unsorted posting offset tables. (#6564) 2020-01-06 14:06:11 +00:00
tombstones Bring back tombstones to Head block (#6542) 2020-01-20 21:08:00 +05:30
tsdbutil Removed extra tsdb/testutil after merge. 2019-08-14 10:12:32 +01:00
wal tsdb/wal: remove unused argument. 2020-02-15 21:54:09 +08:00
.gitignore Moving tsdb into its own subdirectory 2019-08-13 13:58:49 +05:30
block_test.go Unify Iterator interfaces. All point to storage now. 2020-02-17 18:03:54 +00:00
block.go Make head Postings only return series in time range 2020-02-20 22:18:42 +01:00
CHANGELOG.md Spelling (#6517) 2020-01-02 15:54:09 +01:00
compact_test.go Close db properly in tests. 2020-02-17 18:03:57 +00:00
compact.go Make head Postings only return series in time range 2020-02-20 22:18:42 +01:00
db_test.go Addressed Brian's comments, moved metrics to main.go 2020-02-17 18:03:57 +00:00
db.go Moved readyStorage to main. 2020-02-17 18:03:57 +00:00
head_bench_test.go Make head Postings only return series in time range 2020-02-20 22:18:42 +01:00
head_test.go Unify Iterator interfaces. All point to storage now. 2020-02-17 18:03:54 +00:00
head.go Make head Postings only return series in time range 2020-02-20 22:18:42 +01:00
mocks_test.go Stream symbols during compaction. (#6468) 2019-12-17 19:49:54 +00:00
querier_bench_test.go Unify Iterator interfaces. All point to storage now. 2020-02-17 18:03:54 +00:00
querier_test.go Addressed comments. 2020-02-17 18:03:57 +00:00
querier.go Make head Postings only return series in time range 2020-02-20 22:18:42 +01:00
README.md Edit TSDB README badges 2019-10-24 15:35:47 +05:30
repair_test.go Port tsdb to use pkg/labels. (#6326) 2019-11-18 11:53:33 -08:00
repair.go tsdb: add block meta version constant (#5994) 2019-09-09 12:28:01 +03:00
tsdbblockutil.go Unify Iterator interfaces. All point to storage now. 2020-02-17 18:03:54 +00:00
wal_test.go Spelling (#6517) 2020-01-02 15:54:09 +01:00
wal.go Spelling (#6517) 2020-01-02 15:54:09 +01:00

README.md

TSDB

GoDoc

This repository contains the Prometheus storage layer that is used in its 2.x releases.

A writeup of its design can be found here.

Based on the Gorilla TSDB white papers.

Video: Storing 16 Bytes at Scale from PromCon 2017.

See also the format documentation.