prometheus/tsdb
Chris Marchbanks 7c7dafc321
Do not snappy encode if record is too large (#8790)
Snappy cannot encode records larger than ~3.7 GB and will panic if an
encoding is attempted. Check to make sure that the record is smaller
than this before encoding.

In the future, we could improve this behavior to still compress large
records (or break them up into smaller records), but this avoids the
panic for users with very large single scrape targets.

Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>
2021-05-06 12:56:45 -06:00
..
chunkenc Testify: move to require (#8122) 2020-10-29 09:43:23 +00:00
chunks Fixed TestChunkDiskMapper_WriteChunk_Chunk_IterateChunks for go1.16 (#8538) 2021-02-25 14:38:12 +05:30
docs/format Consistent naming and better names for fields in TSDB format docs (#8077) 2020-10-19 14:27:19 +05:30
encoding Avoid WriteAt for Postings. 2019-12-16 15:30:49 +00:00
errors MultiError: Refactored MultiError for more concise and safe usage. (#8066) 2020-10-28 15:24:58 +00:00
fileutil More info in mmap error message (#8058) 2020-10-15 12:58:13 +05:30
goversion tsdb: test for leaks (#7566) 2020-07-21 10:08:06 +02:00
index Add matchers to LabelValues() call (#8400) 2021-02-09 23:08:35 +05:30
record Testify: move to require (#8122) 2020-10-29 09:43:23 +00:00
test labels: Reduce allocated memory and latency of Hash method; Added tests. (#8025) 2020-10-15 11:31:28 +01:00
testdata tsdb: Delete blocks atomically; Remove tmp blocks on start; Added test. (#7772) 2020-08-11 06:56:08 +01:00
tombstones Testify: move to require (#8122) 2020-10-29 09:43:23 +00:00
tsdbutil Testify: move to require (#8122) 2020-10-29 09:43:23 +00:00
wal Do not snappy encode if record is too large (#8790) 2021-05-06 12:56:45 -06:00
.gitignore
block_test.go Combine Appender.Add and AddFast into a single Append method. (#8489) 2021-02-18 17:37:00 +05:30
block.go Rolling tombstones clean up (#8007) 2021-02-17 11:02:43 +05:30
blockwriter_test.go fix failing blockwriter tests 2021-03-01 06:16:05 -08:00
blockwriter.go add changes per comments, fix tests 2021-03-20 12:38:30 -07:00
CHANGELOG.md Rename default branch to main 2021-02-22 20:28:02 +01:00
compact_test.go Combine Appender.Add and AddFast into a single Append method. (#8489) 2021-02-18 17:37:00 +05:30
compact.go Add support for smaller block chunk segment allocations (#8478) 2021-04-15 14:25:01 +05:30
db_test.go Fix DB tests in the default branch 2021-02-18 23:56:27 +01:00
db.go Add support for smaller block chunk segment allocations (#8478) 2021-04-15 14:25:01 +05:30
exemplar_test.go tsdb: move exemplar series labels to index entry (#8783) 2021-05-05 18:51:16 +01:00
exemplar.go tsdb: move exemplar series labels to index entry (#8783) 2021-05-05 18:51:16 +01:00
head_bench_test.go TSDB: optimize series creation on PreCreation() failure (#8620) 2021-03-18 20:53:50 +05:30
head_test.go Fix memSafeIterator.Seek() (#8748) 2021-04-27 00:43:22 +02:00
head.go Fix memSafeIterator.Seek() (#8748) 2021-04-27 00:43:22 +02:00
isolation_test.go Optimise lowWatermark in Isolation (#7332) 2020-06-03 20:09:05 +02:00
isolation.go Optimise lowWatermark in Isolation (#7332) 2020-06-03 20:09:05 +02:00
mocks_test.go Move away from testutil, refactor imports (#8087) 2020-10-22 11:00:08 +02:00
querier_bench_test.go Combine Appender.Add and AddFast into a single Append method. (#8489) 2021-02-18 17:37:00 +05:30
querier_test.go Combine Appender.Add and AddFast into a single Append method. (#8489) 2021-02-18 17:37:00 +05:30
querier.go Correct spelling of "iterable" (#8713) 2021-04-12 21:43:42 +01:00
README.md Link blog post on TSDB queries in tsdb/README.md (#8391) 2021-01-21 17:14:40 +05:30
repair_test.go Testify: move to require (#8122) 2020-10-29 09:43:23 +00:00
repair.go MultiError: Refactored MultiError for more concise and safe usage. (#8066) 2020-10-28 15:24:58 +00:00
tsdbblockutil.go Combine Appender.Add and AddFast into a single Append method. (#8489) 2021-02-18 17:37:00 +05:30
wal_test.go Testify: move to require (#8122) 2020-10-29 09:43:23 +00:00
wal.go Update golangci-lint to 1.36.0 2021-02-04 08:53:33 -06:00

TSDB

GoDoc

This directory contains the Prometheus storage layer that is used in its 2.x releases.

A writeup of its design can be found here.

Based on the Gorilla TSDB white papers.

Video: Storing 16 Bytes at Scale from PromCon 2017.

See also the format documentation.

A series of blog posts explaining different components of TSDB: