Commit Graph

943 Commits

Author SHA1 Message Date
Krasi Georgiev bf2239079d refactor multi errors
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-02-11 12:28:46 +02:00
Krasi Georgiev 07df4fd383 nits
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-02-11 11:25:57 +02:00
Krasi Georgiev d48606827c simplify closers
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-02-08 13:35:32 +02:00
Krasi Georgiev e138c7ed7e Merge remote-tracking branch 'upstream/master' into delete-compact-block-on-reload-error
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-02-08 13:26:28 +02:00
Krasi Georgiev 0b72f9af4c
Merge pull request #270 from codesome/master
Head: don't create stones, delete samples directly
2019-02-08 12:35:01 +02:00
Krasi Georgiev 1e91d619de
Merge pull request #493 from simonpasquier/update-makefile-common
fix static check errors and removed a lot of unused code and vars.
updated Makefile.common
2019-02-08 11:47:06 +02:00
Ganesh Vernekar 5481549324
Merge remote-tracking branch 'upstream/master'
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2019-02-07 22:15:06 +05:30
Simon Pasquier 272fd0eabf Update Makefile.common
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-02-07 15:06:10 +01:00
Simon Pasquier 95334f13c5 Merge branch 'master' into update-makefile-common
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-02-07 12:10:22 +01:00
Krasi Georgiev 77cc2ea210
Update index.md(format of series entry) (#515)
* Update index.md
* more clear format representation
* move dots to the bottom
* update #labels count, #chunks count
* use correct variable types.

Signed-off-by: naivewong <867245430@qq.com>
2019-02-06 12:13:38 +03:00
naivewong 9b227d27e7 update labels count, chunks count
Signed-off-by: naivewong <867245430@qq.com>
2019-02-06 10:00:18 +08:00
naivewong 8e7e2041d3 update #labels count, #chunks count
Signed-off-by: naivewong <867245430@qq.com>
2019-02-06 00:48:25 +08:00
naivewong 42ee59689b move dots to the bottom
Signed-off-by: naivewong <867245430@qq.com>
2019-02-06 00:34:15 +08:00
naivewong 0649dfddf0 more clear format representation
Signed-off-by: naivewong <867245430@qq.com>
2019-02-06 00:05:30 +08:00
naivewong 1f723a8eb5 update #labels and #chunks
Signed-off-by: naivewong <867245430@qq.com>
2019-02-05 23:20:43 +08:00
Alec 8e589474c6 Update index.md
Signed-off-by: naivewong <867245430@qq.com>
2019-02-05 10:34:25 +08:00
Alec 2b6bc9fb32 Update index.md
Signed-off-by: naivewong <867245430@qq.com>
2019-02-05 10:34:25 +08:00
Alec 1bcda9d23f Update tombstones.md (format of tombstone) (#511) 2019-02-04 12:30:47 +03:00
Krasi Georgiev 6c34eb8b63 nits
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-01-30 12:17:15 +02:00
Krasi Georgiev 315de4c782 fix windows tests
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-01-30 11:40:40 +02:00
Krasi Georgiev 1603222bbc small refactor of openTestDB to handle errors properly.
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-01-30 11:40:12 +02:00
Krasi Georgiev 7245c6dc33 Merge remote-tracking branch 'upstream/master' into delete-compact-block-on-reload-error
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-01-30 11:39:19 +02:00
Krasi Georgiev ee99718ff6
rename chunk reader vars to make it easier to follow. (#508)
* rename chunk reader vars to make it easyer to understand.

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-01-29 20:46:12 +03:00
radek_lesniewski eb5034d5b0 Additional logging in compact.go - logged time needed for writing blocks (#505)
* Additional logging in compact.go - logged time needed for writing blocks to disk

Signed-off-by: Radoslaw Lesniewski <Radoslaw.Lesniewski@sabre.com>

* Additional logging in compact.go - code formatted

Signed-off-by: Radoslaw Lesniewski <Radoslaw.Lesniewski@sabre.com>
2019-01-29 16:53:53 +05:30
Krasi Georgiev 3ec08eac50 use camelcase for rangeToTriggerCompaction
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-01-29 10:32:59 +02:00
Krasi Georgiev 53c18e7a41 use a global indexFilename constant
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-01-29 10:32:32 +02:00
Krasi Georgiev b521559c3b do a proper cleanup for a failed reload after a compaction
a failed reload immediately after a compaction should delete the
resulting block to avoid creating blocks with the same time range.

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-01-29 10:31:54 +02:00
Ganesh Vernekar a29ab50a20
Merge pull request #510 from yeya24/patch/fixtypo
fix two typos in db_test
2019-01-29 11:51:03 +05:30
yeya24 6181d1f18f fix two typos in db_test
Signed-off-by: yeya24 <ben.ye@daocloud.io>
2019-01-29 10:25:12 +08:00
Krasi Georgiev dac2b97dfd
make createBlock more generic so it can be used in other tests. (#489)
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-01-28 14:24:49 +03:00
Alec 051a7ae1a7 Missing the length of the encoding byte when calling b.Range
Signed-off-by: naivewong <867245430@qq.com>
2019-01-28 13:33:44 +03:00
Krasi Georgiev a44d15798e
remove a changelog double entry (#507)
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-01-23 17:10:19 +03:00
Brian Brazil 5db162568b Remove _total from prometheus_tsdb_storage_blocks_bytes (#506)
Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2019-01-23 16:46:58 +03:00
Simon Pasquier 2e69508536 Merge branch 'master' into update-makefile-common
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-01-22 10:47:31 +01:00
Krasi Georgiev cc277e398b
fix the refs logic for the addFast path for createBlock (#504)
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-01-21 14:56:26 +03:00
Krasi Georgiev 10ba228e6b
release 0.4.0 (#501)
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-01-18 11:42:59 +03:00
Ganesh Vernekar 1a9d08adc5 Don't write empty blocks (#374)
* Dont write empty blocks when a compaction results in a block with no samples.

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2019-01-18 11:35:16 +03:00
Callum Styan 3929359302
add live reader for WAL (#481)
* add live reader for WAL

Signed-off-by: Callum Styan <callumstyan@gmail.com>
2019-01-16 10:09:08 -08:00
mknapphrt ebf5d74325 Added storage size based retention method and new metrics (#343)
Added methods needed to retain data based on a byte limitation rather than time. Limitation is only applied if the flag is set (defaults to 0). Both blocks that are older than the retention period and the blocks that make the size of the storage too large are removed.

2 new metrics for keeping track of the size of the local storage folder and the amount of times data has been deleted because the size restriction was exceeded.
Signed-off-by: Mark Knapp <mknapp@hudson-trading.com>
2019-01-16 13:03:52 +03:00
naivewong bff5aa4d21 Missing the len of crc32 when calculating maxLen in WriteChunks (#494)
Signed-off-by: naivewong <867245430@qq.com>
2019-01-14 20:58:03 +05:30
Ye Ben a360aa3e86 change variable name metrics to labels (#496)
Signed-off-by: yeya24 <ben.ye@daocloud.io>
2019-01-14 11:44:32 +03:00
Bartek Płotka c065fa6957 Exposed helper methods for reading index bytes. (#492)
Changes:
* Make `NewReader` method useful. It was impossible to use it, because closer was always nil.
* ReadSymbols, TOC and ReadOffsetTable are not public functions (used by Thanos).
* decbufXXX are now functions.
* More verbose errors.
* Removed unused crc32 field.
* Some var name changes to make it more verbose:
  * symbols -> allocatedSymbols
  * symbolsSlice -> symbolsV1
  * symbols -> symbolsV2
  *
* Pre-calculate symbolsTableSize.
* Initialized symbols for Symbols() method with valid length.
* Added test for Symbol method.
* Made Decoder LookupSymbol method public. Kept Decode public as it is useful as helper from index package.

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
2019-01-11 17:31:26 +00:00
Ganesh Vernekar 4592b77035
Patch by Krasi (https://github.com/codesome/tsdb/pull/3)
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2019-01-11 22:04:09 +05:30
Ganesh Vernekar d7e505db34
Dont store stones in head, delete samples directly
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2019-01-08 22:38:41 +05:30
Simon Pasquier 62fca18f0a Add back indexFormatV1
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-01-07 17:37:12 +01:00
Simon Pasquier 597202ae43 Remove more unused code
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-01-07 14:45:17 +01:00
Simon Pasquier 47166a7969 Fix chunkenc/chunk_test.go
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-01-07 12:39:11 +01:00
Krasi Georgiev 8d991bdc1e
Delete temp checkpoint folder on error. (#415) 2019-01-07 11:43:33 +03:00
Simon Pasquier d5d7a097e1 Update Makefile.common
This change also uses the latest staticcheck version which comes with
new verifications, hence some clean up in the code.

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-01-04 16:08:43 +01:00
Brian Brazil 296f943ec4
More efficient Merge implementation. (#486)
Avoid a tree of merge objects, which can result in
what I suspect is n^2 calls to Seek when using Without.

With 100k metrics, and a regex of ^$ in BenchmarkHeadPostingForMatchers:

Before:
BenchmarkHeadPostingForMatchers-8              1        51633185216 ns/op      29745528 B/op      200357 allocs/op

After:
BenchmarkHeadPostingForMatchers-8             10         108924996 ns/op 25715025 B/op     101748 allocs/op

Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
2019-01-03 16:59:52 +00:00