RepoMirrors/prometheus
1
0
Fork 0
mirror of https://github.com/prometheus/prometheus synced 2025-05-10 20:18:33 +00:00
Code Issues Packages Projects Releases Wiki Activity
997 Commits 254 Branches 468 Tags 661 MiB
c3ffdf1a99
Commit Graph

42 Commits

Author SHA1 Message Date
Krasi Georgiev
c3ffdf1a99
Test createBlock and check all os.RemoveAll in the tests for errors. (#549) 
Testing that createBlock creates blocks that can be opened.

and checking the os.RemoveAll for errors will catch errors for un-closed files under windows.

Many missing .Close() calls were added for fixing failing os.RemoveAll

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-03-19 15:31:57 +02:00
Alec
0861a9b14f [bugfix] update 'last' variable in chunks.MergeOverlappingChunks() (#539) 
* update

Signed-off-by: naivewong <867245430@qq.com>

* update comment

Signed-off-by: naivewong <867245430@qq.com>

* Update failing test.

Signed-off-by: naivewong <867245430@qq.com>

* Update comment

Signed-off-by: naivewong <867245430@qq.com>

* Add changelog entry.

Signed-off-by: naivewong <867245430@qq.com>

* Update CHANGELOG.md

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
 2019-03-05 19:18:55 +05:30
Ganesh Vernekar
c59ed492b2 Vertical query merging and compaction (#370) 
* Vertical series iterator

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Select overlapped blocks first in compactor Plan()

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Added vertical compaction

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Code cleanup and comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix review comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix tests

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Add benchmark for compaction

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Perform vertical compaction only when blocks are overlapping.

Actions for vertical compaction:
* Sorting chunk metas
* Calling chunks.MergeOverlappingChunks on the chunks

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Benchmark for vertical compaction

* BenchmarkNormalCompaction => BenchmarkCompaction
* Moved the benchmark from db_test.go to compact_test.go

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Benchmark for query iterator and seek for non overlapping blocks

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Vertical query merge only for overlapping blocks

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Simplify logging in Compact(...)

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Updated CHANGELOG.md

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Calculate overlapping inside populateBlock

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* MinTime and MaxTime for BlockReader.

Using this to find overlapping blocks in populateBlock()

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Sort blocks w.r.t. MinTime in reload()

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Log about overlapping in LeveledCompactor.write() instead of returning bool

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Log about overlapping inside LeveledCompactor.populateBlock()

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix review comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Refactor createBlock to take optional []Series

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* review1

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>

* Updated CHANGELOG and minor nits

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* nits

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Updated CHANGELOG

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Refactor iterator and seek benchmarks for Querier.

Also has as overlapping blocks.

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Additional test case

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* genSeries takes optional labels. Updated BenchmarkQueryIterator and BenchmarkQuerySeek.

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Split genSeries into genSeries and populateSeries

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Check error in benchmark

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix review comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Warn about overlapping blocks in reload()

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
 2019-02-14 14:29:41 +01:00
Krasi Georgiev
bf79c767f0 new line 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-12 11:08:09 +02:00
Krasi Georgiev
beee5c58f3 Merge remote-tracking branch 'upstream/master' into shutdown-during-compaction 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-12 10:56:45 +02:00
Krasi Georgiev
07df4fd383 nits 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-11 11:25:57 +02:00
Krasi Georgiev
0f8f5027ef remove nested for if 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-08 18:09:23 +02:00
Krasi Georgiev
da9da9fbee fix the sleep logic 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-08 12:39:25 +02:00
Krasi Georgiev
457534d5c4 simplify nesting. 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-08 11:36:30 +02:00
Krasi Georgiev
752ab86e4e change the test block series for more stable tests 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-02-04 11:14:39 +02:00
Krasi Georgiev
6c34eb8b63 nits 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-01-30 12:17:15 +02:00
Krasi Georgiev
1603222bbc small refactor of openTestDB to handle errors properly. 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-01-30 11:40:12 +02:00
Krasi Georgiev
b521559c3b do a proper cleanup for a failed reload after a compaction 
a failed reload immediately after a compaction should delete the
resulting block to avoid creating blocks with the same time range.

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-01-29 10:31:54 +02:00
Krasi Georgiev
fa1c00f9e2 no backgorund 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-01-24 14:38:45 +02:00
Krasi Georgiev
1b0d85bbf2 use a db copy instead of creating it again. 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-01-24 14:15:32 +02:00
Krasi Georgiev
8ffd705346 fix the misleading log during compaction cancelation 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-01-24 13:33:12 +02:00
Krasi Georgiev
9638c13ec8 test compares normal vs canceled compaction times 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-01-24 12:48:56 +02:00
Krasi Georgiev
9874377ead nits 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-01-18 19:22:45 +02:00
Krasi Georgiev
921f82cfc1 WIP 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-01-18 18:58:17 +02:00
Krasi Georgiev
0b31b8af88 Merge remote-tracking branch 'upstream/master' into shutdown-during-compaction 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2019-01-18 16:58:18 +02:00
Krasi Georgiev
a2779cc901
fix flaky tests: TestDisableAutoCompactions,TestBlockRanges (#472) 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2018-12-12 14:49:03 +03:00
Krasi Georgiev
db127a60e0 changelog 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2018-12-10 12:48:30 +02:00
Krasi Georgiev
fced260a24 test 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2018-12-10 12:48:30 +02:00
Krasi Georgiev
bd5ccee5c1 use context to cancel compactions 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2018-12-10 12:48:30 +02:00
Krasi Georgiev
bac9cbed2e
no overlapping on compaction when an existing block is not within default boundaries. (#461) 
closes https://github.com/prometheus/prometheus/issues/4643

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2018-12-04 13:30:49 +03:00
Krasi Georgiev
b75d702ceb
fix flaky compaction test (#458) 
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2018-11-21 00:33:14 +02:00
Krasi Georgiev
7f00217d77
Allow manual compaction for tests when compaction is disabled globally. (#412) 
for tests we need to control when a compaction happens so with this
change automated compaction can be disabled, but allow to run it
manually it tests.

fixes failing tests in : https://github.com/prometheus/tsdb/pull/374

Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
 2018-11-20 12:34:26 +02:00
Bartek Płotka
047b1b1357 compact: Verify for chunks outside of compacted time range. Added unit test for populateBlocs. (#349) 
* compact: Verify for chunks outside of compacted time range. 
 Unit test for populateBlocs.

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
Co-authored-by: Krasi Georgiev <kgeorgie@redhat.com>
 2018-10-12 12:45:19 +03:00
Bartek Plotka
2aceac0a12 Fixed some comments. 
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
 2018-03-13 15:30:00 +00:00
Bartek Plotka
328c0ff5b0 Simplified the flow and tests. 
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
 2018-03-13 14:12:32 +00:00
Bartek Plotka
483da43660 compact: Exclude last block with shortest range instead of newest one by ULID. 
Fixes https://github.com/prometheus/prometheus/issues/3943 issue.

Added tests.

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
 2018-03-13 12:30:27 +00:00
Krasi Georgiev
eb6cdbcb5b some test funcitons didn't cleanup after themselves. 2018-02-23 18:45:59 +02:00
Callum Styan
ca4e817372 move test utils to it's own package so we can use it in tests for other packages in tsdb repo 2017-12-06 17:06:14 -08:00
Callum Styan
fa26e2b6c3 use test utils in compact_test 2017-12-06 16:30:24 -08:00
Goutham Veeramachaneni
1fc94a02d1
Don't retry failed compactions. 
Fixes prometheus/prometheus#3487

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-11-23 15:16:20 +05:30
Fabian Reinartz
4cc37eecab Refactor and add tests for compactor 2017-09-01 11:50:58 +02:00
Goutham Veeramachaneni
178f840295 Fix panic caused by 0 division 
Introduced in #108

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-08-18 07:45:29 +02:00
Fabian Reinartz
905af27cf9 Refactor compactor 2017-08-09 11:10:29 +02:00
Fabian Reinartz
3951d8cc29 Fix selection of compactable blocks 2017-08-04 12:02:34 +02:00
Fabian Reinartz
5d7b5994d6 Fix compaction range selection 2017-07-13 16:13:59 +02:00
Goutham Veeramachaneni
9a2ab732b5 Choose ranges by retention and min-block-duration 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-07-12 17:30:34 +02:00
Goutham Veeramachaneni
ce23adb579 Initial compaction implementation for block-ranges 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-07-07 16:32:18 +02:00