4d03c70800
Verify chunk format version ( #544 )
...
Verify chunk format version.
2019-03-24 22:33:08 +02:00
b7cac504c9
Remove unused NewReader function ( #547 )
...
The `NewReader` function is not used anywhere.
2019-03-14 15:05:40 +02:00
0861a9b14f
[bugfix] update 'last' variable in chunks.MergeOverlappingChunks() ( #539 )
...
* update
Signed-off-by: naivewong <867245430@qq.com>
* update comment
Signed-off-by: naivewong <867245430@qq.com>
* Update failing test.
Signed-off-by: naivewong <867245430@qq.com>
* Update comment
Signed-off-by: naivewong <867245430@qq.com>
* Add changelog entry.
Signed-off-by: naivewong <867245430@qq.com>
* Update CHANGELOG.md
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2019-03-05 19:18:55 +05:30
c59ed492b2
Vertical query merging and compaction ( #370 )
...
* Vertical series iterator
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Select overlapped blocks first in compactor Plan()
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Added vertical compaction
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Code cleanup and comments
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Fix review comments
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Fix tests
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Add benchmark for compaction
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Perform vertical compaction only when blocks are overlapping.
Actions for vertical compaction:
* Sorting chunk metas
* Calling chunks.MergeOverlappingChunks on the chunks
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Benchmark for vertical compaction
* BenchmarkNormalCompaction => BenchmarkCompaction
* Moved the benchmark from db_test.go to compact_test.go
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Benchmark for query iterator and seek for non overlapping blocks
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Vertical query merge only for overlapping blocks
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Simplify logging in Compact(...)
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Updated CHANGELOG.md
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Calculate overlapping inside populateBlock
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* MinTime and MaxTime for BlockReader.
Using this to find overlapping blocks in populateBlock()
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Sort blocks w.r.t. MinTime in reload()
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Log about overlapping in LeveledCompactor.write() instead of returning bool
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Log about overlapping inside LeveledCompactor.populateBlock()
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Fix review comments
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Refactor createBlock to take optional []Series
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* review1
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
* Updated CHANGELOG and minor nits
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* nits
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Updated CHANGELOG
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Refactor iterator and seek benchmarks for Querier.
Also has as overlapping blocks.
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Additional test case
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* genSeries takes optional labels. Updated BenchmarkQueryIterator and BenchmarkQuerySeek.
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Split genSeries into genSeries and populateSeries
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Check error in benchmark
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Fix review comments
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Warn about overlapping blocks in reload()
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2019-02-14 14:29:41 +01:00
bf2239079d
refactor multi errors
...
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-02-11 12:28:46 +02:00
95334f13c5
Merge branch 'master' into update-makefile-common
...
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-02-07 12:10:22 +01:00
ee99718ff6
rename chunk reader vars to make it easier to follow. ( #508 )
...
* rename chunk reader vars to make it easyer to understand.
Signed-off-by: Krasi Georgiev <kgeorgie@redhat.com>
2019-01-29 20:46:12 +03:00
051a7ae1a7
Missing the length of the encoding byte when calling b.Range
...
Signed-off-by: naivewong <867245430@qq.com>
2019-01-28 13:33:44 +03:00
2e69508536
Merge branch 'master' into update-makefile-common
...
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-01-22 10:47:31 +01:00
ebf5d74325
Added storage size based retention method and new metrics ( #343 )
...
Added methods needed to retain data based on a byte limitation rather than time. Limitation is only applied if the flag is set (defaults to 0). Both blocks that are older than the retention period and the blocks that make the size of the storage too large are removed.
2 new metrics for keeping track of the size of the local storage folder and the amount of times data has been deleted because the size restriction was exceeded.
Signed-off-by: Mark Knapp <mknapp@hudson-trading.com>
2019-01-16 13:03:52 +03:00
bff5aa4d21
Missing the len of crc32 when calculating maxLen in WriteChunks ( #494 )
...
Signed-off-by: naivewong <867245430@qq.com>
2019-01-14 20:58:03 +05:30
d5d7a097e1
Update Makefile.common
...
This change also uses the latest staticcheck version which comes with
new verifications, hence some clean up in the code.
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2019-01-04 16:08:43 +01:00
1e1b2e163d
Make interval overlap comparisons more explicit
...
Blocks are half-open intervals [a, b), while all other intervals
(chunks, head, ...) are closed intervals [a, b].
Make that distinction explicit by defining `OverlapsClosedInterval()`
methods for blocks and chunks, and using them in place of the more
generic `intervalOverlap()` function.
This change also fixes `db.Querier()` and `db.Delete()`, which could
previously return one extraneous block at the end of the specified
interval.
Signed-off-by: Benoît Knecht <benoit.knecht@fsfe.org>
2018-07-02 10:35:08 +02:00
ee5fe8ea9f
chunks: fix potential "index out of range" error
...
When binary.Uvarint() fails, the returned number of bytes is less than
or equal to zero.
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2018-06-08 10:28:27 +02:00
c0e888e82b
fix megacheck issues: os.SEEK_SET is deprecated: Use io.SeekStart, io.SeekCurrent, and io.SeekEnd.
2018-03-21 22:39:15 +01:00
09142e4dd1
fix unconvert issues: unnecessary conversion
2018-03-21 22:39:14 +01:00
1e55b7987f
Improve comments, handle allPostingsKey properly
2017-12-22 09:43:34 +01:00
67f0ca8f0e
Move index and chunk encoders to own packages
2017-12-21 11:27:54 +01:00
zhulongcheng
Alec
Ganesh Vernekar
Krasi Georgiev
Simon Pasquier
Krasi Georgiev
mknapphrt
Benoît Knecht
Mario Trangoni
Fabian Reinartz