ff0b0ac4b6
use test utils in chunk_test
2017-12-06 19:05:58 -08:00
0585ab3fcd
Use Go 1.9 math/bits implementation for counting leading / trailing zeros
2017-11-08 23:18:33 +01:00
ea817e169b
Return nop iterator for invalid chunk references
2017-10-20 09:43:52 +02:00
905af27cf9
Refactor compactor
2017-08-09 11:10:29 +02:00
66ff7b12e9
Pool Chunk objects during compaction
2017-08-08 17:35:34 +02:00
96d7f540d4
Persist series without allocating the full set
...
Change index persistence for series to not be accumulated in memory
before being written as one large batch. `Labels` and `ChunkMeta`
objects are reused.
This cuts down memory spikes during compaction of multiple blocks
significantly.
As part of the the Index{Reader,Writer} now have an explicit notion of
symbols and series must be inserted in order.
2017-08-06 12:06:41 +02:00
401bdfd4a6
Make Chunks return the number of samples inside
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-07-12 18:45:12 +02:00
0e2f8917ed
Also add original license info to bstream.go
...
Plus, extend the messages a bit to mention the origin repo.
2017-05-03 01:32:09 +02:00
87676f4a26
Add dgryski/go-tsz LICENSE to header
2017-05-02 11:26:01 +02:00
8d1fb4fa01
Minor comment fixes and additions.
2017-04-28 15:41:42 +02:00
778103b450
Add liecence file and headers
2017-04-10 20:59:45 +02:00
761e4768f3
Lint and Vet Fixes
2017-03-19 21:35:01 +05:30
91b65b55e7
Run persistence in separate goroutine
2017-01-02 22:24:35 +01:00
62b8ded0a5
Rename iterator value getters to At()
2017-01-02 13:33:06 +01:00
5d75a3dc7b
Add basic compaction
...
This adds a basic compactor that will merge two persisted blocks into
one. It simply fully rewrites the index and concatenates the chunk
lists.
It just writes into the current working dir and doesn't properly handle
which blocks to compact for now.
2017-01-02 10:34:55 +01:00
675f0886f0
Append to chunks cannot error
2016-12-31 10:10:27 +01:00
ce7f4106c2
Reda correct label number, fix buffered iterator panic
2016-12-20 14:21:50 +01:00
70a0224f19
Change chunk sample number to BigEndian
2016-12-12 08:11:53 +01:00
40a451694f
Refactor persistence into interfaces
2016-12-09 20:45:46 +01:00
b845f8d3a1
Reduce test data allocations
2016-12-08 11:59:54 +01:00
c5945177fb
chunks: helper for bit range
2016-12-07 15:37:37 +01:00
9b459458d0
Docs and interface definitions
2016-12-05 21:26:19 +01:00
0b6d621471
chunks: bring back lead/trail reuse, truncate incomplete sample
2016-11-30 23:01:01 +01:00
8c48dc2ca5
chunks: varint encoding in first full 64bit numbers
...
This saves about 7 bytes per chunk
2016-11-30 22:14:23 +01:00
5e76fd3126
chunks: drop shift, remove branch
...
This removes the need for the shift value and just calculates
everything on the fly through count.
Removes a branch condition.
2016-11-30 21:50:39 +01:00
c9ee572709
chunks: don't mutate in bstream iterator
...
This replaces mutation of underlying bytes in the iterated slice
with a shift counter, which is used when reading the head byte.
This is avoids having to copy the entire slice for every new iterator.
2016-11-30 19:39:22 +01:00
f392c01c12
chunks: add randomized test
2016-11-29 22:43:24 +01:00
fa181a34c1
chunks: cleanup anything but xor encoding
...
xor encoding is fast enough for our purposes and provides
very good compression.
We remove all other ones that partially don't support floats
for the sake of simplicity.
2016-11-29 22:02:58 +01:00
e67cf768dc
chunks: remove intermeidate copy from xor chunk
2016-11-20 16:24:46 +01:00
7874d28f32
chunks: implement xor encoding
2016-11-20 14:33:00 +01:00
ae7b18ae34
Add chunk package
2016-11-15 10:33:34 +01:00
Callum Styan
Daniel Swarbrick
Fabian Reinartz
Goutham Veeramachaneni
Julius Volz