Implement labels.PrefixMatcher and use interface conversion in querier
to optimize label tuples search.
[unit-tests]: Fix bug and populate label index for mock index.
Signed-off-by: Dmitry Ilyevsky <ilyevsky@gmail.com>
Let older head blocks be compacted once the newest once has samples at
50% of its total range. This allows the memory of the compacted blocks
to be released and garbage collected before a new head block gets
created. Thereby the number of head blocks is 1 or 2 instead of 2 or 3
and memory spikes are reduced.
* Expose Stone as it is used in an exported method.
* Move from tombstoneReader to []Stone for the same reason as above.
* Make WAL reading a little cleaner.
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
When we have only one chunk that is out of range, then we are returning
it unpopulated (w/o calling `Chunk(ref)`). This would cause a panic
downstream.
Fixes: prometheus/prometheus#2629
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
Mutating the chunks can change their length. Hence referencing using
previous indices can cause panics.
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
Fixes#43
Added mint, maxt to chunkSeriesIterator. Adding a field there is
inevitable as something similar is required for ignoring deleted
time-ranges.
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
This parallelizes commits to prevent deadlocks across inconsistently
locked heads. As commits are currently not fully atomic across
heads, this does decrease our guarantees.
This adds the Queryable interface to the Block interface. Head and
persisted blocks now implement their own Querier() method and thus
isolate customization (e.g. remapPostings) more cleanly.
This has been a common source of hard to debug issues. Its a premature
and unbenchmarked optimization and semantically, we want ChunkMetas to
be references in all changed cases.
This simplifies some of the iterators by loading chunks from the
ChunkReader earlier, filtering of chunks vs filtering or series is
split into separate iterators for easier testing
Introduce a seperate mutex for the head blocks to avoid a race where
a post-compaction reload may run between switching the DB's base mutex
to create a new head block in an appender.
This adds write path support for segmented chunk data files.
Files of 512MB are pre-allocated and written to. If the file size
is exceeded, the next file is started. On completion, files
are truncated to their final size.
This adds a 4 sample buffer to every head chunk. The XOR
compression scheme may edit bytes in place. The minimum size
of a sample is 2 bits. So keeping the last 4 samples in an in-memory
buffer makes it safe to query the preceeding ones while samples
are added
This adds a position mapper that takes series from a head block
in the order they were appended and creates a mapping representing
them in order of their label sets.
Write-repair of the postings list would cause very expensive writing.
Hence, we keep them as they are and only apply the postition mapping
at the very end, after a postings list has been sufficienctly reduced
through intersections etc.
Dmitry Ilyevsky
Fabian Reinartz
Goutham Veeramachaneni