Commit Graph

28 Commits

Author SHA1 Message Date
Fabian Reinartz 9a5dfadb09 Tweak IndexWriter interface, serialize directly into dir 2017-02-25 07:24:20 +01:00
Fabian Reinartz 78780cd2ba Segment chunk file
This adds write path support for segmented chunk data files.
Files of 512MB are pre-allocated and written to. If the file size
is exceeded, the next file is started. On completion, files
are truncated to their final size.
2017-02-23 10:50:22 +01:00
Fabian Reinartz a3d042b54e Support multiple chunk files in read path 2017-02-18 17:33:20 +01:00
Fabian Reinartz 67d185ceb9 Compact based on compaction generation 2017-01-19 19:45:52 +01:00
Fabian Reinartz d4779b374c Properly track and write meta file 2017-01-19 14:01:38 +01:00
Fabian Reinartz 9ddbd64d00 Move stats into meta.json file, cleanup, docs 2017-01-19 11:22:47 +01:00
Fabian Reinartz dd0b69fe1b Export ErrNotFound 2017-01-16 14:18:32 +01:00
Fabian Reinartz 1943f8d1bb Fix head block stats races 2017-01-07 18:02:17 +01:00
Fabian Reinartz 5d75a3dc7b Add basic compaction
This adds a basic compactor that will merge two persisted blocks into
one. It simply fully rewrites the index and concatenates the chunk
lists.

It just writes into the current working dir and doesn't properly handle
which blocks to compact for now.
2017-01-02 10:34:55 +01:00
Fabian Reinartz c00d17e691 Modify IndexReader API to accomodate compaction
This changes the IndexReader API to expose plain labels
and chunk meta information instead of a Series interface.
Dropping of irrelevant chunks is moved into the querier.

A LabelIndices method is added to query for existing label
value indices.
2016-12-31 16:24:04 +01:00
Fabian Reinartz ede733ab6c Extract labels package 2016-12-21 09:39:01 +01:00
Fabian Reinartz cddc29fa17 Fix labels comparison, fetch correct labels 2016-12-20 14:54:52 +01:00
Fabian Reinartz ce7f4106c2 Reda correct label number, fix buffered iterator panic 2016-12-20 14:21:50 +01:00
Fabian Reinartz d9ca4b47f5 Fix offset errors, fix persisted postings order 2016-12-20 13:14:55 +01:00
Fabian Reinartz 00a503129b Use contextualized and traced errors in reader 2016-12-19 22:29:49 +01:00
Fabian Reinartz 282d9ae6e2 Implement label value queries in all layers. 2016-12-19 12:26:25 +01:00
Fabian Reinartz aabb21f4b9 Add shard series set test 2016-12-19 11:44:11 +01:00
Fabian Reinartz b08f82fa4e Pre-select relevant chunks on series access.
This adds interval metadata to indexed chunks. The queried interval
is used to filter chunks when queried from the index to save
unnecessary accesses of the chunks file.

This is especially relevant for series that come and go often and larger
files.
2016-12-16 12:13:17 +01:00
Fabian Reinartz bd77103a49 Add stats serialization, load querier of all blocks 2016-12-15 16:14:33 +01:00
Fabian Reinartz 9ceed5378e Fix missing bound checks, off-by-ones, typos 2016-12-15 11:24:20 +01:00
Fabian Reinartz d56b281006 Rename Iterator to Postings 2016-12-15 07:05:29 +01:00
Fabian Reinartz ca89080128 Misc fixes for initial Prometheus integration 2016-12-14 18:38:46 +01:00
Fabian Reinartz 6eeb0ef01c Add queriers and partial mocks 2016-12-13 15:26:58 +01:00
Fabian Reinartz 9b400b4c58 Add chunk based series iterator 2016-12-12 19:12:55 +01:00
Fabian Reinartz b334c3ade8 Write chunk skiplist and add series reader 2016-12-12 15:39:55 +01:00
Fabian Reinartz ae379f385b Fix label index write and add read path 2016-12-12 11:38:43 +01:00
Fabian Reinartz 10943b6d88 Add initial index reader implementation 2016-12-12 08:12:19 +01:00
Fabian Reinartz 81b4d570ad Add series file reader 2016-12-11 15:54:25 +01:00