3635569257
Trigger reload correctly on interrupted compaction
2017-03-20 10:41:43 +01:00
2c999836fb
Add Queryable interface to Block
...
This adds the Queryable interface to the Block interface. Head and
persisted blocks now implement their own Querier() method and thus
isolate customization (e.g. remapPostings) more cleanly.
2017-03-20 10:21:21 +01:00
11be2cc585
Add composed Block interfaces, remove head generation
...
This adds more lower-leve interfaces which are used to compose
to different Block interfaces.
The DB only uses interfaces instead of explicit persistedBlock and
headBlock. The headBlock generation property is dropped as the use-case
can be implemented using block sequence numbers.
2017-03-20 09:02:36 +01:00
2c3b56350a
Write meta.json file atomically
2017-03-01 17:19:57 +01:00
306831f151
Add per-block state ULID
2017-02-27 10:46:15 +01:00
9a5dfadb09
Tweak IndexWriter interface, serialize directly into dir
2017-02-25 07:24:20 +01:00
78780cd2ba
Segment chunk file
...
This adds write path support for segmented chunk data files.
Files of 512MB are pre-allocated and written to. If the file size
is exceeded, the next file is started. On completion, files
are truncated to their final size.
2017-02-23 10:50:22 +01:00
f734773214
Replace per-file locking with single PID lock file
...
File locks have a multitude of problems that make them hard to use
correctly. As they are just advisory, they are only meaningful to
prevent accidents like running the same process twice.
A simple PID file lock works reliably in those cases and is simpler.
2017-02-19 13:01:19 +01:00
a3d042b54e
Support multiple chunk files in read path
2017-02-18 17:33:20 +01:00
30efe4a58c
Support writing to multiple head blocks
...
This is an initial (and hacky) first pass on allowing
appending to multiple blocks simultaniously to avoid
dropping samples right after cutting a new head block.
It's also required for cases like the PGW, where a scrape may
contain varying timestamps.
2017-02-01 15:57:28 +01:00
c20cc44b06
Add docs, write sequence number to meta.json
2017-01-29 08:11:47 +01:00
30d8866c13
Limit compaction range, make cut and compact concurrent
2017-01-20 07:58:19 +01:00
67d185ceb9
Compact based on compaction generation
2017-01-19 19:45:52 +01:00
472c618c39
Drop out-of-bound samples
2017-01-19 15:03:57 +01:00
d4779b374c
Properly track and write meta file
2017-01-19 14:01:38 +01:00
9ddbd64d00
Move stats into meta.json file, cleanup, docs
2017-01-19 11:22:47 +01:00
e006bc6dc6
Improve error messages, create regular dir for block
2017-01-19 08:40:15 +01:00
d2322f6095
Improve compaction processing
2017-01-18 06:18:32 +01:00
c32a94d409
Unexport HeadBlock, export Block interface
2017-01-10 15:41:57 +01:00
1943f8d1bb
Fix head block stats races
2017-01-07 18:02:17 +01:00
2eb544c98e
Change file names and maker parsing safer
2017-01-06 13:13:22 +01:00
937cdb579c
Switch to sequential block names
...
This changes block directory names from the int64 timestamp
to sequential numbering.
2017-01-06 10:45:03 +01:00
ac49f8c15e
Consolidate persistence and compaction
2017-01-03 16:49:37 +01:00
e7f04d14d5
Lock mmapped files
2017-01-03 10:09:20 +01:00
91b65b55e7
Run persistence in separate goroutine
2017-01-02 22:24:35 +01:00
beb842a856
Change block interface
...
This changes the block interface to directly expose index
and series readers rather than the direct querier
2017-01-02 11:12:28 +01:00
8aba95048a
Rename to OpenHeadBlock
2016-12-22 20:00:24 +01:00
9c6a72aadd
Load head with WALs correctly
2016-12-22 15:54:39 +01:00
d9ca4b47f5
Fix offset errors, fix persisted postings order
2016-12-20 13:14:55 +01:00
00a503129b
Use contextualized and traced errors in reader
2016-12-19 22:29:49 +01:00
bd77103a49
Add stats serialization, load querier of all blocks
2016-12-15 16:14:33 +01:00
9ceed5378e
Fix missing bound checks, off-by-ones, typos
2016-12-15 11:24:20 +01:00
9873e18b75
Add loading of persisted blocks
2016-12-15 08:36:07 +01:00
6eeb0ef01c
Add queriers and partial mocks
2016-12-13 15:26:58 +01:00
14dbc59f2b
cleanup and switching removal of unsafe calls.
2016-12-10 18:09:57 +01:00
40a451694f
Refactor persistence into interfaces
2016-12-09 20:45:46 +01:00
62f9dc311c
misc
2016-12-09 16:54:38 +01:00
74f8dfd95d
Persist blocks periodically
2016-12-09 13:41:38 +01:00
0cf8bb9e53
Move sub-indexes into single index structure
2016-12-09 10:41:51 +01:00
8aa99a3ebd
misc
2016-12-09 10:00:14 +01:00
2c34a15fe6
Add initial seriailization of block data
2016-12-08 17:43:10 +01:00
Fabian Reinartz