Commit Graph

35 Commits

Author SHA1 Message Date
Fabian Reinartz
87805fb83f Remove Partitioned* code 2017-03-06 17:34:49 +01:00
Fabian Reinartz
cc0a7c8279 Create alloc and inuse space heap profile 2017-03-02 21:54:30 +01:00
Fabian Reinartz
ffb24a98f4 Add missing unlock, run debug endpoint in benchmark 2017-03-02 21:54:02 +01:00
Fabian Reinartz
db5c88ea9a Misc compaction fixes 2017-02-28 08:40:51 +01:00
Fabian Reinartz
b281e4e39b Accept prometheus.Registerer in constructor 2017-02-28 07:26:24 +01:00
Fabian Reinartz
a3b47c4929 Create default logger for DB 2017-02-19 16:04:37 +01:00
Fabian Reinartz
9c7a88223e Add full encode/decode WAL cycle test 2017-02-14 21:55:50 -08:00
Fabian Reinartz
79944a5912 Break out WAL into segment files 2017-02-13 23:53:19 -08:00
Fabian Reinartz
9c76624df2 Add initial retention cutoff 2017-02-09 17:54:26 -08:00
Fabian Reinartz
012cf4ef25 Count writer references on head blocks 2017-02-04 11:53:52 +01:00
Fabian Reinartz
5a1c8eaa0e Fix missing appends after reference lookups 2017-02-02 11:10:17 +01:00
Fabian Reinartz
30efe4a58c Support writing to multiple head blocks
This is an initial (and hacky) first pass on allowing
appending to multiple blocks simultaniously to avoid
dropping samples right after cutting a new head block.
It's also required for cases like the PGW, where a scrape may
contain varying timestamps.
2017-02-01 15:57:28 +01:00
Fabian Reinartz
c20cc44b06 Add docs, write sequence number to meta.json 2017-01-29 08:11:47 +01:00
Fabian Reinartz
5fb01d41aa Use new Prometheus text format parser 2017-01-16 21:29:53 +01:00
Fabian Reinartz
fde69dab49 Use buffer pool for head appenders 2017-01-12 20:03:44 +01:00
Fabian Reinartz
a317f252b9 Expose series references to clients
This exposes a reference number of a series represented by a label set
to clients.
Subsequent samples can be directly added via the reference rather than
repeatedly passing in the full labels. This drasitcally speeds up the
append process.

The appender chain uses different sections of the reference number for
assignment to child appenders and invalidating reference numbers as
necessary.

Clients can either pass out reference numbers themselves or have their
own optimized lookup, i.e. by directly associating unparsed metric
descriptors strings with reference numbers.
2017-01-12 20:00:54 +01:00
Fabian Reinartz
80affd98a8 Add barrier to benchmark writer
This adds a barrier to avoid issues with unfair goroutine scheduling
that causes some fake scrapers to run away from the other ones.
2017-01-11 13:01:30 +01:00
Fabian Reinartz
c32a94d409 Unexport HeadBlock, export Block interface 2017-01-10 15:41:57 +01:00
Fabian Reinartz
d86e8a63c7 Report correct number of appended samples 2017-01-10 11:17:37 +01:00
Fabian Reinartz
300f4e2abf Use separate lock for series creation
This uses the head block's own lock to only lock if new series were
encountered.
In the general append case we just need to hold a
2017-01-06 18:10:50 +01:00
Fabian Reinartz
c61b310210 Naive size-based compaction
This adds naive compaction that tries to compact three
blocks of roughly equal size.
It decides based on samples present in a block and has no
safety measures considering the actual file size.
2017-01-06 13:53:05 +01:00
Fabian Reinartz
3ed2c2a14b Rename Partition to regular DB, DB to PartitionedDB 2017-01-06 11:40:09 +01:00
Fabian Reinartz
cabd7e4ebc Add small testdata file 2016-12-22 19:15:37 +01:00
Fabian Reinartz
ede733ab6c Extract labels package 2016-12-21 09:39:01 +01:00
Fabian Reinartz
ee217adc7e Redfine append interface, remove old Prometheus storage from bench 2016-12-21 00:02:37 +01:00
Fabian Reinartz
e561c91d53 Implement proper buffered iterator
This adds a proper duration based lookback buffer for series iterators
to allow advancing sequentially while remaining able to calculate time
aggregating functions such as `rate` backwards.

It uses an array ring buffer to minimize heap allocations for
potentially hundreds of thousands of series for a single query.
2016-12-14 21:14:44 +01:00
Fabian Reinartz
3a528c3078 Write plain postings list index 2016-12-10 09:44:00 +01:00
Fabian Reinartz
40a451694f Refactor persistence into interfaces 2016-12-09 20:45:46 +01:00
Fabian Reinartz
74f8dfd95d Persist blocks periodically 2016-12-09 13:41:38 +01:00
Fabian Reinartz
8aa99a3ebd misc 2016-12-09 10:00:14 +01:00
Fabian Reinartz
63b887eb62 Add Makefile 2016-12-08 12:00:05 +01:00
Fabian Reinartz
b845f8d3a1 Reduce test data allocations 2016-12-08 11:59:54 +01:00
Fabian Reinartz
ce82bdb71a Add write benchmark utility 2016-12-07 17:30:10 +01:00
Fabian Reinartz
ff29705571 Delete old benchmarking tool 2016-12-04 13:15:49 +01:00
Fabian Reinartz
40c27a1c2f Add benchmark utility 2016-11-15 10:35:42 +01:00