Commit Graph

692 Commits

Author SHA1 Message Date
Goutham Veeramachaneni
5b242f35ba Expose a Querier with manually passed in readers.
Allows people to not copy the querying code.

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-08-25 16:06:45 +05:30
Goutham Veeramachaneni
7438ed7035 Expose Intervals type for use by TombstoneReader.
TombstoneReader is exposed but Intervals is not.

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-08-25 16:06:36 +05:30
Goutham Veeramachaneni
79dc0ed420 Merge pull request #126 from ankon/pr/issue-125
Log the directory when the WAL cannot be opened
2017-08-22 14:52:30 +05:30
Andreas Kohn
4003da0fdd
Log the directory when the WAL cannot be opened
Fixes #125
2017-08-22 11:17:07 +02:00
Fabian Reinartz
c4ca881685 Merge pull request #120 from Gouthamve/fix-0-division
Fix panic caused by 0 division
2017-08-18 09:54:27 +02:00
Goutham Veeramachaneni
178f840295 Fix panic caused by 0 division
Introduced in #108

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-08-18 07:45:29 +02:00
Fabian Reinartz
e0aca4bee9 Merge pull request #119 from prometheus/allocsave
Allocsave
2017-08-11 12:42:10 +02:00
Fabian Reinartz
f9f439c6db Merge pull request #118 from prometheus/allocsave2
Avoid chunk allocations and refactor compactions
2017-08-10 14:45:54 +02:00
Fabian Reinartz
912302877b Merge pull request #116 from prometheus/allocsave
Persist series without allocating the full set
2017-08-10 10:08:49 +02:00
Fabian Reinartz
905af27cf9 Refactor compactor 2017-08-09 11:10:29 +02:00
Fabian Reinartz
66ff7b12e9 Pool Chunk objects during compaction 2017-08-08 17:35:34 +02:00
Fabian Reinartz
2644c8665c Don't allocate ChunkMetas, reuse postings slices 2017-08-06 20:41:24 +02:00
Fabian Reinartz
96d7f540d4 Persist series without allocating the full set
Change index persistence for series to not be accumulated in memory
before being written as one large batch. `Labels` and `ChunkMeta`
objects are reused.
This cuts down memory spikes during compaction of multiple blocks
significantly.

As part of the the Index{Reader,Writer} now have an explicit notion of
symbols and series must be inserted in order.
2017-08-06 12:06:41 +02:00
Fabian Reinartz
1875d05e00 Merge pull request #115 from prometheus/compactionfix
Fix compactions
2017-08-04 13:40:00 +02:00
Fabian Reinartz
3951d8cc29 Fix selection of compactable blocks 2017-08-04 12:02:34 +02:00
Fabian Reinartz
fa04455cd7 Properly detect compactable heads
The fullness of the wrong head was being tracked, causing wrong
lists of compactable heads.
2017-08-03 13:58:11 +02:00
Julius Volz
15baaa5677 Merge pull request #112 from joehandzik/patch-3
Fix typo in index.md
2017-07-24 23:38:54 +02:00
Joe Handzik
e6cb86c029 Fix typo in index.md
Minor change, monotinically -> monotonically.

Signed-Off-By: Joe Handzik <joseph.t.handzik@hpe.com>
2017-07-24 15:23:29 -05:00
Goutham Veeramachaneni
1373a0ea9f Merge pull request #111 from dilyevsky/add-prefix-matcher
Add prefix label matcher.
2017-07-23 17:59:31 +05:30
Dmitry Ilyevsky
37194b7a30 Add prefix label matcher.
Implement labels.PrefixMatcher and use interface conversion in querier
to optimize label tuples search.

[unit-tests]: Fix bug and populate label index for mock index.

Signed-off-by: Dmitry Ilyevsky <ilyevsky@gmail.com>
2017-07-22 01:06:30 -07:00
Goutham Veeramachaneni
1378338aab Merge pull request #110 from WJTian/master
Fix DefaultOptions
2017-07-22 00:07:38 +05:30
tianwenjie01
b64054365d Fix DefaultOptions 2017-07-18 17:42:58 +08:00
Fabian Reinartz
8d3a398113 Merge pull request #109 from prometheus/fixes
Fix compactions
2017-07-14 13:45:46 +02:00
Fabian Reinartz
74f67e8271 Fix compacting disable/enable
Enabling and disabling compaction no longer blocks are potentially
causes panics.
2017-07-14 10:08:05 +02:00
Fabian Reinartz
3065be97d8 Fix and document locking order for DB 2017-07-14 09:00:22 +02:00
Fabian Reinartz
47afc8e00f Reduce test timeout on Travis 2017-07-14 08:00:18 +02:00
Fabian Reinartz
9c4235532e Fix compaction selection after creating new heads
This fixes the case where between block creations no compaction
plans are ran. We were not compacting anything in these
cases since the on creation the most recent head block always had
a high timestamp of 0.
2017-07-13 16:15:13 +02:00
Fabian Reinartz
5d7b5994d6 Fix compaction range selection 2017-07-13 16:13:59 +02:00
Goutham Veeramachaneni
3c22488157 Merge pull request #108 from Gouthamve/compact-large-dels-2
Recompact blocks with large number of tombstones
2017-07-12 18:54:01 +02:00
Goutham Veeramachaneni
401bdfd4a6 Make Chunks return the number of samples inside
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-07-12 18:45:12 +02:00
Goutham Veeramachaneni
1263a68875 Recompact blocks with large number of tombstones
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-07-12 18:27:31 +02:00
Goutham Veeramachaneni
7b9c536883 Merge pull request #103 from Gouthamve/block-ranges
Compaction implementation for block-ranges
2017-07-12 17:32:49 +02:00
Goutham Veeramachaneni
9a2ab732b5 Choose ranges by retention and min-block-duration
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-07-12 17:30:34 +02:00
Goutham Veeramachaneni
969c407335 Merge pull request #106 from Gouthamve/snapshot-time
Persist the right MaxTime when snapshotting
2017-07-12 13:54:31 +02:00
Goutham Veeramachaneni
f1ae239c20 Persist the right MaxTime when snapshotting
This is because we cut a new block from where the snapshotted block ends
if we restore from backups and highTimestamp would be where we should be
 starting from.

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-07-12 13:48:13 +02:00
Goutham Veeramachaneni
ce23adb579 Initial compaction implementation for block-ranges
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-07-07 16:32:18 +02:00
Fabian Reinartz
bc3fee4820 Add benchmark for mergedSeriesSet 2017-07-05 16:19:28 +02:00
Fabian Reinartz
d492bfd973 Merge pull request #101 from prometheus/chunkit
Fix omitting of chunk on Seek()
2017-06-30 15:17:34 +02:00
Fabian Reinartz
344a50b9d8 Merge pull request #100 from prometheus/fullness
Compact head block early
2017-06-30 15:07:22 +02:00
Fabian Reinartz
bda3ed20ac Fix omitting of chunk on Seek() 2017-06-30 15:06:27 +02:00
Fabian Reinartz
1e74c155eb Return empty string to signal non-caching 2017-06-26 14:58:00 +02:00
Fabian Reinartz
3410559c1b Compact head block early
Let older head blocks be compacted once the newest once has samples at
50% of its total range. This allows the memory of the compacted blocks
to be released and garbage collected before a new head block gets
created. Thereby the number of head blocks is 1 or 2 instead of 2 or 3
and memory spikes are reduced.
2017-06-26 08:52:59 +02:00
Brian Brazil
60c823c663 Fix broken test (#99) 2017-06-22 15:24:38 +01:00
Fabian Reinartz
a2948f3c5f Merge pull request #97 from Gouthamve/nit
Fix bug with Seek() and optimise bounding params.
2017-06-13 10:11:10 +02:00
Fabian Reinartz
3a36551b90 Merge pull request #98 from prometheus/balance
Properly balance k-way operations
2017-06-13 10:09:55 +02:00
Goutham Veeramachaneni
a280169427
Add regression test
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-06-13 13:21:22 +05:30
Fabian Reinartz
eb8c9759fc Properly balance k-way operations 2017-06-13 08:25:13 +02:00
Goutham Veeramachaneni
9e1f34dae3
Fix bug with Seek() and optimise bounding params.
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-06-13 11:24:04 +05:30
Fabian Reinartz
9963a4c7c3 Merge pull request #95 from Gouthamve/wal-ahead
Fix race condition for 2 appenders having same ts
2017-06-12 11:17:49 +02:00
Goutham Veeramachaneni
73cc5bae51 Colocate defer statements near relevant functions
Signed-off-by: Goutham Veeramachaneni <goutham@boomerangcommerce.com>
2017-06-12 14:37:58 +05:30