1698c516ad
[WIP]: WAL implementation
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-09-04 14:52:40 +02:00
893b6ec506
Add tests for GC and chunk truncation
2017-09-01 14:38:49 +02:00
4cc37eecab
Refactor and add tests for compactor
2017-09-01 11:50:58 +02:00
5cf2662074
Refactor WAL into Head and misc improvements
2017-09-01 11:50:58 +02:00
8209e3ec23
Add various metrics
2017-09-01 11:50:58 +02:00
3901b6e70b
Remove multiple heads
...
This changes the structure to a single WAL backed by a single head
block.
Parts of the head block can be compacted. This relieves us from any head
amangement and greatly simplifies any consistency and isolation concerns
by just having a single head.
2017-09-01 11:50:58 +02:00
4003da0fdd
Log the directory when the WAL cannot be opened
...
Fixes #125
2017-08-22 11:17:07 +02:00
905af27cf9
Refactor compactor
2017-08-09 11:10:29 +02:00
66ff7b12e9
Pool Chunk objects during compaction
2017-08-08 17:35:34 +02:00
fa04455cd7
Properly detect compactable heads
...
The fullness of the wrong head was being tracked, causing wrong
lists of compactable heads.
2017-08-03 13:58:11 +02:00
b64054365d
Fix DefaultOptions
2017-07-18 17:42:58 +08:00
74f67e8271
Fix compacting disable/enable
...
Enabling and disabling compaction no longer blocks are potentially
causes panics.
2017-07-14 10:08:05 +02:00
3065be97d8
Fix and document locking order for DB
2017-07-14 09:00:22 +02:00
9c4235532e
Fix compaction selection after creating new heads
...
This fixes the case where between block creations no compaction
plans are ran. We were not compacting anything in these
cases since the on creation the most recent head block always had
a high timestamp of 0.
2017-07-13 16:15:13 +02:00
7b9c536883
Merge pull request #103 from Gouthamve/block-ranges
...
Compaction implementation for block-ranges
2017-07-12 17:32:49 +02:00
9a2ab732b5
Choose ranges by retention and min-block-duration
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-07-12 17:30:34 +02:00
f1ae239c20
Persist the right MaxTime when snapshotting
...
This is because we cut a new block from where the snapshotted block ends
if we restore from backups and highTimestamp would be where we should be
starting from.
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-07-12 13:48:13 +02:00
ce23adb579
Initial compaction implementation for block-ranges
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-07-07 16:32:18 +02:00
1e74c155eb
Return empty string to signal non-caching
2017-06-26 14:58:00 +02:00
3410559c1b
Compact head block early
...
Let older head blocks be compacted once the newest once has samples at
50% of its total range. This allows the memory of the compacted blocks
to be released and garbage collected before a new head block gets
created. Thereby the number of head blocks is 1 or 2 instead of 2 or 3
and memory spikes are reduced.
2017-06-26 08:52:59 +02:00
03f95ee096
Fix directory file descriptor leak.
2017-06-12 00:05:04 +02:00
d4cb579443
Add DB.Dir method
2017-06-08 12:14:13 +02:00
ff4ccb6eb0
Remove unnecessary error from ToggleCompaction fns
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-06-06 23:45:23 +05:30
261cd9f393
Incorporate feedback.
...
Move back to {Enable, Disable}Compactions.
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-06-06 20:23:20 +05:30
a110a64abd
Add full Snapshot support
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-06-06 18:15:54 +05:30
a1c8425357
Initial implementation of HeadBlock Snapshots
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-06-05 13:48:31 +05:30
3a5ae6b1a4
Merge pull request #82 from Gouthamve/deletes-1
...
Implement deletions
2017-05-27 22:06:45 +05:30
b3b9245ad6
Add and fix metrics
2017-05-26 15:13:03 +02:00
6febabeb28
Final delete fixes.
...
* Make sure no reads happen on the block when delete is in progress.
* Fix bugs in compaction.
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-26 16:31:45 +05:30
9bf7aa9af1
Misc. fixes incorporating feedback.
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-23 18:13:30 +05:30
8434019ad9
Merge branch 'master' into deletes-1
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-22 12:58:38 +05:30
0b70333ef6
Add tests for tombstones.
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-21 23:20:05 +05:30
d32eb25662
Synchronise deletes and compactions.
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-20 13:21:10 +05:30
3dbb400bef
Initial implementation of Delete on DB.
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-20 00:35:50 +05:30
debfe9b1e5
Simplify AddFast and avoid errgroup Commits in general
2017-05-18 17:17:56 +02:00
fb5c5535fc
Misc cleanup
2017-05-18 16:38:45 +02:00
39df7e2bba
Switch blocks to ULID directories, drop sequenc numbers
2017-05-18 16:09:30 +02:00
285bc07030
Switch append refs to string
2017-05-18 10:56:57 +02:00
8b51b7e2be
Make WAL for HeadBlock composeable.
2017-05-13 18:14:18 +02:00
5534e6c53c
Make HeadBlock impl public, make interface private
2017-05-12 16:34:41 +02:00
44769c1654
Allow disabling lock file
2017-05-09 12:52:47 +02:00
35b62f001e
Change offset table layout, add TOC, ...
2017-04-30 10:18:07 +02:00
54a7b98cff
Fix and test ref handling in DB appender
2017-04-28 15:24:28 +02:00
d04bd947ea
Merge pull request #51 from prometheus/rollback
...
Make rollback actually rollback.
2017-04-19 11:29:03 +01:00
dba222b53c
Make rollback actually rollback.
...
Also add tests for basic commit/rollback
functionality.
Fixes #39
2017-04-19 11:28:36 +01:00
46c947d94f
Merge pull request #45 from mattbostock/document_appender_not_goroutine_safe
...
db: Add comment: Appender not goroutine safe
2017-04-17 12:12:20 +02:00
4d5ca4b9c1
db: Fix typo: reference
2017-04-14 22:02:06 +01:00
3dd5a4ee09
db: Add comment: Appender not goroutine safe
2017-04-14 21:39:08 +01:00
778103b450
Add liecence file and headers
2017-04-10 20:59:45 +02:00
10c7c9acbe
Adjust import names to new repository organisation
2017-04-04 11:27:26 +02:00
Goutham Veeramachaneni
Fabian Reinartz
Andreas Kohn
tianwenjie01
Brian Brazil
Brian Brazil
Matt Bostock