66ff7b12e9
Pool Chunk objects during compaction
2017-08-08 17:35:34 +02:00
fa04455cd7
Properly detect compactable heads
...
The fullness of the wrong head was being tracked, causing wrong
lists of compactable heads.
2017-08-03 13:58:11 +02:00
b64054365d
Fix DefaultOptions
2017-07-18 17:42:58 +08:00
74f67e8271
Fix compacting disable/enable
...
Enabling and disabling compaction no longer blocks are potentially
causes panics.
2017-07-14 10:08:05 +02:00
3065be97d8
Fix and document locking order for DB
2017-07-14 09:00:22 +02:00
9c4235532e
Fix compaction selection after creating new heads
...
This fixes the case where between block creations no compaction
plans are ran. We were not compacting anything in these
cases since the on creation the most recent head block always had
a high timestamp of 0.
2017-07-13 16:15:13 +02:00
7b9c536883
Merge pull request #103 from Gouthamve/block-ranges
...
Compaction implementation for block-ranges
2017-07-12 17:32:49 +02:00
9a2ab732b5
Choose ranges by retention and min-block-duration
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-07-12 17:30:34 +02:00
f1ae239c20
Persist the right MaxTime when snapshotting
...
This is because we cut a new block from where the snapshotted block ends
if we restore from backups and highTimestamp would be where we should be
starting from.
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-07-12 13:48:13 +02:00
ce23adb579
Initial compaction implementation for block-ranges
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-07-07 16:32:18 +02:00
1e74c155eb
Return empty string to signal non-caching
2017-06-26 14:58:00 +02:00
3410559c1b
Compact head block early
...
Let older head blocks be compacted once the newest once has samples at
50% of its total range. This allows the memory of the compacted blocks
to be released and garbage collected before a new head block gets
created. Thereby the number of head blocks is 1 or 2 instead of 2 or 3
and memory spikes are reduced.
2017-06-26 08:52:59 +02:00
03f95ee096
Fix directory file descriptor leak.
2017-06-12 00:05:04 +02:00
d4cb579443
Add DB.Dir method
2017-06-08 12:14:13 +02:00
ff4ccb6eb0
Remove unnecessary error from ToggleCompaction fns
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-06-06 23:45:23 +05:30
261cd9f393
Incorporate feedback.
...
Move back to {Enable, Disable}Compactions.
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-06-06 20:23:20 +05:30
a110a64abd
Add full Snapshot support
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-06-06 18:15:54 +05:30
a1c8425357
Initial implementation of HeadBlock Snapshots
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-06-05 13:48:31 +05:30
3a5ae6b1a4
Merge pull request #82 from Gouthamve/deletes-1
...
Implement deletions
2017-05-27 22:06:45 +05:30
b3b9245ad6
Add and fix metrics
2017-05-26 15:13:03 +02:00
6febabeb28
Final delete fixes.
...
* Make sure no reads happen on the block when delete is in progress.
* Fix bugs in compaction.
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-26 16:31:45 +05:30
9bf7aa9af1
Misc. fixes incorporating feedback.
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-23 18:13:30 +05:30
8434019ad9
Merge branch 'master' into deletes-1
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-22 12:58:38 +05:30
0b70333ef6
Add tests for tombstones.
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-21 23:20:05 +05:30
d32eb25662
Synchronise deletes and compactions.
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-20 13:21:10 +05:30
3dbb400bef
Initial implementation of Delete on DB.
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-20 00:35:50 +05:30
debfe9b1e5
Simplify AddFast and avoid errgroup Commits in general
2017-05-18 17:17:56 +02:00
fb5c5535fc
Misc cleanup
2017-05-18 16:38:45 +02:00
39df7e2bba
Switch blocks to ULID directories, drop sequenc numbers
2017-05-18 16:09:30 +02:00
285bc07030
Switch append refs to string
2017-05-18 10:56:57 +02:00
8b51b7e2be
Make WAL for HeadBlock composeable.
2017-05-13 18:14:18 +02:00
5534e6c53c
Make HeadBlock impl public, make interface private
2017-05-12 16:34:41 +02:00
44769c1654
Allow disabling lock file
2017-05-09 12:52:47 +02:00
35b62f001e
Change offset table layout, add TOC, ...
2017-04-30 10:18:07 +02:00
54a7b98cff
Fix and test ref handling in DB appender
2017-04-28 15:24:28 +02:00
d04bd947ea
Merge pull request #51 from prometheus/rollback
...
Make rollback actually rollback.
2017-04-19 11:29:03 +01:00
dba222b53c
Make rollback actually rollback.
...
Also add tests for basic commit/rollback
functionality.
Fixes #39
2017-04-19 11:28:36 +01:00
46c947d94f
Merge pull request #45 from mattbostock/document_appender_not_goroutine_safe
...
db: Add comment: Appender not goroutine safe
2017-04-17 12:12:20 +02:00
4d5ca4b9c1
db: Fix typo: reference
2017-04-14 22:02:06 +01:00
3dd5a4ee09
db: Add comment: Appender not goroutine safe
2017-04-14 21:39:08 +01:00
778103b450
Add liecence file and headers
2017-04-10 20:59:45 +02:00
10c7c9acbe
Adjust import names to new repository organisation
2017-04-04 11:27:26 +02:00
767b88a3a5
Parallelize commits to prevent deadlocks
...
This parallelizes commits to prevent deadlocks across inconsistently
locked heads. As commits are currently not fully atomic across
heads, this does decrease our guarantees.
2017-04-04 11:21:19 +02:00
a52980e0a8
Add workaround for deadlocks
...
This adds a workaround to avoid deadlocks for inconsistent write lock
order across headBlocks.
Things keep working if transactions only append data for the same
timestamp, which is generally the case for Prometheus.
Full behavior should be restored in a subsequent change.
2017-03-27 19:05:34 +02:00
e478d0e3bc
Actually close olds blocks in reloadBlocks
...
This fixes a bug leaking memory because blocks were not actually closed
as the closing call references the initial, empty slice
2017-03-23 18:27:20 +01:00
70909ca8ad
Ensure GC runs after each compactor call
...
GC is triggered rarely, which may cause unnecessarily high memory
spikes when running several compaction cycles in a row. Explicitly run
GC so we don't have idle bytes marked as used from the previous cycle.
2017-03-21 12:21:02 +01:00
55ee4b5b3b
Merge branch 'master' of github.com:fabxc/tsdb
2017-03-21 10:11:39 +01:00
c18e055d7c
Fix races and add comments on remaining ones
2017-03-21 10:11:23 +01:00
e837034360
Merge pull request #14 from Gouthamve/log-update
...
Update kit/log To New API
2017-03-21 09:56:32 +01:00
9c93f8f2aa
Fix various races
...
This fixes different race condition encoutnered when running Prometheus.
It reduces the overall performance in the synthetic benchmark a fair bit
but has no indiciations of impacting a real-world setup notably.
2017-03-20 14:45:27 +01:00
Fabian Reinartz
tianwenjie01
Goutham Veeramachaneni
Brian Brazil
Brian Brazil
Matt Bostock