27f1b8aac3
Merge pull request #162 from BasPH/fsync-duration
...
Instrument WAL fsync
2017-10-05 08:18:36 +02:00
5e1c258a98
Instrument WAL fsync
2017-10-04 22:17:02 +02:00
da565f975e
Merge pull request #161 from prometheus/fileutil
...
Remove dependency on etcd/pkg/fileutil
2017-10-04 17:08:54 +05:30
f04ec031eb
compact: sync temporary directory
2017-10-04 12:22:09 +02:00
665d1fd451
Merge pull request #158 from prometheus/cachesym
...
Allocate and cache strings for persisted blocks
2017-10-04 12:19:50 +02:00
bbe72dccb9
Remove dependency on etcd/pkg/fileutil
2017-10-04 10:23:41 +02:00
3b7e71fee9
Merge pull request #159 from dim/fix/wal-flush
...
Use configurable WAL flush interval
2017-10-03 16:17:55 +05:30
83c984d9a3
Use configurable WAL flush inteval
2017-10-03 11:35:38 +01:00
e19b648377
Merge pull request #155 from Gouthamve/level-log
...
Add levels to all log lines.
2017-10-02 16:34:49 +02:00
78df406dac
Allocate and cache strings for persisted blocks
...
This change loads the full symbol table when we open a persisted block
and allocates a string for each. This ensures that strings retrieved
through the index can be used after the block was closed.
Before we backed the strings by the mmap'd byte regions which would
segfault in this case.
Also remove an inconsistency in the disk format and move both offset
tables to the end (breaking change).
2017-10-02 15:56:57 +02:00
c521ac495f
Merge pull request #152 from criteo-forks/master
...
Add db.Head() method
2017-09-28 18:10:53 +05:30
c35d3a65bd
Add levels to all log lines.
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-09-28 12:49:34 +05:30
0ff8f91ec8
Add db.Head() method
...
Signed-off-by: Thibault Chataigner <t.chataigner@criteo.com>
2017-09-25 16:45:24 +02:00
87c01dd5fb
Merge pull request #134 from criteo-forks/master
...
Expose `Blocks()` on DB
2017-09-25 17:41:25 +05:30
69f105f4f9
Merge pull request #151 from prometheus/waltrunc
...
Use boolean function instead of postings to drop WAL series
2017-09-21 15:04:43 +02:00
1e88ba06b4
Use boolean function instead of postings to drop WAL series
...
There is not guarantee or requirement for WAL writers to only add
series entries in increasing order of IDs. A postings list cannot look
back and thus unordered WAL entries would skip over IDs to not truncate
from the WAL.
We replace it with a simple boolean check function that does not require
order.
2017-09-21 13:31:01 +02:00
5ca18837e9
Make DB blocks list public.
...
This enable computing the first timestamp in the DB
only accessing public interfaces.
Signed-off-by: Thibault Chataigner <t.chataigner@criteo.com>
2017-09-21 11:56:12 +02:00
5fa1c993b9
Merge pull request #150 from prometheus/postingssort
...
Ensure postings are always sorted
2017-09-21 09:37:31 +02:00
6ee254e353
Ensure postings are always sorted
...
IDs for new series are handed out before the postings are locked. Thus
series are not indexed in order of their IDs, which could result in only
partially sorted postings list.
Iterating over those silently skipped elements as the sort invariant was
violated.
2017-09-21 09:38:18 +02:00
f39388c9af
Merge pull request #147 from prometheus/deadlock
...
Simplify series create logc in head
2017-09-19 19:11:53 +05:30
162a48e4f2
Create series with ID recorded in WAL when reading it back
2017-09-19 11:31:16 +02:00
7ada9cd805
Simplify series create logic in head
2017-09-18 12:38:39 +02:00
2187388292
Merge pull request #146 from prometheus/deadlock
...
Add missing unlock on early return
2017-09-18 15:02:32 +05:30
ab8d9b9706
Add missing unlock on early return
2017-09-18 11:23:22 +02:00
99d39174f6
Merge branch 'master' of github.com:prometheus/tsdb
2017-09-18 11:20:45 +02:00
8214dc82a7
Remove infinite block in benchmark
2017-09-18 11:20:25 +02:00
643563068b
Merge pull request #144 from Gouthamve/misc-
...
Expose NewIndexReader() and cleanups
2017-09-13 10:23:07 +02:00
8919baef03
Expose NewIndexReader() and cleanups
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-09-13 13:47:20 +05:30
e45bb1d328
Merge pull request #142 from krasi-georgiev/107-swap-cobra-with-kingpin
...
replaced cobra with kingpin for the tsdb cli tool
2017-09-12 19:28:35 +02:00
3870ec285c
Merge pull request #140 from prometheus/locks
...
Fix various races
2017-09-11 10:41:33 +02:00
3d8be398d6
Merge branch 'locks' of github.com:prometheus/tsdb into locks
2017-09-11 10:39:55 +02:00
24362567b9
Fix test flakes
2017-09-11 10:33:17 +02:00
30d29b889c
Merge pull request #141 from prometheus/dectmpstr
...
Fixup allocation regression during compaction
2017-09-11 10:06:49 +02:00
bfe75cec35
Merge pull request #138 from Gouthamve/chunk-compress
...
Compress the series chunk details in index.
2017-09-11 10:01:42 +02:00
92d0414993
replaced cobra with kingpin
...
Signed-off-by: Krasi Georgiev <krasi.root@gmail.com>
2017-09-10 12:35:02 +03:00
b09d90c79c
Add decoding method to retrieve unsafe strings
...
When decoding data from mmaped blocks, we would like to retrieve
a string backed by the mmaped region. As the underlying byte slice
never changes, this is safe.
2017-09-08 18:41:43 +02:00
f904cd385f
Do not build a superflous 'all' postings
2017-09-08 18:41:43 +02:00
6892fc6dcb
Finish old WAL segment async, default to no fsync
...
We were still fsyncing while holding the write lock when we cut a new
segment. Given we cannot do anything but logging errors, we might just
as well complete segments asynchronously.
There's not realistic use case where one would fsync after every WAL
entry, thus make the default of a flush interval of 0 to never fsync
which is a much more likely use case.
2017-09-08 18:41:12 +02:00
afaf12fe45
Compress the series chunk details in index.
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-09-08 20:25:19 +05:30
1d5f85817d
Fix various races
2017-09-08 08:48:19 +02:00
c2916736be
Re-use slices on deocding the WAL. Fix tests.
2017-09-08 08:17:01 +02:00
0db4c227b7
Fix min/max time handling and concurrent crc32 usage
2017-09-07 13:04:02 +02:00
970bffec8d
Fix WAL errors and add tests for it
2017-09-07 10:58:34 +02:00
81222849bc
Filter WAL data in Head, misc fixes
2017-09-06 16:20:37 +02:00
33e9bdf403
WAL refactoring and truncation fixes and test
2017-09-06 14:59:25 +02:00
c36d574290
Replace single head lock with granular locks
...
This adds various new locks to replace the single big lock on
the head. All parts now must be COW as they may be held by clients
after initial retrieval.
Series by ID and hashes are now held in a stripe lock to reduce
contention and total holding time during GC. This should reduce
starvation of readers.
2017-09-05 14:41:39 +02:00
1ddedf2b30
Change series ID from uint32 to uint64
2017-09-04 16:08:38 +02:00
af2c2f9674
Re-enable head block delet test
2017-09-04 15:07:30 +02:00
1698c516ad
[WIP]: WAL implementation
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-09-04 14:52:40 +02:00
9f41d9fd3c
Instrument chunks on level 1 compactions
2017-09-01 16:10:10 +02:00
Fabian Reinartz
Bas Harenslak
Goutham Veeramachaneni
Dimitrij Denissenko
Thibault Chataigner
Krasi Georgiev