1e88ba06b4
Use boolean function instead of postings to drop WAL series
...
There is not guarantee or requirement for WAL writers to only add
series entries in increasing order of IDs. A postings list cannot look
back and thus unordered WAL entries would skip over IDs to not truncate
from the WAL.
We replace it with a simple boolean check function that does not require
order.
2017-09-21 13:31:01 +02:00
3d8be398d6
Merge branch 'locks' of github.com:prometheus/tsdb into locks
2017-09-11 10:39:55 +02:00
24362567b9
Fix test flakes
2017-09-11 10:33:17 +02:00
6892fc6dcb
Finish old WAL segment async, default to no fsync
...
We were still fsyncing while holding the write lock when we cut a new
segment. Given we cannot do anything but logging errors, we might just
as well complete segments asynchronously.
There's not realistic use case where one would fsync after every WAL
entry, thus make the default of a flush interval of 0 to never fsync
which is a much more likely use case.
2017-09-08 18:41:12 +02:00
c2916736be
Re-use slices on deocding the WAL. Fix tests.
2017-09-08 08:17:01 +02:00
0db4c227b7
Fix min/max time handling and concurrent crc32 usage
2017-09-07 13:04:02 +02:00
970bffec8d
Fix WAL errors and add tests for it
2017-09-07 10:58:34 +02:00
33e9bdf403
WAL refactoring and truncation fixes and test
2017-09-06 14:59:25 +02:00
1ddedf2b30
Change series ID from uint32 to uint64
2017-09-04 16:08:38 +02:00
1698c516ad
[WIP]: WAL implementation
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-09-04 14:52:40 +02:00
5cf2662074
Refactor WAL into Head and misc improvements
2017-09-01 11:50:58 +02:00
215a856f34
Just check for general error in WAL test
...
Go 1.9 changed the returned error. Checking the exact message is too
fragile.
2017-08-27 16:57:43 +02:00
7438ed7035
Expose Intervals type for use by TombstoneReader.
...
TombstoneReader is exposed but Intervals is not.
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-08-25 16:06:36 +05:30
44e9ae38b5
Incorporate PR feedback.
...
* Expose Stone as it is used in an exported method.
* Move from tombstoneReader to []Stone for the same reason as above.
* Make WAL reading a little cleaner.
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-26 21:26:31 +05:30
bacb143b7e
Simplify tombstone and WAL Delete formats.
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-24 13:42:56 +05:30
f29fb62fba
Make TombstoneReader a Getter.
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-24 11:24:24 +05:30
3eb4119ab1
Make HeadBlock use WAL.
...
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-05-23 16:15:16 +05:30
fb5c5535fc
Misc cleanup
2017-05-18 16:38:45 +02:00
4862b261d0
Abstract WAL into interface
2017-05-13 17:09:26 +02:00
535532ca02
Export refdSample
...
The type was part of a exported method signatures and should therefore
be exported as well.
2017-05-12 17:06:26 +02:00
778103b450
Add liecence file and headers
2017-04-10 20:59:45 +02:00
10c7c9acbe
Adjust import names to new repository organisation
2017-04-04 11:27:26 +02:00
ad5812d83a
Handle WAL corruption by truncating
...
This adds handling for various corruption scenarios of the WAL.
If corruption is encountered, we truncate the WAL after the last valid
entry transparently and continue appending after the offset.
2017-03-15 11:16:16 +01:00
d3edfb5540
Test multiple full write/restore cylcles in WAL
2017-03-08 21:02:08 +01:00
9c7a88223e
Add full encode/decode WAL cycle test
2017-02-14 21:55:50 -08:00
2c97428a79
Add basic WAL tests
2017-02-14 16:12:56 -08:00
9ddbd64d00
Move stats into meta.json file, cleanup, docs
2017-01-19 11:22:47 +01:00
2f02f86b62
Fix WAL tests
2017-01-19 08:48:11 +01:00
1943f8d1bb
Fix head block stats races
2017-01-07 18:02:17 +01:00
937cdb579c
Switch to sequential block names
...
This changes block directory names from the int64 timestamp
to sequential numbering.
2017-01-06 10:45:03 +01:00
7280533c42
Add basic shard metrics
2016-12-31 09:48:49 +01:00
8aba95048a
Rename to OpenHeadBlock
2016-12-22 20:00:24 +01:00
1dde3b6d31
Add WAL decoder+loading and benchmarks
2016-12-22 15:18:33 +01:00
Fabian Reinartz
Goutham Veeramachaneni