RepoMirrors/prometheus
1
0
Fork 0
mirror of https://github.com/prometheus/prometheus synced 2024-12-29 02:02:17 +00:00
Code Issues Packages Projects Releases Wiki Activity
693 Commits 243 Branches 0 Tags 653 MiB
6e5164fe44
Commit Graph

106 Commits

Author SHA1 Message Date
Bartek Plotka
328c0ff5b0 Simplified the flow and tests. 
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
 2018-03-13 14:12:32 +00:00
Bartek Plotka
483da43660 compact: Exclude last block with shortest range instead of newest one by ULID. 
Fixes https://github.com/prometheus/prometheus/issues/3943 issue.

Added tests.

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
 2018-03-13 12:30:27 +00:00
Fabian Reinartz
8cd32d8fd4 Improve usability of snapshots 2018-02-28 12:07:34 +01:00
Fabian Reinartz
d09283f60a Add repair of bad index version 2018-02-09 13:11:03 +01:00
Goutham Veeramachaneni
6c62437101 Fix metric unit (#275) 
Fixes #274

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2018-02-05 08:36:18 +00:00
Simon Pasquier
5d47710f8a Fix panic on nil index writer 2018-01-24 15:55:04 +01:00
Shubheksha Jalan
44052bc937 Make version for index format configurable 2018-01-11 00:49:16 +05:30
Fabian Reinartz
1e55b7987f Improve comments, handle allPostingsKey properly 2017-12-22 09:43:34 +01:00
Fabian Reinartz
7087f68647 Return new block ULID in compactor 2017-12-21 12:02:05 +01:00
Fabian Reinartz
67f0ca8f0e Move index and chunk encoders to own packages 2017-12-21 11:27:54 +01:00
Goutham Veeramachaneni
f35e9c7fec
Merge branch 'master' into fix-dels 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-11-30 13:30:13 +05:30
Goutham Veeramachaneni
24ff293dc9
Merge pull request #207 from Gouthamve/compact-fail 
Don't retry failed compactions.
 2017-11-30 12:58:19 +05:30
Fabian Reinartz
cf7f6108fd Order postings lists in index file by key 
Aligning postings list for similar keys close to each other improves
page cache hit rates in typical queries that select postings for
multiple label pairs with the same name.
 2017-11-29 08:28:55 +01:00
Goutham Veeramachaneni
1fc94a02d1
Don't retry failed compactions. 
Fixes prometheus/prometheus#3487

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-11-23 15:16:20 +05:30
Goutham Veeramachaneni
1627a47640
A copy is being assigned newChunk not the original 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-11-22 17:58:06 +05:30
Fabian Reinartz
a031cf7424 Return ULID in Compactor 2017-11-14 15:25:30 +01:00
Fabian Reinartz
f1512a368a Expose ChunkSeriesSet and lookups methods. 2017-11-13 14:02:32 +01:00
Fabian Reinartz
3ef4326114 Refactor tombstone reader types 2017-11-13 13:38:07 +01:00
ranbochen
a27cf34a36 fix bugs on platform windows to pass all test case. (#192) 
* fix bugs on platform windows to pass all test case.

* fix bugs on platform windows to pass all test case

* clean up codes
 2017-10-31 15:37:41 +01:00
Fabian Reinartz
d17104f1f0 Prefix all metrics with prometheus_* 2017-10-20 12:32:32 +02:00
Fabian Reinartz
fb9da52b11 Add more verbose error handling for closing, reduce locking 
This commit introduces error returns in various places and is explicit
about closing persisted blocks.
{Index,Chunk,Tombstone}Readers are more consistent about their Close()
method. Whenever a reader is retrieved, the corresponding close method
must eventually be called. We use this to track pending readers against
persisted blocks.

Querier's against the DB no longer hold a read lock for their entire
lifecycle. This avoids long running queriers to starve new ones when we
have to acquire a write lock when reloading blocks.
 2017-10-10 12:13:37 +02:00
Goutham Veeramachaneni
da565f975e Merge pull request #161 from prometheus/fileutil 
Remove dependency on etcd/pkg/fileutil
 2017-10-04 17:08:54 +05:30
Fabian Reinartz
f04ec031eb compact: sync temporary directory 2017-10-04 12:22:09 +02:00
Fabian Reinartz
bbe72dccb9 Remove dependency on etcd/pkg/fileutil 2017-10-04 10:23:41 +02:00
Goutham Veeramachaneni
c35d3a65bd
Add levels to all log lines. 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-09-28 12:49:34 +05:30
Fabian Reinartz
6ee254e353 Ensure postings are always sorted 
IDs for new series are handed out before the postings are locked. Thus
series are not indexed in order of their IDs, which could result in only
partially sorted postings list.
Iterating over those silently skipped elements as the sort invariant was
violated.
 2017-09-21 09:38:18 +02:00
Fabian Reinartz
f904cd385f Do not build a superflous 'all' postings 2017-09-08 18:41:43 +02:00
Fabian Reinartz
c36d574290 Replace single head lock with granular locks 
This adds various new locks to replace the single big lock on
the head. All parts now must be COW as they may be held by clients
after initial retrieval.
Series by ID and hashes are now held in a stripe lock to reduce
contention and total holding time during GC. This should reduce
starvation of readers.
 2017-09-05 14:41:39 +02:00
Fabian Reinartz
1ddedf2b30 Change series ID from uint32 to uint64 2017-09-04 16:08:38 +02:00
Fabian Reinartz
9f41d9fd3c Instrument chunks on level 1 compactions 2017-09-01 16:10:10 +02:00
Fabian Reinartz
4cc37eecab Refactor and add tests for compactor 2017-09-01 11:50:58 +02:00
Fabian Reinartz
3901b6e70b Remove multiple heads 
This changes the structure to a single WAL backed by a single head
block.
Parts of the head block can be compacted. This relieves us from any head
amangement and greatly simplifies any consistency and isolation concerns
by just having a single head.
 2017-09-01 11:50:58 +02:00
Goutham Veeramachaneni
7438ed7035 Expose Intervals type for use by TombstoneReader. 
TombstoneReader is exposed but Intervals is not.

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-08-25 16:06:36 +05:30
Goutham Veeramachaneni
178f840295 Fix panic caused by 0 division 
Introduced in #108

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-08-18 07:45:29 +02:00
Fabian Reinartz
905af27cf9 Refactor compactor 2017-08-09 11:10:29 +02:00
Fabian Reinartz
66ff7b12e9 Pool Chunk objects during compaction 2017-08-08 17:35:34 +02:00
Fabian Reinartz
2644c8665c Don't allocate ChunkMetas, reuse postings slices 2017-08-06 20:41:24 +02:00
Fabian Reinartz
96d7f540d4 Persist series without allocating the full set 
Change index persistence for series to not be accumulated in memory
before being written as one large batch. `Labels` and `ChunkMeta`
objects are reused.
This cuts down memory spikes during compaction of multiple blocks
significantly.

As part of the the Index{Reader,Writer} now have an explicit notion of
symbols and series must be inserted in order.
 2017-08-06 12:06:41 +02:00
Fabian Reinartz
3951d8cc29 Fix selection of compactable blocks 2017-08-04 12:02:34 +02:00
Fabian Reinartz
5d7b5994d6 Fix compaction range selection 2017-07-13 16:13:59 +02:00
Goutham Veeramachaneni
401bdfd4a6 Make Chunks return the number of samples inside 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-07-12 18:45:12 +02:00
Goutham Veeramachaneni
1263a68875 Recompact blocks with large number of tombstones 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-07-12 18:27:31 +02:00
Goutham Veeramachaneni
9a2ab732b5 Choose ranges by retention and min-block-duration 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-07-12 17:30:34 +02:00
Goutham Veeramachaneni
ce23adb579 Initial compaction implementation for block-ranges 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-07-07 16:32:18 +02:00
Fabian Reinartz
03f95ee096 Fix directory file descriptor leak. 2017-06-12 00:05:04 +02:00
Fabian Reinartz
05e411a8eb Improve heuristic to spread chunks across block 2017-06-08 11:30:32 +02:00
Fabian Reinartz
f006e2d1ab Merge pull request #92 from prometheus/sources 
Add list of source head blocks to meta.json
 2017-06-07 11:50:44 +02:00
Fabian Reinartz
683b4c8c2a Add list of source head blocks to meta.json 2017-06-07 11:06:45 +02:00
Goutham Veeramachaneni
a1c8425357
Initial implementation of HeadBlock Snapshots 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-06-05 13:48:31 +05:30
Goutham Veeramachaneni
3a5ae6b1a4 Merge pull request #82 from Gouthamve/deletes-1 
Implement deletions
 2017-05-27 22:06:45 +05:30