42198c1f1c
Merge pull request #311 from prometheus/fix/watermarking/on-first-write
...
Ensure new metrics are watermarked early.
2013-06-25 03:13:58 -07:00
4137c75523
Shrink default LRU cache sizes.
...
Observing Prometheus in production confirms we can lower these values
safely.
2013-06-24 12:09:16 +02:00
ecb9c7bb9d
Code Review: Swap ordering of elements.
2013-06-21 21:17:50 +02:00
5daa0a09ea
Code Review: Swap ordering of watermark getting.
...
A test for Julius.
2013-06-21 18:34:08 +02:00
ee840904d2
Code Review: !Before -> After.
2013-06-21 18:26:40 +02:00
2d5de99fbf
Regard in-memory series as new.
...
This commit ensures that series that exist only in-memory and not
on-disk are not regarded as too old for operation exclusion.
2013-06-21 18:26:39 +02:00
81c406630a
Merge pull request #312 from prometheus/fix/sample-append-logging
...
Log correct sample count when appending to disk.
2013-06-21 08:55:51 -07:00
a1a23fbaf8
Ensure new metrics are watermarked early.
...
With the checking of fingerprint freshness to cull stale metrics
from queries, we should write watermarks early to aid in more
accurate responses.
2013-06-21 16:38:46 +02:00
ba8c122147
Log correct sample count when appending to disk.
2013-06-21 12:23:27 +02:00
f2b4067b7b
Speedup and clean up operation optimization.
2013-06-20 03:01:13 +02:00
008bc09da8
Move check for empty memory series to separate method.
2013-06-19 14:19:53 +02:00
16364eda37
Drop empty series from memory after flushing.
2013-06-19 12:14:23 +02:00
71199e2c93
Cache disk fingerprint->metric lookups in memory.
2013-06-18 14:08:58 +02:00
a73f061d3c
Persist solely Protocol Buffers.
...
An design question was open for me in the beginning was whether to
serialize other types to disk, but Protocol Buffers quickly won out,
which allows us to drop support for other types. This is a good
start to cleaning up a lot of cruft in the storage stack and
can let us eventually decouple the various moving parts into
separate subsystems for easier reasoning.
This commit is not strictly required, but it is a start to making
the rest a lot more enjoyable to interact with.
2013-06-08 11:02:35 +02:00
95400cb785
Merge pull request #290 from prometheus/fix/go-vet
...
Minor "go tool vet" cleanups
2013-06-07 06:52:48 -07:00
558281890b
Minor "go tool vet" cleanups
2013-06-07 15:34:41 +02:00
615972dd01
Merge pull request #288 from prometheus/fix/curator/fallthrough-compaction-ordering
...
Fix fallthrough compaction value ordering.
2013-06-07 05:46:15 -07:00
86f63b078b
Fix fallthrough compaction value ordering.
...
We discovered a regression whereby data chunks could be appended out
of order if the fallthrough case was hit.
2013-06-07 14:41:00 +02:00
7b9ee95030
Minor LevelDB watermark handling cleanups.
2013-06-06 23:56:31 +02:00
84741b227d
Use LRU cache to avoid querying stale series.
2013-06-06 23:56:19 +02:00
f98853d7b7
Fix type error in watermark list handling.
2013-06-06 23:56:14 +02:00
ef1d5fd8a2
Introduce semaphores for tiered storage.
...
This commit wraps the tiered storage access componnets in semaphores,
since we can handle several concurrent memory reads.
2013-06-06 18:16:18 +02:00
819045541e
Code Review: Make double-drain a panic.
2013-06-06 12:40:06 +02:00
e217a9fb41
Race Work: Make memory arena locks more coarse.
...
We can optimize these as needed later.
2013-06-06 12:08:20 +02:00
beaaf386e7
Add storage state guards and transition callbacks.
...
To ensure that we access tiered storage in the proper way, we have
guards now.
2013-06-06 11:52:09 +02:00
abb5353ade
Merge pull request #283 from prometheus/feature/storage/consult-watermark
...
Include LRU cache for fingerprint watermarks.
2013-06-06 02:33:45 -07:00
2c3df44af6
Ensure database access waits until it is started.
...
This commit introduces a channel message to ensure serving
state has been reached with the storage stack before anything attempts
to use it.
2013-06-06 10:42:21 +02:00
cbe2f3a7b1
Include LRU cache for fingerprint watermarks.
2013-06-06 10:13:18 +02:00
51689d965d
Add debug timers to instant and range queries.
...
This adds timers around several query-relevant code blocks. For now, the
query timer stats are only logged for queries initiated through the UI.
In other cases (rule evaluations), the stats are simply thrown away.
My hope is that this helps us understand where queries spend time,
especially in cases where they sometimes hang for unusual amounts of
time.
2013-06-05 18:32:54 +02:00
8339a189cb
Code Review: Fix seriesPresent scope.
...
The seriesPresent scope should be constrained to the scope of a
scanJob, since this is keyed to given series.
2013-06-04 13:16:59 +02:00
fe41ce0b19
Conditionalize disk initializations.
...
This commit conditionalizes the creation of the diskFrontier and
seriesFrontier along with the iterator such that they are provisioned
once something is actually required from disk.
2013-06-04 12:53:57 +02:00
a8468a2e5e
Fix reversed disk flush cutoff behavior.
2013-05-28 16:14:30 +02:00
eb1f956909
Revert "Revert "Ensure that all extracted samples are added to view.""
...
This reverts commit 4b30fb86b4
2013-05-28 14:36:03 +02:00
4b30fb86b4
Revert "Ensure that all extracted samples are added to view."
...
This reverts commit 008314b5a8https://gist.github.com/matttproud-soundcloud/22a371a8d2cba382ea64
this commit was found.
2013-05-23 13:36:22 +02:00
750f862d9a
Use GetBoundaryValues() for non-counter deltas.
2013-05-22 19:13:47 +02:00
f2b48b8c4a
Make getValuesAtIntervalOp consume all chunk data in one pass.
...
This is mainly a small performance improvement, since we skip past the last
extracted time immediately if it was also the last sample in the chunk, instead
of trying to extract non-existent values before the chunk end again and again
and only gradually approaching the end of the chunk.
2013-05-22 18:14:45 +02:00
83d60bed89
extractValuesAroundTime() code simplification.
2013-05-22 18:14:45 +02:00
008314b5a8
Ensure that all extracted samples are added to view.
...
The current behavior only adds those samples to the view that are extracted by
the last pass of the last processed op and throws other ones away. This is a
bug. We need to append all samples that are extracted by each op pass.
This also makes view.appendSamples() take an array of samples.
2013-05-22 18:14:37 +02:00
b586801830
Code Review: Fix to-disk queue infinite growth.
...
We discovered a bug while manually testing this branch on a live
instance, whereby the to-disk queue was never actually dumped to
disk.
2013-05-22 17:59:53 +02:00
285a8b701b
Code Review: Extend lock.
2013-05-22 17:59:53 +02:00
2526ab8c81
Code Review: Extend lock scope for appending.
2013-05-22 17:59:53 +02:00
f994482d15
Code Review: Avenues for future improvemnet noted.
2013-05-22 17:59:53 +02:00
298a90c143
Code Review: Initial arena size name.
2013-05-22 17:59:53 +02:00
c07abf8521
Initial move away from skiplist.
2013-05-22 17:59:53 +02:00
74a66fd938
Spawn grouping of fingerprints with free semaphore.
...
The previous implementation spawned N goroutines to group samples
together and would not start work until the semaphore unblocked.
While this didn't leak, it polluted the scheduling space. Thusly,
the routine only starts after a semaphore has been acquired.
2013-05-21 16:11:35 +02:00
5b105c77fc
Repointerize fingerprints.
2013-05-21 14:28:14 +02:00
ec5b5bae28
Fuck you, Travis.
2013-05-21 09:42:00 +02:00
e5ac91222b
Benchmark memory arena; simplify map generation.
...
The one-off keys have been replaced with ``model.LabelPair``, which is
indexable. The performance impact is negligible, but it represents
a cognitive simplification.
2013-05-21 09:39:12 +02:00
360477f66c
Merge pull request #257 from prometheus/feature/better-memory-behaviors
...
Pointerize memorySeriesArena.
2013-05-16 07:36:40 -07:00
e1f20de2e9
Pointerize memorySeriesArena.
2013-05-16 17:09:28 +03:00
juliusv
Matt T. Proud
Matt T. Proud
Julius Volz