Commit Graph

261 Commits

Author SHA1 Message Date
Brian Brazil
c77c3a8c56 promql: Limit extrapolation of delta/rate/increase
The new implementation detects the start and end of a series by
looking at the average sample interval within the range. If the first
(last) sample in the range is more than 1.1*interval distant from the
beginning (end) of the range, it is considered the first (last) sample
of the series as a whole, and extrapolation is limited to half the
interval (rather than all the way to the beginning (end) of the
range). In addition, if the extrapolated starting point of a counter
(where it is zero) is within the range, it is used as the starting
point of the series.

Fixes #581
2016-01-08 15:32:43 +01:00
Brian Brazil
89760dd77d Handle NaN for min/max.
Similar to topk and sort, prefer not returning NaN
where possible.
2016-01-06 12:41:40 +00:00
Brian Brazil
bac1f28cad Similar to topk/bottomk, have sort/sort_desc put NaN at end.
This makes topk and bottomk consistent with the sorting functions,
as per #1271.
2015-12-31 14:52:48 +00:00
Fabian Reinartz
4209ec6864 Change WITH keyword to LABELS 2015-12-23 14:54:02 +01:00
Brian Brazil
88ca82304c Make topk/bottomk prefer returning real numbers over NaN. 2015-12-22 13:53:43 +00:00
Brian Brazil
edf3e123f5 Move topk/bottomk tests from legacy. 2015-12-22 12:38:32 +00:00
Fabian Reinartz
af3a6661ed Implement new alerting rule syntax 2015-12-11 17:02:34 +01:00
James Sanford
5b53262b7a promql: Add clamp_max/clamp_min functions. 2015-11-26 13:38:06 -08:00
Brian Brazil
a287264989 Print offsets in promql. 2015-11-15 16:24:29 +00:00
Fabian Reinartz
33aab4169c Anchor regexes in vector matching
This commit makes the regex behavior of vector matching consistent with
configuration and label_replace() by anchoring it.

Fixes #1200
2015-11-05 11:23:43 +01:00
Fabian Reinartz
51e8badc7f Merge pull request #1159 from prometheus/scalar-bool
promql: Remove scalar/scalar comparisons.
2015-10-16 12:28:56 +02:00
Brian Brazil
c36961130b promql: Remove scalar/scalar comparisons.
This change is breaking, use the 'bool' modifier for such comprisons.

After this change all comparisons without 'bool' will filter, and all
comparisons with 'bool' will return 0/1. This makes the language more
consistent and orthogonal, and ultimately easier to learn and use.

If we ever figure out sane semantics for filtering scalar/scalar
comparisons we can add them in, which will most likely come out of how
the new vector() function is used.
2015-10-11 08:51:04 +01:00
Brian Brazil
5740a8fade promql: Remove deprecated 2nd argument to delta()
This change is breaking, use increase() instead.

I'm not cleaning up the function in this PR, as my solution to #581 will
rewrite and simplify increase/rate/delta.
2015-10-10 15:41:23 +01:00
Brian Brazil
965a71dc4d Merge pull request #1155 from prometheus/irate
promql: Add irate() function
2015-10-10 08:05:05 +01:00
Brian Brazil
f08abdb48b promql: Add irate() function
irate is a rate function that only looks at the most
recent two data points, and calucaltes a per-second value
from that. This produces much more granular graphs for
fast moving data, and works sanely across many scrape intervals.

It doesn't do so well for slowly moving data.
2015-10-09 21:44:35 +01:00
Julius Volz
0088aa4d45 Merge pull request #1132 from prometheus/fix-quoting-and-escaping
Support escape sequences in strings and add raw strings
2015-10-08 20:51:18 +02:00
Julius Volz
46c5260761 Support escape sequences in strings and add raw strings.
This adapts some functionality from the Go standard library for string
literal lexing and unquoting/unescaping.

The following string types are now supported:

Double- or single-quoted strings:

  These support all escape sequences that Go supports in double-quoted
  string literals. The difference is that Prometheus also has
  single-quoted strings (instead of single-quoted runes in Go). Raw
  newlines are not allowed.

Backtick-quoted raw strings:

  Strings quoted in backticks are treated as raw strings just like in Go
  and may contain raw newlines and other special characters directly.

Fixes https://github.com/prometheus/prometheus/issues/1122
Fixes https://github.com/prometheus/prometheus/issues/1121
2015-10-08 19:17:21 +02:00
Fabian Reinartz
e3b6ec9784 Switch to common/log 2015-10-03 10:21:43 +02:00
Brian Brazil
653ff71f1f promql: Reduce flakiness of concurrency test 2015-09-23 10:07:30 +01:00
Fabian Reinartz
171f50706a Fix unkeyed field errors. 2015-09-18 17:00:08 +02:00
Fabian Reinartz
36ec8ba460 Fix missing return on error 2015-09-18 16:50:13 +02:00
Fabian Reinartz
e005f939fd Fix scalar construction in function 2015-09-18 16:49:32 +02:00
Fabian Reinartz
eca41f5319 Run gofmt 2015-09-16 14:33:12 +02:00
Brian Brazil
fa793d917e Merge pull request #1080 from prometheus/query-timeout-test
promql: Bump sleep in query timeout test
2015-09-14 13:00:47 +01:00
Brian Brazil
ce7f31e03c promql: Bump sleep in query timeout test
This test is flaky, I'm presuming the time.AfterFunc
call is being delayed so the evaluation isn't getting
cancelled.
2015-09-14 11:49:18 +01:00
Julius Volz
347630431c Merge pull request #1077 from prometheus/cleanups
Fix some dead code, missing error checks, shadowings.
2015-09-14 12:37:26 +02:00
Julius Volz
af513468eb Fix some dead code, missing error checks, shadowings.
I applied
https://medium.com/@jgautheron/quality-pipeline-for-go-projects-497e34d6567
and was greeted with a deluge of warnings, most of which were not
applicable or really fixable realistically. These are some of the first
ones I decided to fix.
2015-09-14 12:21:34 +02:00
Brian Brazil
29de4ee2b0 Merge pull request #1078 from prometheus/whats-our-vector-victor
Remove optional vector() 2nd argument
2015-09-13 14:14:20 +01:00
Brian Brazil
9b382647b5 Remove optional vector() 2nd argument 2015-09-13 09:13:22 +01:00
Fabian Reinartz
a1617d90f4 Merge pull request #1073 from prometheus/whats-our-vector-victor
promql: Add vector function.
2015-09-12 08:36:13 +02:00
Brian Brazil
69f5fa0c1e promql: Add vector function.
Currently the only way to convert a scalar to a vector is to
use absent(), which isn't very clean. This adds a vector()
function that's the inverse of scalar() and lets your optionally
set labels.

Example usage would be
vector(time() % 86400) < 3600
to filter to only the first hour of the day.
2015-09-11 12:09:34 +01:00
Julius Volz
6d3e054692 Fix bool modifier in recording rules and printing.
Fixes https://github.com/prometheus/prometheus/issues/1065
2015-09-10 01:37:05 +02:00
Brian Brazil
9ec11b1847 Merge pull request #1049 from prometheus/bool-nofilter
promql: Add 'bool' modifier to comparison functions
2015-09-03 15:08:38 +01:00
Brian Brazil
29e8dc2c49 promql: Add 'bool' modifier to comparison functions
When doing comparison operations on vectors, filtering
sometimes gets in the way and you have to go to a fair bit of
effort to workaround it in order to always return a result.
The 'bool' modifier instead of filtering returns 0/1 depending
on the result of the compairson.

This is also a prerequisite to removing plain scalar/scalar comparisons,
as it maintains the current behaviour under a new syntax.
2015-09-02 14:51:44 +01:00
Julius Volz
61c42c8da0 Change relabel_replace() to do full-string matches.
THIS IS A BREAKING CHANGE.

Fixes part of https://github.com/prometheus/prometheus/issues/996
2015-09-01 15:49:28 +02:00
Julius Volz
744d5d5a7a Merge pull request #1029 from prometheus/vet-fixes
Fix "go vet" errors.
2015-08-26 12:50:18 +02:00
Julius Volz
995d3b831d Fix most golint warnings.
This is with `golint -min_confidence=0.5`.

I left several lint warnings untouched because they were either
incorrect or I felt it was better not to change them at the moment.
2015-08-26 12:44:46 +02:00
Julius Volz
963ad82dcb Fix "go vet" errors.
I ignored all errors of the type "composite literal uses unkeyed
fields". Most of them are wrong because of
https://github.com/golang/go/issues/9171.
2015-08-26 02:05:04 +02:00
Julius Volz
077a753e6b Merge pull request #1006 from prometheus/true-values
promql: Remove interpolation of vector values.
2015-08-25 16:11:07 +02:00
Fabian Reinartz
d6b8da8d43 Switch promql types to common/model 2015-08-25 13:49:14 +02:00
Brian Brazil
fb585e4591 promql: Remove interpolation of vector values.
The current behaviour produces values that are not
from rules or scrapes. So if for example I have
a boolean 0/1 it can be returned as 0.2344589. This
prevents a number of advanced use cases, introduces
race conditions and can produce misleading graphs.
2015-08-24 17:37:31 +01:00
Fabian Reinartz
1535ef1457 Replace metric.SamplePair with model.SamplePair 2015-08-22 14:52:35 +02:00
Fabian Reinartz
438e232c9b Fix grouping of import blocks 2015-08-22 09:42:45 +02:00
Fabian Reinartz
306e8468a0 Switch from client_golang/model to common/model 2015-08-21 13:33:38 +02:00
Brian Brazil
296f551418 Merge pull request #1014 from prometheus/scalar-rules
rules: Allow recorded rules expressions to be scalars.
2015-08-19 22:10:49 +01:00
Brian Brazil
e6a67476c2 rules: Allow recorded rules expressions to be scalars.
This is useful if you want to build up a constant metric,
such as a set of alert thresholds that vary by label value.
2015-08-19 21:09:00 +01:00
Laurie Malau
cdf38ab93a Log runtime errors during query evaluation instead of panicking. 2015-08-19 16:56:41 +02:00
Julius Volz
27ed874358 Implement label_replace()
Implements part of https://github.com/prometheus/prometheus/issues/959.
2015-08-18 14:20:07 +02:00
Fabian Reinartz
690b5f1575 Remove multi-statement queries
This commit removes the possibility to have multi-statement queries
which had no full support anyway. This makes the caller responsible
for multi-statement semantics.
Multiple tests are no longer timing-dependent.
2015-08-10 14:26:20 +02:00
Julius Volz
e324910ff2 Merge pull request #936 from prometheus/predict
promql: Add support for predict(my_timeseries[1h], 2h)
2015-08-05 16:40:51 +02:00
Brian Brazil
d6a80c2b76 promql: Add support for predict_linear(my_timeseries[1h], 7200)
This will give a prediction for the value of my_timeseries in 2 hours,
based on the last hour of data.
2015-08-05 15:16:49 +01:00
Fabian Reinartz
579fdf65e2 Implement unary expression for vector types.
Closes #956
2015-08-04 15:46:36 +02:00
Fabian Reinartz
c322422412 Merge pull request #954 from prometheus/fabxc/fuzz-fix
Add missing check for nil expression
2015-08-03 16:48:20 +02:00
Fabian Reinartz
adf109795c forbid unexpected (runtime) errors in parse tests 2015-08-03 12:53:31 +02:00
Fabian Reinartz
c20e25f718 Add missing check for nil expression 2015-08-03 12:28:40 +02:00
Brian Brazil
a0f0b82348 promql: Test errors aren't always ParseErr 2015-08-02 23:26:21 +01:00
Fabian Reinartz
5279d50d92 Handle parser runtime panics gracefully 2015-08-02 13:42:18 +02:00
Julius Volz
4e4b468fba Fix lexer bug treating non-Latin Unicode digits as digits.
Fixes https://github.com/prometheus/prometheus/issues/939
2015-07-29 02:11:13 +02:00
Fabian Reinartz
3d67d75935 promql: implement JSON array format for scalar and string 2015-07-06 13:09:26 +02:00
Fabian Reinartz
77e8983221 promql: add MarshalJSON method for SamplePair 2015-07-06 10:29:59 +02:00
Fabian Reinartz
c1d37bc55b Merge pull request #843 from prometheus/fabxc/runbook
promql: add runbook to alert statement.
2015-06-25 14:07:45 +02:00
Fabian Reinartz
70d7a987a7 promql: add json tags, fix query constructor. 2015-06-25 13:44:05 +02:00
Fabian Reinartz
749ae450c5 promql: add runbook to alert statement.
This commit adds the RUNBOOK keyword to alert statements. The field
is optional and expected to be a link.
2015-06-25 13:00:52 +02:00
Fabian Reinartz
7f85b9b215 promql: add MarshalJSON method for ExprType. 2015-06-25 12:01:26 +02:00
Fabian Reinartz
1eff186555 Merge pull request #810 from prometheus/fabxc/lmatch
Match empty labels.
2015-06-22 15:45:50 +02:00
Fabian Reinartz
5b91ea9b36 storage: improve label matching and allow unset matching.
Matching of empty labels now also matches metrics where the label
was not explicitly set to the empty string.
2015-06-22 15:33:44 +02:00
Fabian Reinartz
94cd321be1 promql: error if all label matchers are empty. 2015-06-22 15:33:44 +02:00
Fabian Reinartz
fe301d7946 promql: remove global flags 2015-06-15 19:01:06 +02:00
Julius Volz
5e2d1c1464 Deprecate keeping_extra, rename it to keep_common.
`keep_common` is more in line with the function name
`drop_common_labels()` terminology-wise, and also more in line with
`group_left`/`group_right` (no `...ing` verb suffix).

We could also go the full way and call it `keep_common_labels`. That
would have the benefit of being even more consistent with the function
`drop_common_labels()` and would be more explanatory, but it also seems
quite long.
2015-06-12 14:21:05 +02:00
Fabian Reinartz
e7659f908c promql: remove DotGraph methods from nodes. 2015-06-12 09:48:14 +02:00
Fabian Reinartz
c716d8a47b promql: fix aggregation expression String() method.
Fixes #794.
2015-06-12 09:48:01 +02:00
Fabian Reinartz
c32ae22119 promql: fix missing metric in range results. 2015-06-11 23:50:53 +02:00
Fabian Reinartz
0acd44b0e3 promql: expose ParseMetric and ParseMetricSelector 2015-06-11 12:22:11 +02:00
Fabian Reinartz
cb10ceac18 promql: allow scalar expressions in range queries, improve errors.
These changes allow to do range queries over scalar expressions.
Errors on bad types for range queries are now raised on query creation
rather than evaluation.
2015-06-10 18:36:02 +02:00
Fabian Reinartz
ab9c98acac web/api: add initial API v1 implementation. 2015-06-06 21:47:36 +02:00
Fabian Reinartz
03b737322e promql: expose storage and query engine for tests 2015-06-04 19:00:22 +02:00
Fabian Reinartz
319068a7a6 promql: fix parsing ambiguity for serial repetitions 2015-06-04 19:00:22 +02:00
Fabian Reinartz
c60e3387ef promql: fix lexing of \r as whitespaces 2015-06-02 18:33:49 +02:00
Fabian Reinartz
0de6edbdfc Move pkg/ to util/ 2015-06-01 21:12:32 +02:00
Fabian Reinartz
dbc0d30e3e Move string functionality to pkg/strutil 2015-06-01 21:12:32 +02:00
Fabian Reinartz
ccf51b132e Move stats package to pkg/stats 2015-06-01 21:12:31 +02:00
Fabian Reinartz
3c8fbf1e15 Move test package to pkg/testutil 2015-06-01 21:12:31 +02:00
Brian Brazil
f34de493d5 Add increase() function, to replace delta(..., 1).
This calculates how much a counter increases over
a given period of time, which is the area under the curve
of it's rate.

increase(x[5m]) is equivilent to rate(x[5m]) * 300.
2015-05-26 22:49:21 +01:00
Julius Volz
d44a89c6e8 Implement changes() function.
changes() takes a range vector and returns the number of times a value
has changed in the given time window for each time series as an instant
vector.
2015-05-26 19:06:22 +02:00
Julius Volz
6f33ed9e59 Add resets() function to count counter resets.
resets() returns for every range vector element how many counter
resets there have been in the specified range.
2015-05-26 17:56:52 +02:00
Fabian Reinartz
b7eb105ffa Remove test flags in help output 2015-05-23 12:23:33 +02:00
beorn7
3b9c421a69 Weed out all the [Gg]et* method names.
The only exception is getNumChunksToPersist to avoid naming the struct
member numChunksToPersist in a weird way.
2015-05-20 19:13:06 +02:00
Fabian Reinartz
ac4d63b833 Merge pull request #689 from prometheus/fabxc/qltest
Add basic testing language, migrate tests
2015-05-18 19:22:48 +02:00
Fabian Reinartz
0d3012a605 Migrate matrix tests, remove old test files. 2015-05-18 17:50:12 +02:00
Fabian Reinartz
71ef7ab405 Migrate remaining vector evaluation tests to new testing language. 2015-05-18 17:47:47 +02:00
Fabian Reinartz
3c22eded97 Migrate literal tests to testing language. 2015-05-18 17:47:47 +02:00
Fabian Reinartz
eba07a7d3d Migrate histogram tests to test language. 2015-05-18 17:47:47 +02:00
Fabian Reinartz
03094eff04 Migrate parsing error tests.
The promql_test checks failure of various bad syntaxed queries.
Those are moved into the parser tests as the new testing language
only deals with valid queries.
2015-05-18 17:47:47 +02:00
Fabian Reinartz
6321964738 Add parsing and execution of new test format.
This commit adds a new test structure that parses and executes
the new testing language.
2015-05-18 17:47:47 +02:00
Fabian Reinartz
a236c01457 Add time series description parsing.
This commit adds parsing of time series description to the exisiting
query language parser. Time series descriptions are defined by a
metric followed by a sequence of values.
2015-05-18 17:29:32 +02:00
Fabian Reinartz
d122749b39 Enhance lexer testing output 2015-05-18 17:19:17 +02:00
Fabian Reinartz
ce487f763e Simplify vector binary evaluation logic 2015-05-17 00:02:34 +02:00
Fabian Reinartz
8a109e061b Extract OR operation into own eval method. 2015-05-16 14:00:11 +02:00
Fabian Reinartz
2c3e9e2e87 Extract AND operation into own eval method. 2015-05-16 13:33:03 +02:00
Fabian Reinartz
4234a45d2e Merge pull request #693 from prometheus/fabxc/preload_timeout
Check context before each preloading.
2015-05-12 12:25:22 +02:00
Fabian Reinartz
8ea94ebb58 Check context before each preloading. 2015-05-12 12:03:30 +02:00
Fabian Reinartz
969c231191 Make parser more strict about identifiers, extract number parsing 2015-05-11 11:45:23 +02:00
Fabian Reinartz
8707c54508 Fix single quote parsing, add tests 2015-05-08 16:43:02 +02:00
Fabian Reinartz
9ab1f6c690 Limit maximum number of concurrent queries.
A high number of concurrent queries can slow each other down
so that none of them is reasonbly responsive. This commit limits
the number of queries being concurrently executed.
2015-05-06 11:34:17 +02:00
Fabian Reinartz
d59d1cb2c1 Fix Error() methods. 2015-05-01 17:58:58 +02:00
Fabian Reinartz
6649306e63 Merge pull request #660 from prometheus/fabxc/pql/parse-errs
Fix and improve parsing error output.
2015-04-30 13:31:13 +02:00
Fabian Reinartz
279831cdf1 Fix and improve parsing error output. 2015-04-30 12:19:39 +02:00
Fabian Reinartz
fe935179cd Stop routing rule statements through the engine. 2015-04-29 18:01:43 +02:00
Fabian Reinartz
25cdff3527 Remove name arg from Parse* functions, enhance parsing errors. 2015-04-29 16:38:41 +02:00
Fabian Reinartz
5602328c7c Refactor query evaluation.
This copies the evaluation logic from the current rules/ package.
The new engine handles the execution process from query string to final result.
It provides query timeout and cancellation and general flexibility for
future changes.

functions.go: Add evaluation implementation. Slight changes to in/out data but
	not to the processing logic.
quantile.go: No changes.
analyzer.go: No changes.
engine.go: Actually new part. Mainly consists of evaluation methods
	which were not changed.
setup_test.go: Copy of rules/helpers_test.go to setup test storage.
promql_test.go: Copy of rules/rules_test.go.
2015-04-28 14:19:05 +02:00
Fabian Reinartz
32b7595c47 Create promql package with lexer/parser.
This commit creates a (so far unused) package. It contains the a custom
lexer/parser for the query language.

ast.go: New AST that interacts well with the parser.
lex.go: Custom lexer (new).
lex_test.go: Lexer tests (new).
parse.go: Custom parser (new).
parse_test.go: Parser tests (new).
functions.go: Changed function type, dummies for parser testing (barely changed/dummies).
printer.go: Adapted from rules/ and adjusted to new AST (mostly unchanged, few additions).
2015-04-23 16:04:50 +02:00