Commit Graph

956 Commits

Author SHA1 Message Date
Neeraj Gartia
661856cb65 removes the added tests from engine_test.go
Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>
2024-05-13 22:58:25 +05:30
Neeraj Gartia
6119124d0e some nits
Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>
2024-05-13 22:55:38 +05:30
Neeraj Gartia
adf5a36c1e adds test for sum, count, stddev, stdvar, quantile and fraction func to promql testing framework
Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>
2024-05-13 22:55:38 +05:30
Neeraj Gartia
8b838a05d9 adds test for native histogram rate func in promql testing framework
Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>
2024-05-13 22:55:38 +05:30
Neeraj Gartia
548bd9d6fb adds TestNativeHistogramRate func to promql test framework
Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>
2024-05-13 22:55:38 +05:30
Bryan Boreham
3fd24d1cd7
Merge pull request #13999 from bboreham/extract-promqltest
[Test] Extract most PromQL test code into separate packages
2024-05-09 13:23:11 +01:00
Bryan Boreham
786e0e706c test: PromQL: stop using internal fields of engine
* set enablePerStepStats and lookback duration via
  `NewTestEngine` parameters.
* check maxSamples by recreating query engine
* check lookback without modifying internals

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-05-08 19:58:41 +01:00
Bryan Boreham
0dbfd20b69 test: move most PromQL tests into separate test package
So that they can import promqltest which imports promql.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-05-08 16:28:56 +01:00
Bryan Boreham
4a72607c4a refactor: extract some PromQL Engine tests which use unexported structs
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-05-08 16:27:17 +01:00
Bryan Boreham
e7c77f7b40 promql: export NewTestQuery
So that tests can call it from another package.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-05-08 16:08:04 +01:00
Bryan Boreham
babfcfdd91 refactor: Move NewTestEngine into promqltest
And export `DefaultMaxSamplesPerQuery` so callers can replicate previous
behaviour.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-05-08 16:07:28 +01:00
Bryan Boreham
b3c15d2246 test: clean up promqltest package references
So it nearly compiles.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-05-08 13:42:55 +01:00
Bryan Boreham
9aa321d672 test: make field initializers explicit
Lint started complaining after I moved the file.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-05-08 13:42:55 +01:00
Bryan Boreham
ccf73266f0 test: move promqltest tests together with the implementation
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-05-08 13:42:55 +01:00
Bryan Boreham
11b27d5d22 test: move test files into new promqltest package
So that promql package does not bring in test-only dependencies.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-05-08 13:42:55 +01:00
Bryan Boreham
a6e427660a test: check for @-modifier without using engine internals
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-05-08 13:42:55 +01:00
Bryan Boreham
2b0c87b1b6 test: turn TestKahanSum into scripted test
This saves having a function solely to call kahanSumInc.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-05-08 13:42:55 +01:00
Bryan Boreham
a1af3c27d4 refactor: extract almost.Equal() to new package
To avoid a circular reference between promql and promqltest.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-05-08 13:42:10 +01:00
Arve Knudsen
a25160e6a4
[REFACTOR] PromQL: simplify rangeEvalTimestampFunctionOverVectorSelector (#14021)
The function `rangeEvalTimestampFunctionOverVectorSelector` appeared to be checking histogram size, however the value it used was always 0 due to subtle variable shadowing.
However we don't need to pass sample values to the `timestamp` function, since the latter only cares about timestamps. This also affects peak sample count in statistics, since we are no longer copying histogram samples.

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-05-08 10:39:44 +01:00
Oleg Zaytsev
4b7a44c7a0
Fix parser.VectorSelector.String() with empty name matcher (#14015)
The check fell into "this matcher equals vector selector's name" case when vector selector doesn't have a name and the matcher is an explicit matcher for an empty __name__ label.

To provide some context about why this is important: some downstream projects use the promql.Parse(expr.String()) to clone an expression's AST, and with this bug that matcher disappears in the cloning.

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
2024-05-06 11:51:08 +02:00
guangwu
9fda9443d4
fix(promql/query_logger): close file in error handling (#13948)
Signed-off-by: guoguangwu <guoguangwug@gmail.com>
2024-04-30 10:47:10 +02:00
Owen Williams
4a6f8704ef
parser: remake generated_parser output (#13923)
In a previous PR, the generated parser was created using an old version of goyacc.

Also adds -l to disable line directives, which fixes debug processing and reduces diffs at the expense of making it more difficult to reason about the generated output.

Signed-off-by: Owen Williams <owen.williams@grafana.com>
2024-04-13 12:59:54 +02:00
Neeraj Gartia
612de026da
Adds Inf and NaN as Numbers to Histogram in Promql Testing Framework (#13916)
includes Inf and NaN as numbers to histogram

---------

Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>
Signed-off-by: Björn Rabenstein <github@rabenste.in>
Co-authored-by: Björn Rabenstein <github@rabenste.in>
2024-04-11 12:53:28 +02:00
Matthieu MOREL
6f595c6762
golangci-lint: enable whitespace linter (#13905)
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2024-04-11 09:27:54 +01:00
Bryan Boreham
12961c6a37 promql: refactor: eliminate one 'else'
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham
0ac927515b promql: move group-seen into group struct
Save allocating an auxilliary array.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham
7499d90913 promql: remove pointer to aggregation groups
Just allocate in one slice.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham
cfbeb6681b promql: re-use one heap for topk and bottomk
Slightly ugly casting saves memory.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham
5e3914a27c promql: remove histogramMean from groupedAggregation
Re-use histogramValue since we don't need them separately.

Tidy up initialization.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham
2cf3c9de8f promql: store labels per-group only for count_values
This saves memory in other kinds of aggregation.

We don't need `orderedResult` in `aggregationCountValues`; the ordering
is not guaranteed.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham
185290a0d2 promql: pull checking of q and k out of loop
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham
4584f67e17 promql: inline nextSample function
Move Sample out of loop to reduce allocations, otherwise it escapes to
the heap.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham
526ce4ee7a promql: simplify data collection in aggregations
We don't need a Sample, just the float and histogram values.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham
2f03acbafc promql: refactor: split topk/bottomk from sum/avg/etc
They aggregate results in different ways.
topk/bottomk don't consider histograms so can simplify data collection.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham
74eed67ef6 promql: refactor: pull fetching input data out of rangeEvalAgg
This is a cleaner split of responsibilities.
We now check the sample count after calling rangeEvalAgg.
Changed re-use of samples to use `Clone` and `defer`.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham
602eb69edf promql: refactor: extract function nextSample
With sub-function nextValues which we shall use shortly.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham
eb41e770b7 promql: refactor: extract function addToSeries
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham
53a3138eeb promql aggregations: pre-generate mapping from inputs to outputs
So we don't have to re-create it on every time step.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham
cb6c4b3092 promql: simplify k/q parameter to topk/bottomk/quantile
Pass it as a float64 not as interface{}.
Make k a simple int, since that is the parameter to make().
Pull invalid quantile warning out of the loop.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham
b3bda7df4b promql: aggregations: skip copying input to a Vector
We can work directly from the inputMatrix on each timestep.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham
c9b6c4c55a promql: aggregations: output directly to matrix for instant queries
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham
3851b74db1 promql: aggregations: skip result vector in range queries
Adjust test to match the lower count, since samples in the vector
are no longer counted.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham
59548b8a0b promql: refactor: move collection of results into aggregation()
We don't need to check for duplicates as aggregation cannot generate them.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham
bd9bdccb22 promql: refactor: simplify internal data structures
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham
5f10d17cef promql: refactor: split out aggregations over range
The new function `rangeEvalAgg` is mostly a copy of `rangeEval`, but
without `initSeries` which we don't need and inlining the callback to
`aggregation()`.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham
e5f667537c promql: refactor: initialize aggregation before storing in map
This seems more consistent to me.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham
29244fb841 promql: refactor: extract count_values implementation
The existing aggregation function is very long and covers very different
cases.

`aggregationCountValues` is just for `count_values`, which differs from
other aggregations in that it outputs as many series per group as there
are values in the input.

Remove the top-level switch on string parameter type; use the same `Op`
check there as elswehere.

Pull checking parameters out to caller, where it is only executed once.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham
8e04ab6dd4 promql: refactor: extract generateGroupingLabels function
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-05 15:47:54 +01:00
Bryan Boreham
7c28521451 [TESTS] Truncate some long test names, for readability
The strings produced by these tests can run to thousands of characters,
which makes test logs difficult to read.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-04-03 10:10:39 +01:00
Charles Korn
cd72ebb05f
promql: include more details in error message when creating test query fails or an unexpected series is returned (#13847)
* promql: include more details in error message when creating test query fails

Signed-off-by: Charles Korn <charles.korn@grafana.com>

* Include more details when an unexpected metric is returned

Signed-off-by: Charles Korn <charles.korn@grafana.com>

---------

Signed-off-by: Charles Korn <charles.korn@grafana.com>
2024-04-03 10:57:08 +02:00