prometheus/promql
Callum Styan 8fd73b1d28
Add Exemplar Remote Write support (#8296)
* Write exemplars to the WAL and send them over remote write.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Update example for exemplars, print data in a more obvious format.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Add metrics for remote write of exemplars.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Fix incorrect slices passed to send in remote write.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* We need to unregister the new metrics.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Address review comments

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Order of exemplar append vs write exemplar to WAL needs to change.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Several fixes to prevent sending uninitialized or incorrect samples with an exemplar. Fix dropping exemplar for missing series. Add tests for queue_manager sending exemplars

Signed-off-by: Martin Disibio <mdisibio@gmail.com>

* Store both samples and exemplars in the same timeseries buffer to remove the alloc when building final request, keep sub-slices in separate buffers for re-use

Signed-off-by: Martin Disibio <mdisibio@gmail.com>

* Condense sample/exemplar delivery tests to parameterized sub-tests

Signed-off-by: Martin Disibio <mdisibio@gmail.com>

* Rename test methods for clarity now that they also handle exemplars

Signed-off-by: Martin Disibio <mdisibio@gmail.com>

* Rename counter variable. Fix instances where metrics were not updated correctly

Signed-off-by: Martin Disibio <mdisibio@gmail.com>

* Add exemplars to LoadWAL benchmark

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* last exemplars timestamp metric needs to convert value to seconds with
ms precision

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Process exemplar records in a separate go routine when loading the WAL.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Address review comments related to clarifying comments and variable
names. Also refactor sample/exemplar to enqueue prompb types.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Regenerate types proto with comments, update protoc version again.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Put remote write of exemplars behind a feature flag.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Address some of Ganesh's review comments.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Move exemplar remote write feature flag to a config file field.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Address Bartek's review comments.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Don't allocate exemplar buffers in queue_manager if we're not going to
send exemplars over remote write.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Add ValidateExemplar function, validate exemplars when appending to head
and log them all to WAL before adding them to exemplar storage.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Address more reivew comments from Ganesh.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Add exemplar total label length check.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Address a few last review comments

Signed-off-by: Callum Styan <callumstyan@gmail.com>

Co-authored-by: Martin Disibio <mdisibio@gmail.com>
2021-05-06 13:53:52 -07:00
..
fuzz-data textparse: Add fuzzing and fix bug caught 2017-07-07 11:12:17 +02:00
parser Add circular in-memory exemplars storage (#6635) 2021-03-16 15:17:45 +05:30
testdata promql: Add sgn, clamp and last_over_time functions (#8457) 2021-02-20 16:34:52 +01:00
bench_test.go Optimize aggregations in PromQL engine (#8594) 2021-03-19 17:52:29 +01:00
engine.go improve grouping label match logic 2021-04-16 22:04:58 -04:00
engine_test.go Fix memSafeIterator.Seek() (#8748) 2021-04-27 00:43:22 +02:00
functions.go Speed delta value without loop to calculate resultValue 2021-02-23 22:28:04 +08:00
functions_test.go Combine Appender.Add and AddFast into a single Append method. (#8489) 2021-02-18 17:37:00 +05:30
fuzz.go Fuzz: limit input size (#7317) 2020-05-31 09:42:56 +02:00
promql_test.go Testify: move to require (#8122) 2020-10-29 09:43:23 +00:00
quantile.go Fix off-by-one error in funcHistogramQuantile / ensureMonotonic (#7393) 2020-06-15 11:32:10 +01:00
query_logger.go promql: Swap order of parseBrokenJSON. (#7718) 2020-08-02 09:48:57 +01:00
query_logger_test.go Testify: move to require (#8122) 2020-10-29 09:43:23 +00:00
test.go Add Exemplar Remote Write support (#8296) 2021-05-06 13:53:52 -07:00
test_test.go Testify: move to require (#8122) 2020-10-29 09:43:23 +00:00
value.go Move away from testutil, refactor imports (#8087) 2020-10-22 11:00:08 +02:00