Up until this point, if a scrape was done with the protobuf format Prometheus would always try to ingest native histograms even with the feature flag disabled. This causes problems with other feature-flags that depend on the protobuf format, like 'created-timestamp-zero-ingestion'. This commit decouples native histogram parsing from ingestion, making sure ingestion only happens when the 'native-histogram' feature-flag is enabled.
Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>
* fix the bug of setting native histogram min bucket factor
Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>
* Add unit test for checking that min_bucket_factor is correctly applied
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
---------
Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
Co-authored-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
When Prometheus scrapes a target and it sees the same time series repeated multiple times it currently silently ignores that. This change adds a test for that and fixes the scrape loop so that:
* Only first sample for each unique time series is appended
* Duplicated samples increment the prometheus_target_scrapes_sample_duplicate_timestamp_total metric
This allows one to identify such scrape jobs and targets.
Also fix some tests and benchmark.
This call bloats checkAddError signature and logic, we can and should call it from the main scrape logic.
Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
This test calls Rollback() which is normally called from within append code.
Doing so means that staleness tracking data is outdated and need to by cycled manually.
Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
OpenMetrics requires EOF comment at the end of metrics body, but the makeTestMetrics() function doesn't append it.
This means this benchmark tests a response with errors but I don't think that was the intention.
Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
When the scrape tolerance is bigger than 1% of the scrape interval, take
1% of the scrape interval as the tolerance instead of not aligning the
scrape at all.
Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>
This adds support for the new grammar of `{"metric_name", "l1"="val"}` to promql and some of the exposition formats.
This grammar will also be valid for non-UTF-8 names.
UTF-8 names will not be considered valid unless model.NameValidationScheme is changed.
This does not update the go expfmt parser in text_parse.go, which will be addressed by https://github.com/prometheus/common/issues/554/.
Part of https://github.com/prometheus/prometheus/issues/13095
Signed-off-by: Owen Williams <owen.williams@grafana.com>
Normally, a NaN value is never equal to any other value. Compare sample
values via `Float64bits` so that NaN values which are exactly the same
will compare equal.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
Native Histograms: support native_histogram_min_bucket_factor in scrape_config
---------
Signed-off-by: Ziqi Zhao <zhaoziqi9146@gmail.com>
Signed-off-by: Björn Rabenstein <github@rabenste.in>
Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
Co-authored-by: Björn Rabenstein <github@rabenste.in>
They are used in multiple repos, so common is a better place for them.
Several packages now don't depend on `model/textparse`, e.g.
`storage/remote`.
Also remove `metadata` struct from `api.go`, since it was identical to
a struct in the `metadata` package.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
* Append created timestamps.
Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>
* Log when created timestamps are ignored
Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>
* Proposed changes to Append CT PR.
Changes:
* Changed textparse Parser interface for consistency and robustness.
* Changed CT interface to be more explicit and handle validation.
* Simplified test, change scrapeManager to allow testability.
* Added TODOs.
Signed-off-by: bwplotka <bwplotka@gmail.com>
* Updates.
Signed-off-by: bwplotka <bwplotka@gmail.com>
* Addressed comments.
Signed-off-by: bwplotka <bwplotka@gmail.com>
* Refactor head_appender test
Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>
* Fix linter issues
Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>
* Use model.Sample in head appender test
Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>
---------
Signed-off-by: Arthur Silva Sens <arthur.sens@coralogix.com>
Signed-off-by: bwplotka <bwplotka@gmail.com>
Co-authored-by: bwplotka <bwplotka@gmail.com>
The 'ToFloat' method on integer histograms currently allocates new memory
each time it is called.
This commit adds an optional *FloatHistogram parameter that can be used
to reuse span and bucket slices. It is up to the caller to make sure the
input float histogram is not used anymore after the call.
Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
The scrape loop and scrape cache should use the same instance.
This brings the tests' behavior more in line with production.
Signed-off-by: Paulin Todev <paulin.todev@gmail.com>
Previously we had one per scrapePool, and one of those per configured
scraping job. Each pool holds a few unused buffers, so sharing one
across all scrapePools reduces total heap memory.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
Too confusing to have `MetadataList` and `ListMetadata`, etc.
I standardised on the ones which are in an interface.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
Arthur Silva Sens
Matthieu MOREL
David Ashpole
Ziqi Zhao
Bryan Boreham
Julien
machine424
Łukasz Mierzwa
Owen Williams
Paweł Szulik
Kumar Kalpadiptya Roy
Filip Petkovski
Björn Rabenstein
Paulin Todev