RepoMirrors/prometheus
1
0
Fork 0
mirror of https://github.com/prometheus/prometheus synced 2024-12-29 02:02:17 +00:00
Code Issues Packages Projects Releases Wiki Activity
13,419 Commits 243 Branches 0 Tags 653 MiB
1daf7cdd62
Commit Graph

13419 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Björn Rabenstein
1daf7cdd62
Merge pull request #14626 from cuiweiyuan/main 
chore: fix some function names
 2024-08-15 11:46:21 +02:00
cuiweiyuan
1800af54f0 chore: fix some function names 
Signed-off-by: cuiweiyuan <cuiweiyuan@aliyun.com>
 2024-08-15 13:57:21 +08:00
Bryan Boreham
d1ea6eb35d
Merge pull request #14354 from bboreham/ooo-head-together 
[PERF] TSDB: Query head and ooo-head together

The current implementation of out-of-order querying runs two queriers which each iterate all series in the head, then runs a merge operation on the output.

This PR adds HeadAndOOOQuerier which iterates just once over series, then where necessary merges chunks from in-order and out-of-order lists.

In order to distinguish in-order from out-of-order chunk references I set bit 23 (i.e. 1<<23) on ooo references; this reduces the maximum number of chunks from 16 million to 8 million.

Note one side-effect of this change is that results may come in a different order - the merge operation done previously required a sort of all series. This only changes where Prometheus does not guarantee the order.

Fixes #11628
 2024-08-14 14:39:30 +01:00
Bryan Boreham
512c67ec26 TSDB: Never go over maximum number of OOO chunks 
In `mmapCurrentOOOHeadChunk`, check if the number is at the maximum and
drop the data with an error log. This is not expected to happen as the
maximum is over 8 million; that's 8 years of 1 sample every second.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2024-08-14 13:41:59 +01:00
Bryan Boreham
9135da1e4f TSDB: Review feedback 
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* Re-enable check in `createHeadWithOOOSamples` which wasn't really broken.
* Move code making `Block` into a `Queryable` into test file.
* Make `getSeriesChunks` return a slice (renamed `appendSeriesChunks`).
* Rename `oooMergedChunks` to `mergedChunks`.
* Improve comment on `ChunkOrIterableWithCopy`.
* Name return values from unpackHeadChunkRef.

Co-authored-by: Oleg Zaytsev <mail@olegzaytsev.com>
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2024-08-14 13:41:44 +01:00
Bryan Boreham
7ffd3ca280 TSDB: Cosmetic: move HeadAndOOO implementations where old code was 
This makes the diffs easier to follow.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2024-08-14 13:41:13 +01:00
Bryan Boreham
e95607b276 TSDB: Lock round access to labels, where necessary 
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2024-08-14 13:41:13 +01:00
Bryan Boreham
26b3de0438 TSDB: Remove OOOHeadIndexReader 
Use headIndexReader instead.

OOOCompactionHeadIndexReader needs to be expanded slightly, because it previously delegated to OOOHeadIndexReader.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2024-08-14 13:41:13 +01:00
Bryan Boreham
a299c7b6d6 TSDB: Remove OOOHeadChunkReader 
Use HeadAndOOOChunkReader instead.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2024-08-14 13:41:13 +01:00
Bryan Boreham
e7e50a3afd TSDB: Remove code for querying OOO-head only 
Just query via `HeadAndOOOQuerier`, which will skip series where no
in-order chunks are in range.

Now we don't need `OOORangeHead`.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2024-08-14 13:41:13 +01:00
Bryan Boreham
0a2ff76881 TSDB tests: Fix up BenchmarkQueries 
Was not working even on main.  Some cases still error.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2024-08-14 13:41:04 +01:00
Björn Rabenstein
9849418fac
Merge pull request #14611 from charleskorn/sum-and-avg-over-mixed-custom-exponential-histograms 
promql: fix incorrect results and panics in `sum` and `avg` over mixed custom and exponential buckets, or incompatible custom buckets
 2024-08-14 13:56:47 +02:00
suntala
ce4eac859a Link to specific feature flag entry 
Signed-off-by: suntala <arati.rana@grafana.com>
 2024-08-14 13:22:12 +02:00
Bryan Boreham
f261597944 TSDB: Fix up LabelValues to work for OOO-only head 
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2024-08-14 11:19:02 +01:00
Bryan Boreham
6529d6336c TSDB: NewHeadAndOOOChunkReader takes headChunkReader 
So we can pass nil and have it read just OOO chunks.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2024-08-14 11:19:02 +01:00
Bryan Boreham
e04d137649 [PERF] TSDB: Query head and ooo-head together 
Add `HeadAndOOOQuerier` which iterates just once over series, then
where necessary merges chunks from in-order and out-of-order lists.

Add a ChunkQuerier for in-order and ooo together

Add copy-last-chunk behaviour to HeadAndOOOChunkReader

Out-of-order chunk IDs are distinguished from in-order by setting bit 23.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2024-08-14 11:19:02 +01:00
Bryan Boreham
2936ab80d7 [Tests] Promtool: Sort output where Prometheus does not guarantee the order. 
Previously this was working because iout-of-order chunks forced a sort and merge.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2024-08-14 11:19:02 +01:00
Bryan Boreham
da31da3ea6 Refactor: extract selectSeriesSet and selectChunkSeriesSet 
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2024-08-14 11:19:02 +01:00
Bryan Boreham
7e24844d08 Refactor: extract headChunkReader.chunkFromSeries() 
For when you have a series locked already.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2024-08-14 11:19:02 +01:00
Bryan Boreham
a32aca0cd7 Refactoring: extract getOOOSeriesChunks 
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2024-08-14 11:19:02 +01:00
Bryan Boreham
c75c8f8329 Refactoring: extract getSeriesChunks 
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2024-08-14 11:19:02 +01:00
Bryan Boreham
0c852680bf [Benchmark] TSDB: Add BenchmarkQuerierSelectWithOutOfOrder 
Refactor existing BenchmarkQuerierSelect to provide the set-up.

Note that Head queries now run faster because they use a RangeHead.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2024-08-14 11:19:02 +01:00
George Krajcsovits
ff0a1e5e11
Merge pull request #14671 from krajorama/prep-ooo-native-histograms 
tsdb: prepare inserting native histograms into OOO head
 2024-08-14 12:02:37 +02:00
György Krajcsovits
41656162fc tsdb: prepare inserting native histograms into OOO head 
Rename a variable.
Add parameters to memSeries.insert function.

No effect on how float samples are handled.

Related to #14546

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
 2024-08-14 11:13:47 +02:00
Charles Korn
52818a97e2
Merge branch 'main' into sum-and-avg-over-mixed-custom-exponential-histograms 
# Conflicts:
#	promql/promqltest/testdata/native_histograms.test
 2024-08-14 07:52:08 +10:00
Björn Rabenstein
c2bc6cfe97
Merge pull request #14621 from charleskorn/panic-message 
promql: clarify error message logged when panic occurs during query evaluation
 2024-08-13 23:02:43 +02:00
George Krajcsovits
fe5354df88
Merge pull request #14658 from krajorama/zero-rate-nh-quantile 
Native histograms: define behavior when rate is null.
 2024-08-13 16:05:43 +02:00
György Krajcsovits
386fc8b9f6 Update from review comments. 
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
 2024-08-13 15:26:07 +02:00
Bryan Boreham
aa4b056ad0
Merge pull request #13200 from bboreham/wlog-defer 
tsdb/wlog: close segment files sooner
 2024-08-13 14:11:38 +01:00
Arve Knudsen
6f1fd4be96
Merge pull request #14663 from aknuds1/arve/promql-comment 
PromQL engine: Fix comment regarding non-nil histogram pointer
 2024-08-13 09:11:40 +02:00
Arve Knudsen
0503d4f372 PromQL: Fix comment regarding non-nil histogram pointer 
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
 2024-08-13 08:55:24 +02:00
György Krajcsovits
6aee5b4b38 fix typo 
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
 2024-08-12 12:04:45 +02:00
György Krajcsovits
06a8886b94 Native histograms: define behavior when rate is null. 
Histogram quantile returns NaN in this case, which might be
surprising, so add a unit test that clarifies that this is
intentional.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
 2024-08-12 10:40:21 +02:00
Arve Knudsen
5fd66ba855
Merge pull request #14510 from aknuds1/arve/add-normalize-name-tests 
otlptranslator: Add tests for BuildCompliantName
 2024-08-09 18:21:16 +02:00
Julien
616038f2b6
Merge pull request #14553 from prometheus/dependabot/go_modules/github.com/aws/aws-sdk-go-1.55.5 
build(deps): bump github.com/aws/aws-sdk-go from 1.54.19 to 1.55.5
 2024-08-09 12:06:36 +02:00
Julien
7b0f6f1aed
Merge pull request #14573 from prometheus/dependabot/github_actions/scripts/actions/setup-go-5.0.2 
build(deps): bump actions/setup-go from 5.0.1 to 5.0.2 in /scripts
 2024-08-09 12:06:05 +02:00
Julien
c210917b91
Merge pull request #14571 from prometheus/dependabot/github_actions/actions/upload-artifact-4.3.4 
build(deps): bump actions/upload-artifact from 4.3.3 to 4.3.4
 2024-08-09 12:05:37 +02:00
Julien
4387de8189
Merge pull request #14568 from prometheus/dependabot/github_actions/bufbuild/buf-setup-action-1.35.1 
build(deps): bump bufbuild/buf-setup-action from 1.34.0 to 1.35.1
 2024-08-09 12:05:24 +02:00
Charles Korn
f992f81bd0
Merge branch 'main' into sum-and-avg-over-mixed-custom-exponential-histograms 
Signed-off-by: Charles Korn <charleskorn@users.noreply.github.com>
 2024-08-09 13:58:54 +10:00
Charles Korn
5cfdde327c
Address PR feedback: add extra test case 
Signed-off-by: Charles Korn <charles.korn@grafana.com>
 2024-08-09 13:57:37 +10:00
Charles Korn
82bb35fabb
Address PR feedback: fix typo and rename variable 
Signed-off-by: Charles Korn <charles.korn@grafana.com>
 2024-08-09 13:51:31 +10:00
George Krajcsovits
cf62fb5c44
Merge pull request #14629 from krajorama/fix-to-encoded-chunks 
Fix ToEncodedChunks minT for recoded chunks
 2024-08-08 20:00:31 +02:00
György Krajcsovits
1ea3781699 Fix ToEncodedChunks minT for recoded chunks 
Discovered while working on #14546 OOO native histograms.
Not triggered on main before #14546 as the code path is unused.

There was a bug where the min time of a chunk was adjusted even
if it was only recoded and not completely new.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
 2024-08-08 15:22:46 +02:00
Björn Rabenstein
b5cdb4e479
Merge pull request #14628 from prometheus/beorn7/doc 
docs: Be more explicit about `promql-experimental-functions`
 2024-08-08 15:00:04 +02:00
Julien
5857888926
Merge pull request #14570 from prometheus/dependabot/github_actions/golangci/golangci-lint-action-6.1.0 
build(deps): bump golangci/golangci-lint-action from 6.0.1 to 6.1.0
 2024-08-08 14:45:43 +02:00
Julien
339889f60f
Merge pull request #14567 from prometheus/dependabot/github_actions/ossf/scorecard-action-2.4.0 
build(deps): bump ossf/scorecard-action from 2.3.3 to 2.4.0
 2024-08-08 14:44:46 +02:00
Julien
d7e4ba6cf1
Merge pull request #14566 from prometheus/dependabot/github_actions/actions/setup-go-5.0.2 
build(deps): bump actions/setup-go from 5.0.1 to 5.0.2
 2024-08-08 14:43:55 +02:00
beorn7
2360ce8d2b docs: Be more explicit about promql-experimental-functions 
We have not mentioned that experimental PromQL functions might get
removed entirely, although that's one of the most important properties
of functions declared experimental.

Signed-off-by: beorn7 <beorn@grafana.com>
 2024-08-08 14:37:29 +02:00
Björn Rabenstein
41f8a0223c
Merge pull request #14609 from charleskorn/rate-over-mixed-custom-exponential-histograms 
promql: fix "cannot reduce resolution to custom buckets schema" panic in `rate` over native histograms with mix of custom and exponential buckets
 2024-08-08 14:03:41 +02:00
Manik Rana
02c465bf58
textparse: Implement CreatedTimestamp() in openmetricsparse.go (#14356) 
* feat: initial implement of createedTimestamp() with tests

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* feat: return ct after finding it

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: remove unneeded test

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: add comments

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* feat: multiple changes

- implement changes from pair programming session
- use newParse.val()
- advance parser p if ct is found

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* fix: check if err from p.Next()

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* feat: advance parser and parse histograms + summary

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* fix: restore previous tests

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* fix: retore failing tests

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: remove unneeded comments

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* fix: return nil when mtype doesn't match

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: update go fmt version

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: cleanup

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* fix: comments

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* feat: document deepcopyparser

Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: lint

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* fix: cover edgecase of `gauge_created` in CreatedTimestamp()

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* refac: readability updates

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* refac: dedeuplicate labeldiff checks

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* tests: add tests for new label functions

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* feat: document CreatedTimestamp func

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* refac: optimize `CreatedTimestamp()`

 - Use refactored CreatedTimestamp function with bug fixes
 - Remove unused code in labels.go
 - Improve code documentation

 Signed-off-by: Manik Rana <manikrana54@gmail.com>

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: add tests and lint fixes

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: remove mName

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: lint

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: comments

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* tests: add tests for CT parse failures and deepCopy

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* refac: edit expectCT struct

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: lint

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: add new label in deepCopy

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* fix: use p.builder in deepCopy

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* fix: add NewMetricsParserWithOpts

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: lint

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: comments

Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>

* chore: comments

Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>

* chore: rename var

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* fix: add condition for OM fuzzing

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* fix: build tags

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* refac: default skipCT to false

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* refac: rename skipCT to skipCTSeries

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: formatting

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: comments and readability updates

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: comments

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>

* refac: remove NewOpenMetricsParserWithOpts

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: lint

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* refac: extract skipCTSeries logic from parseMetricSuffix

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* refac: inline create a NewOpenMetricsParser

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: comments

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: comments

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>

* refac: improve error handling

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* fix: return error instead of nil

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* fix: remove skipCT check from tBraceOpen

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* Pair programming with Manik, Arthur and Daniel.

Signed-off-by: bwplotka <bwplotka@gmail.com>

* chore: comments and use helper funcs

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: lint

Signed-off-by: Manik Rana <manikrana54@gmail.com>

---------

Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
Signed-off-by: bwplotka <bwplotka@gmail.com>
Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
 2024-08-08 12:35:35 +01:00