RepoMirrors/prometheus
1
0
Fork 0
mirror of https://github.com/prometheus/prometheus synced 2024-12-28 09:42:22 +00:00
Code Issues Packages Projects Releases Wiki Activity
12,371 Commits 243 Branches 444 Tags 653 MiB
55dcaab41b
Commit Graph

12371 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Łukasz Mierzwa
55dcaab41b Fix TestScrapeLoopDiscardDuplicateLabels test 
This test calls Rollback() which is normally called from within append code.
Doing so means that staleness tracking data is outdated and need to by cycled manually.

Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
 2024-02-27 11:36:16 +00:00
Łukasz Mierzwa
50c81bed86 Check for duplicated series on a scrape 
When Prometheus scrapes a target and it sees the same time series repeated multiple times it currently silently ignores that.
This change adds a test for that and fixes the scrape loop so that:

- Only first sample for each unique time series is appended
- Duplicated samples increment the prometheus_target_scrapes_sample_duplicate_timestamp_total metric

This allows one to identify such scrape jobs and targets.

Benchmark results:

```
name                            old time/op    new time/op    delta
ScrapeLoopAppend-8                64.8µs ± 2%    71.1µs ±20%   +9.75%  (p=0.000 n=10+10)
ScrapeLoopAppendOM-8              64.2µs ± 1%    68.5µs ± 7%   +6.71%  (p=0.000 n=9+10)
TargetsFromGroup/1_targets-8      14.2µs ± 1%    14.5µs ± 1%   +1.99%  (p=0.000 n=10+10)
TargetsFromGroup/10_targets-8      149µs ± 1%     152µs ± 1%   +2.05%  (p=0.000 n=9+10)
TargetsFromGroup/100_targets-8    1.49ms ± 4%    1.48ms ± 1%     ~     (p=0.796 n=10+10)

name                            old alloc/op   new alloc/op   delta
ScrapeLoopAppend-8                19.9kB ± 1%    17.8kB ± 3%  -10.23%  (p=0.000 n=8+10)
ScrapeLoopAppendOM-8              19.9kB ± 1%    18.3kB ±10%   -8.14%  (p=0.001 n=9+10)
TargetsFromGroup/1_targets-8      2.43kB ± 0%    2.43kB ± 0%   -0.15%  (p=0.045 n=10+10)
TargetsFromGroup/10_targets-8     24.3kB ± 0%    24.3kB ± 0%     ~     (p=0.083 n=10+9)
TargetsFromGroup/100_targets-8     243kB ± 0%     243kB ± 0%     ~     (p=0.720 n=9+10)

name                            old allocs/op  new allocs/op  delta
ScrapeLoopAppend-8                  9.00 ± 0%      9.00 ± 0%     ~     (all equal)
ScrapeLoopAppendOM-8                10.0 ± 0%      10.0 ± 0%     ~     (all equal)
TargetsFromGroup/1_targets-8        40.0 ± 0%      40.0 ± 0%     ~     (all equal)
TargetsFromGroup/10_targets-8        400 ± 0%       400 ± 0%     ~     (all equal)
TargetsFromGroup/100_targets-8     4.00k ± 0%     4.00k ± 0%     ~     (all equal)
```

Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
 2024-02-27 11:36:16 +00:00
Łukasz Mierzwa
1a8ea78207 Fix BenchmarkScrapeLoopAppendOM 
OpenMetrics requires EOF comment at the end of metrics body, but the makeTestMetrics() function doesn't append it.
This means this benchmark tests a response with errors but I don't think that was the intention.

Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
 2024-02-27 11:36:16 +00:00
Julien
0474b0bc39
Merge pull request #13621 from bboreham/rw-docs 
Docs: update remote-write defaults
 2024-02-27 12:28:16 +01:00
Julien
6ef810f5f8
Merge pull request #13650 from prometheus/dependabot/go_modules/github.com/docker/docker-25.0.3incompatible 
build(deps): bump github.com/docker/docker from 25.0.0+incompatible to 25.0.3+incompatible
 2024-02-27 12:18:17 +01:00
Julien
32a3b2e50d
Merge pull request #13648 from prometheus/dependabot/go_modules/k8s-io-a2ca04ec63 
build(deps): bump the k8s-io group with 3 updates
 2024-02-27 12:18:06 +01:00
Julien
8d1c3f8df5
Merge pull request #13649 from prometheus/dependabot/go_modules/github.com/aws/aws-sdk-go-1.50.25 
build(deps): bump github.com/aws/aws-sdk-go from 1.50.0 to 1.50.25
 2024-02-27 12:17:56 +01:00
Julien
d753fd3fde
Merge pull request #13651 from prometheus/dependabot/go_modules/github.com/linode/linodego-1.29.0 
build(deps): bump github.com/linode/linodego from 1.27.1 to 1.29.0
 2024-02-27 12:17:46 +01:00
Björn Rabenstein
dbb4055633
Merge pull request #13593 from machine424/no-index 
discovery: kubernetes: Avoid creating unnecessary Kubernetes indexers…
 2024-02-27 10:48:37 +01:00
Bryan Boreham
17edf98083
Merge pull request #13634 from bboreham/slim-parse-tests 
refactor: metrics parsing: extract common checking code
 2024-02-26 18:14:00 +00:00
gotjosh
0e25418eb6
Merge pull request #13652 from rnaveiras/rnaveiras-update-remote-write-docs-queue-default 
[docs] Update remote write queue defaults
 2024-02-26 17:38:00 +00:00
Bryan Boreham
ba6aa95222 refactor: metrics parsing: extract common checking code 
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2024-02-26 16:37:41 +00:00
Raúl Naveiras
58f528dcbc
[docs] Update remote write queue defaults 
Update docs to reflect the default values in the
`config.DefaultQueueConfig`

Signed-off-by: Raúl Naveiras <me@raulnaveiras.com>
 2024-02-26 15:59:29 +00:00
dependabot[bot]
851a28203c
build(deps): bump github.com/linode/linodego from 1.27.1 to 1.29.0 
Bumps [github.com/linode/linodego](https://github.com/linode/linodego) from 1.27.1 to 1.29.0.
- [Release notes](https://github.com/linode/linodego/releases)
- [Commits](https://github.com/linode/linodego/compare/v1.27.1...v1.29.0)

---
updated-dependencies:
- dependency-name: github.com/linode/linodego
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-02-26 15:04:44 +00:00
dependabot[bot]
4a0541178d
build(deps): bump github.com/docker/docker 
Bumps [github.com/docker/docker](https://github.com/docker/docker) from 25.0.0+incompatible to 25.0.3+incompatible.
- [Release notes](https://github.com/docker/docker/releases)
- [Commits](https://github.com/docker/docker/compare/v25.0.0...v25.0.3)

---
updated-dependencies:
- dependency-name: github.com/docker/docker
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-02-26 15:04:22 +00:00
dependabot[bot]
67f13f3541
build(deps): bump github.com/aws/aws-sdk-go from 1.50.0 to 1.50.25 
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.50.0 to 1.50.25.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.50.0...v1.50.25)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-02-26 15:03:53 +00:00
dependabot[bot]
3fc6577ec6
build(deps): bump the k8s-io group with 3 updates 
Bumps the k8s-io group with 3 updates: [k8s.io/api](https://github.com/kubernetes/api), [k8s.io/apimachinery](https://github.com/kubernetes/apimachinery) and [k8s.io/client-go](https://github.com/kubernetes/client-go).


Updates `k8s.io/api` from 0.28.6 to 0.29.2
- [Commits](https://github.com/kubernetes/api/compare/v0.28.6...v0.29.2)

Updates `k8s.io/apimachinery` from 0.28.6 to 0.29.2
- [Commits](https://github.com/kubernetes/apimachinery/compare/v0.28.6...v0.29.2)

Updates `k8s.io/client-go` from 0.28.6 to 0.29.2
- [Changelog](https://github.com/kubernetes/client-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/kubernetes/client-go/compare/v0.28.6...v0.29.2)

---
updated-dependencies:
- dependency-name: k8s.io/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: k8s-io
- dependency-name: k8s.io/apimachinery
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: k8s-io
- dependency-name: k8s.io/client-go
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: k8s-io
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-02-26 15:03:41 +00:00
Julien
e6d3d6abd9
Merge pull request #13604 from arukiidou/chore/bump-to-go1.21 
ci: Update to Go version 1.22/ old to 1.21
 2024-02-26 16:02:11 +01:00
Bryan Boreham
d2817e9c91
Merge pull request #12304 from bboreham/labels-symboltable 
Labels: reduce memory by de-duplicating strings in SymbolTables
 2024-02-26 14:06:27 +00:00
Julien
4284c29597
Merge pull request #13645 from bboreham/merge-2.50.1 
Merge 2.50.1 into main
 2024-02-26 14:10:43 +01:00
Bryan Boreham
518ab179fb Merge branch 'release-2.50' into merge-2.50.1 2024-02-26 11:50:06 +00:00
Bryan Boreham
e1a741a0d7 labels: update copyright dates 
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2024-02-26 11:45:25 +00:00
Bryan Boreham
55e7de04f8 model/labels (stringlabels): use strings.Clone 
Suggestion from @colega.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2024-02-26 11:45:25 +00:00
Bryan Boreham
d16ce3c9bd model/labels (dedupelabels): small clarifications 
Suggestion from @colega.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2024-02-26 11:45:25 +00:00
Bryan Boreham
c8c28efcb3 Run Go tests with -tags dedupelabels 
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2024-02-26 11:45:25 +00:00
Bryan Boreham
0347148628 promql: fuzz test needs symbol table for parser 
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2024-02-26 11:45:25 +00:00
Bryan Boreham
5f50d974c9 scraping: reset symbol table periodically 
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2024-02-26 11:45:25 +00:00
Bryan Boreham
4e748b9cd8 scraping: re-use labels Builder in scrape report metrics 
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2024-02-26 11:45:25 +00:00
Bryan Boreham
abb3a62f04 scraping: re-use symbol table for scrape loops 
One symbol table for all loops in the same scrape pool, i.e. from the
same job.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2024-02-26 11:45:25 +00:00
Bryan Boreham
0403d098e1 scraping: re-use symbolTable for target discovery 
Call labels.NewBuilderWithSymbolTable.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2024-02-26 11:45:25 +00:00
Bryan Boreham
9ba13de220 scraping: pass a Builder to get Target labels 
This saves memory allocations from making a new Builder every time.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2024-02-26 11:45:25 +00:00
Bryan Boreham
eff3a13e19 model/textparse: parsers take a labels SymbolTable 
This allows strings to be interned to save memory.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2024-02-26 11:45:25 +00:00
Bryan Boreham
123109e967 config: make work with SymbolTable labels 
Need a SymbolTable when we expand ExternalLabels.

And we can't use labels.NewBuilder because we need to add blank values.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2024-02-26 11:45:25 +00:00
Bryan Boreham
3716326f3f rules: call NewScratchBuilder 
Need to initialize ScratchBuilder with a SymbolTable.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2024-02-26 11:45:25 +00:00
Bryan Boreham
925134e6de tsdb tests: make work with labels SymbolTable 
Need to initialize decoders with SymbolTable.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2024-02-26 11:45:25 +00:00
Bryan Boreham
93b72ec5dd tsdb: create SymbolTables for labels as required 
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2024-02-26 11:45:25 +00:00
Bryan Boreham
8c9b028536
Merge pull request #13641 from bboreham/pick-13633 
Cut 2.50.1 with metadata api bugfix (cherry-picked #13633)
 2024-02-26 11:31:35 +00:00
Bryan Boreham
e537d6d1c1 Cut 2.50.1 with metadata api bugfix 
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2024-02-26 10:00:21 +00:00
ismail simsek
eecaa71ff1 Fix: metadata API using wrong field names (#13633) 
Fix is to add json tags to `Metadata` struct. Absence of these tags
causes Go to use the field name, which starts with an upper-case
letter and breaks the protocol.

Extend tests to verify the JSON response.

Signed-off-by: ismail simsek <ismailsimsek09@gmail.com>
 2024-02-26 09:54:39 +00:00
ismail simsek
2507469291
Fix: metadata API using wrong field names (#13633) 
Fix is to add json tags to `Metadata` struct. Absence of these tags
causes Go to use the field name, which starts with an upper-case
letter and breaks the protocol.

Extend tests to verify the JSON response.

Signed-off-by: ismail simsek <ismailsimsek09@gmail.com>
 2024-02-26 09:53:39 +00:00
junya koyama
37613ef3b6
Merge branch 'chore/bump-to-go1.21' of https://github.com/arukiidou/prometheus into chore/bump-to-go1.21 2024-02-24 00:34:33 +09:00
junya koyama
e171f6c8aa
go: fix go.mod version from 1.21.7 to 1.21 
Signed-off-by: junya koyama <arukiidou@yahoo.co.jp>
 2024-02-24 00:33:35 +09:00
Bryan Boreham
2ac1632eec storage/remote: improve symbol-table handling 
On the incoming path, `writeHandler.write()` creates a new table for
each request.

`labelProtosToLabels` takes a `ScratchBuilder` now.

Call `NewScratchBuilder` as required in tests.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2024-02-23 13:50:27 +00:00
Bryan Boreham
8f525b4ba4 storage/remote tests: refactor: extract function newTestQueueManager 
To reduce repetition.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2024-02-23 13:50:27 +00:00
Bryan Boreham
b39286fd1f Add dedupelabels tag to not build regular labels 
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2024-02-23 13:50:27 +00:00
Bryan Boreham
d51a5344cd labels: new version de-duplicating strings in SymbolTables 
The individual strings for label names and values are held in a table,
and each Labels value is a run of varint-encoded indexes into that table.

When creating new labels, a sync.Mutex is locked around reads and writes.
When reading labels, there is no locking because the table of strings
used by those labels is immutable.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2024-02-23 13:50:27 +00:00
Bryan Boreham
28191109a8 Labels: add fake versions of SymbolTable apis 
So we can use them where necessary for internlabels implementation.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2024-02-23 13:50:27 +00:00
Bryan Boreham
d1af84f6ee Labels: move Builder and Reset out of common 
New internstrings implementation is different.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2024-02-23 13:50:27 +00:00
Bryan Boreham
5aa4473894 labels tests: extend TestBuilder 
Start with empty base labels, also check new and re-used symbol tables

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2024-02-23 13:50:27 +00:00
Bryan Boreham
bb82a57e64 Labels: Call NewScratchBuilder in test_utils 
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
 2024-02-23 13:50:27 +00:00