Commit Graph

8350 Commits

Author SHA1 Message Date
Björn Rabenstein
ffb3ef6765
Merge pull request #8315 from yeya24/create-dir
Create output directory before backfilling
2020-12-28 14:36:00 +01:00
Julien Pivotto
a033845d87
Merge pull request #8317 from ArthurSens/fuzzit_todo
Change seconds()*1000 to milliseconds()
2020-12-25 17:03:49 +01:00
arthursens
8493704b9b Change seconds()*1000 to milliseconds()
Signed-off-by: arthursens <arthursens2005@gmail.com>
2020-12-25 10:45:23 -03:00
yeya24
cedd2dbec9 create output directory before backfilling
Signed-off-by: yeya24 <yb532204897@gmail.com>
2020-12-24 23:36:36 -05:00
Bartlomiej Plotka
70c69aaa6c
Small cleanup of API code. (#8258)
* Small cleanup of API code.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Fixed typo spotted by Julien.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
2020-12-24 00:27:13 +00:00
AdaephonBen
dca6954b0a
promtool: Add URL scheme when not provided (#7956)
Signed-off-by: AdaephonBen <ma18btech11011@iith.ac.in>
2020-12-23 19:52:04 +01:00
Julien Pivotto
3e8a96e652
Merge pull request #8294 from prometheus/dependabot/npm_and_yarn/web/ui/react-app/ini-1.3.8
Bump ini from 1.3.5 to 1.3.8 in /web/ui/react-app
2020-12-23 15:12:11 +01:00
Alfred Krohmer
4efca5ab70
EC2 service discovery: reuse EC2 client instead of recreating it every time (#8311)
This also caches credentials that are obtained e.g. via IRSA on AWS EKS.
Previously, every refresh cycle would request the credentials again.

Signed-off-by: Alfred Krohmer <alfred.krohmer@logmein.com>
2020-12-22 15:47:44 +00:00
Ganesh Vernekar
faa1554aa1
Don't call runtime.GC() after compaction (#8276)
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2020-12-22 14:44:17 +00:00
Ben Ye
caa173d2aa
Support matchers for Labels API (#8301)
Signed-off-by: Ben Ye <yb532204897@gmail.com>
Co-authored-by: Erik Klockare <eklockare@gmail.com>
2020-12-22 11:02:19 +00:00
Julien Pivotto
e5478983c4
Merge pull request #8299 from roidelapluie/fixdocker
Fix docker image build
2020-12-16 21:49:53 +01:00
Julien Pivotto
041d0bcfd6 Fix docker image build
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-12-16 21:21:22 +01:00
Nicolas Lamirault
aa1ca13025
Add: Custom tags and prefix in Prometheus Mixin (#8287)
* Add: custom tags and prefix

Signed-off-by: Nicolas Lamirault <nicolas.lamirault@gmail.com>

* Fix: fmt

Signed-off-by: Nicolas Lamirault <nicolas.lamirault@gmail.com>
2020-12-16 18:49:06 +01:00
dependabot[bot]
abe85ce052
Bump ini from 1.3.5 to 1.3.8 in /web/ui/react-app
Bumps [ini](https://github.com/isaacs/ini) from 1.3.5 to 1.3.8.
- [Release notes](https://github.com/isaacs/ini/releases)
- [Commits](https://github.com/isaacs/ini/compare/v1.3.5...v1.3.8)

Signed-off-by: dependabot[bot] <support@github.com>
2020-12-15 22:50:00 +00:00
Julien Pivotto
8dc53c2218
Api: Verify that at every matcher in api/v1/series is not empty (#8288)
Fixed #8286

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-12-15 17:24:57 +00:00
Atibhi Agrawal
0dd6e0cb76
change string(model.metricnamelabel) to model.metricnamelabel (#8289)
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
2020-12-15 20:17:31 +05:30
Atibhi Agrawal
7a519fb294
add fullstop (#8290)
Signed-off-by: aSquare14 <atibhi.a@gmail.com>
2020-12-15 20:13:44 +05:30
Atibhi Agrawal
c08009139a
Add docs for backfill (#8230)
* Add docs for backfill

Signed-off-by: aSquare14 <atibhi.a@gmail.com>
2020-12-10 16:29:44 +01:00
lzhfromustc
27a6e1e174
test: add buffer to channel to avoid goroutine leak (#8274)
Signed-off-by: lzhfromustc <lzhfromustc@gmail.com>
2020-12-10 09:09:21 +00:00
Julien Pivotto
66f47e116e
Merge pull request #8269 from roidelapluie/fuzz
Remove fuzzit, add oss-fuzz on PR
2020-12-09 21:58:04 +01:00
Julien Pivotto
7bc48123e5 Restrict fuzzing on go paths
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-12-09 21:37:14 +01:00
Julien Pivotto
4a4ea309b6 Add oss-fuzz on PR's
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-12-09 21:13:18 +01:00
Julien Pivotto
c666879e3f Remove fuzzit
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-12-09 21:10:31 +01:00
Björn Rabenstein
e13f4a5f6e
Merge pull request #8232 from roidelapluie/eof
TSDB backfill: enforce # EOF
2020-12-09 13:22:03 +01:00
Julien Pivotto
7957731339 Inline defer
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-12-09 09:23:39 +01:00
Julien Pivotto
82b5f1d8b1 Backfill: Use mmap to reuse parser code
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-12-08 23:48:31 +01:00
Björn Rabenstein
511511324a
Merge pull request #8235 from Allex1/master
Update remote-write grafana mixin
2020-12-08 14:50:47 +01:00
Julien Pivotto
87e142c02f
remote_write documentation: fix blank lines (#8264)
The blank line is not efficiently placed in the remote_write config,
affecting readability.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-12-08 14:27:50 +01:00
Björn Rabenstein
404b2980cf
Merge pull request #8254 from prometheus/beorn7/mixin
mixin: Add a capability to exclude non-prod AM instances
2020-12-08 13:33:27 +01:00
Arthur Silva Sens
64a106c5dd
Logging added for when compaction takes more than the block time range (#8151)
* Logging added for when compaction takes more than the block time range

Signed-off-by: arthursens <arthursens2005@gmail.com>

* Log only if no errors were already logged

Signed-off-by: arthursens <arthursens2005@gmail.com>

* Log duration as human readable string

Signed-off-by: arthursens <arthursens2005@gmail.com>

* Move logging from compactHead() to Compact()

Signed-off-by: arthursens <arthursens2005@gmail.com>

* Compute duration of all head compactions plus wal truncation

Signed-off-by: arthursens <arthursens2005@gmail.com>

* Remove named return added os first commits

Signed-off-by: arthursens <arthursens2005@gmail.com>

* Address nits

Signed-off-by: arthursens <arthursens2005@gmail.com>

* Change miliseconds to seconds to make fuzzit tests happy

Signed-off-by: ArthurSens <arthursens2005@gmail.com>
2020-12-07 21:29:43 +00:00
beorn7
553f904f2d mixin: Add a capability to exclude non-prod AM instances
Signed-off-by: beorn7 <beorn@grafana.com>
2020-12-03 20:59:53 +01:00
Thibault Jamet
f055690b34
Clarify: all rules in a group are concomitant (#8248)
Improve the documentation to clarify the differences beetween rules in a
group and outside a group.

Signed-off-by: Thibault Jamet <tjamet@users.noreply.github.com>
2020-12-03 10:32:10 +00:00
Mitsuo Heijo
8b64b70fe4
Guard closing quitCh with sync.Once to prevent double close (#8242)
* Guard closing quitCh with sync.Once to prevent double close

Signed-off-by: Mitsuo Heijo <mitsuo.heijo@gmail.com>
2020-12-02 09:39:54 +01:00
birca
3ec4161575 Update remote-write grafana mixin
Signed-off-by: birca <birca@adobe.com>
2020-12-02 09:50:15 +02:00
Matt Berther
acee998df6
CNAME responses can occur with "Type: A" dns_sd_config requests (#8216)
Signed-off-by: Matt Berther <mattberther@users.noreply.github.com>
2020-12-01 09:32:15 +00:00
johncming
a6e18916ab
tsdb: Remove duplicate variables. (#8239)
Signed-off-by: johncming <johncming@yahoo.com>
2020-11-30 08:55:33 +00:00
Mike DeSimone
759150878c
fix trivial typos (#8238)
Signed-off-by: Mike DeSimone <michael.2.desimone@gmail.com>
2020-11-29 01:23:10 +00:00
Julien Pivotto
868184b33a
Merge pull request #8234 from prometheus/release-2.23
Merge release 2.23 to master
2020-11-28 00:41:46 +01:00
Ganesh Vernekar
26d89b4b07
Cut v2.23.0 (#8223)
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2020-11-26 15:41:54 +05:30
Atibhi Agrawal
b317b6ab9c
Backfill from OpenMetrics format (#8084)
* get parser working

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* import file created

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Find min and max ts

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* make two passes over file and write to tsdb

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* print error messages

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fix Max and Min initializer

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Start with unit tests

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* reset file read

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* align blocks to two hour range

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Add cleanup test

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* remove .ds_store

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* add license to import_test

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fix Circle CI error

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Refactor code
Move backfill from tsdb to promtool directory

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix gitignore

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Remove panic
Rename ContenType

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* adjust mint

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix return statement

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix go modules

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Added unit test for backfill

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix CI error

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fix file handling

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Close DB

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Close directory

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Error Handling

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* inline err

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fix command line flags

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* add spaces before func
fix pointers

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Add defer'd calls

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* move openmetrics.go content to backfill

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* changed args to flags

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* add tests for wrong OM files

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Added additional tests

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Add comment to warn of func reuse

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Make input required in main.go

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* defer blockwriter close

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix defer

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* defer

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Remove contentType

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* remove defer from backfilltest

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fix defer remove in backfill_test

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* changes to fix CI errors

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix go.mod

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* change package name

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* assert->require

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* remove todo

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix format

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix todo

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix createblock

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix tests

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix defer

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix return

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* check err for anon func

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* change comments

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* update comment

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fix for the Flush Bug

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix formatting, comments, names

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Print Blocks

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* cleanup

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* refactor test to take care of multiple samples

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* refactor tests

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* remove om

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* I dont know what I fixed

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fix tests

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fix tests, add test description, print blocks

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* commit after 5000 samples

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* reviews part 1

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Series Count

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix CI

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* remove extra func

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* make timestamp into sec

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Reviews 2

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Add Todo

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fixes

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fixes reviews

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* =0

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* remove backfill.om

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* add global err var, remove stuff

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* change var name

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* sampleLimit pass as parameter

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Add test when number of samples greater than batch size

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Change name of batchsize

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* revert export

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* nits

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* remove

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* add comment, remove newline,consistent err

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Print Blocks

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Modify comments

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* db.Querier

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* add sanity check , get maxt and mint

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* ci error

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* comment change

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* nits

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* NoError

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix

Signed-off-by: aSquare14 <atibhi.a@gmail.com>
2020-11-26 10:37:06 +05:30
Ganesh Vernekar
dff967286e
Set the min time of Head properly after truncation (#8212)
* Set the min time of Head properly after truncation

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix lint

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Enhance compaction plan logic for completely deleted small block

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix review comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2020-11-25 18:33:30 +05:30
Julien Pivotto
f50dde7031
Remove Krasi as TSDB maintainer (#8213)
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-11-24 20:24:25 +01:00
Ganesh Vernekar
9972e69c43
Cut v2.23.0-rc.0 (#8206)
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2020-11-23 14:52:37 +05:30
George Brighton
76cd5f4c7f
Fix typo in EC2 SD docs (#8210)
Signed-off-by: George Brighton <george@gebn.co.uk>
2020-11-20 19:22:03 +00:00
Muhammad Falak R Wani
587fd740be
discovery: ec2: expose IPv6 as label (#7086)
* discovery: ec2: expose IPv6 as label

Signed-off-by: Muhammad Falak R Wani <falakreyaz@gmail.com>
2020-11-20 17:13:52 +00:00
Justin Toh
5f103aaecc
docs: Fix "time duration" link (#8209)
Signed-off-by: Justin Toh <tohjustin@hotmail.com>
2020-11-20 12:25:45 +00:00
Chris Marchbanks
c8f810083d
Cache label name and last value when adding series (#8192)
Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>
2020-11-19 11:18:12 -07:00
Ganesh Vernekar
c2ce236de1
Upgrade react deps (#8207)
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2020-11-19 17:10:29 +01:00
gotjosh
4eca4dffb8
Allow metric metadata to be propagated via Remote Write. (#6815)
* Introduce a metadata watcher

Similarly to the WAL watcher, its purpose is to observe the scrape manager and pull metadata. Then, send it to a remote storage.

Signed-off-by: gotjosh <josue@grafana.com>

* Additional fixes after rebasing.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Rework samples/metadata metrics.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Use more descriptive variable names in MetadataWatcher collect.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Fix issues caused during rebasing.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Fix missing metric add and unneeded config code.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Address some review comments.

Signed-off-by: Callum Styan <callumstyan@gmail.com>

* Fix metrics and docs

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Replace assert with require

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Bring back max_samples_per_send metric

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix tests

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

Co-authored-by: Callum Styan <callumstyan@gmail.com>
Co-authored-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2020-11-19 20:53:03 +05:30
Ganesh Vernekar
3ad25a6dc3
Merge pull request #8204 from prometheus/release-2.22
Backport v2.22.2 to master
2020-11-19 19:57:52 +05:30