RepoMirrors/prometheus
1
0
Fork 0
mirror of https://github.com/prometheus/prometheus synced 2024-12-27 00:53:12 +00:00
Code Issues Packages Projects Releases Wiki Activity
9,268 Commits 243 Branches 444 Tags 653 MiB
60d0990886
Commit Graph

95 Commits

Author SHA1 Message Date
fpetkovski
449f874679 promtool: add extended flag for tsdb analysis 
The compaction analysis which runs under promtool tsdb analyze can be an
intensive process which slows down the entire command.

This commit adds an --extended flag to tsdb analyze which can be toggled
for running long running tasks, such as compaction analysis.

Signed-off-by: fpetkovski <filip.petkovsky@gmail.com>
 2021-09-08 10:50:01 +02:00
Julien Pivotto
bd24e2fb92
Merge pull request #9303 from LeviHarrison/backfill-return-1 
Rules backfill: return 1 if unsuccessful
 2021-09-07 18:00:42 +02:00
Levi Harrison
34e1b47968
Fixed error handling 
Signed-off-by: Levi Harrison <git@leviharrison.dev>
 2021-09-06 21:55:57 -04:00
Holger Hans Peter Freyther
5edec40d60 promtool: Speed up checking for duplicate rules 
Trade space for speed. Convert all rules into our temporary struct, sort
and then iterate. This is a significant when having many rules.

Signed-off-by: Holger Hans Peter Freyther <holger@moiji-mobile.com>
 2021-09-06 23:10:26 +08:00
Levi Harrison
3b5257d869
Changed disabled_features to feature_flags 
Signed-off-by: Levi Harrison <git@leviharrison.dev>
 2021-07-13 22:03:51 -04:00
Filip Petkovski
7c125aa5fb
Promtool: Add support for compaction analysis (#8940) 
* Extend promtool to support compaction analysis

This commit extends the promtool tsdb analyze command to help
troubleshoot high Prometheus disk usage. The command now plots a
distribution of how full chunks are relative to the maximum capacity of
120 samples per chunk.

Signed-off-by: fpetkovski <filip.petkovsky@gmail.com>

* Update cmd/promtool/tsdb.go

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
 2021-07-02 11:08:52 +01:00
Levi Harrison
90976e7505
Promtool: Add feature flags to unit tests (#8958) 
* Added feature flag support to unit tests

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Added/fixed tests

Signed-off-by: Levi Harrison <git@leviharrison.dev>

* Addressed review comments

Signed-off-by: Levi Harrison <git@leviharrison.dev>
 2021-06-30 22:43:39 +01:00
Levi Harrison
ca1896c15b
Promtool: Validate service discovery files (#8950) 
* Check SD files in promtool

Signed-off-by: Levi Harrison <git@leviharrison.dev>
 2021-06-29 17:32:59 +02:00
Steve Kuznetsov
fd6c852567
promtool: backfill: allow configuring block duration (#8919) 
* promtool: backfill: allow configuring block duration

When backfilling large amounts of data across long periods of time, it
may in certain circumstances be useful to use a longer block duration to
increase the efficiency and speed of the backfilling process. This patch
adds a flag --block-duration-power to allow a user to choose the power N
where the block duration is 2^(N+1)h.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

* promtool: use sub-tests in backfill testing

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

* backfill: add messages to tests for clarity

When someone new breaks a test, seeing "expected: false, got: true" is
really not useful. A nice message helps here.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

* backfill: test long block durations

A test that uses a long block duration to write bigger blocks is added.
The check to make sure all blocks are the default duration is removed.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>
 2021-06-29 14:53:38 +05:30
Julien Pivotto
ba76bceb6b
Merge pull request #8917 from stevekuznetsov/skuznets/silence-backfill 
promtool: backfill: allow silencing output
 2021-06-14 23:27:18 +02:00
Levi Harrison
b5f6f8fb36 Switched to go-kit/log 
Signed-off-by: Levi Harrison <git@leviharrison.dev>
 2021-06-11 12:28:36 -04:00
Steve Kuznetsov
ee771a2a66
promtool: backfill: allow silencing output 
When using the backfill command to add data to an ephemeral/test
Prometheus instance, it is not important to see which data was added as
it is often generated ahead of time and mostly irrelevant to the
use-case. The current approach prints information about each block that
is written, but does so in a generally inefficient and costly manner.
This patch adds a `--quiet` flag that allows a user to opt out of this
behavior.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>
 2021-06-10 15:31:16 -07:00
Jess G
731545ad34
Add documentation for recording rule backfiller (#8674) 
* add docs for rule backfiller

Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
 2021-04-01 22:38:00 +02:00
Julien Pivotto
e635ca834b Add environment variable expansion in external label values 
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
 2021-03-30 01:36:28 +02:00
Björn Rabenstein
9549a15c6f
Merge pull request #7675 from JessicaGreben/jg/11-retroactive-rule-eval 
Add rule importer to backfill
 2021-03-29 19:09:21 +02:00
Ben Kochie
f0bccba1c3
Update Go modules for 2.26 (#8636) 
* Update Go modules for 2.26

Bump all Go modules to the latest upstream.

Signed-off-by: Ben Kochie <superq@gmail.com>

* Fix promtool for new client_golang

LabelValues now requires a list of string matchers.

Signed-off-by: Ben Kochie <superq@gmail.com>
 2021-03-24 09:41:12 +00:00
jessicagreben
8de4da3716 add changes per comments, fix tests 
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
 2021-03-20 12:38:30 -07:00
jessicagreben
e3a8132bb3 fix block alignment, add sample alignment 
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
 2021-03-15 12:44:58 -07:00
jessicagreben
7c26642460 add block alignment and write in 2 hr blocks 
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
 2021-03-14 10:10:55 -07:00
Pau Freixes
b1ac4a45e6 Add num scrapes as tsdb write benchmark command flag 
By default same value that was hardcoded is used, but with the
new flag added the number of scrapes can be increased to any value.

Signed-off-by: Pau Freixes <pfreixes@gmail.com>
 2021-03-10 11:17:07 +01:00
Arthur Silva Sens
537c0aff49
Prometheus and Promtool binaries now print help and usage to stdout (#8542) 
Signed-off-by: ArthurSens <arthursens2005@gmail.com>
 2021-02-25 19:52:34 +01:00
jessicagreben
78e84aed89 resolve merge conflict 
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
 2021-02-24 09:47:29 -08:00
Łukasz Hryniuk
ab41de68b4 Print details of API errors 
Signed-off-by: Łukasz Hryniuk <code@hryniuk.pl>
 2021-02-15 23:42:06 +01:00
Julien Pivotto
c1f8bd9944
Merge pull request #8432 from roidelapluie/backfillpanic 
backfill: move checkErr before we close the mmaped file
 2021-02-03 16:32:35 +01:00
Julien Pivotto
9334269f2b backfill: move checkErr before we close the mmaped file 
When printing the error, we still need access to the mmapped byte array
of the file. Therefore, we make sure that we run it before closing the
file.

I could have done something more complex like a defer, or not closing
the file, knowing that we would exit the program anyway. However, I
think that in case we extend this in the future, or this is copy/paster
elsewhere, we should continue closing the file. As it is small enough, I
went for the solution to call the function 3 times instead of playing
with a defer.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
 2021-02-01 21:18:42 +01:00
Jeremy Albinet
4a1f2c097e Typo on plural in checkRules/checkDuplicates 
Signed-off-by: Jeremy Albinet <jalbinet@synthesio.com>
 2021-02-01 15:43:05 +01:00
Julien Pivotto
ac2626757c Update exporter-toolkit to 0.5.0 
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
 2021-01-13 21:49:54 +01:00
Julien Pivotto
003d6451fc Promtool: add web config validation 
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
 2020-12-29 16:55:29 +01:00
beorn7
651b57b9ab Merge branch 'backfillhr' of git://github.com/roidelapluie/prometheus into review 2020-12-28 16:18:00 +01:00
Julien Pivotto
53480c168d Backfill: print created blocks only, add human-readable option 
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
 2020-12-23 20:42:30 +01:00
AdaephonBen
dca6954b0a
promtool: Add URL scheme when not provided (#7956) 
Signed-off-by: AdaephonBen <ma18btech11011@iith.ac.in>
 2020-12-23 19:52:04 +01:00
jessicagreben
cec3515fa3 fix linter 
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
 2020-11-30 08:17:51 -08:00
jessicagreben
2e9946e4d7 add test 
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
 2020-11-28 07:58:33 -08:00
jessicagreben
ac06d0a657 merge master/resolve conflict 
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
 2020-11-26 08:43:07 -08:00
jessicagreben
ee85c22adb flush samples to disk every 5k samples 
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
 2020-11-26 08:30:06 -08:00
Atibhi Agrawal
b317b6ab9c
Backfill from OpenMetrics format (#8084) 
* get parser working

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* import file created

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Find min and max ts

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* make two passes over file and write to tsdb

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* print error messages

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fix Max and Min initializer

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Start with unit tests

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* reset file read

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* align blocks to two hour range

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Add cleanup test

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* remove .ds_store

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* add license to import_test

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fix Circle CI error

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Refactor code
Move backfill from tsdb to promtool directory

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix gitignore

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Remove panic
Rename ContenType

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* adjust mint

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix return statement

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix go modules

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Added unit test for backfill

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix CI error

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fix file handling

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Close DB

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Close directory

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Error Handling

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* inline err

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fix command line flags

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* add spaces before func
fix pointers

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Add defer'd calls

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* move openmetrics.go content to backfill

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* changed args to flags

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* add tests for wrong OM files

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Added additional tests

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Add comment to warn of func reuse

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Make input required in main.go

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* defer blockwriter close

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix defer

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* defer

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Remove contentType

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* remove defer from backfilltest

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fix defer remove in backfill_test

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* changes to fix CI errors

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix go.mod

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* change package name

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* assert->require

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* remove todo

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix format

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix todo

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix createblock

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix tests

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix defer

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix return

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* check err for anon func

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* change comments

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* update comment

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fix for the Flush Bug

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix formatting, comments, names

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Print Blocks

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* cleanup

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* refactor test to take care of multiple samples

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* refactor tests

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* remove om

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* I dont know what I fixed

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fix tests

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fix tests, add test description, print blocks

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* commit after 5000 samples

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* reviews part 1

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Series Count

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix CI

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* remove extra func

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* make timestamp into sec

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Reviews 2

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Add Todo

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Fixes

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fixes reviews

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* =0

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* remove backfill.om

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* add global err var, remove stuff

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* change var name

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* sampleLimit pass as parameter

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Add test when number of samples greater than batch size

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Change name of batchsize

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* revert export

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* nits

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* remove

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* add comment, remove newline,consistent err

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Print Blocks

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* Modify comments

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* db.Querier

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* add sanity check , get maxt and mint

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* ci error

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* comment change

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* nits

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* NoError

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix

Signed-off-by: aSquare14 <atibhi.a@gmail.com>

* fix

Signed-off-by: aSquare14 <atibhi.a@gmail.com>
 2020-11-26 10:37:06 +05:30
jessicagreben
5dd3577424 change name of promtool subcommand to create-blocks-from 
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
 2020-11-22 15:05:02 -08:00
jessicagreben
75654715d3 fix panics 
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
 2020-11-01 07:54:04 -08:00
jessicagreben
6980bcf671 unexport backfiller 
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
 2020-10-31 06:40:56 -07:00
jessicagreben
3ed6457dd4 use blockwriter, rm multiwriter code 
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
 2020-10-31 06:32:07 -07:00
Julien Pivotto
4e5b1722b3
Move away from testutil, refactor imports (#8087) 
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
 2020-10-22 11:00:08 +02:00
jessicagreben
36ac0b68f1 merge master, fix conflicts 2020-10-17 08:20:21 -07:00
jessicagreben
2e526cf2a7 add output dir parameter 
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
 2020-09-13 08:38:32 -07:00
jessicagreben
dfa510086b add alignment, mv rule importer to promtool dir, add queryRange 
Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
 2020-09-13 08:07:59 -07:00
Julien Pivotto
442b3364d7
Promtool: add evaluation time to instant query (#7829) 
* Promtool: add evaluation time to instant query

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>

* Apply suggestion

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
 2020-08-25 11:32:25 +01:00
Andy Bursavich
4e6a94a27d
Invert service discovery dependencies (#7701) 
This also fixes a bug in query_log_file, which now is relative to the config file like all other paths.

Signed-off-by: Andy Bursavich <abursavich@gmail.com>
 2020-08-20 13:48:26 +01:00
jessicagreben
7504b5ce7c add rule importer with tsdb block writer 
Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>
 2020-07-27 07:44:49 -07:00
Ben Ye
50c261502e
add tsdb cmds into promtool (#6088) 
Signed-off-by: yeya24 <yb532204897@gmail.com>

update tsdb cli in makefile and promu

Signed-off-by: yeya24 <yb532204897@gmail.com>

remove building tsdb bin

Signed-off-by: yeya24 <yb532204897@gmail.com>

remove useless func

Signed-off-by: yeya24 <yb532204897@gmail.com>

refactor analyzeBlock

Signed-off-by: yeya24 <yb532204897@gmail.com>

Fix Makefile

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
 2020-07-23 19:35:50 +01:00
Ben Ye
e6ea798c32
promtool range query should exit when fail to parse time (#7505) 
Signed-off-by: yeya24 <yb532204897@gmail.com>
 2020-07-16 23:53:04 +01:00
yeya24
797e48c1a3 support time range in promtool query labels 
Updated prometheus/client_golang and json-iterator/go

Signed-off-by: yeya24 <yb532204897@gmail.com>
 2020-07-03 11:29:39 -04:00