Commit Graph

82 Commits

Author SHA1 Message Date
David Kaltschmidt
87c46ea6c3 Renamed TotalEvalTime to EvalTotalTime
* TotalFoo suggested a comprehensive timing, but TotalEvalTime was part
of the Exec timings, together with Queue timings
* The other option was to rename ExecTotalTime to TotalExecTime, but
 there was already ExecQueueTime, suggesting Exec to be some sort of
group
2017-11-17 17:46:51 +01:00
David Kaltschmidt
af75ce02c1 Review feedback
* renamed MakeQueryStats
* added stats to query() as well
* gofmt
2017-11-16 16:30:48 +01:00
David Kaltschmidt
c93e54d240 Adds execution timer stats to the range query
API consumers should be able to get insight into the query run times.
The UI currently measures total roundtrip times. This PR allows for more
fine grained metrics to be exposed.

* adds new timer for total execution time (queue + eval)

* expose new timer, queue timer, and eval timer in stats field of the
 range query response:
```json
{
  "status": "success",
  "data": {
    "resultType": "matrix",
    "result": [],
    "stats": {
      "execQueueTimeNs": 4683,
      "execTotalTimeNs": 2086587,
      "totalEvalTimeNs": 2077851
    }
  }
}
```

* stats field is optional, only set when query parameter `stats` is not
empty

Try it via
```sh
curl 'http://localhost:9090/api/v1/query_range?query=up&start=1486480279&end=1486483879&step=14000&stats=true'
```

Review feedback

* moved query stats json generation to query_stats.go
* use seconds for all query timers
* expose all timers available
* Changed ExecTotalTime string representation from Exec queue total time to Exec total time
2017-11-16 16:05:10 +01:00
Julius Volz
c3d6abc8e6 Fix some lint errors (#3334)
I left the promql ones and some others untouched as I remember that @fabxc
prefers them that way.
2017-10-23 14:57:30 +01:00
Alexander Kazarin
2c163f32a5 fix for issue 2976 (#3313)
fix for null pointer exception in ZookeeperLogger
2017-10-18 17:02:20 +01:00
Thibault Chataigner
bf4a279a91 Remote storage reads based on oldest timestamp in primary storage (#3129)
Currently all read queries are simply pushed to remote read clients.
This is fine, except for remote storage for wich it unefficient and
make query slower even if remote read is unnecessary.
So we need instead to compare the oldest timestamp in primary/local
storage with the query range lower boundary. If the oldest timestamp
is older than the mint parameter, then there is no need for remote read.
This is an optionnal behavior per remote read client.

Signed-off-by: Thibault Chataigner <t.chataigner@criteo.com>
2017-10-18 12:08:14 +01:00
Julien Levesy
8c1b9e45cf add a NotOk helper method in the testing package 2017-10-12 13:14:58 +02:00
Fabian Reinartz
269b5cf936 Merge pull request #3245 from jlevesy/use-require-web
Imports benbjohnson/testing and use it in web package
2017-10-10 10:58:27 +02:00
Marc Sluiter
6a633eece1 Added go-conntrack for monitoring http connections (#3241)
Added metrics for in- and outgoing traffic with go-conntrack.
2017-10-06 11:22:19 +01:00
Julien Levesy
332910d02d import assertion helpers in testutil 2017-10-05 22:22:40 +02:00
Fabian Reinartz
2d0b8e8b94 Merge branch 'master' into dev-2.0 2017-10-05 13:09:18 +02:00
Alberto Cortés
2c9739162e utils/stats: remove dead code 2017-09-26 16:32:45 +02:00
Bryan Boreham
e0a4d18301 Allow http keep-alive setting to be overridden in config 2017-09-11 09:07:14 +00:00
Fabian Reinartz
d21f149745 *: migrate to go-kit/log 2017-09-08 22:01:51 +05:30
Fabian Reinartz
87918f3097 Merge branch 'master' into dev-2.0 2017-09-04 14:09:21 +02:00
Lynn Lin
1bf25dc1b2 fix issues reported by gofmt and spelling typo (#3127) 2017-08-29 09:00:11 +01:00
Fabian Reinartz
921c6ebb3b Merge branch 'dev-2.0' into mergemaster 2017-08-10 17:27:52 +02:00
Fabian Reinartz
25f3e1c424 Merge branch 'master' into mergemaster 2017-08-10 17:04:25 +02:00
Fabian Reinartz
9516d04472 util: Add idle timeout for scrape connections 2017-08-10 14:47:51 +02:00
Pablo Andres Fuente
c79a4db812 Adding tests for util/httputil/client (#3002)
Adding tests for util/httputil/client with a 100% coverage.
Removing the NewDeadlineRoundTripper from util/httputil/client because
is not used.
Adding a new test util to check http.Request in http.RoundTrip interface
implementors.
2017-08-09 13:23:57 +01:00
Fabian Reinartz
dba7586671 Merge branch 'master' into dev-2.0 2017-07-11 17:22:14 +02:00
Fabian Reinartz
ccf9e62972 *: add admin grpc API 2017-07-10 09:14:14 +02:00
Fuente, Pablo Andres
902fafb8e7 Fixing tests for Windows
Fixing the config/config_test, the discovery/file/file_test and the
promql/promql_test tests for Windows. For most of the tests, the fix involved
correct handling of path separators. In the case of the promql tests, the
issue was related to the removal of the temporal directories used by the
storage. The issue is that the RemoveAll() call returns an error when it
tries to remove a directory which is not empty, which seems to be true due to
some kind of process that is still running after closing the storage. To fix
it I added some retries to the remove of the temporal directories.
Adding tags file from Universal Ctags to .gitignore
2017-07-09 01:59:30 -03:00
Goutham Veeramachaneni
e3701077c3
Move promtool to kingpin
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-06-21 17:42:57 +05:30
Goutham Veeramachaneni
d407bd150c Consolidate the duration params in CLI
* All CLI params moved to model.Duration

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-06-16 20:20:57 +05:30
Goutham Veeramachaneni
507790a357
Rework logging to use explicitly passed logger
Mostly cleaned up the global logger use. Still some uses in discovery
package.

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
2017-06-16 15:52:44 +05:30
Fabian Reinartz
669075c6b9 Merge branch 'master' into dev-2.0 2017-06-06 09:36:51 +02:00
Conor Broderick
6766123f93 Replace regex with Secret type and remarshal config to hide secrets (#2775) 2017-05-29 12:46:23 +01:00
Fabian Reinartz
d289dc55c3 storage: update TSDB 2017-05-22 11:53:08 +02:00
Fabian Reinartz
6e804b3497 Merge branch 'master' into dev-2.0 2017-05-12 13:29:58 +02:00
Stephan Erb
0b9fca983b Fix reload of ZooKeeper service discovery config (#2669)
Rational:

* When the config is reloaded and the provider context is canceled, we need to
  exit the current ZK `TargetProvider.Run` method as a new provider will be
  instantiated.
* In case `Stop` is called on the `ZookeeperTreeCache`, the update/events
  channel may not be closed as it is shared by multiple caches and would
  thus be double closed.
* Stopping all `zookeeperTreeCacheNode`s on teardown ensures all associated
  watcher go-routines will be closed eagerly rather than implicityly on
  connection close events.
2017-05-02 18:21:37 -05:00
Fabian Reinartz
0f3110487d Merge remote-tracking branch 'origin/dev-2.0' into dev-2.0 2017-04-27 10:25:04 +02:00
Fabian Reinartz
73b8ff0ddc Merge branch 'master' into dev-2.0 2017-04-27 10:19:55 +02:00
Brian Brazil
5c9a6ce747 Add license to files.
This should fix CI for dev-2.0.
2017-04-19 13:46:22 +01:00
Matt Layher
1faf33acac Add promlint check for histogram/summary reserved names (#2626) 2017-04-15 22:38:01 +01:00
Matt Layher
cc4198f421
Simplify promlint problems gathering, use protobuf accessors 2017-04-14 16:40:40 -04:00
Matt Layher
34a4813464 Initial promlint counter _total suffix check (#2624) 2017-04-14 22:09:54 +02:00
Matt Layher
254cb1ec29 Use untyped metrics for some promlint tests (#2623) 2017-04-14 19:38:57 +01:00
Matt Layher
283756c503 Initial commit of 'promtool check-metrics', promlint package (#2605) 2017-04-13 23:53:41 +02:00
Fabian Reinartz
8ffc851147 Merge branch 'master' into dev-2.0 2017-04-04 15:17:56 +02:00
Julius Volz
815762a4ad Move retrieval.NewHTTPClient -> httputil.NewClientFromConfig 2017-03-20 14:17:04 +01:00
Fabian Reinartz
c389193b37 Merge branch 'master' into dev-2.0 2017-03-17 16:27:07 +01:00
Stephan Erb
3038d0eb9b Prevent deadlock in ZK TreeCache constructor by deferring the initial sync.
Fixes #2254
2017-03-03 23:58:46 +01:00
Fabian Reinartz
4397b4d508 *: pass Prometheus registry into storage 2017-02-28 09:33:14 +01:00
Fabian Reinartz
311e7b5069 storage/vendor: update to latest fabxc/tsdb 2017-02-20 11:11:44 +01:00
Fabian Reinartz
5772f1a7ba retrieval/storage: adapt to new interface
This simplifies the interface to two add methods for
appends with labels or faster reference numbers.
2017-02-02 13:05:46 +01:00
Fabian Reinartz
71fe0c58a8 promql: misc fixes 2016-12-28 11:32:15 +01:00
Fabian Reinartz
fecf9532b9 *: fix misc compile errors 2016-12-25 11:42:57 +01:00
Fabian Reinartz
622ece6273 *: fix recording tests, migrate matcher types 2016-12-25 11:12:57 +01:00
Tom Wilkie
4520e12440 Add HTTP Basic Auth & TLS support to the generic write path. (#1957)
* Add config, HTTP Basic Auth and TLS support to the generic write path.

- Move generic write path configuration to the config file
- Factor out config.TLSConfig -> tlf.Config translation
- Support TLSConfig for generic remote storage
- Rename Run to Start, and make it non-blocking.
- Dedupe code in httputil for TLS config.
- Make remote queue metrics global.
2016-09-19 22:47:51 +02:00