Commit Graph

851 Commits

Author SHA1 Message Date
Fabian Reinartz
30e777d10d tsdb: default too small max block duration 2017-10-30 12:09:56 +01:00
Tom Wilkie
48a7a00a38 Fast path the merge querier (#3358)
* Fast path the merge querier such that it is completely removed from query path when there is no remote storage.

* Add NoopQuerier

* Add copyright notice.

* Avoid global, use a function.
2017-10-27 13:29:05 +02:00
Tom Wilkie
0e572686db Revert "Bypass the fanout storage merging if no remote storage is configured." 2017-10-26 16:09:39 +01:00
Tom Wilkie
1af3ef431d s/TestRemoveLabels/TestSeriesSetFilter/ 2017-10-26 13:50:39 +01:00
Tom Wilkie
9c3c98e8de Revert "Port 'Don't disable HTTP keep-alives for remote storage connections.' to 2.0 (see #3173)"
This reverts commit 0997191b18.
2017-10-26 13:43:48 +01:00
Tom Wilkie
746752b946 Merge external labels in order. 2017-10-26 11:44:49 +01:00
Tom Wilkie
6e4d4ea402 Initialise some counters in remote storage API. 2017-10-26 11:09:45 +01:00
Tom Wilkie
2ae04d0e79 Add license header. 2017-10-26 11:09:16 +01:00
Tom Wilkie
e8c264e47a Add comment. 2017-10-26 11:09:16 +01:00
Tom Wilkie
ee011d906d Port remote read server to 2.0. 2017-10-26 11:09:14 +01:00
Bryan Boreham
0997191b18 Port 'Don't disable HTTP keep-alives for remote storage connections.' to 2.0 (see #3173)
Removes configurability introduced in #3160 in favour of hard-coding,
per advice from @brian-brazil.
2017-10-26 11:08:33 +01:00
Tom Wilkie
56820726fa Move a couple of the encoding/decoding functions into codec.go 2017-10-26 11:08:33 +01:00
Conor Broderick
08b7328669 Port Metric name validation to 2.0 (see #2975) 2017-10-26 11:08:33 +01:00
Tom Wilkie
8fe0212ff7 Port 'Make queue manager configurable.' to 2.0, see #2991 2017-10-26 11:08:33 +01:00
Tom Wilkie
3760f56c0c remote: Expose ClientConfig type (see #3165) 2017-10-26 11:08:33 +01:00
Tom Wilkie
16f71a7723 Port codec.go over form 1.8 branch. 2017-10-26 11:08:33 +01:00
Fabian Reinartz
e53040e2ac Merge pull request #3339 from tomwilkie/3065-remote-read-bypass
Bypass the fanout storage merging if no remote storage is configured.
2017-10-26 09:14:26 +02:00
Fabian Reinartz
bf56ad4233 Merge branch 'master' into master 2017-10-26 09:06:12 +02:00
Paul Gier
c4c3205d76 storage/tsdb: check that max block duration is larger than min
If the user accidentally sets the max block duration smaller than the min,
the current error is not informative.  This change just performs the check
earlier and improves the error message.
2017-10-25 19:24:49 -05:00
Fabian Reinartz
ce63a5a855 Merge pull request #3352 from prometheus/rc2
Cut v2.0.0-rc.2
2017-10-25 20:39:39 +02:00
Thibault Chataigner
fc4406201e Tsdb StartTime : Use a simplier way to compute StartTime 2017-10-25 17:41:00 +02:00
Julius Volz
099df0c5f0 Migrate "golang.org/x/net/context" -> "context" (#3333)
In some places, where ctxhttp or gRPC are concerned, we still need to use the
old contexts.
2017-10-24 21:21:42 -07:00
Tom Wilkie
4bbef0ec30 Bypass the fanout storage merging if no remote storage is configured. 2017-10-23 21:34:53 +01:00
Fabian Reinartz
a57ea79660 Close index reader properly 2017-10-23 21:59:18 +02:00
Julius Volz
c3d6abc8e6 Fix some lint errors (#3334)
I left the promql ones and some others untouched as I remember that @fabxc
prefers them that way.
2017-10-23 14:57:30 +01:00
Julius Volz
2846d62573 Fix staticcheck issue in test (#3331)
staticcheck fails with:
storage/remote/read_test.go:199:27: do not pass a nil Context, even if a function permits it; pass context.TODO if you are unsure about which Context to use (SA1012)
2017-10-23 11:51:48 +01:00
Brian Brazil
4a50f547c8 removeLabels needs a pointer to work. (#3326) 2017-10-21 08:29:03 +01:00
Thibault Chataigner
bf4a279a91 Remote storage reads based on oldest timestamp in primary storage (#3129)
Currently all read queries are simply pushed to remote read clients.
This is fine, except for remote storage for wich it unefficient and
make query slower even if remote read is unnecessary.
So we need instead to compare the oldest timestamp in primary/local
storage with the query range lower boundary. If the oldest timestamp
is older than the mint parameter, then there is no need for remote read.
This is an optionnal behavior per remote read client.

Signed-off-by: Thibault Chataigner <t.chataigner@criteo.com>
2017-10-18 12:08:14 +01:00
Julius Volz
9ef8518b37 Remove "package remote" garbage from license headers (#3304) 2017-10-17 02:26:38 +01:00
Tobias Schmidt
721050c6cb Update prometheus/tsdb dependency 2017-10-16 15:36:25 +02:00
Julius Volz
33c1171b9c Don't add anchoring to exported Value matcher field
Instead, just make the anchoring part of the internal regex. This helps because
some users will want to read back the `Value` field and expect it to be the
same as the input value (e.g. some tests in Cortex), or use the value in
another context which is already expected to add its own anchoring, leading to
superfluous double anchoring (such as when we translate matchers into remote
read request matchers).
2017-10-10 10:10:21 -07:00
Brian Brazil
73dc96e7f5 Fix leak of ticker in remote storage queue manager. 2017-10-09 19:44:03 +01:00
Brian Brazil
ee88f0d222 Ensure all values are used or _ 2017-10-09 19:44:03 +01:00
Brian Brazil
37ec2d5283 Fix off by one error in concreteSeriesSet (#3262) 2017-10-09 13:37:58 +01:00
Marc Sluiter
6a633eece1 Added go-conntrack for monitoring http connections (#3241)
Added metrics for in- and outgoing traffic with go-conntrack.
2017-10-06 11:22:19 +01:00
Julius Volz
f7e8348a88 Re-add contexts to storage.Storage.Querier() (#3230)
* Re-add contexts to storage.Storage.Querier()

These are needed when replacing the storage by a multi-tenant
implementation where the tenant is stored in the context.

The 1.x query interfaces already had contexts, but they got lost in 2.x.

* Convert promql.Engine to use native contexts
2017-10-04 21:04:15 +02:00
Fabian Reinartz
7b02bfee0a web: start web handler while TSDB is starting up 2017-09-20 15:03:19 +02:00
Fabian Reinartz
d21f149745 *: migrate to go-kit/log 2017-09-08 22:01:51 +05:30
Fabian Reinartz
0efecea6d4 Adapt storage APIs to uint64 references 2017-09-07 14:14:41 +02:00
Fabian Reinartz
0c81d5f719 storage: instantiate correct block ranges 2017-08-24 12:36:07 +02:00
Fabian Reinartz
2037778d14 vendor: update TSDB 2017-08-10 14:51:02 +02:00
Tom Wilkie
b11bc8ae24 Fix some comments. 2017-08-01 11:19:35 +01:00
Tom Wilkie
ec999ff397 Prevent number of remote write shards from going negative.
This can happen in the situation where the system scales up the number of shards massively (to deal with some backlog), then scales it down again as the number of samples sent during the time period is less than the number received.
2017-07-19 16:32:09 +01:00
Tom Wilkie
a09acdcc5b Make concreteSeriersIterator behave. 2017-07-13 18:33:08 +01:00
Tom Wilkie
994a7f27d6 Propagate errors through mergeSeriesSet correctly. 2017-07-13 15:02:01 +01:00
Tom Wilkie
2e0d8487e3 Return zeros if At() is called after Next() returns false. 2017-07-13 14:40:29 +01:00
Tom Wilkie
014bd31a86 Remove unnecessary whitespace changes, add comment. 2017-07-13 11:26:46 +01:00
Tom Wilkie
98ac07f86a Add unit test for the merging on the read path. 2017-07-13 11:05:38 +01:00
Tom Wilkie
b568ace7ce Move protos to ./prompb 2017-07-12 22:06:35 +01:00
Tom Wilkie
96e25adc8d Introduce 'primary' storage in fanout, and have Add return the ref from the primary.
Also, ensure all append batches are rolled back when a commit or rollback fails.
2017-07-12 15:51:05 +01:00