Commit Graph

20 Commits

Author SHA1 Message Date
Fabian Reinartz 7ccd4b39b8 *: implement query params
This adds a parameter to the storage selection interface which allows
query engine(s) to pass information about the operations surrounding a
data selection.
This can for example be used by remote storage backends to infer the
correct downsampling aggregates that need to be provided.
2018-02-13 12:17:22 +01:00
Tom Wilkie 749781edf3 Also, don't make a mergeSeriesSet if there is only one SeriesSet. 2018-01-25 11:17:16 +00:00
Tom Wilkie 48e39068bd Don't allocate a mergeSeries if there is only one series to merge. 2018-01-25 11:11:55 +00:00
Ed Schouten bb724f1bef Deprecate DeduplicateSeriesSet() in favor of NewMergeSeriesSet().
Federation makes use of dedupedSeriesSet to merge SeriesSets for every
query into one output stream. If many match[] arguments are provided,
many dedupedSeriesSet objects will get chained. This has the downside of
causing a potential O(n*k) running time, where n is the number of series
and k the number of match[] arguments.

In the mean time, the storage package provides a mergeSeriesSet that
accomplishes the same with an O(n*log(k)) running time by making use of
a binary heap. Let's just get rid of dedupedSeriesSet and change all
existing callers to use mergeSeriesSet.
2017-12-10 19:51:20 +01:00
Fabian Reinartz 83cd270ea4 *: adapt to storage interface changes 2017-11-23 19:05:04 +01:00
Tom Wilkie 48a7a00a38 Fast path the merge querier (#3358)
* Fast path the merge querier such that it is completely removed from query path when there is no remote storage.

* Add NoopQuerier

* Add copyright notice.

* Avoid global, use a function.
2017-10-27 13:29:05 +02:00
Tom Wilkie 0e572686db Revert "Bypass the fanout storage merging if no remote storage is configured." 2017-10-26 16:09:39 +01:00
Tom Wilkie 4bbef0ec30 Bypass the fanout storage merging if no remote storage is configured. 2017-10-23 21:34:53 +01:00
Thibault Chataigner bf4a279a91 Remote storage reads based on oldest timestamp in primary storage (#3129)
Currently all read queries are simply pushed to remote read clients.
This is fine, except for remote storage for wich it unefficient and
make query slower even if remote read is unnecessary.
So we need instead to compare the oldest timestamp in primary/local
storage with the query range lower boundary. If the oldest timestamp
is older than the mint parameter, then there is no need for remote read.
This is an optionnal behavior per remote read client.

Signed-off-by: Thibault Chataigner <t.chataigner@criteo.com>
2017-10-18 12:08:14 +01:00
Julius Volz 9ef8518b37 Remove "package remote" garbage from license headers (#3304) 2017-10-17 02:26:38 +01:00
Julius Volz f7e8348a88 Re-add contexts to storage.Storage.Querier() (#3230)
* Re-add contexts to storage.Storage.Querier()

These are needed when replacing the storage by a multi-tenant
implementation where the tenant is stored in the context.

The 1.x query interfaces already had contexts, but they got lost in 2.x.

* Convert promql.Engine to use native contexts
2017-10-04 21:04:15 +02:00
Fabian Reinartz d21f149745 *: migrate to go-kit/log 2017-09-08 22:01:51 +05:30
Fabian Reinartz 0efecea6d4 Adapt storage APIs to uint64 references 2017-09-07 14:14:41 +02:00
Tom Wilkie 994a7f27d6 Propagate errors through mergeSeriesSet correctly. 2017-07-13 15:02:01 +01:00
Tom Wilkie 2e0d8487e3 Return zeros if At() is called after Next() returns false. 2017-07-13 14:40:29 +01:00
Tom Wilkie 014bd31a86 Remove unnecessary whitespace changes, add comment. 2017-07-13 11:26:46 +01:00
Tom Wilkie 98ac07f86a Add unit test for the merging on the read path. 2017-07-13 11:05:38 +01:00
Tom Wilkie 96e25adc8d Introduce 'primary' storage in fanout, and have Add return the ref from the primary.
Also, ensure all append batches are rolled back when a commit or rollback fails.
2017-07-12 15:51:05 +01:00
Tom Wilkie db8128ceeb Add label set as first parameter to AddFast, ingored by TSDB adapter. 2017-07-12 15:20:12 +01:00
Tom Wilkie 2dda5775e3 Initial port of remote storage to v2. 2017-07-12 12:27:57 +01:00