Commit Graph

46 Commits

Author SHA1 Message Date
Ganesh Vernekar f34aeefe6e
Allow overlapping blocks by default (#11331)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2022-09-28 19:17:54 +05:30
Rodrigo Queiro 70c1446a64
Clarify units of --storage.tsdb.retention.size (#10154)
The flag uses Base2Bytes: 129ed4ec8b/cmd/prometheus/main.go (L1476)

Signed-off-by: Rodrigo Queiro <rodrigoq@google.com>
2022-01-13 00:55:57 +01:00
beorn7 8fdfa52976 API: Promote remote-write-receiver to stable
Since `/api/v1/write` is a mutating endpoint, we should still activate
the remote-write-receiver explicitly. But we should do it in the same
way as the other mutating endpoints, i.e. via a flag
`--web.enable-remote-write-receiver`.

This commit marks the feature flag as deprecated, i.e. it still works
but logs a warning on startup. This enables users to seamlessly
migrate. With the next minor release, we can start ignoring the
feature flag (but still warn a user that is trying to use it).

Signed-off-by: beorn7 <beorn@grafana.com>
2022-01-05 15:36:07 +01:00
Dean Meehan 5751bfb61f Removed Duplication Typo (to to)
Signed-off-by: Dean Meehan <dean@dean.technology>
2021-09-13 19:34:07 +01:00
Ganesh Vernekar 3d4c5f890d
Clarify in docs about the min disk space requirement when using size retention (#9245)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-08-24 17:26:10 +05:30
Ankit Goel d437cee73a
Move storage.tsdb.retention.size out of experimental #8728 (#9004)
* Move storage.tsdb.retention.size out of experimental #8728

Signed-off-by: Ankit Goel <ankit.goel@deliveryhero.com>
2021-06-30 01:30:11 +02:00
Steve Kuznetsov fd6c852567
promtool: backfill: allow configuring block duration (#8919)
* promtool: backfill: allow configuring block duration

When backfilling large amounts of data across long periods of time, it
may in certain circumstances be useful to use a longer block duration to
increase the efficiency and speed of the backfilling process. This patch
adds a flag --block-duration-power to allow a user to choose the power N
where the block duration is 2^(N+1)h.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

* promtool: use sub-tests in backfill testing

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

* backfill: add messages to tests for clarity

When someone new breaks a test, seeing "expected: false, got: true" is
really not useful. A nice message helps here.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

* backfill: test long block durations

A test that uses a long block duration to write bigger blocks is added.
The check to make sure all blocks are the default duration is removed.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>
2021-06-29 14:53:38 +05:30
Julien Pivotto 2c9875d0e7 Storage: Create a code block for example command.
The current doc is broken:
https://prometheus.io/docs/prometheus/2.27/storage/

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2021-06-17 12:21:45 +02:00
Ikko Ashimine 446e5cc160 Fix typo in storage.md
mulitple -> multiple

Signed-off-by: Ikko Ashimine <eltociear@gmail.com>
2021-05-20 20:32:03 +09:00
nberkley f9e2dd0697
Add support for smaller block chunk segment allocations (#8478)
* Add support for --storage.tsdb.max-chunk-size to suport small chunks for space limited prometheus instances.

Signed-off-by: Nathan Berkley <nberkley@tripadvisor.com>

* Update tsdb/compact.go

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Nathan Berkley <nberkley@tripadvisor.com>

* Update tsdb/db.go

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Nathan Berkley <nberkley@tripadvisor.com>

* Update cmd/prometheus/main.go

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Nathan Berkley <nberkley@tripadvisor.com>

* Change naming scheme to

Signed-off-by: Nathan Berkley <nberkley@tripadvisor.com>

* Add a lower bound to --storage.tsdb.max-block-chunk-segment-size

Signed-off-by: Nathan Berkley <nberkley@tripadvisor.com>

* Update storage.md to explain what a chunk segment is

Signed-off-by: Nathan Berkley <nberkley@tripadvisor.com>

* Apply suggestions from code review

Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>
Signed-off-by: Nathan Berkley <nberkley@tripadvisor.com>

* Force tests

Signed-off-by: Nathan Berkley <nberkley@tripadvisor.com>

* Fix code style

Signed-off-by: Nathan Berkley <nberkley@tripadvisor.com>

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>
2021-04-15 14:25:01 +05:30
Jess G 731545ad34
Add documentation for recording rule backfiller (#8674)
* add docs for rule backfiller

Signed-off-by: jessicagreben <jessicagrebens@gmail.com>
2021-04-01 22:38:00 +02:00
Julien Pivotto 432d5ebc6c Rename default branch to main
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2021-02-22 20:28:02 +01:00
Tom Wilkie d479151f1f Various enhancements and refactorings for remote write receiver:
- Remove unrelated changes
- Refactor code out of the API module - that is already getting pretty crowded.
- Don't track reference for AddFast in remote write.  This has the potential to consume unlimited server-side memory if a malicious client pushes a different label set for every series.  For now, its easier and safer to always use the 'slow' path.
- Return 400 on out of order samples.
- Use remote.DecodeWriteRequest in the remote write adapters.
- Put this behing the 'remote-write-server' feature flag
- Add some (very) basic docs.
- Used named return & add test for commit error propagation

Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2021-02-08 20:41:23 +00:00
Atibhi Agrawal c08009139a
Add docs for backfill (#8230)
* Add docs for backfill

Signed-off-by: aSquare14 <atibhi.a@gmail.com>
2020-12-10 16:29:44 +01:00
Mike DeSimone 759150878c
fix trivial typos (#8238)
Signed-off-by: Mike DeSimone <michael.2.desimone@gmail.com>
2020-11-29 01:23:10 +00:00
Anthony D'Atri 2cbc0f9bfe
Various doc tweaks (#8111)
Signed-off-by: Anthony D'Atri <anthony.datri@gmail.com>
2020-10-27 09:50:37 +00:00
venkatbvc 6f28c46f4c
Updating storage.md for retention.size (#7942)
Signed-off-by: venkatbvc <chalapathirao@gmail.com>
2020-09-16 10:49:02 +05:30
showuon 791b84830d
Update the storage.md to fix typo, and some grammar/format issue (#7871)
* Update the storage.md to fix typo, and some grammar/format issue

Signed-off-by: Luke Chen <showuon@gmail.com>
2020-08-31 09:50:42 +01:00
Julien Pivotto 32c2d897ec
Storage: add chunks_head and fix links (#7693)
The links fix makes the links point to the current version of the files,
e.g. the relase branch.

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-07-31 19:17:16 +02:00
Julien Pivotto 01e3bfcd1a
Add warnings about NFS (#7691)
* Add warnings about NFS

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-07-30 11:22:44 +02:00
Ben Kochie 8d3c2f6829
Enable WAL compression by default (#7410)
Enable the `--storage.tsdb.wal-compression` flag by defualt.

Signed-off-by: Ben Kochie <superq@gmail.com>
2020-06-18 17:59:40 +01:00
Arthur Silva Sens 7727b9012e
Correction of misleading help text(#5142) (#7231)
* Correction of misleading help text(#5142)

Signed-off-by: arthursens <arthursens2005@gmail.com>
2020-05-11 12:15:01 +01:00
Nguyen Hai Truong 26574eb04d
docs: Fix typo in docs/storage.md (#6799)
`availiablity` -> `availability`

Signed-off-by: Nguyen Hai Truong <truongnh@vn.fujitsu.com>
2020-02-10 18:32:58 +00:00
Stuart Spence 5ade524a1b grammar typo (#6617)
Signed-off-by: Stuart Spence <stuart.spence@canada.ca>
2020-01-13 22:14:52 +00:00
Kien Nguyen 34e2f15e18 Correct the default max block duratioh value
The default value is 31 days [1]

[1] https://github.com/prometheus/prometheus/blob/master/cmd/prometheus/main.go#L312

Signed-off-by: Kien Nguyen <kiennt2609@gmail.com>
2019-11-25 11:12:26 +07:00
yuxiaobo 7850f1b35c new world spelling mistake
Signed-off-by: yuxiaobo <yuxiaobogo@163.com>
2019-10-17 19:09:54 +08:00
Ben Kochie 16370e6880
Update storage considerations documentation (#6134)
* Improve clarity of internal storage durability.
* Add notes about remote storage.

Signed-off-by: Ben Kochie <superq@gmail.com>
2019-10-16 21:45:01 +02:00
abejidefemi1@gmail.com 5264b9628b fix punctuation mark
fix typo
Signed-off-by: abejidefemi1@gmail.com <abejidefemi1@gmail.com>
2019-10-01 09:02:11 +01:00
yuxiaobo96 5f6a56a0bb Third correction (#6012)
Signed-off-by: yuxiaobo <yuxiaobogo@163.com>
2019-09-20 10:48:56 +01:00
Ganesh Vernekar 5ecef3542d
Cleanup after merging tsdb into prometheus
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2019-08-13 14:04:14 +05:30
Ben Kochie ff40de7ca6 Add more details about retention to storage docs (#5842)
* Make compaction docs a little more clear, easy to find.
* Expand compaction docs slightly.
* Add notes about block cleanup to operational section.

Signed-off-by: Ben Kochie <superq@gmail.com>
2019-08-07 17:04:48 +01:00
Chris Marchbanks 08f6a43619
Add documentation for the WAL compression flag
Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>
2019-07-09 10:33:59 -06:00
Hrishikesh Barman 27404e6226 better readability for data directory tree in storage docs (#5128)
* better readabilty for data dir tree in storage docs

Signed-off-by: geekodour <hrishikeshbman@gmail.com>
2019-04-16 11:40:13 +01:00
Frederic Branczyk 32079f351f
docs: Specifically call out NFS and POSIX
Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com>
2019-01-31 12:57:48 +01:00
Frederic Branczyk 3de734d8de
docs: Add filesystem POSIX requirement
Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com>
2019-01-29 13:51:16 +01:00
Goutham Veeramachaneni 384cba1211
Add flag for size based retention (#5109)
* Add flag for size based retention

Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>

* Deprecate the old retention flag for a new one.

Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>

* Add ability to take a suffix for size flag

Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>

* Address feedback

Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>
2019-01-18 19:18:36 +05:30
Daniel Lo Nigro edaa4835a2 Update WAL directory structure, and mention that the last three are always kept (#5096)
* Small tweaks to storage doc page

Signed-off-by: Daniel Lo Nigro <d@d.sb>
2019-01-15 10:32:29 +00:00
nilsocket fe0f0da6b3 docs: add missing word (time) (#4797)
Signed-off-by: nilsocket <nilsocket@gmail.com>
2018-10-28 07:36:09 +00:00
Catalin Patulea 50850c0ad9 Add link to TSDB format page. (#4402)
Signed-off-by: Catalin Patulea <catalinp@google.com>
2018-07-28 08:02:03 +01:00
Ben Kochie 390e260bd9 Improve wording of remote write documentation. (#3817)
Reduce the use of the term `long-term`, when what we're really talking
about is remote clustered storage for increased capacity and durability.

Signed-off-by: Ben Kochie <superq@gmail.com>
2018-05-05 16:38:45 +01:00
Brian Brazil 9083d41d3a
Add 2.0 stability guarantees (#3484)
As discussed generally consider SDs as unstable, as realistically they
are never going to be. Drop the words "experimental/beta" from most
places in the docs, as users are getting the wrong impression from this.
2017-12-14 12:54:32 +00:00
Or Elimelech 6e8d192ba0 Wrong URL for remote.proto (#3431)
Change wrong URL for remote.proto
2017-12-14 12:20:28 +00:00
Julius Volz ef08df0e6f
Add 2.0 storage docs (#3423)
* Add 2.0 storage docs

* Review fixups

* More review fixups
2017-11-07 22:00:38 +01:00
Fabian Reinartz a32e4cbdd8 docs: remove 1.x storage docs
The only section that still aplies was the one on the default storage
directory so those docs seem obsolete.
We'll probably have a similar page on the new storage but we'll only
find out what caveats etc. we'll have to point out as we get people
reporting problems or notable behavior.
2017-10-28 12:11:35 +02:00
Tobias Schmidt f432b8176d Consolidate configuration and rules docs in docs/configuration/ 2017-10-27 09:54:02 +02:00
Tobias Schmidt 4d30a11ab6 Import storage and federation documentation from docs 2017-10-26 22:36:47 +02:00