Commit Graph

1373 Commits

Author SHA1 Message Date
Kefu Chai
bb351842bc doc/rados/configuration/general-config-ref: use describe to define non-options
max_open_files is not generated from yaml files, so define it using
"describe" directive instead.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-05-06 22:21:43 +08:00
Sridhar Seshasayee
0ad2b2c8ee
Merge pull request #41004 from sseshasa/wip-mclock-max-backfill-misc-fix
osd: Override recovery, backfill and sleep related config options during OSD and mclock scheduler initialization.

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sunny Kumar <sunkumar@redhat.com>
2021-04-30 22:58:21 +05:30
Zac Dover
87a1ae7a44 doc/rados: rename "Running Multiple Clusters"
This PR corrects a title so that the title
more accurately describes the content in the
section it names.

Fixes: https://tracker.ceph.com/issues/50517 (partially)
Signed-off-by: Zac Dover <zac.dover@gmail.com>
2021-04-28 22:45:30 +10:00
Sridhar Seshasayee
8876d27f38 osd: Override recovery/backfill/sleep options with mclock scheduler.
Make the osd_*_sleep options modifiable during runtime and add them
to the set of tracked conf keys. This is to ensure that the sleep
options can be disabled/overridden during OSD bring-up if mclock
scheduler is employed.

Introduce OSD::maybe_override_options_for_qos():
  This method does the following if the mclock scheduler is enabled:
  - overrides the "recovery_max_active" to a high limit of 1000,
  - overrides "osd_max_backfills" option to a high limit of 1000 and
    sets the corresponding Async local and remote reserver objects also
    to the same value (1000),
  - disables osd_*_sleep options so that appropriate QoS may be
    provided with the mclock scheduler.

The above method is called in the following scenarios:
- After all the op shards are brought up during OSD initialization.
- In OSD::handle_conf_change() to override any settings related to
  QoS that the user intended to change.

Modify the mclock config reference to accurately reflect what options
can be changed when using mclock's "custom" profile and clean up
some whitespaces.

Fixes: https://tracker.ceph.com/issues/50501
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
2021-04-27 11:11:58 +05:30
Kefu Chai
ecb8e5e9c7 doc/rados/operations/pools: define pool settings using "describe"
better readability and more structured this way

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-26 21:01:44 +08:00
Kefu Chai
4e27388ba7 doc/rados/configuration: reference options using :confval:
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-26 21:01:44 +08:00
Kefu Chai
c34cc03684 doc/rados/configuration/mon-config-ref: link by ref instead of relative hyperlink
the latter is more fragile.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-22 12:08:54 +08:00
Kefu Chai
ae5f2b2ca3 doc/rados/configuration/mon-config-ref: use confval directive
for defining options

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-22 12:08:54 +08:00
Kefu Chai
be07a5b407 doc/rados/configuration: add missing options
and link to them when appropriate

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-22 12:08:54 +08:00
Kefu Chai
52a5655296 doc/rados/configuration/mon-config-ref: use confval directive
for defining options

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-21 12:37:03 +08:00
Kefu Chai
4481f4ac75 doc/rados/configuration/general-config-ref: use confval directive
for defining options

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-21 12:33:19 +08:00
Kefu Chai
4a4dc9f5a1 doc/rados/configuration/osd-config-ref: use confval directive
for defining options

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-21 12:33:19 +08:00
Kefu Chai
d72fb1f3e6 doc/rados/configuration/osd-config-ref: use confval directive
for defining options

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-20 22:49:20 +08:00
Kefu Chai
8e0158a047 doc/rados/configuration/mon-config-ref: use confval directive
for defining options

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-19 22:03:45 +08:00
Kefu Chai
e5722a46e1 doc/rados/configuration/auth-config-ref: drop section of keyrings
as per Sage, the caps are out of date (should be based on profiles) and
the keyring locations aren't useful information

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-19 21:45:07 +08:00
Kefu Chai
cdd059b3e4 doc/rados/configuration/mon-config-ref: use confval directive
for defining options

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-19 17:35:31 +08:00
Kefu Chai
536f096055 doc/rados/configuration/msgr2: use confval directive
for defining options

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-19 17:08:46 +08:00
Kefu Chai
3a62ba9463 doc/rados/configuration/auth-config-ref: use "describe" to define default keyrings
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-19 17:08:46 +08:00
Kefu Chai
3767e0cb16 doc/rados/configuration: merge ms-ref into network-conf-ref
the latter provides more context, and "messenger" is how the networking
infra is implemented.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-19 17:08:46 +08:00
Kefu Chai
1ecd8e7f97 doc/rados/configuration/ceph-conf: use "describe" to define option types
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-19 17:08:46 +08:00
Kefu Chai
a67451d05d doc/rados/configuration: fix formatting of code block
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-19 17:08:46 +08:00
Kefu Chai
76fb3d147d doc: use "confsec" role for configure section
and rename confval_section to confsec for shorter name

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-19 17:08:45 +08:00
Kefu Chai
66a86596a3 doc/rados/configuration/ceph-conf: use confval directive
for defining options

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-19 17:08:45 +08:00
Kefu Chai
02a15a81d8 doc: use ":confval:" role to reference options defined using confval
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-19 17:08:45 +08:00
Kefu Chai
a37f5a5eea doc/rados/configuration/osd-config-ref: use confval directive
for defining options

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-19 17:08:45 +08:00
Kefu Chai
23c7081dab doc/rados/configuration/mclock-config-ref: use confval directive
for defining options

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-19 17:08:43 +08:00
Kefu Chai
20676f00c9 doc/rados/configuration/pool-pg-config-ref: use confval directive
for defining options.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-19 17:08:11 +08:00
Kefu Chai
65925d78c6 doc/rados/configuration/bluestore-config-ref: use confval directive
for defining options.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-19 17:08:07 +08:00
Kefu Chai
f86cc95ecf doc/rados/configuration/mon-osd-interaction: use confval directive to define options
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-17 00:02:45 +08:00
Kefu Chai
4da18aaf57 common/options,doc: extract formatted desc into .yaml.in
* add a field named "fmt_desc", which is the description formatted using
  reStructuredText. it is preserved as it is if it's different from the
  desc or long_desc of an option. we can consolidate it with long_desc
  in future, and use pretty printer which has minimal support for
  reStructuredText for printing the formatted descriptions for a better
  user experience of command line. but at this moment, fmt_desc has
  only one consumer: the "ceph_confval" sphinx extension which extracts
  and translate the options yaml file to reStructuredText, which is in
  turn rendered by sphinx.
* remove unused options from the doc
  - journal_queue_max_ops
  - journal_queue_max_bytes

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-17 00:02:45 +08:00
Kefu Chai
bfb4d7720a doc: use "describe" directive for meta variables
they are not options, so we should not use confval for defining them.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-16 20:27:24 +08:00
Kefu Chai
bd9b597242 doc: add "confval_section" directive
instead of reusing "confval" directive, define a new directive just
for configration sections

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-16 20:27:24 +08:00
Ilya Dryomov
7eb9c5ddb2 Merge branch 'master' into wip-unauthorized-gids
Sync up with master up to commit 3d8e73b266 ("Merge pull request
#40731 from tchaikov/wip-yamlize-options").  Specifically, bring in
src/common/options.cc yamlization and move new auth-related options
into src/common/options/global.yaml.in.

Conflicts:
	src/common/options.cc
	src/common/options/global.yaml.in

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2021-04-13 15:42:06 +02:00
Zac Dover
f3180ad722 doc/rados: rewrite mclock docs (2 of 2)
This is my second attempt to rewrite the
second half of the mclock docs. The first attempt
is enshrined in https://github.com/ceph/ceph/pull/40571,
in which I got cute with git and got burned.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2021-04-10 06:49:43 +10:00
Adam Kupczyk
a2067a02d7 doc: Add BlueStore sharding documentation
Provide sharding explanation, configuration parameters explanation.
Provide resharding tool example into Pacific format.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
Co-authored-by: Josh Durgin <jdurgin@redhat.com>
2021-04-08 09:27:58 +02:00
Sage Weil
18b343b06e mon/HealthMonitor: raise AUTH_INSECURE_GLOBAL_ID_RENEWAL[_ALLOWED]
Two new alerts:

- AUTH_INSECURE_GLOBAL_ID_RENEWAL_ALLOWED if we are allowing clients to reclaim
global_ids in an insecure manner (for backwards compatibility until
clients are upgraded)

- AUTH_INSECURE_GLBOAL_ID_RENEWAL if there are currently clients connected that
do not know how to securely renew their global_id, as exposed by
auth_expose_insecure_global_id_reclaim=true.  The client auth names and IPs
are listed the alert details (up to a limit, at least).

The docs recommend operators mute these alerts instead of silencing, but
we still include option that allow the alerts to be disabled entirely.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-06 17:28:55 -04:00
Zac Dover
e262e85606 doc/rados: rewrite mclock-config-ref
This PR rewrites the material provided in
PR#40531, improving its elegance and
readability.

This PR rewrites the first 48 percent of the
material provided in PR#40531.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2021-04-07 03:17:51 +10:00
Kefu Chai
f26515fef7
Merge pull request #40168 from amathuria/wip-amathuria-fix-mon-trimming
mon: Modifying trim logic to change paxos_service_trim_max dynamically

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-04-03 22:12:26 +08:00
Sridhar Seshasayee
0ddeccf4b5 doc: Add mclock configuration reference documentation
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
2021-04-01 19:03:14 +05:30
Aishwarya Mathuria
2e1141e439 mon: Modifying trim logic to change paxos_service_trim_max dynamically
Currently, the Paxos Service trim logic is bounded by a max value (paxos_service_trim_max). This change dynamically modifies the max value when the number of logs to be trimmed is higher than paxos_service_trim_max.

 The paxos_service_trim_max_multiplier has been added in case we want to increase paxos_service_trim_max by a certain factor. If this option is enabled we get a new upper bound when trim sizes are high.

Fixes: https://tracker.ceph.com/issues/50004

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
2021-03-30 00:13:37 +05:30
Kefu Chai
b60d041efa doc/rados/configuration: format confval with "confval" directive
for better formatting

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-03-28 23:38:02 +08:00
Sage Weil
f755e353e8 os/bluestore: separate omap per-pool vs per-pg alerts
Currently the health alert raised does not match the docs, and the docs
do not describe what the health alert indicates.

Octopus added per-pool omap storage.  This improves space accounting
and reporting.

Pacific added per-pg omap storage (object hash in key).  This speeds up
PG removal.

Separate everthing out into two distinct alerts raised from bluestore
and surfaced as health alerts, with corresponding config options to
disable, and update the docs accordingly.

Also update the fsck options for warn vs error, and raise separate
errors for the per-pg and per-pool cases.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-03-24 07:34:39 -05:00
Patrick Donnelly
6ad5901422
Merge PR #39191 into master
* refs/pull/39191/head:
	pybind/mgr/snap_schedule: use ceph VFS
	pybind/mgr/snap_schedule: idempotentize table creation
	mgr: add ceph sqlite VFS
	doc: add libcephsqlite
	ceph.spec,debian: package libcephsqlite
	test/libcephsqlite,qa: add tests for libcephsqlite
	libcephsqlite: rework architecture and backend
	SimpleRADOSStriper: wait for finished aios after write
	SimpleRADOSStriper: add new minimal async striper
	mon: define simple-rados-client-with-blocklist profile
	librados: define must renew lock flag
	common: add timeval conversion for durations
	Revert "libradosstriper: add function to read into char*"
	test_libcephsqlite: test random inserts
	cephsqlite: fix compiler errors
	cmake: improve build inst for cephsqlite
	libcephsqlite: sqlite interface to RADOS
	libradosstriper: add function to read into char*

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-03-22 10:06:11 -07:00
Kefu Chai
1dbc932195 doc: use :ref: instead relative path for links
path is fragile when we move the file around, but link is more stable.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-03-22 01:41:53 +08:00
Patrick Donnelly
d08a977a76
doc: add libcephsqlite
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-03-19 08:52:55 -07:00
Patrick Donnelly
108f486afe
mon: define simple-rados-client-with-blocklist profile
A mon profile that grants the ability for rados clients to blocklist
others (similar to rbd).

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-03-19 08:52:55 -07:00
Jonas Jelten
76a67d7753 doc: update osd-class-path error messages
the return value is "95 Operation not supported"

Signed-off-by: Jonas Jelten <jj@sft.lol>
2021-03-16 16:16:02 +01:00
Jonas Jelten
4f98499bfa doc/mgr/crash: explain needed crash upload permissions
Signed-off-by: Jonas Jelten <jj@sft.lol>
2021-03-09 14:12:32 +01:00
Sebastian Wagner
089938a1af
Merge pull request #39558 from sebastian-philipp/doc-rados-rm-toctree-cephadm
doc/rados: avoid including cephadm into the toctree

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Zac Dover <zac.dover@gmail.com>
2021-03-03 16:17:51 +01:00
Sage Weil
67e842d519 Merge PR #39654 into master
* refs/pull/39654/head:
	common/options: drop ms_async_max_op_threads
	msg/async: drop Stack::num_workers
	msg/async: s/num_workers/workers.size()/
	msg/async: use range-based loop in NetworkStack
	msg/async: do not pass worker id to Stack::spawn_worker()
	async/Stack: pass Worker* to NetworkStack::add_thread()
	async/rdma: do not reference worker id in RDMAStack::spawn_worker()
	async/dpdk: do not use worker id when creating worker
	async/PosixStack: do not reference worker id in ctor
	async/rdma: initialize worker in RDMAStack::create_worker()
	async/rdma: move RDMAStack::create_worker() to .cc

Reviewed-by: luo runbing <luo.runbing@zte.com.cn>
Reviewed-by: Haomai Wang <haomai@xsky.com>
2021-03-03 08:38:21 -05:00