Commit Graph

122514 Commits

Author SHA1 Message Date
Patrick Donnelly
4f6adfd148
Merge PR #40892 into master
* refs/pull/40892/head:
	qa/crontab: reduce pacific priority

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-04-17 10:10:15 -07:00
Kefu Chai
d9631287dc
Merge pull request #40890 from tchaikov/wip-yamlize-options-doc
common/options,doc: extract formatted desc into .yaml.in

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-04-17 10:43:58 +08:00
Josh Durgin
0d44a6bd0e common/options/global.yaml.in: remove unused mon options
These were removed in a5f9ca1571

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2021-04-16 18:16:31 -04:00
Sage Weil
1a55d82229 Merge PR #40888 into master
* refs/pull/40888/head:
	qa/tasks/cephadm: ignore --keep-logs failure
	qa/tasks/cephadm: use yaml.dump_all()
	qa/suites/rados/cephadm/smoke-*: use cephadm.wait_for_service
	qa/tasks/cephadm: tear down clsuter before gathering logs
	qa/suites/rados/cephadm/smoke-roleless: test rgw-ingress
	mgr/cephadm: remove virtual_ip check during scheduling
	mgr/orchestrator: orch ls: leave off virtual_ip prefixlen
	qa/tasks/cephadm: add wait_for_service
	qa/tasks/cephadm: allow skip_monitor_stack=true
	qa/tasks/cephadm: do subst_vip for cephadm.shell and .apply
	qa/tasks/vip: add vip task to allocate virtual IPs
	qa/suites/rados/cephadm/smoke-roleless: add rgw-ingress test case
	qa/tasks/cephadm: shell: take 'all-roles' or 'all-hosts'
	qa/tasks/cephadm: let cephadm.shell take string or list

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2021-04-16 18:10:46 -04:00
Josh Durgin
e39ad08b1b common/options/y2c.py: add newline after set_enum
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2021-04-16 17:29:25 -04:00
Josh Durgin
5277ec62d3 common/options/immutable-object-cache.in.yaml: fix watermark after yaml conversion
This was set higher by 2093c88229

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2021-04-16 17:15:52 -04:00
Josh Durgin
aa1d431578 common/options/global.yaml.in: fix bluestore cache ratios from yaml conversion
These were updated two years ago in 303d42533d

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2021-04-16 17:14:33 -04:00
Neha Ojha
d8a37fe4fa
Merge pull request #40732 from neha-ojha/wip-50217
common/options/global.yaml.in: increase default value of bluestore_cache_trim_max_skip_pinned

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Mark Nelson <mnelson@redhat.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse
2021-04-16 13:00:52 -07:00
Sage Weil
73af779d31 mgr/MgrStandby: fix config observer
Otherwise handle_conf_change() is never called.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 16:00:31 -04:00
Sage Weil
efc29fb363 mgr/MgrStandby: respawn if mgr_standby_modules changes
If we have standby modules and this goes false, or do not have modules
and this goes true, then respawn.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 16:00:31 -04:00
Sage Weil
a2c9333a0b qa/tasks/mgr/test_dashboard: skip test_standby if mgr_standby_modules=false
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 16:00:31 -04:00
Sage Weil
9c1f128885 qa/suites/rados/cephadm/smoke-*: use cephadm.wait_for_service
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 16:00:31 -04:00
Sage Weil
c20323d114 qa/suites/rados/cephadm/smoke-singlehost: test --single-host-defaults
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 16:00:31 -04:00
Sage Weil
ae899d3769 cephadm: add --single-host-defaults option to bootstrap
[global] osd_crush_choose_leaf_type = 0
[global] osd_pool_default_size = 2
[mgr] mgr_standby_modules = false

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 16:00:31 -04:00
Sage Weil
77e0f5b45c mgr/cephadm: allow mgr colo if mgr_standby_modules=false
If the standby mgr daemons' modules aren't listening on any ports, then we
can schedule multiple on the same host.

Note that this may make 'orch ps' output misleading, as ports will be
reported for each mgr instance, but only one of them will actually be
listening at any one time (if they are behaving, at least!).  Treat a
mgr port check error as non-fatal.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 16:00:31 -04:00
Sage Weil
35d2146c1a mgr/MgrStandby: add mgr_standby_modules option
Add config option to control whether the standby modules are started.
Default to true (no change in behavior), but if set to false the standby
mgr modules don't do the redirect business.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 16:00:31 -04:00
Josh Durgin
0c44c3ef01 common/options/global.yaml.in: remove obsolete bluefs options
These were removed in d17cd6604b and
1f04b4589d

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2021-04-16 15:17:00 -04:00
Sage Weil
65f2848e97 qa/tasks/cephadm: ignore --keep-logs failure
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 14:12:49 -05:00
Josh Durgin
4c02c102b2 common/options/global.yaml.in: fix some defaults that got changed by the yaml conversion
Presumably git didn't complain during a rebase because the whole file
was removed.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2021-04-16 15:08:57 -04:00
Josh Durgin
726a4dae77 common/options/global.yaml.in: remove duplicate line
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2021-04-16 15:06:57 -04:00
Jiffin Tony Thottan
c2af1cdc13 doc/radosgw/vault: add documentation for ssl authentication
Now RGW server can use custom ssl certs to authenticate with vault
server, adding steps for the same

Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com>
2021-04-16 23:07:59 +05:30
Jiffin Tony Thottan
5b9139bd36 rgw: add support for client cert and key for vault
Authenticate vault with help of user supplied client cert and keys.

Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com>
2021-04-16 23:07:49 +05:30
Neha Ojha
26d7336d6b common/options/global.yaml.in: increase default value of bluestore_cache_trim_max_skip_pinned
This option controls the rate of trimming of onodes and the earlier default of
64 has been seen to be too low for large clusters, leading to buildup of
onodes resulting in memory growth.

Increase the default value to 1000, since there are no known downsides to it.

Fixes: https://tracker.ceph.com/issues/50217
Signed-off-by: Neha Ojha <nojha@redhat.com>
2021-04-16 17:35:43 +00:00
Jiffin Tony Thottan
d88ce5100e rgw: update debug log for curl error
The req_data->error_buf stores error returned by the curl command, add
that information to existing debug log.

Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com>
2021-04-16 22:58:39 +05:30
Jiffin Tony Thottan
424dca0f38 rgw: add support to consume user given ca cert for vault
Currently RGW can authenticate with vault via SSL using system certs.
With this patch user can provide custom ca cert and location of the file
can be specified in ceph.conf like this :
	rgw_crypt_require_ssl = <file path>

Fixes: https://tracker.ceph.com/issues/47776
Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com>
2021-04-16 22:58:19 +05:30
Jiffin Tony Thottan
4a93faa2ee rgw: add seperate option for verify ssl for vault KMS engine
Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com>
2021-04-16 22:54:38 +05:30
Daniel Gryniewicz
b5188a46dc
Merge pull request #40189 from dang/wip-dang-zipper-rename
RGW Zipper - The Great Rename
2021-04-16 13:23:21 -04:00
Ilya Dryomov
8a5b329e06
Merge pull request #40891 from idryomov/wip-y2c-reject-duplicate-keys
common: make y2c.py choke on duplicate keys

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-04-16 18:51:52 +02:00
Adam King
5bdf50e531 mgr/cephadm: don't list non ceph daemons as needing upgrade in upgrade check
Fixes: https://tracker.ceph.com/issues/50401

Signed-off-by: Adam King <adking@redhat.com>
2021-04-16 12:49:59 -04:00
Patrick Donnelly
1605af807d
qa/crontab: reduce pacific priority
Now that it is released, we can reduce its priority to be in line with
other nightlies (but still higher, for now).

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-04-16 09:15:05 -07:00
Kefu Chai
f86cc95ecf doc/rados/configuration/mon-osd-interaction: use confval directive to define options
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-17 00:02:45 +08:00
Kefu Chai
4da18aaf57 common/options,doc: extract formatted desc into .yaml.in
* add a field named "fmt_desc", which is the description formatted using
  reStructuredText. it is preserved as it is if it's different from the
  desc or long_desc of an option. we can consolidate it with long_desc
  in future, and use pretty printer which has minimal support for
  reStructuredText for printing the formatted descriptions for a better
  user experience of command line. but at this moment, fmt_desc has
  only one consumer: the "ceph_confval" sphinx extension which extracts
  and translate the options yaml file to reStructuredText, which is in
  turn rendered by sphinx.
* remove unused options from the doc
  - journal_queue_max_ops
  - journal_queue_max_bytes

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-17 00:02:45 +08:00
Kefu Chai
11fe444b83
Merge pull request #40748 from tchaikov/wip-selftest-ann
pybind/mgr/selftest: add selftest to mypy and cleanups

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2021-04-16 23:51:12 +08:00
Sage Weil
b0f1146a6c Merge PR #40712 into master
* refs/pull/40712/head:
	doc/cephadm: wrong command for single daemon events

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
2021-04-16 11:31:24 -04:00
Sage Weil
97c143c344 qa/tasks/cephadm: use yaml.dump_all()
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 10:30:05 -05:00
Ilya Dryomov
7bef7e9b3b common: make y2c.py choke on duplicate keys
Commit 5505fc0051 ("common: generate legacy_config_opts.h from
.yaml.in files") inadvertently reverted a change of a default value by
adding a second "default" key with the old value.  This was corrected
in commit 75e07f8638 ("common/options/global: correct default of
auth_mon_ticket_ttl"), but highlights that mis-merging a yaml file
is rather easy.

To prevent this happening again, fail the build if duplicate keys
exist in any of src/common/options/*.yaml.in files.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2021-04-16 16:53:49 +02:00
Sage Weil
16b30f2858 qa/suites/rados/cephadm/smoke-*: use cephadm.wait_for_service
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 09:49:45 -05:00
Sage Weil
deec9074bb qa/tasks/cephadm: tear down clsuter before gathering logs
We dont' always stop all services, because teuthology doesn't know about
things it didn't start.  Use rm-cluster to tear things down, but do not
remove the logs themselves.  After we get logs, we'll clean up completely.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 09:49:45 -05:00
Sage Weil
3ff3f697b4 qa/suites/rados/cephadm/smoke-roleless: test rgw-ingress
Test this properly by downing each rgw and haproxy in turn and ensuring
that things remain up.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 09:49:45 -05:00
Sage Weil
2382603162 mgr/cephadm: remove virtual_ip check during scheduling
In 2f33c6ebbc we made the keepalived task
set the necessary sysctls  to add a virtual_ip, so we don't need this
check anymore.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 09:49:45 -05:00
Sage Weil
4498bbe77f mgr/orchestrator: orch ls: leave off virtual_ip prefixlen
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 09:49:45 -05:00
Sage Weil
ced2f7fe4a qa/tasks/cephadm: add wait_for_service
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 09:49:45 -05:00
Sage Weil
bb825157dc qa/tasks/cephadm: allow skip_monitor_stack=true
(Useful for roleless when we want to go faster)

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 09:49:45 -05:00
Sage Weil
0d561e2741 qa/tasks/cephadm: do subst_vip for cephadm.shell and .apply
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 09:49:45 -05:00
Sage Weil
2a8ab2d2b8 qa/tasks/vip: add vip task to allocate virtual IPs
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 09:49:45 -05:00
Sage Weil
3a7c72a9ed Merge PR #40411 into master
* refs/pull/40411/head:
	doc: add note about removal of the `cephfs` nfs cluster type
	mgr/volumes/nfs: drop `type` param during cluster create

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Varsha Rao <varao@redhat.com>
2021-04-16 10:48:04 -04:00
Sage Weil
84e01cb9fe Merge PR #40376 into master
* refs/pull/40376/head:
	mgr/cephadm: place maximum on placement count based on host count

Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2021-04-16 10:47:22 -04:00
Sage Weil
7e16bf3468 qa/suites/rados/cephadm/smoke-roleless: add rgw-ingress test case
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 07:36:47 -05:00
Sage Weil
0803957695 qa/tasks/cephadm: shell: take 'all-roles' or 'all-hosts'
'all' is ambiguous

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 07:36:47 -05:00
Sage Weil
779af3da6f qa/tasks/cephadm: let cephadm.shell take string or list
Make it a bit more forgiving.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 07:36:47 -05:00