Commit Graph

122241 Commits

Author SHA1 Message Date
Kefu Chai
65925d78c6 doc/rados/configuration/bluestore-config-ref: use confval directive
for defining options.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-19 17:08:07 +08:00
Kefu Chai
f415cec292
Merge pull request #40906 from tchaikov/rzarzynski/wip-crimson-internal_client_request-cleanup
crimson/osd: various cleanups

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-04-19 16:45:46 +08:00
Kefu Chai
4fd963d68f
Merge pull request #40874 from rhcs-dashboard/fix-cephfs-pool-size-stat
mgr/dashboard: filesystem pool size should use stored stat

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
2021-04-19 16:36:04 +08:00
Kefu Chai
43940b1b91
Merge pull request #40879 from myoungwon/wip-fix-50299
osd: fix reference leak when ManifestOp is not used

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-04-19 16:29:08 +08:00
Kefu Chai
9ae03a788a
Merge pull request #40878 from tchaikov/wip-mop-cleanup
osd: remove C_SetManifestRefCountDone::mop

Reviewed-by: Myoungwon Oh <myoungwon.oh@samsung.com>
Reviewed-by: Sam Just <sjust@redhat.com>
2021-04-19 16:26:56 +08:00
Kefu Chai
671af3ca85
Merge pull request #40896 from jdurgin/wip-yaml-options-fix
common/options: fix several out of date defaults and options added during yaml conversion

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-04-19 16:26:04 +08:00
Kefu Chai
20de7dd55c
Merge pull request #40884 from majianpeng/osd-watch-remove-unused-code
osd/Watch.h: remove unused code.

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-04-19 16:25:30 +08:00
Radoslaw Zarzynski
a28cc0d629 crimson/osd: fix formatting around ClientRequest::do_process().
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-04-19 15:58:19 +08:00
Radoslaw Zarzynski
713bb9188f crimson/osd: fix header inclusion order in ops_executer.h.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-04-19 15:58:19 +08:00
Radoslaw Zarzynski
46bd515410 crimson/osd: PG::with_locked_obc() doesn't depend on Op anymore.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-04-19 15:58:19 +08:00
Radoslaw Zarzynski
1364a46826 crimson/osd: drop namespace for arg in PG::with_locked_obc().
It's unnecessary.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-04-19 15:58:19 +08:00
Radoslaw Zarzynski
8b90df0dd2 crimson/osd: avoid unneeded obc catching in PG::do_osd_ops_execute().
Actually, `OpsExecuter` already holds `ObjectContextRef` and even
has a (private till now) getter for `hobject_t` extraction.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-04-19 15:58:19 +08:00
Kefu Chai
7200e5286e
Merge pull request #40872 from wjwithagen/wjw-fix-pidfile-order
core: fix compiler warning due to difference in order of struct members

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-04-19 15:50:33 +08:00
Kefu Chai
f057ea202f
Merge pull request #40749 from tchaikov/wip-python3-spec
ceph.spec: consolidate python3-{setuptools,Cython} builds deps

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2021-04-19 15:49:37 +08:00
Kefu Chai
1f4525062b
Merge pull request #40869 from tchaikov/wip-run-make-check.sh-py3
run-make-check.sh: do not specify -DWITH_PYTHON=3

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-04-19 15:48:02 +08:00
Kefu Chai
d8a1d1f93a cmake: rearrange configure_file()
AddCephTest and googletest's CMake scripts also call
find_package(Python3...), but they do not specify the required minor
version of Python3. by default, find_package(Python3...) picks the highest
available python3. so, if we have multiple python3 versions installed in the
system, and the highest python3 version is not the one specified by the
-DWITH_PYTHON3=3.x.y in the cmake command line, we might end up using a
different python3 for the ceph CLI. and even worse, the required python3
package might not available for the picked python3 interpreter found by
googletest. as, in general, only a single python3 has the full access to
prepackaged python3-* shipped by a GNU/Linux distro.

in this change, the configure_file() calls are rearranged to the top of
src/CMakeLists.txt, so they have less chance to use the "polluted" cmake
variable for their subvars.

this change address the test failure where we have, for instance, python3.8
installed on RHEL8/CentOS8, where python3.6 is the python3 which has
the access to the python3-* packages.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-19 12:09:32 +08:00
Kefu Chai
bcd596994f run-make-check.sh: do not specify -DWITH_PYTHON=3
should leave it to do_cmake.sh to decide which python3 version to use,
there is case that we have multiple python3 installed, but only one of them
is fully supported by the distro, in the sense that python3-* packages
are packaged for that python3.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-19 12:04:18 +08:00
Kefu Chai
3903a71a60
Merge pull request #40901 from tchaikov/wip-mgr-rook
cmake: let WITH_MGR_ROOK_CLIENT depend on WITH_MGR

Reviewed-by: Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-04-19 11:25:23 +08:00
Kefu Chai
63d50348c6 cmake: let WITH_MGR_ROOK_CLIENT depend on WITH_MGR
it does not depend on WITH_MGR_DASHBOARD_FRONTEND, which is disabled by
default and is used for enable/disable the inclusion of dashboard
support. while root client is used by orchestrator. so it should depend
on WITH_MGR not WITH_MGR_DASHBOARD_FRONTEND.

this change addresses the regression introduced by
1003f1ffee

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-18 21:53:29 +08:00
Patrick Donnelly
4f6adfd148
Merge PR #40892 into master
* refs/pull/40892/head:
	qa/crontab: reduce pacific priority

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-04-17 10:10:15 -07:00
Kefu Chai
d9631287dc
Merge pull request #40890 from tchaikov/wip-yamlize-options-doc
common/options,doc: extract formatted desc into .yaml.in

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-04-17 10:43:58 +08:00
Josh Durgin
0d44a6bd0e common/options/global.yaml.in: remove unused mon options
These were removed in a5f9ca1571

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2021-04-16 18:16:31 -04:00
Sage Weil
1a55d82229 Merge PR #40888 into master
* refs/pull/40888/head:
	qa/tasks/cephadm: ignore --keep-logs failure
	qa/tasks/cephadm: use yaml.dump_all()
	qa/suites/rados/cephadm/smoke-*: use cephadm.wait_for_service
	qa/tasks/cephadm: tear down clsuter before gathering logs
	qa/suites/rados/cephadm/smoke-roleless: test rgw-ingress
	mgr/cephadm: remove virtual_ip check during scheduling
	mgr/orchestrator: orch ls: leave off virtual_ip prefixlen
	qa/tasks/cephadm: add wait_for_service
	qa/tasks/cephadm: allow skip_monitor_stack=true
	qa/tasks/cephadm: do subst_vip for cephadm.shell and .apply
	qa/tasks/vip: add vip task to allocate virtual IPs
	qa/suites/rados/cephadm/smoke-roleless: add rgw-ingress test case
	qa/tasks/cephadm: shell: take 'all-roles' or 'all-hosts'
	qa/tasks/cephadm: let cephadm.shell take string or list

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2021-04-16 18:10:46 -04:00
Josh Durgin
e39ad08b1b common/options/y2c.py: add newline after set_enum
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2021-04-16 17:29:25 -04:00
Josh Durgin
5277ec62d3 common/options/immutable-object-cache.in.yaml: fix watermark after yaml conversion
This was set higher by 2093c88229

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2021-04-16 17:15:52 -04:00
Josh Durgin
aa1d431578 common/options/global.yaml.in: fix bluestore cache ratios from yaml conversion
These were updated two years ago in 303d42533d

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2021-04-16 17:14:33 -04:00
Neha Ojha
d8a37fe4fa
Merge pull request #40732 from neha-ojha/wip-50217
common/options/global.yaml.in: increase default value of bluestore_cache_trim_max_skip_pinned

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Mark Nelson <mnelson@redhat.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse
2021-04-16 13:00:52 -07:00
Josh Durgin
0c44c3ef01 common/options/global.yaml.in: remove obsolete bluefs options
These were removed in d17cd6604b and
1f04b4589d

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2021-04-16 15:17:00 -04:00
Sage Weil
65f2848e97 qa/tasks/cephadm: ignore --keep-logs failure
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 14:12:49 -05:00
Josh Durgin
4c02c102b2 common/options/global.yaml.in: fix some defaults that got changed by the yaml conversion
Presumably git didn't complain during a rebase because the whole file
was removed.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2021-04-16 15:08:57 -04:00
Josh Durgin
726a4dae77 common/options/global.yaml.in: remove duplicate line
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2021-04-16 15:06:57 -04:00
Neha Ojha
26d7336d6b common/options/global.yaml.in: increase default value of bluestore_cache_trim_max_skip_pinned
This option controls the rate of trimming of onodes and the earlier default of
64 has been seen to be too low for large clusters, leading to buildup of
onodes resulting in memory growth.

Increase the default value to 1000, since there are no known downsides to it.

Fixes: https://tracker.ceph.com/issues/50217
Signed-off-by: Neha Ojha <nojha@redhat.com>
2021-04-16 17:35:43 +00:00
Daniel Gryniewicz
b5188a46dc
Merge pull request #40189 from dang/wip-dang-zipper-rename
RGW Zipper - The Great Rename
2021-04-16 13:23:21 -04:00
Ilya Dryomov
8a5b329e06
Merge pull request #40891 from idryomov/wip-y2c-reject-duplicate-keys
common: make y2c.py choke on duplicate keys

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-04-16 18:51:52 +02:00
Patrick Donnelly
1605af807d
qa/crontab: reduce pacific priority
Now that it is released, we can reduce its priority to be in line with
other nightlies (but still higher, for now).

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-04-16 09:15:05 -07:00
Kefu Chai
f86cc95ecf doc/rados/configuration/mon-osd-interaction: use confval directive to define options
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-17 00:02:45 +08:00
Kefu Chai
4da18aaf57 common/options,doc: extract formatted desc into .yaml.in
* add a field named "fmt_desc", which is the description formatted using
  reStructuredText. it is preserved as it is if it's different from the
  desc or long_desc of an option. we can consolidate it with long_desc
  in future, and use pretty printer which has minimal support for
  reStructuredText for printing the formatted descriptions for a better
  user experience of command line. but at this moment, fmt_desc has
  only one consumer: the "ceph_confval" sphinx extension which extracts
  and translate the options yaml file to reStructuredText, which is in
  turn rendered by sphinx.
* remove unused options from the doc
  - journal_queue_max_ops
  - journal_queue_max_bytes

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-17 00:02:45 +08:00
Kefu Chai
11fe444b83
Merge pull request #40748 from tchaikov/wip-selftest-ann
pybind/mgr/selftest: add selftest to mypy and cleanups

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2021-04-16 23:51:12 +08:00
Sage Weil
b0f1146a6c Merge PR #40712 into master
* refs/pull/40712/head:
	doc/cephadm: wrong command for single daemon events

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
2021-04-16 11:31:24 -04:00
Sage Weil
97c143c344 qa/tasks/cephadm: use yaml.dump_all()
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 10:30:05 -05:00
Ilya Dryomov
7bef7e9b3b common: make y2c.py choke on duplicate keys
Commit 5505fc0051 ("common: generate legacy_config_opts.h from
.yaml.in files") inadvertently reverted a change of a default value by
adding a second "default" key with the old value.  This was corrected
in commit 75e07f8638 ("common/options/global: correct default of
auth_mon_ticket_ttl"), but highlights that mis-merging a yaml file
is rather easy.

To prevent this happening again, fail the build if duplicate keys
exist in any of src/common/options/*.yaml.in files.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2021-04-16 16:53:49 +02:00
Sage Weil
16b30f2858 qa/suites/rados/cephadm/smoke-*: use cephadm.wait_for_service
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 09:49:45 -05:00
Sage Weil
deec9074bb qa/tasks/cephadm: tear down clsuter before gathering logs
We dont' always stop all services, because teuthology doesn't know about
things it didn't start.  Use rm-cluster to tear things down, but do not
remove the logs themselves.  After we get logs, we'll clean up completely.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 09:49:45 -05:00
Sage Weil
3ff3f697b4 qa/suites/rados/cephadm/smoke-roleless: test rgw-ingress
Test this properly by downing each rgw and haproxy in turn and ensuring
that things remain up.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 09:49:45 -05:00
Sage Weil
2382603162 mgr/cephadm: remove virtual_ip check during scheduling
In 2f33c6ebbc we made the keepalived task
set the necessary sysctls  to add a virtual_ip, so we don't need this
check anymore.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 09:49:45 -05:00
Sage Weil
4498bbe77f mgr/orchestrator: orch ls: leave off virtual_ip prefixlen
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 09:49:45 -05:00
Sage Weil
ced2f7fe4a qa/tasks/cephadm: add wait_for_service
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 09:49:45 -05:00
Sage Weil
bb825157dc qa/tasks/cephadm: allow skip_monitor_stack=true
(Useful for roleless when we want to go faster)

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 09:49:45 -05:00
Sage Weil
0d561e2741 qa/tasks/cephadm: do subst_vip for cephadm.shell and .apply
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 09:49:45 -05:00
Sage Weil
2a8ab2d2b8 qa/tasks/vip: add vip task to allocate virtual IPs
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 09:49:45 -05:00