Commit Graph

105121 Commits

Author SHA1 Message Date
Tatjana Dehler
6a988938ac qa/tasks/mgr/dashboard/test_health: update schema
Fixes: https://tracker.ceph.com/issues/43195
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
2019-12-09 16:12:12 +01:00
Sage Weil
105428e8f4 Merge PR #29687 into master
* refs/pull/29687/head:
	test/store_test: test coverage for anti-spillover framework.
	os/bluestore: introduce bluestore_volume_selection_reserved cfg option.
	os/bluestore: beautify RocksDBBlueFSVolumeSelector::dump output
	os/bluestore: introduce 'bluestore bluefs stats' admin socket command
	os/bluestore: implement more intelligent DB volume space managment for BlueFS.
	os/kv: add static method to parse RocksDB options
	os/bluestore: introduce volume selector abstraction to BlueFS

Reviewed-by: Sage Weil <sage@redhat.com>
2019-12-08 15:24:13 -06:00
Sage Weil
d5d4868b9f Merge PR #32108 into master
* refs/pull/32108/head:
	ceph-daemon: remove prepare-host

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2019-12-08 13:26:32 -06:00
Sage Weil
ade391513c ceph-daemon: remove prepare-host
I thought I took this out of the PR but somehow it got merged in... must
have repushed and old branch and not realized.  :/

Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-08 11:26:14 -06:00
Sage Weil
e2f8187b4c Merge PR #31959 into master
* refs/pull/31959/head:
	os/bluestore/BlueStore.cc: set priorities for compression stats

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2019-12-08 10:15:48 -06:00
Sage Weil
1542776e44 Merge PR #31995 into master
* refs/pull/31995/head:
	msg/async: No need lock for func _filter_addrs

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-12-08 10:15:17 -06:00
Sage Weil
9634b20908 Merge PR #32101 into master
* refs/pull/32101/head:
	mgr/orchestrator_cli: make rgw rm take realm+zone
	qa/tasks/mgr/test_orchestrator_cli: fix rgw add test

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2019-12-08 10:14:50 -06:00
Sage Weil
99eb52b68a mgr/orchestrator_cli: make rgw rm take realm+zone
And adjust test accordingly

Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-07 23:41:36 -06:00
Sage Weil
28e9e25e93 Merge PR #32043 into master
* refs/pull/32043/head:
	os/bluestore: default bluestore_block_size 1T -> 100G

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2019-12-07 14:43:07 -06:00
Sage Weil
d73fe9cbd6 Merge PR #32044 into master
* refs/pull/32044/head:
	mon/OSDMonitor: make memory autotune disable itself if no rocksdb

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-12-07 14:42:49 -06:00
Sage Weil
552f5109ce Merge PR #31581 into master
* refs/pull/31581/head:
	mon: fix mon_sync_max_payload_size type
	mon: cap keys in mon_sync messages
	mon/MonitorDBStore: improve get_chunk_tx limits
	mon/MonitorDBStore: better size estimation for Transaction

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-12-07 12:35:58 -06:00
Sage Weil
d91cb4270d Merge PR #32048 into master
* refs/pull/32048/head:
	mon/PGMap: use NONE for 'pg ls[-*]' output too

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-12-07 12:35:20 -06:00
Sage Weil
86950ce9aa Merge PR #32039 into master
* refs/pull/32039/head:
	test: Improve races by using kill_daemons which waits for OSDs terminate
	test: run-standalone.sh: Only run execs in the subdirectories of qa/standalone
	test: Use activate_osd() when restarting OSDs
	test: osd-scrub-snaps.sh: Fix race with osd restart and doing a scrub

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-12-07 12:28:15 -06:00
Sage Weil
7e1f445422 qa/tasks/mgr/test_orchestrator_cli: fix rgw add test
Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-07 08:14:59 -06:00
Sage Weil
b35dbcd582 Merge PR #32010 into master
* refs/pull/32010/head:
	mgr/ssh: Add module to static type checking from tox.
	mgr/orchestrator: Add mypy static type checking

Reviewed-by: Thomas Bechtold <tbechtold@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2019-12-07 08:05:49 -06:00
Sage Weil
0d1c74a578 Merge PR #32023 into master
* refs/pull/32023/head:
	qa/tasks/mgr/test_orchestrator_cli: test with valid network or IP
	qa/orchestrator: do not test "mon update 3 host1"

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Joshua Schmid <jschmid@suse.de>
2019-12-07 08:05:24 -06:00
David Zafman
676d882649 test: Improve races by using kill_daemons which waits for OSDs terminate
osd-backfill-space.sh: More sleep time to make sure the backfill gets started

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-12-06 19:44:06 -08:00
Sage Weil
fbee608dee qa/tasks/mgr/test_orchestrator_cli: test with valid network or IP
Passing 'network' will fail with

Error EINVAL: Failed to parse host list: '['host1:network', 'host2:network', 'host3:network']': u'network' does not appear to be an IPv4 or IPv6 address

Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-06 21:26:13 -06:00
Sage Weil
409608704c Merge PR #32096 into master
* refs/pull/32096/head:
	ceph-daemon: fix firewalld error case

Reviewed-by: Michael Fritch <mfritch@suse.com>
2019-12-06 21:23:55 -06:00
Sage Weil
a4727bdc71 Merge PR #32097 into master
* refs/pull/32097/head:
	ceph-daemon: fix bootstrap ownership of tmp monmap file

Reviewed-by: Michael Fritch <mfritch@suse.com>
2019-12-06 21:23:43 -06:00
Sage Weil
eee4c2ff8f Merge PR #32051 into master
* refs/pull/32051/head:
	mgr: simply exit on SIGINT or SIGTERM

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-12-06 16:19:09 -06:00
Sage Weil
32e06caa35 ceph-daemon: fix bootstrap ownership of tmp monmap file
The file is created with mode 0644 by the ceph:ceph user, but root
cannot write to that.  Instead, create the tmp file owned by root, write
to it, then chown it to user ceph.

Fixes e92929d86d

Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-06 15:14:31 -06:00
Sage Weil
474309804b ceph-daemon: fix firewalld error case
Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-06 14:55:03 -06:00
Neha Ojha
9d4e7e2f27
Merge pull request #32021 from falcon78921/wip-doc-43130
doc: fixed broken reference link for Graphviz

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-12-06 11:52:05 -08:00
David Zafman
7d09908e6a
Merge pull request #32017 from dzafman/wip-revert-verify-upmap
Revert "crush: remove invalid upmap items"

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Xie Xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-12-06 09:29:40 -08:00
David Zafman
8d3cfc6bc5 test: run-standalone.sh: Only run execs in the subdirectories of qa/standalone
This will ignore scripts placed at the qa/standalone level, though
I'm not sure if we should be putting any tests there.  It does
allow support scripts present like ceph-helper.sh without modifying
run-standalone.sh to ignore it.

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-12-06 09:19:54 -08:00
Jason Dillaman
4cce6e9ee3
Merge pull request #31589 from zhengchengyao/mv_snapshot
librbd:move all snapshot API functions in internal.cc over to api/Snapshot.cc

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-12-06 10:22:22 -05:00
Casey Bodley
53ad9d2983
Merge pull request #31767 from pritha-srivastava/wip-rgw-mfa-set-versioning
rgw: adding mfa code validation when bucket versioning status is changed

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2019-12-06 10:18:56 -05:00
Lenz Grimmer
64009101f8
mgr/dashboard: adapt to create_osds parameters change (#31928)
mgr/dashboard: adapt to create_osds parameters change

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
2019-12-06 14:57:27 +00:00
Laura Paduano
843ab6161a
Merge pull request #31062 from dillaman/wip-42355
mgr/dashboard: support RBD mirroring bootstrap create/import

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
2019-12-06 10:42:03 +01:00
Lenz Grimmer
1af00460f8
mgr/dashboard: Cleanup code (#32031)
mgr/dashboard: Cleanup code

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
2019-12-06 08:57:49 +00:00
Mykola Golub
d8600f9383
Merge pull request #31949 from dillaman/wip-42748
rbd: incorporate rbd-mirror daemon status in mirror pool status

Reviewed-by: Mykola Golub <mgolub@suse.com>
2019-12-06 10:43:47 +02:00
Patrick Donnelly
88ef8147bc
Merge PR #29964 into master
* refs/pull/29964/head:
	qa/cephfs: test conf file reading for CephFS shell
	qa/cephfs: allow passing conf to methods that runs shell command
	qa/vstart_runner: add a method to create temporary file
	test_cephfs_shell: update test since shell can now read ceph.conf
	src/vstart.sh: add cephfs-shell section to ceph.conf
	cephfs-shell: read shell variables from ceph.conf

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-12-05 21:42:12 -08:00
Rishabh Dave
d6f768b24e qa/cephfs: test conf file reading for CephFS shell
Adds tests that checks -
* reading conf option
* reading conf option after setting it twice
* reading conf option after setting it after a reset

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2019-12-06 08:54:06 +05:30
Rishabh Dave
a1e3e270b9 qa/cephfs: allow passing conf to methods that runs shell command
Add an argument to the methods that issues CephFS shell command to
accept path to the ceph.conf file.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2019-12-06 08:54:06 +05:30
Rishabh Dave
8d7c45d3b0 qa/vstart_runner: add a method to create temporary file
The method's signature is kept identical to the method in teuthology.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2019-12-06 08:54:05 +05:30
Rishabh Dave
2f85a03b56 test_cephfs_shell: update test since shell can now read ceph.conf
In the test
test_cephfs_shell.TestMisc.test_issue_cephfs_shell_cmd_at_invocation,
don't set the shell option with in the command to be tested since
CephFS shell can now read it's value from ceph.conf.

Also, add methods to set config options for vstart as well teuthology
testing.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2019-12-06 08:52:59 +05:30
Rishabh Dave
5f74a44d36 src/vstart.sh: add cephfs-shell section to ceph.conf
And set "debug" to true, so that developers can see tracebacks in case
cephfs-shell crashes.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2019-12-06 08:52:59 +05:30
Rishabh Dave
33f5a59ac8 cephfs-shell: read shell variables from ceph.conf
Read ceph.conf and initialize shell variables by reading options set
under section cephfs-shell. If there are sections/options that are
duplicated in ceph.conf, CephFS shell overrides the options with the
values last.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2019-12-06 08:52:59 +05:30
Patrick Donnelly
45907ee258
Merge PR #31897 into master
* refs/pull/31897/head:
	doc: obsolete entries for allow_standby_replay
	wip-doc: entries obsoleted by allow_standby_replay

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-12-05 16:05:59 -08:00
Patrick Donnelly
30dc6aae08
Merge PR #31403 into master
* refs/pull/31403/head:
	vstart.sh: highlight presence of stray conf

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-12-05 15:58:26 -08:00
Sage Weil
3363a1001c mgr: simply exit on SIGINT or SIGTERM
The python modules don't reliable shut down.  We've fixed a few shutdown
bugs chasing https://tracker.ceph.com/issues/42744.

This was compounded by https://tracker.ceph.com/issues/42981, which seems
to suggest something in the python module teardown is screwing with the
signal handlers.

Now that df507cde8d is there, the mgr will
blacklist all rados clients the mgr instantiates, which means we can just
exit immediately and let the blacklisting clean things up.

Works-around: https://tracker.ceph.com/issues/42981
Fixes: https://tracker.ceph.com/issues/42744
Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-05 17:18:07 -06:00
David Zafman
43f6218993 test: Use activate_osd() when restarting OSDs
Signed-off-by: David Zafman <dzafman@redhat.com>
2019-12-05 15:13:31 -08:00
David Zafman
cca541d0f9 test: osd-scrub-snaps.sh: Fix race with osd restart and doing a scrub
Fixes: https://tracker.ceph.com/issues/43150

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-12-05 15:12:43 -08:00
Patrick Donnelly
e8368d61be
Merge PR #29421 into master
* refs/pull/29421/head:
	qa/cephfs: add tests for ACLs
	qa/cephfs: allow running tests from xfstests-dev
	qa/tasks: add methods to get monitor's sockets
	qa/cephfs: don't crash if mountpoint dir is already deleted
	vstart_runner.py: set omit_sudo's default value to False
	qa/vstart_runner.py: fix get_keyring_path()
	qa/cephfs: don't abort if mountpoint is already present
	qa/cephfs: allow specifying mountpoint for kernel mounts
	qa/cephfs: allow specifying mountpoints for FUSE mounts
	qa/vstart_runner.py: allow specifying mountpoint for local FUSE mounts
	qa/mount.py: allow setting mountpoint
	qa/vstart_runner.py: add a method to create a temporary directory

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-12-05 13:25:03 -08:00
Sage Weil
81e6a58d2b mon/PGMap: use NONE for 'pg ls[-*]' output too
We already show CRUSH_ITEM_NONE as NONE in the 'pg dump' output as of
57bac8ea88e5cdcb3854852d439f7e6cbf9df900; do the same for 'pg ls' (which
is the friendlier of the two and what users should be using anyway!).

Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-05 14:37:02 -06:00
Sage Weil
68eea44c11 mon/OSDMonitor: make memory autotune disable itself if no rocksdb
Currently, we crash (failed assertion). Instead, just disable autotuning.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-05 13:34:39 -06:00
Sage Weil
57890fce70 os/bluestore: default bluestore_block_size 1T -> 100G
This makes vstart *way* faster.  This option is only really relevant
for dev test environments.  We bumped it up back in dbdd1d9b6e
from 10G just to make ENOSPC less common in dev/test.  Let's see if 100G
is a better balance.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-12-05 12:58:12 -06:00
Casey Bodley
f6bda33bc9
Merge pull request #31926 from yuvalif/wip-yuval-fix-pubsub-record-format
rgw/pubsub: fix records/event json format to match documentation

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-12-05 13:55:31 -05:00
Kefu Chai
b6aa85cfb1
Merge pull request #32036 from zdover23/wip-doc-troubleshooting-pg-verb-fix
doc: s/achieve/achieves/ (Fixed a verb disagreement)

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-12-06 01:45:36 +08:00