Commit Graph

5485 Commits

Author SHA1 Message Date
Casey Bodley
aa283c5464 qa/rgw: use config overrides for 'testing' kms backend
and configure the 'testing' backend for the rgw/verify suite

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-10-15 11:30:15 -04:00
Mykola Golub
a3b1373535
Merge pull request #30558 from dillaman/wip-41555
rbd-mirror: image status should report remote status

Reviewed-by: Mykola Golub <mgolub@suse.com>
2019-10-15 14:20:12 +03:00
Patrick Donnelly
3ebc5465d4
Merge PR #30816 into master
* refs/pull/30816/head:
	qa: use small default pg count for CephFS pools
	*: remove config of "mon pg warn min per osd" in testing

Reviewed-by: Laura Paduano <lpaduano@suse.com>
2019-10-14 17:59:12 -07:00
Patrick Donnelly
463d8731d7
Merge PR #30873 into master
* refs/pull/30873/head:
	qa: get rid of iterkeys for py3 compatibility

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
2019-10-14 13:01:36 -07:00
Patrick Donnelly
fc88e6c6c5
qa: use small default pg count for CephFS pools
The pg count needs to be a power-of-two since
dff5697464.

Also, mon_pg_warn_min_per_osd is disabled by default now (or set to a
low value in vstart/testing) so there's no need to base the pg count on
this value.

Ideally someday we can remove this so that the default cluster value is
used but we need to keep this for deployments of older versions of Ceph.

Fixes: https://tracker.ceph.com/issues/42228
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-10-14 10:45:05 -07:00
Kefu Chai
f9d1612273
Merge pull request #30723 from vumrao/wip-vumrao-42111
mon/OSDMonitor: add check for crush rule size in pool set size command

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-10-14 22:56:15 +08:00
Kefu Chai
1e0bc26e5c
Merge pull request #29908 from batrick/rhel7.7
qa: add RHEL 7.7 and use as RHEL7 default

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
2019-10-14 22:33:59 +08:00
Kefu Chai
e660ca0880
Merge pull request #30792 from kshtsk/wip-python3-no-iteritems-ceph-task
tasks/ceph: get rid of iteritems for python3

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-10-14 22:26:36 +08:00
Sage Weil
71d74aa8c6 qa: more tries for mon tell when injecting msgr failures
With failure injection the default 2 tries isn't quite enough

Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-11 14:16:42 -05:00
Kyr Shatskyy
5f95b532aa qa: get rid of iterkeys for py3 compatibility
Fixes: https://tracker.ceph.com/issues/42287

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
2019-10-11 18:54:29 +02:00
Sage Weil
cbaa6c079b qa/suites/rados/singleton*: symlink to same msgr-failures
Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-11 09:10:48 -05:00
Ricardo Dias
fefb6b2971
Merge pull request #30789 from s0nea/wip-dashboard-exception-errorcodes
mgr/dashboard: fix existing issues in user integration tests

Reviewed-by: Stephan Müller <smueller@suse.com>
2019-10-11 14:28:59 +01:00
Ricardo Dias
048f0543b2
Merge pull request #30864 from s0nea/wip-dashboard-comment-failing-suites-out
mgr/dashboard: comment failing QA suites out

Ricardo Dias <rdias@suse.com>
2019-10-11 14:27:50 +01:00
Tatjana Dehler
a747efad29 mgr/dashboard: comment questionable suites out
We're currently facing some issues with our integration
tests. Because of that we agreed on commenting questionable
suites out to be able to run all other suites on open pull
requests.

'test_health' and 'test_perf_counters' are commented out
because they led to issues in relation to
https://tracker.ceph.com/issues/41538
As soon as the issue has been fixed, we need to re-add
these two suites again.

Signed-off-by: Tatjana Dehler <tdehler@suse.com>
2019-10-11 10:58:41 +02:00
Tatjana Dehler
9eb64d758f mgr/dashboard: sort QA suites alphabetically
Sort QA suites alphabetically to recognize a missing
suite faster.

Signed-off-by: Tatjana Dehler <tdehler@suse.com>
2019-10-11 10:56:45 +02:00
Tatjana Dehler
dcb70a3c0d mgr/dashboard: wait a moment after module load
After a dashboard module load we need to wait (10 secs)
for the module to be back again. Otherwise the module
might not be able to answer requests properly. It could
lead to issues in some cases, e.g.: 'Failed to establish
a new connection: [Errno 111] Connection refused'.

Signed-off-by: Tatjana Dehler <tdehler@suse.com>
2019-10-11 10:46:26 +02:00
Tatjana Dehler
a14163a589 mgr/dashboard: reset login to admin user
After logging in with a separate testuser, especially a read-only
user, we need to reset the login to the admin user. This is important
in order to be able to continue with further tests. If not doing so
further tests might fail with the following error response:
{"status": "401 Unauthorized",
 "detail": "You are not authorized to access that resource", ...}

Fixes: https://tracker.ceph.com/issues/42226,
https://tracker.ceph.com/issues/42227
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
2019-10-11 10:46:26 +02:00
Tatjana Dehler
4dd83fd6aa mgr/dashboard: check for correct DashboardException error codes
We need to check for the correct error codes.
Otherwise the integration tests are going to fail.

Fixes: https://tracker.ceph.com/issues/42222
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
2019-10-11 10:46:26 +02:00
Sage Weil
984f0204c1 Merge PR #30817 into master
* refs/pull/30817/head:
	qa/suites/rados/thrash-old-clients: exclude ceph-daemon on nautilus installs
2019-10-10 18:10:41 -05:00
Patrick Donnelly
518168f996
*: remove config of "mon pg warn min per osd" in testing
Follows: 609ab5aa85
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-10-10 09:54:33 -07:00
Ricardo Marques
d611a6abc3
Merge pull request #30400 from rhcs-dashboard/fix-41867-master
mgr: change perf-counter precision to float

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2019-10-10 11:45:13 +01:00
Kefu Chai
202b4d3359
Merge pull request #30787 from zhangdaolong/master
doc, qa: remove invalid option mon_pg_warn_max_per_osd

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-10-10 14:04:56 +08:00
Sage Weil
4e50520efe qa/suites/rados/thrash-old-clients: exclude ceph-daemon on nautilus installs
i did the original addition by grepping for ceph-mgr-ssh, but that's
included in nautilus so I missed this one!

Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-09 13:12:26 -05:00
Venky Shankar
e646524567 qa: tolerate ECONNRESET errcode during logrotate
Fixes: http://tracker.ceph.com/issues/41800
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2019-10-09 08:46:35 -04:00
zhangdaolong
e5d91c9872 doc, qa:remove invalid option mon_pg_warn_max_per_osd
The older mon_pg_warn_max_per_osd option has been removed in v12.2.1 Luminous
https://ceph.com/releases/v12-2-1-luminous-released/

Fixes: https://tracker.ceph.com/issues/42221

Signed-off-by: zhang daolong <zhangdaolong@fiberhome.com>
2019-10-09 16:09:18 +08:00
Patrick Donnelly
559032d7b3
Merge PR #30476 into master
* refs/pull/30476/head:
	doc: list FS subvolumes, subvolume groups and their snapshots
	qa/tasks: tests for ls
	mgr/volumes: list FS subvolumes, subvolume groups and their snapshots

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-10-08 11:43:24 -07:00
Vikhyat Umrao
e1451c92d8 qa/workunits/mon: test for crush rule size in pool set size command
Signed-off-by: Vikhyat Umrao <vumrao@redhat.com>
2019-10-08 09:06:22 -07:00
Jason Dillaman
eaec1786ee qa/workunits/rbd: tweak mirror test case to handle peer status
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-10-08 11:17:15 -04:00
Kyr Shatskyy
9b5bd6c715 tasks/ceph: get rid of iteritems for python3
For python3 compatibility use items() instead of iteritems().

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
2019-10-08 16:41:32 +02:00
Jos Collin
1bdd75f999
qa/tasks: tests for ls
tests for:
* fs subvolume ls
* 'fs subvolume ls' for non-existent default group
* fs subvolumegroup ls
* 'fs subvolumegroup ls' for not-existent /volume
* fs subvolume snapshot ls
* fs subvolumegroup snapshot ls

Fixes: https://tracker.ceph.com/issues/41842
Signed-off-by: Jos Collin <jcollin@redhat.com>
2019-10-08 11:53:46 +05:30
Sage Weil
6fd67f19e3 Merge PR #30603 into master
* refs/pull/30603/head:
	ceph-daemon: -n type.id instead of -i id
	ceph-daemon: drop unused VERSION
	ceph-daemon: clean up dir helpers, tighten up permissions
	ceph-daemon: fchmod before writing to keyring file
	test_ceph_daemon.sh: skip ssh until container image has remoto
	ceph-daemon: decode utf-8 in run() helper
	mgr/ssh: clean up debug cruft
	mgr/ssh: clean up bare except: block
	ceph-daemon: clean up bare except: blocks
	ceph-daemon: all imports to top
	ceph-volume: no_tmpfs -> tmpfs
	doc/bootstrap: add new bootstrap documentation
	ceph-daemon: add --output-pub-ssh-key for bootstrap
	ceph-daemon: make 'shell' easier to use
	ceph-daemon: support docker; prefer podman
	qa: add ceph-daemon
	debian: ceph-daemon package, required by ceph-mgr-ssh
	ceph.spec.in: ceph-daemon package, required by ceph-mgr
	common/options: cleanup whitespace
	mgr/ssh: simplify getting the cluster fsid
	mgr/ssh: pipe ceph-daemon script to stdin of python3
	ceph-daemon: add support for args and/or stdin from top of script
	ceph-daemon: make ceph-volume use get_config_and_keyring
	ceph-daemon: ls: behave if /var/log/ceph doesn't exist
	ceph-daemon: implement 'adopt' for legacy style daemons
	ceph-daemon: fix fsid detection for legacy osds
	ceph-daemon: make rm-cluster clean up system-ceph*.slice too
	ceph-daemon: configure ssh orchestrator
	ceph-daemon: be more restrictive with file permissions
	mgr/ssh: create osd with ceph-daemon
	mgr/ssh: pass daemon id separately to _create_daemon
	ceph-daemon: add --config-and-keyring to ceph-volume command
	ceph-daemon: create log path for shell (if needed)
	mgr/ssh: use _run_ceph_daemon for _create_daemon
	mgr/ssh: factor _run_ceph_daemon out of _get_device_inventory
	mon/ConfigMonitor: allow entity type only for 'config get'
	ceph-daemon: add ceph-volume subcommand
	ceph-daemon: remove unused CephContainer dname property
	ceph-daemon: drop useless uid/gid checks
	mgr/ssh: deploy new mgrs with ceph-daemon
	mgr/ssh: factor _create_daemon out of create_mon
	mon/MonCap: allow mgr to create new auth keys
	mgr/ssh: run c-v with podman when getting inventory
	mgr/ssh: simplify ssh connection management
	mgr/ssh: use ceph-daemon for deploying mon
	ceph-daemon: allow --mon-network for deploying new mon (vs specifying IP)
	ceph-daemon: --config-and-keyring (not key)
	common/options: add 'image' config option
	test_ceph_daemon: specify image name
	vstart.sh: add --ssh to enable+configure ssh orchestrator
	mgr/ssh: use ssh identity from config-key, if present
	mgr/ssh: hardcode default ssh_config
	ceph-daemon: store ssh identity in mon config-key store
	ceph-daemon: --privileged arg for 'exec'
	ceph-daemon: make deploy work for osd (do a c-v prepare)
	ceph-daemon: make shell privileged
	ceph-daemon: move get_container_mounts to a helper
	ceph-daemon: pass full path for entrypoint
	ceph-daemon: make id portion of 'shell' optional
	ceph-volume: accept --no-tmpfs argument for bluestore
	ceph-daemon: 'unit' command
	ceph-daemon: fix run command to use call(), not check_output()
	src/ceph-daemon: whitespace
	ceph-daemon: add 'enter', 'exec' commands
	ceph-daemon: bind config to default location
	test_ceph_daemon.sh: test deploy mds too
	ceph-daemon: generate ssh keys
	ceph-daemon: --config, not --conf
	ceph-daemon: long lines
	ceph-daemon: add --config to bootstrap
	ceph-daemon: add 'shell' command
	ceph-daemon: do not import subprocess symbols directly
	ceph-daemon: add mons with 'deploy mon.x ...'
	ceph-daemon: add 'ls'
	ceph-daemon: simplify uid/gid a bit
	ceph-daemon: fix libudev
	ceph-daemon: autodetect uid/gid from container image
	ceph-daemon: default to empty log files, log to stderr (systemd journal)
	ceph-daemon: rm-{daemon,cluster}
	ceph-daemon: fix bootstrap config
	ceph-daemon: fix args.fsid usage
	ceph-daemon: be careful overwriting live files
	ceph-daemon: slurp some options over from the standard systemd unit
	ceph-daemon: add ceph.target and ceph-$fsid.target units
	test_ceph_daemon.sh: stupid test script
	ceph-daemon: bootstrap and deploy (mgr) work
	ceph-daemon: initial checkin
	ceph-mon: fix debug print of public_addr
2019-10-07 15:31:14 -05:00
Mykola Golub
8ec5f6c8d7
Merge pull request #30711 from dillaman/wip-41938
librbd: v1 clones are restricted to the same namespace

Reviewed-by: Mykola Golub <mgolub@suse.com>
2019-10-07 11:30:02 +03:00
Ali Maredia
d4872ce97a
Merge pull request #29783 from scarvalhojr/ssevault
rgw: add SSE-KMS with Vault using token auth
2019-10-06 22:33:36 -04:00
Sage Weil
bbc7bb5a22 Merge PR #30217 into master
* refs/pull/30217/head:
	crimson: common/admin_socket kludge so that it builds
	mon/MonClient: fix sending mon command to a specific rank
	src/.gitignore: ignore .tox
	mon/MonClient: interpret numeric mon target name as rank
	mgr,mgr/MgrClient: use fsid to signal mon-mgr vs cli MCommands
	qa/workunits/cephtool: fix errpr checks for 'ceph daemon' commands
	common/ceph_context: make 'config unset' idempotent
	qa/tasks/dump_stuck: mon.a, not mon.0
	qa/suites/rados/singleton/all/admin-socket: fix test
	common/config: EPERM setting config option after startup
	qa/workunits/cephtool/test.sh: fix tell output error check
	common/admin_socket: pass Formatter from generic infrastructure
	common/admin_socket: pass ostream to call() for error output
	os/bluestore: fix asok hook return value
	rgw: fix asok return value
	common/ceph_context: return error code from asok commands
	test/pybind/test_rados: fix accidental mon tell test
	mon: print entity_name along with caps to debug log
	PendingReleaseNotes: notes about asok changes
	mgr/MgrClient: empty target string for 'tell' means active mgr
	common/admin_socket: report error code as part of output string
	osd: change trigger_[deep_]scrub tommands to a pg tell command
	osd: remove old command workqueue, threadpool
	osd: drop MMonCommand handling
	osdc/Objecter: resend OSD tell commands on EAGAIN
	osd: route tell commands to asok; migrate commands
	osd: use unique_ptr<Formatter> for asok_command
	common/ceph_context: add generic asok 'injectargs'
	common/admin_socket: allow dup prefixes
	common/admin_socket: refactor with sync and async execute_command variants
	common/admin_socket: pass input bufferlist
	osd: transition to call_async() for asok
	common/admin_socket: support alternative call_async()
	mon/MonClient: send tell commands out of band via MCommand
	mon: accept tell commands via MCommand and send them to asok handler
	common/admin_socket: return int from hook call()
	mgr/DaemonServer: route MCommand (for octopus+) to asok commands
	do not use 'ceph tell mgr'
	pybind/ceph_argparse: disambiguate mgr tell and CLI commands
	ceph: make 'ceph tell mgr.*' send to the active mgr
	ceph: send 'ceph tell mgr.X' to the right mgr
	librados: add rados_mgr_command_target
	mgr/MgrClient: add start_command variant that takes a target
	common/admin_socket: drop unregister_command(); use per-hook variant
	common/admin_socket: drop explicit prefix arg to register_command
	common/admin_socket: simplify command routing
	common/admin_socket: add ability to process MCommand via asok queue
	common/admin_socket: pass cmdvec to execute_command
	common/admin_socket: use pipe for general wakeup
	include/compat: add flags arg to pipe_cloexec
	common/admin_socket: drop unused args

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-10-06 09:08:28 -05:00
Sage Weil
f2e2cb1541 qa: add ceph-daemon
Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-04 20:33:35 -05:00
David Zafman
fdf93add0b
Merge pull request #30714 from dzafman/wip-41743
test: Ignore OSD_SLOW_PING_TIME* if injecting socket failures

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-10-04 18:28:48 -07:00
Jason Dillaman
547ebccaf9 librbd: v1 clones are restricted to the same namespace
Therefore the v1 parent spec should not attempt to populate the
namespace.

Fixes: https://tracker.ceph.com/issues/41938
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-10-04 13:27:50 -04:00
Sage Weil
99f940641f qa/workunits/cephtool: fix errpr checks for 'ceph daemon' commands
Error output has changed slightly.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-04 09:07:03 -05:00
Sage Weil
4a4897414d qa/tasks/dump_stuck: mon.a, not mon.0
Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-04 09:07:03 -05:00
Sage Weil
7b644f599b qa/suites/rados/singleton/all/admin-socket: fix test
We can't set the filestore setting because filestore isn't active and so
the option isn't observed, so it isn't changeable.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-04 09:07:03 -05:00
Sage Weil
05bc99f82e qa/workunits/cephtool/test.sh: fix tell output error check
It goes to stderr instead of stdout now.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-04 09:07:03 -05:00
Sage Weil
1e44d86b2c osd: change trigger_[deep_]scrub tommands to a pg tell command
This is cleaner.  All users are currently standalone tests; updated.

It also means that *all* commands that have a name=pgid arg are pg tell
commands.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-10-04 09:07:02 -05:00
Kefu Chai
86b9036c6c
Merge pull request #30614 from tchaikov/wip-no-more-osdmap-full
librados,osd,mon: remove traces of CEPH_OSDMAP_FULL

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-10-04 10:27:21 +08:00
Neha Ojha
dc473ec733
Merge pull request #30699 from neha-ojha/wip-more-recovery-tests
qa/suites/rados/thrash: increase async and partial recovery test coverage

Reviewed-by: Xie Xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-10-03 16:11:55 -07:00
Neha Ojha
375217a3d7 qa/suites/rados: add more-async-partial-recovery.yaml in the thrash suite
This will increase both async and partial recovery.

Signed-off-by: Neha Ojha <nojha@redhat.com>
2019-10-03 11:15:51 -07:00
Neha Ojha
e953719f2c qa/suites/rados: add more-partial-recovery.yaml in the thrash suite
Increasing osd_object_clean_region_max_num_intervals to track more
clean regions, resulting in more partial recovery.

Signed-off-by: Neha Ojha <nojha@redhat.com>
2019-10-03 11:15:51 -07:00
Neha Ojha
a2eb45507a qa/suites/rados: add more-async-recovery.yaml in the thrash suite
Reducing osd_async_recovery_min_cost to 1 should allow more
asynchronous recovery to happen.

Signed-off-by: Neha Ojha <nojha@redhat.com>
2019-10-03 11:15:51 -07:00
David Zafman
ded58ef91d test: Ignore OSD_SLOW_PING_TIME* if injecting socket failures
Fixes: https://tracker.ceph.com/issues/41743

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-10-03 09:09:10 -07:00
Sage Weil
4bfd1c5114 Merge PR #30627 into master
* refs/pull/30627/head:
	qa/suites/rados/verify: whitelist MON_DOWN when using valgrind
	qa/suits/rados/basic/tasks/rados_api_tests: pgs can go degraded

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-10-02 15:46:15 -05:00
Andrea Baglioni
955d3b517a rgw: add SSE-KMS with Vault using token auth
Restructure SSE-KMS tests which now has 3 scenarios for each KMS
backend: Barbican, Vault, and testing (keys stored in ceph.conf).

Signed-off-by: Andrea Baglioni <andrea.baglioni@workday.com>
Signed-off-by: Sergio de Carvalho <sergio.carvalho@workday.com>
2019-10-02 16:38:01 +01:00