Commit Graph

6025 Commits

Author SHA1 Message Date
Sage Weil
65fbc620b6 qa/standalone/mon/osd-create-pool: fix utf-8 grep LANG
This needs en_US.UTF-8... en_US does not work.

Fixes: https://tracker.ceph.com/issues/43422
Signed-off-by: Sage Weil <sage@redhat.com>
2020-01-17 14:19:53 -06:00
Kefu Chai
7b0973e33b
Merge pull request #32624 from alimaredia/wip-s3-tests-python-3
qa/suites: use s3-tests with python3 support

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-01-17 23:47:51 +08:00
Sebastian Wagner
bbd18f1986
Merge pull request #32049 from mgfritch/wip-cd-timeout
cephadm: add ability to specify a timeout

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-01-17 08:32:06 +01:00
Sage Weil
4b7f5de235 Merge PR #32411 into master
* refs/pull/32411/head:
	qa/tasks/ceph_manager: enable ceph-objectstore-tool via cephadm
	qa/tasks/cephadm: support crush_tunables config option

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-01-15 08:39:48 -06:00
Sage Weil
2739ac4a60 qa/tasks/ceph_manager: enable ceph-objectstore-tool via cephadm
- drop support for keyvaluestore
- leave a few paths non-cephadm specific (filestore, upgrade workaround)

Signed-off-by: Sage Weil <sage@redhat.com>
2020-01-15 07:53:01 -06:00
Kefu Chai
d8a3c605cb
Merge pull request #32625 from mgfritch/cephadm-workunit-test-cleanup
qa/workunits/cephadm/test_cephadm: workunit test cleanup

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
2020-01-15 12:33:52 +08:00
Patrick Donnelly
dc018d8b79
Merge PR #31763 into master
* refs/pull/31763/head:
	test: auto-upgrade subvolume test
	mgr/volumes: tie everything together to implement versioned subvolumes
	mgr/volumes: provide subvolume create/remove/open APIs
	mgr/volumes: implement subvolume based on subvolume template
	mgr/volumes: implement subvolume group based on group template
	mgr/volumes: implement trash as a subvolume group
	mgr/volumes: snapshot util module
	mgr/volumes: template for implementing groups and subvolumes
	mgr/volumes: implement filesystem volume module
	mgr/volumes: lock module to serialize volume operations
	mgr/volumes: introduce volume specification module
	mgr/volumes: add fs_util helper module

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-01-14 20:32:44 -08:00
Patrick Donnelly
167563c59e
Merge PR #32582 into master
* refs/pull/32582/head:
	qa/cephfs: don't run kclient tests on latest RHEL
	qa/cephfs: don't test kclient on RHEL 7

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-01-14 12:43:28 -08:00
Casey Bodley
6a6d4a4ffa
Merge pull request #32620 from cbodley/wip-qa-rgw-hadoop-versions
qa/rgw: s3a-hadoop task defaults to maven-version 3.6.3

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2020-01-14 13:36:29 -05:00
Sage Weil
ef91c1e70b Merge PR #32622 into master
* refs/pull/32622/head:
	qa/workunits/cephadm/test_cephadm: drop stray 'exit 0'

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-01-14 08:45:30 -06:00
Casey Bodley
b10873142e qa/rgw: remove test against hadoop v2.8.5
the hadoop branch rel/release-2.8.5 fails to build with:

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  01:37 min
[INFO] Finished at: 2020-01-14T13:09:02Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (create-parallel-tests-dirs) on project hadoop-aws: An Ant BuildException has occured: Unable to create javax script engine for javascript

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2020-01-14 09:42:54 -05:00
Casey Bodley
5bb03f143e qa/rgw: remove test against hadoop trunk
hadoop's trunk fails to build with many 'package does not exist' errors:

[INFO] 100 errors
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  48.792 s
[INFO] Finished at: 2020-01-14T13:07:36Z
[INFO] ------------------------------------------------------------------------

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2020-01-14 09:39:33 -05:00
Laura Paduano
f22247fd28
Merge pull request #32032 from votdev/issue_43088_passwd_async_validator
mgr/dashboard: Process password complexity checks immediately

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2020-01-14 11:15:10 +01:00
Sage Weil
eba385b715 qa/tasks/cephadm: support crush_tunables config option
Signed-off-by: Sage Weil <sage@redhat.com>
2020-01-13 19:49:57 -06:00
Michael Fritch
19e20c2e89
qa/workunits/cephadm/test_cephadm: add SUDO to vgchange
2020-01-13T10:48:14.235 INFO:tasks.workunit.client.0.smithi177.stderr:+ vgchange -an test_cephadm
2020-01-13T10:48:14.239 INFO:tasks.workunit.client.0.smithi177.stderr:  WARNING: Running as a non-root user. Functionality may be unavailable.
2020-01-13T10:48:14.239 INFO:tasks.workunit.client.0.smithi177.stderr:  /run/lvm/lvmetad.socket: access failed: Permission denied
2020-01-13T10:48:14.240 INFO:tasks.workunit.client.0.smithi177.stderr:  WARNING: Failed to connect to lvmetad. Falling back to device scanning.
2020-01-13T10:48:14.260 INFO:tasks.workunit.client.0.smithi177.stderr:  /dev/mapper/control: open failed: Permission denied
2020-01-13T10:48:14.261 INFO:tasks.workunit.client.0.smithi177.stderr:  Failure to communicate with kernel device-mapper driver.
2020-01-13T10:48:14.261 INFO:tasks.workunit.client.0.smithi177.stderr:  Incompatible libdevmapper 1.02.145 (2017-11-03) and kernel driver (unknown version).
2020-01-13T10:48:14.301 INFO:tasks.workunit.client.0.smithi177.stderr:  /run/lock/lvm/V_test_cephadm:aux: open failed: Permission denied
2020-01-13T10:48:14.301 INFO:tasks.workunit.client.0.smithi177.stderr:  Can't get lock for test_cephadm
2020-01-13T10:48:14.301 INFO:tasks.workunit.client.0.smithi177.stderr:  Cannot process volume group test_cephadm

Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-01-13 15:18:24 -07:00
Michael Fritch
57f3dc87b1
qa/workunits/cephadm/test_cephadm: need to add quotes for PYTHONS var
2020-01-13T10:48:13.374 INFO:tasks.workunit.client.0.smithi177.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh: line 46: [: python3: binary operator expected

Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-01-13 15:18:24 -07:00
Ali Maredia
0ed77d1c1a qa: remove s3readwrite & s3roundtrip tasks
As a result of the s3-tests python3 port, the
s3readwrite & s3roundtrip testing files were
deleted

Signed-off-by: Ali Maredia <amaredia@redhat.com>
2020-01-13 15:46:31 -05:00
Ali Maredia
07a8ac5778 qa: s3tests/s3tests-website tasks no longer only use ubunutu
Signed-off-by: Ali Maredia <amaredia@redhat.com>
2020-01-13 15:46:28 -05:00
Sage Weil
fe8f423345 qa/workunits/cephadm/test_cephadm: drop stray 'exit 0'
Introduced in 40b70c632d22dbd4cf68f8ad0df1b30ee28b9f19

Signed-off-by: Sage Weil <sage@redhat.com>
2020-01-13 14:40:51 -06:00
Casey Bodley
abe5277269
Merge pull request #32188 from cbodley/wip-qa-rgw-obj-manifest
qa/rgw: add rgw_obj and throttle tests to rgw verify suite

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2020-01-13 14:45:19 -05:00
Casey Bodley
9d4d273af7
Merge pull request #32550 from cbodley/wip-qa-rgw-multisite-admin-rest
qa/rgw: remove failing radosgw_admin_rest from multisite suite

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
2020-01-13 14:43:37 -05:00
Casey Bodley
82351b9874 qa/rgw: s3a-hadoop task defaults to maven-version 3.6.3
binaries for v3.6.2 are no longer available under
http://www-us.apache.org/dist/maven/maven-3/

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2020-01-13 14:15:03 -05:00
Rishabh Dave
853504bf8e qa/cephfs: don't run kclient tests on latest RHEL
Fixes: https://tracker.ceph.com/issues/43541
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-01-13 10:27:02 +05:30
Rishabh Dave
e0335ee6e9 qa/cephfs: don't test kclient on RHEL 7
https://tracker.ceph.com/issues/43539
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-01-13 10:27:02 +05:30
Kefu Chai
62a79c08bd
Merge pull request #32588 from tchaikov/wip-qa-mgr-setup-with-force
qa/tasks/mgr: set mgr module option with --force

Reviewed-by: Laura Paduano <lpaduano@suse.com>
2020-01-10 22:58:42 +08:00
Kefu Chai
14a138e676
Merge pull request #32519 from tchaikov/wip-qa-skip-if-no-pool
qa/tasks/ceph_manager: do not pick a pool is there is no pools

Reviewed-By: Neha Ojha <nojha@redhat.com>
2020-01-10 21:33:32 +08:00
Kefu Chai
ba98e06c61
Merge pull request #32549 from tchaikov/wip-qa-rados-whitelist-health-warning
qa: whitelist FS_DEGRADED

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-01-10 21:28:33 +08:00
Kefu Chai
492b41bb2a
Merge pull request #32512 from tchaikov/wip-cbt-py3
Revert "qa/tasks/cbt: include py2 deps on ubuntu for now"

Reviewed-By: Neha Ojha <nojha@redhat.com>
2020-01-10 21:26:49 +08:00
Kefu Chai
7210d7f951 qa/tasks/mgr: set mgr module option with --force
if mgr is not active, monitor will refuse to set any option consumed by
mgr modules.

the reason the tests pass somtimes is that, we have a racing here:

1. stop all mgr daemons
2. MgrMonitor gets updated and updates its mgr_module_options
accordingly.
3. in TestDashboard.setUp(), we reset the port number for dashboard
using "ceph config set mgr mgr/dashboard/y/ssl_server_port 7789"
4. restart all mgr daemons

but the 2nd step and 3rd step could race with each other, if the 2nd
step happens after 3rd step, the test passes. otherwise it fails.

in this change, "--force" is passed to the "ceph config set" command,
so ConfigMonitor can bypass the sanity test for the option, and just
set this option.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-01-10 13:28:34 +08:00
Kefu Chai
1bd1328ad7
Merge pull request #32571 from tchaikov/wip-qa-test-runner-run-testcase
qa/tasks/cephfs_test_runner: setattr to class not instance

Reviewed-by: Laura Paduano <lpaduano@suse.com>
2020-01-10 11:28:52 +08:00
Casey Bodley
cfa5922fd8 qa/rgw: add rgw_obj and throttle tests to rgw verify suite
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2020-01-09 16:51:37 -05:00
Patrick Donnelly
cf2d411251
Merge PR #32532 into master
* refs/pull/32532/head:
	qa: only restart MDS between tests

Reviewed-by: Jos Collin <jcollin@redhat.com>
2020-01-09 12:24:28 -08:00
Casey Bodley
c0a1b533f9
Merge pull request #32462 from alimaredia/wip-s3-tests-branch-override-remove
qa: remove force-branch from overrides of s3-tests

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2020-01-09 14:48:44 -05:00
Patrick Donnelly
bae0c67395
Merge PR #32533 into master
* refs/pull/32533/head:
	qa: fix various py3 cephfs qa bugs x2

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Thomas Bechtold <tbechtold@suse.com>
2020-01-09 11:13:53 -08:00
Patrick Donnelly
148e99be09
Merge PR #32507 into master
* refs/pull/32507/head:
	qa: detect RHEL8 for yum package installation

Reviewed-by: Rishabh Dave <ridave@redhat.com>
2020-01-09 11:12:32 -08:00
Kefu Chai
2fe9771842
Merge pull request #32575 from tchaikov/wip-43227
qa/tasks/mgr/dashboard: set pg_num to 16

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
2020-01-09 23:23:15 +08:00
Michael Fritch
906bfb3251
cephadm: add timeout for run, shell, enter, logs
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-01-09 07:59:40 -07:00
Kefu Chai
15a5ebdce2 qa/tasks/mgr/dashboard: set pg_num to 16
as the default pg_num is now 16 since 78bf9244, and the minimum pg_num
is also 16 in autoscaler since 78bf9244.

Fixes: https://tracker.ceph.com/issues/43227
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-01-09 20:54:56 +08:00
Kefu Chai
204263b829 qa/tasks/cephfs_test_runner: setattr to class not instance
before this change, `setattr()` sets the instance specialized with a certain method
of test case, so in `MgrTestCase.setUpClass()`

assert cls.mgr_cluster is not None

fails,

after this change, instead of test case, the class of test suite is updated with the
specified params, even if we pass a certain test to test runner.
so we can

./run-backend-api-tests.sh tasks.mgr.test_dashboard.TestDashboard.test_standby

now. before this change, we can only:

./run-backend-api-tests.sh tasks.mgr.test_dashboard.TestDashboard

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-01-09 20:48:23 +08:00
Tatjana Dehler
1c18ca31fd
Merge pull request #32234 from votdev/better_orch_exception
mgr/dashboard: Throw a more meaningful exception

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2020-01-09 13:41:43 +01:00
Venky Shankar
03ee966b6c test: auto-upgrade subvolume test
Fixes: https://tracker.ceph.com/issues/43349
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2020-01-09 06:33:31 -05:00
Kefu Chai
e67034b95a qa/tasks/ceph_manager: do not pick a pool is there is no pools
random.choice(seq) raises IndexError if seq is empty. we cannot ensure
there is always one or more pools in the cluster while using pool
related thrasher. so skip the thrasher action if there is no pools at
that moment.

Fixes: https://tracker.ceph.com/issues/43412
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-01-09 10:26:32 +08:00
Casey Bodley
f51a7cae7d qa/rgw: remove failing radosgw_admin_rest from multisite suite
this was added to test that admin apis forward relevent requests to the
master zone, but radosgw_admin_rest.py tries to create an admin user
with 'radosgw-admin user create'. this fails with:

Please run the command on master zone. Performing this operation on
non-master zone leads to inconsistent metadata between zones
Are you sure you want to go ahead? (requires --yes-i-really-mean-it)

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2020-01-08 10:27:33 -05:00
Patrick Donnelly
1d727267e5
Merge PR #32535 into master
* refs/pull/32535/head:
	qa: define centos version for fs:verify

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-01-08 07:27:06 -08:00
Kefu Chai
44fb077978 qa: whitelist FS_DEGRADED
`admin_socket_output --all` sends "respawn" to mds, so when the mds
restarts, FS_DEGRADED.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-01-08 22:20:35 +08:00
Kefu Chai
93fd11dd47
Merge pull request #32461 from s0nea/wip-dashboard-fix-user-test
mgr/dashboard: fix failing user test

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Patrick Seidensal <pseidensal@suse.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-01-08 11:40:42 +08:00
Patrick Donnelly
423dcb4c54
qa: only restart MDS between tests
"fs fail" will only fail the MDS that are part of the file system which
will generally allow us to avoid spurious MDS_INSUFFICIENT_STANDBY
warnings. Further, only restart the MDS, there's no reason to leave them
offline.

Fixes: https://tracker.ceph.com/issues/43514
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2020-01-07 18:47:23 -08:00
Patrick Donnelly
6adcace694
Merge PR #30802 into master
* refs/pull/30802/head:
	cephfs-shell: Modify Quota Test
	cephfs-shell: Fix permission errors on quota tests
	cephfs-shell: Add tests for quota
	cephfs-shell: Print max_bytes and max_files as string on get Quota
	cephfs-shell: Remove extra length argument passed to setxattr()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
2020-01-07 14:11:16 -08:00
Patrick Donnelly
e83aa733b9
Merge PR #32459 into master
* refs/pull/32459/head:
	qa/tasks: Nothing to clean up if the volume was not created
	mgr/volumes: drop obsolete comment in _cmd_fs_volume_create
	mgr/volumes: cleanup on fs create error

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-01-07 14:08:41 -08:00
Patrick Donnelly
593d3b1779
qa: detect RHEL8 for yum package installation
Fixes: https://tracker.ceph.com/issues/43487
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2020-01-07 13:54:17 -08:00