Commit Graph

2057 Commits

Author SHA1 Message Date
Patrick Donnelly
c99a5e56a6 Merge PR #42201 into master
* refs/pull/42201/head:
	qa: fold frag confs into conf/mds.yaml

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
2021-07-30 14:00:19 -07:00
Patrick Donnelly
665b36de4e Merge PR #42349 into master
* refs/pull/42349/head:
	mon/MDSMonitor: propose if FSMap struct_v is too old
	mon/MDSMonitor: give a proper error message if FSMap struct_v is too old
	mds/FSMap: use DECODE_OLDEST to gate FSMap version
	qa: add tests for fs dump of epoch and trimming
	qa: add file system support for dumping epoch
	mon/MDSMonitor: return mon_mds_force_trim_to even if equal to current epoch
	mon: add debugging for trimming methods
	mon: fix debug spacing
	qa: add nofs upgrade suite

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
2021-07-28 10:45:08 -07:00
Patrick Donnelly
4f0f51e4cb Merge PR #41025 into master
* refs/pull/41025/head:
	qa: wait pgs to be clean before using the pools
	qa: ignore PG_RECOVERY_FULL and PG_DEGRADED for mds-full
	qa: wait more time since there have many more pgs than before
	qa: do not multiple the full ratio twice
	qa: do not raise for kclient for _fsync test
	qa: use the pg autoscale mode to calcuate the pg_num
	qa: set the object_size to 1M
	qa: move the is_full() to parent class

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-07-28 10:34:12 -07:00
Patrick Donnelly
9941188116 qa: add nofs upgrade suite
This adds an upgrade suite to ensure that a Ceph cluster without a
CephFS file system does not blow up on upgrade (in particular, that the
MDSMonitor does not trip). This was developed to potentially reproduce
tracker 51673 but the actual cause for that issue was an old encoding
for the MDSMap which was obsoleted in Pacific. You must create a cluster
older than the FSMap (~Hammer or Infernalis) to reproduce. In any case,
this upgrade suite may be useful in the future so let's keep it!

Related-to: https://tracker.ceph.com/issues/51673
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-07-28 07:07:05 -07:00
Sage Weil
ecc0b4597b Merge PR #42406 into master
* refs/pull/42406/head:
	mgr/nfs: remove unused 'realm' arg for 'nfs export create rgw'
	doc/mgr/rook: update title
	doc/mgr/nfs: reference customizing ingress
	doc/mgr/nfs: add section for manual ganesha config; reframe
	doc/mgr/nfs: document ingress in more detail
	doc/mgr/nfs: typo
	doc/mgr/nfs: add note about incomplete ingress
	qa/suites/orch/cephadm: add rgw nfs export test
	mgr/cephadm: ingress: tolerate no daemons
	mgr/nfs: add --squash option to 'nfs export create rgw ...'
	mgr/nfs: use bucket owner creds for rgw bucket export
	mgr/cephadm: use new CEPH_IMAGE_TYPES for all daemons using ceph container image
	qa/tasks/python: simple task to run python code
	doc/mgr/nfs: revisions
	mgr/nfs/export: nicer exceptions on cap update

Reviewed-by: Varsha Rao <varao@redhat.com>
2021-07-27 14:11:56 -04:00
Xiubo Li
3456ff2628 qa: ignore PG_RECOVERY_FULL and PG_DEGRADED for mds-full
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2021-07-27 09:48:56 +08:00
Sage Weil
f8f7b86571 Merge PR #42292 into master
* refs/pull/42292/head:
	qa/suites/upgrade: log_to_journald=false

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-07-26 19:23:26 -04:00
Sage Weil
ac63ab6125 Merge PR #42489 into master
* refs/pull/42489/head:
	qa/suites/upgrade/pacific-x/stress-split: do not avoid_pacific_features

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-07-26 19:23:03 -04:00
Sage Weil
cd089ee74e qa/suites/orch/cephadm: add rgw nfs export test
Signed-off-by: Sage Weil <sage@newdream.net>
2021-07-26 16:23:17 -04:00
Patrick Donnelly
83d252cc30 qa: fold frag confs into conf/mds.yaml
These overrides are standard for all configurations. The config to
enable fragmentation is also long removed.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-07-26 07:14:38 -07:00
Neha Ojha
c9f8846b7f
Merge pull request #41907 from kamoltat/wip-ksirivad-progress-time-interval
pybind/mgr/progress: introduce 5 second sleep interval

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-07-21 16:53:38 -07:00
Casey Bodley
e3a6377099
Merge pull request #42196 from cbodley/wip-qa-rgw-rm-cephadm
qa/rgw: remove rgw_cephadm.yaml from rgw/singleton suite

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2021-07-21 13:14:35 -04:00
Casey Bodley
255293bd80
Merge pull request #42317 from cbodley/wip-39657
rgw multisite: metadata sync treats all errors as 'transient' for retry

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
2021-07-21 13:13:54 -04:00
Casey Bodley
1acee2ab76
Merge pull request #42361 from cbodley/wip-49747
qa/rgw: add failing tempest test to blocklist

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2021-07-21 13:10:30 -04:00
Casey Bodley
d373d07a9c
Merge pull request #42413 from galsalomon66/fix_s3tests_yaml
RGW:revert force-branch to be ceph-master (s3-tests)

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-07-20 08:04:59 -04:00
galsalomon66
7d4bc5a72c revert force-branch to be ceph-master
Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
2021-07-20 12:48:00 +03:00
Ernesto Puerta
64dbe17fdb
Merge pull request #42188 from votdev/issue_51408_motd
mgr/dashboard: Add configurable MOTD or wall notification

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Paul Cuzner <pcuzner@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: sebastian-philipp <NOT@FOUND>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2021-07-19 19:56:50 +02:00
Sage Weil
69a834d21f qa/suites/upgrade/pacific-x/stress-split: do not avoid_pacific_features
We can use pacific features when installing pacific.

Otherwise, we end up with the default keyring rule for client.admin,
which uses mode 0600, which makes teuthology jobs fail.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-07-16 11:23:36 -05:00
Casey Bodley
84909e7a50 qa/rgw: add failing tempest test to blocklist
Fixes: https://tracker.ceph.com/issues/49747

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-07-15 14:10:14 -04:00
Sage Weil
a07c24423a Merge PR #42343 into master
* refs/pull/42343/head:
	qa/suites/upgrade/pacific-x: do distro before tasks
	qa/suites/orch/cephadm/smoke: centos 8.2 -> 8.3

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-07-15 11:23:28 -04:00
Sage Weil
10cdb238d2 qa/suites/upgrade/pacific-x: do distro before tasks
Otherwise, we isntall new podman at the end, and the
container-selinux-policy package install triggers a bunch of selinux
errors.

Fixes: https://tracker.ceph.com/issues/50151
Signed-off-by: Sage Weil <sage@newdream.net>
2021-07-14 15:24:07 -05:00
Sage Weil
52f6989fff qa/suites/orch/cephadm/mgr-nfs-upgrade: add test for nfs migration
Signed-off-by: Sage Weil <sage@newdream.net>
2021-07-14 16:20:11 -04:00
Sage Weil
44d0f0f6b5 qa/suites/orch/cephadm/smoke: centos 8.2 -> 8.3
Signed-off-by: Sage Weil <sage@newdream.net>
2021-07-14 10:25:28 -05:00
Volker Theile
f7f163e75c mgr/dashboard: Add configurable MOTD or wall notification
Fixes: https://tracker.ceph.com/issues/51408

Signed-off-by: Volker Theile <vtheile@suse.com>
2021-07-14 10:48:49 +02:00
Kamoltat
5f33f2f6e0 mgr/test_progress.py: Delay recover in test_progress
Changes some the tests in teuthology to make
the test more deterministic.
Using:

`ceph osd set norecover` and
`ceph osd set nobackfill` when marking osds in
or out. As this will delay the recovery and make
sure it the test cases get the chance to check
that there is actually events poping up in
the progress module.

took out test_osd_cannot_recover from
tasks/mgr/test_progress.py since it is no longer
a relevant test case since recovery will get
triggered regardless if pg is unmoved.

Ignoring `OSDMAP_FLAGS` in teuthology
because we are using norecover and nobackfill
to delay the recovery process, therefore, it
will create a health warning and fails the
teuthology test.

Signed-off-by: Kamoltat <ksirivad@redhat.com>
2021-07-13 19:33:20 +00:00
Casey Bodley
cff6bf37e6 qa/rgw: run multisite tests with metadata sync error injection
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-07-13 14:17:00 -04:00
Sage Weil
58a90d1724 qa/suites/upgrade: log_to_journald=false
In 8b95c4b7c5 we set log_to_journald=false
in the cephadm config.  However, that's not present in pre-quincy builds,
which means that when we upgrade the new daemons start spamming the
teuthology.log.  Set this (with --force, since it's not valid pre-quincy)
in the config before we start the ugprade.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-07-12 13:32:51 -05:00
Mark Kogan
d9256eeab7
Merge pull request #36266 from amathuria/wip-rgw-d3n-cache-upstreaming
rgw: D3N Cache changes for Upstream
2021-07-07 12:11:47 +03:00
Kefu Chai
2582e6f89a
Merge pull request #42158 from tchaikov/wip-e2e-centos
qa: run e2e test on centos only

Reviewed-by: Laura Paduano <lpaduano@suse.com>
2021-07-07 16:37:08 +08:00
Casey Bodley
b7d4ea481c qa/rgw: remove rgw_cephadm.yaml from rgw/singleton suite
this cephadm task was merged without testing in
https://github.com/ceph/ceph/pull/39855/ and fails consistently with an
error in kernel.py. the teuthology issue
https://tracker.ceph.com/issues/50338 has gone unfixed for months, so
removing rgw_cephadm.yaml to clean up the rgw suite

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-07-06 11:01:20 -04:00
Ali Maredia
1a6255530f qa: add workunit for testing rgw datacache
Add a workunit for testing the rgw object cache
by using s3cmd to write objects and then
verify the objects in the cache.

Also move the 0-install.yaml file out of tasks and
into the main dir for the rgw/verify subsuite.

Signed-off-by: Ali Maredia <amaredia@redhat.com>

qa: d3n: add debug logs

Signed-off-by: Ali Maredia <amaredia@redhat.com>

rgw: s3n: qa: fix netstat search for rgw process

Signed-off-by: Mark Kogan <mkogan@redhat.com>
2021-07-04 14:18:06 +03:00
Kefu Chai
15fa32dc86 qa: run e2e test on centos only
this change is a follow up of 02b8b0f490,
which failed to remove the random facet for distro.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-07-02 23:06:27 +08:00
Ali Maredia
0bc9b868e2
Merge pull request #42023 from TRYTOBE8TME/wip-rgw-notification-tests
rgw: Modification in the usage of force-branch

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2021-07-01 12:44:20 -04:00
Kalpesh Pandya
9bb68a55ca rgw: Modification in the usage of force-branch
Signed-off-by: Kalpesh Pandya <kapandya@redhat.com>
2021-07-01 13:54:51 +05:30
Kefu Chai
e5c9315b11
Merge pull request #42084 from tchaikov/wip-49638
qa: run e2e test on centos only

Reviewed-by: Laura Paduano <lpaduano@suse.com>
2021-06-30 19:26:42 +08:00
Kefu Chai
812e58c597
Merge pull request #42013 from ronen-fr/wip-ronenf-scrubs-config
qa/suites/rados: add simultaneous scrubs to the thrasher

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-06-29 16:21:52 +08:00
Kefu Chai
02b8b0f490 qa: run e2e test on centos only
it's a regression introduced by the restrcuture of the test suites,
let's pin the test to CentOS8.

See-also: https://tracker.ceph.com/issues/49638
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-29 13:09:53 +08:00
Sage Weil
4537b14b34 Merge PR #41574 into master
* refs/pull/41574/head:
	qa/tasks/vstart_runner: add LocalCluster.run
	qa/tasks/cephfs/test_nfs: fiddle with sudo
	mgr/nfs/export: some cleanup, minor refactoring
	mgr/nfs/cluster: remove unused @cluster_setter
	nfs/mgr: fix help message case
	doc/cephfs/fs-nfs-export: add note about export update behavior
	mgr/nfs: move user create/delete into helper
	mgr/nfs: refactor _delete_user helper
	mgr/nfs: refactor create_export_from_dict() helper
	mgr/nfs: keep 'nfs export get' around for backward-compat
	mgr/nfs: rename method
	qa/tasks/cephfs/test_nfs: test new export via apply
	doc/cephfs/fs-nfs-export: be consistent with cluster_id and _ vs -
	mgr/nfs: addr -> client_addr for 'nfs export create ...'
	mgr/nfs: fix tests
	mgr/nfs: 'nfs export get' -> 'nfs export info'
	mgr/nfs: binding -> pseudo_path
	mgr/nfs: more revisions based on review
	mgr/nfs: adjust NFSExceptoin errno arg
	doc/cephfs: update 'nfs export {get,apply}' docs
	mgr/nfs: merge FSExport back into ExportMgr
	doc/radosgw/nfs: document mgr/nfs way to add/remove rgw exports
	mgr/nfs: merge 'nfs export {update,import}' -> 'nfs export apply'
	mgr/nfs: test export creation and list
	mgr/nfs: test export_update (+ fixes)
	mgr/nfs: test Export.validate(); several fixes
	mgr/nfs: test that export <-> block+dict conversions go both ways
	mgr/nfs: clean up test a bit
	mgr/nfs/export: fix export validation
	mgr/nfs/export: fix tests
	mgr/nfs: handle option addr/client block in create_export()
	mgr/nfs: allow multiple addrs for new exports
	mgr/nfs: fix/finish rgw export
	mgr/nfs/module: clusterid -> cluster_id
	mgr/nfs/export: fix export_update_1 to type check
	mgr/nfs/cluster: fix type error
	mgr/nfs/export: wrap long lines
	mgr/nfs: ExportMgr._delete_export only works for cephfs for now
	mgr/nfs: Remove pool_ns from NFSCluster
	mgr/nfs: Remove ExportMgr.rados_namespace
	mgr/nfs: flake8
	mgr/nfs: Add type checking
	mgr/nfs: Add __eq__ method to Export
	mgr/nfs: Add some compatibility to mgr/dashboard
	mgr/nfs: Fix whitespace handling
	mgr/nfs: Copy unit tests from mgr/dashboard
	mgr/nfs: partially implement rgw export support
	mgr/nfs: abstract FSAL; add RGWFSAL
	mgr/nfs: refactor to merge 'update' and 'import' code
	mgr/nfs: add 'nfs export import' command
	mgr/nfs: refactor 'nfs export update' and export validation
	mgr/nfs: fix _fetch_export to distinguish between clusters
	mgr/nfs: move export ganesha conf translation into caller
	mgr/nfs: name nfs cephfs client key 'nfs.{cluster_id}.{export_id}'
	mgr/nfs: add --addr to 'nfs export create'
	mgr/nfs: add --squash to 'nfs export create'
	mgr/nfs/export_utils: include false but non-None items in config
	vstart.sh: enable nfs module
	mgr/cephadm: nfs: drop attr_expiration_time from top-level config
	mgr/cephadm: remove Dir_Chunk = 0

Reviewed-by: Michael Fritch <mfritch@suse.com>
2021-06-26 10:41:27 -04:00
Kefu Chai
29064f1bf8
Merge pull request #41937 from liewegas/mgr-crash
mgr: generate crash dumps for Python exceptions in mgr modules

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-26 22:18:14 +08:00
Sage Weil
3edc04a46b qa/suites/rados/mgr: whitelist module crash during selftest
One of the selftests triggers an exception from serve().

Signed-off-by: Sage Weil <sage@newdream.net>
2021-06-25 13:48:45 -04:00
Ronen Friedman
d232c4e8d8 qa/suites/rados: add simultaneous scrubs (multiple options) to the thrasher
Setting osd-max-scrubs to either 2 or 3.

Triggered by https://tracker.ceph.com/issues/50346

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2021-06-24 18:53:50 +03:00
Sage Weil
fe9963b03c qa/suites/rados/dashboard: fix e2e test
Move roles into task yaml.  Rename e2e.

Fixes: https://tracker.ceph.com/issues/51292
Signed-off-by: Sage Weil <sage@newdream.net>
2021-06-23 09:54:40 -05:00
Sage Weil
5a2382a5a8 mgr/nfs: binding -> pseudo_path
Signed-off-by: Sage Weil <sage@newdream.net>
2021-06-21 14:13:15 -04:00
Patrick Donnelly
b946d5a6b2
qa: use centos latest for fs:upgrade
Fixes: https://tracker.ceph.com/issues/51250
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-16 12:16:36 -07:00
Sage Weil
091a32e130 Merge PR #41844 into master
* refs/pull/41844/head:
	qa/suites/orch/cephadm/dashboard: remove remaining bits

Reviewed-by: Michael Fritch <mfritch@suse.com>
2021-06-15 15:42:26 -04:00
Patrick Donnelly
03674f5197
Merge PR #41821 into master
* refs/pull/41821/head:
	qa: specify distro for fs:bugs

Reviewed-by: Rishabh Dave <ridave@redhat.com>
2021-06-15 10:33:38 -07:00
Sage Weil
ebb5a3f0bc qa/suites/orch/cephadm/dashboard: remove remaining bits
Signed-off-by: Sage Weil <sage@newdream.net>
2021-06-14 13:00:45 -05:00
Sage Weil
9074e87611 Merge PR #41827 into master
* refs/pull/41827/head:
	qa: move dashboard e2e from cephadm -> rados suite

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2021-06-14 09:11:04 -04:00
Sage Weil
ac05b3568f qa: move dashboard e2e from cephadm -> rados suite
This test fails ~20% of the time.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-06-12 07:52:54 -05:00
Patrick Donnelly
d6c66f3fa6
qa,pybind/mgr: allow disabling .mgr pool
This is mostly for testing: a lot of tests assume that there are no
existing pools. These tests relied on a config to turn off creating the
"device_health_metrics" pool which generally exists for any new Ceph
cluster. It would be better to make these tests tolerant of the new .mgr
pool but clearly there's a lot of these. So just convert the config to
make it work.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-06-11 19:35:17 -07:00