Commit Graph

1991 Commits

Author SHA1 Message Date
Sage Weil
9ab9cc26e2 Merge PR #41007 into master
* refs/pull/41007/head:
	qa/tasks/cephfs/test_nfs: fix info test
	doc/cephfs/fs-nfs-exports: document --ingress --virtual-ip
	mgr/nfs: move ingress vs virtual_ip check to cluster interface
	PendingReleaseNotes: clarify deprecated
	PendingReleaseNotes: note breaking CLI changes
	doc/cephadm/nfs: document nfs+ingress
	qa/suites/rados/cephadm/smoke-roleless: test nfs, nfs + ingress
	mgr/nfs: take --ingress argument to 'nfs cluster create'
	mgr/cephadm: adjust debug output for device refresh
	mgr/cephadm: ingress: fix log msg
	mgr/cephadm: fix logging of config/placement errors
	common/options: enable nfs module for new clusters
	cephadm: --stop-signal=SIGTERM
	mgr/orchestrator: default nfs pool, namespaces
	mgr/cephadm: nfs: create pool if it doesn't yet exist
	doc/cephadm/nfs: update
	mgr/nfs: change 'nfs cluster info'
	mgr/nfs: take optional virtual_ip for deploying ingress
	mgr/nfs: remove 'nfs cluster update'
	mgr/nfs: factor out ganesha pool creation
	mgr/nfs: delete -> rm for CLI
	mgr/nfs: add some type annotations
	python-common: fix IngressSpec yaml dump
	mgr/cephadm: ingress: remove eth0 default
	qa/tasks/cephadm: allow mounting volumes in shell
	cephadm: add -v arg to shell
	qa/tasks/vip: add 'vip.exec' task
	mgr/orchestrator: add --port arg to 'orch apply nfs'
	mgr/cephadm: nfs: add purge
	mgr/cephadm: ingress: support nfs
	mgr/cephadm: do not reconfigure daemons on deleted services
	mgr/cephadm: nfs: shell out to rados tool for conf creation
	mgr/cephadm: nfs: add rank to grace file from mgr module
	mgr/cephadm: nfs: bind ganesha to appropriate ip:port
	mgr/cephadm: enable ranked daemons for nfs
	mgr/cephadm: support creation of daemons with ranks
	mgr/cephadm: make _plan show removed daemon names
	mgr/cephadm/schedule: assign/map ranks
	mgr/cephadm: add rank[_generation] properties
	mgr/cephadm/inventory: store optional rank_map along with specs
	mgr/cephadm: include service_name is generated DaemonDescription
	mgr/orchestrator: include service_name in DaemonDescription dump
	mgr/cephadm/inventory: fix deleted check
	mgr/cephadm: simplify
	mgr/cephadm/schedule: make placement shuffle deterministic
	mgr/cephadm: document CephadmService flags

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Varsha Rao <varao@redhat.com>
2021-05-25 16:17:44 -04:00
Sage Weil
e2db373698 qa/suites/rados/cephadm/smoke-roleless: test nfs, nfs + ingress
Still missing a full client mount test, though!

Signed-off-by: Sage Weil <sage@newdream.net>
2021-05-25 10:15:45 -04:00
Neha Ojha
9241144022
Merge pull request #41487 from neha-ojha/wip-toc
qa/suites/rados/thrash-old-clients: remove luminous and mimic and use centos_latest

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-05-24 14:44:18 -07:00
Neha Ojha
ece5ed1ac9
Merge pull request #41486 from neha-ojha/wip-49139-new
qa: use ubuntu_latest for perf suites and remove cosbench workloads

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-05-24 12:53:46 -07:00
Yuri Weinstein
9305b8c7ea
Merge pull request #41504 from yuriw/wip-yuriw-master
qa/tests - removed ref to 18.04 distro as it's not supported on master+

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-05-24 12:21:00 -07:00
Neha Ojha
30eb7467aa qa/suites/rados/thrash-old-clients: use centos_latest.yaml
use centos_latest instead of bionic because this is only common
distro for which we build packages for nautilus and above.

Signed-off-by: Neha Ojha <nojha@redhat.com>
2021-05-24 18:34:04 +00:00
Yuri Weinstein
d8794db5b0 qa/tests - removed ref to 18.04 distro as it's not supported on master+
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2021-05-24 10:40:23 -07:00
Sage Weil
04ce0496e8 Merge PR #41451 into master
* refs/pull/41451/head:
	qa/suites/rados: include rook test in rados

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
2021-05-24 13:30:27 -04:00
Matt Benjamin
5276c2a3d0
Merge pull request #40910 from galsalomon66/update_s3select_submodule_and_s3tests_head_10apr
rgw/s3select: update s3select submodule to last commit, (new features), update for test coverage(s3test)
2021-05-21 19:17:53 -04:00
Neha Ojha
b7237c9e2d qa/suites/rados/thrash-old-clients: remove luminous and mimic
We support N-3 client versions.

Signed-off-by: Neha Ojha <nojha@redhat.com>
2021-05-21 22:03:41 +00:00
Neha Ojha
fd350fd015 qa: remove cosbench workloads from perf suites
Due to https://tracker.ceph.com/issues/49139

Signed-off-by: Neha Ojha <nojha@redhat.com>
2021-05-21 20:17:11 +00:00
gal salomon
7c9ac4ed3d update to s3select/master (new features)
Signed-off-by: gal salomon <gal.salomon@gmail.com>

force-branch to s3test/master

Signed-off-by: gal salomon <gal.salomon@gmail.com>

update of s3select (== -> =)

Signed-off-by: gal salomon <gal.salomon@gmail.com>

update of s3select (== -> =)

Signed-off-by: gal salomon <gal.salomon@gmail.com>

closing documentation gaps between previous and current functionalities(WIP)

Signed-off-by: gal salomon <gal.salomon@gmail.com>

Update s3select.rst

additional features

Update s3select.rst

Signed-off-by: gal salomon <gal.salomon@gmail.com>

Update s3select.rst

editorial
Signed-off-by: gal salomon <gal.salomon@gmail.com>

Update s3select.rst

editorial
Signed-off-by: gal salomon <gal.salomon@gmail.com>

Update s3select.rst

adding cast to bool
Signed-off-by: gal salomon <gal.salomon@gmail.com>

editorial; more description for substring and trim

Signed-off-by: gal salomon <gal.salomon@gmail.com>
2021-05-21 21:58:20 +03:00
Neha Ojha
5957d1797a qa: use ubuntu_latest for perf suites
Signed-off-by: Neha Ojha <nojha@redhat.com>
2021-05-21 17:39:44 +00:00
Sage Weil
5db5c8c292 qa/suites/rados: include rook test in rados
This just to make sure we don't break mgr/orchestrator.

Note that we already symlink ../orch/cephadm, so this makes rados
include all of orch/.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-05-20 12:41:52 -05:00
Sage Weil
b9d8dc483a Merge PR #41286 into master
* refs/pull/41286/head:
	qa/suites/orch/rook: disable centos for now
	qa/suites/orch/rook/smoke: initial smoke suite
	qa/tasks/rook: ROOK_HOSTPATH_REQUIRES_PRIVILEGED=true on centos
	qa/tasks/rook: simplify shutdown
	qa/tasks/rook: archive logs
	qa/tasks/rook: more orderly cluster teardown
	qa/tasks/rook: deploy ceph via rook on top of kubernetes
	qa/tasks/kubeadm: install kubernetes with kubeadm
	qa/suites: move rados/cephadm -> orch/cephadm; symlink
	qa/tasks/cephadm: add whitespace between functions
	qa/tasks/cephadm: clean up ctx.manager setup

Reviewed-by: Sébastien Han <seb@redhat.com>
2021-05-19 07:55:30 -04:00
Sage Weil
ad8096e0a0 qa/suites/orch/rook: disable centos for now
We are hitting an selinux denial:

type=AVC msg=audit(1621353663.595:6016): avc: denied { write } for pid=28812 comm="cephcsi" name="pids.max" dev="cgroup" ino=2933 scontext=system_u:system_r:container_t:s0:c43,c177 tcontext=system_u:object_r:cgroup_t:s0 tclass=file permissive=1

Signed-off-by: Sage Weil <sage@newdream.net>
2021-05-18 15:19:16 -05:00
Sage Weil
ab33ca3655 qa/suites/orch/rook/smoke: initial smoke suite
- os: 18.04, 20.04, centos, centos+kubic
- nodes: 1 or 3
- rook: 1.6.2 or master
- k8s: 1.21 always (for now)
- net: calico always (flannel + centos is wonky)
- workload: none, rados bench

Signed-off-by: Sage Weil <sage@newdream.net>
2021-05-18 15:19:16 -05:00
Sage Weil
5b25f8a2e5 qa/suites: move rados/cephadm -> orch/cephadm; symlink
Move cephadm under orch/ top-level suite.  Symlink so that we
still include it in a rados run.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-05-18 11:14:14 -05:00
Casey Bodley
46c0292870 rgw: remove the civetweb frontend from src and qa
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-05-17 15:00:31 -04:00
Neha Ojha
d3692a3e92
Merge pull request #40016 from neha-ojha/wip-default-mclock
use mclock_scheduler as the default scheduler

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sridhar Seshasayee <sseshasa@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Sunny Kumar <sunkumar@redhat.com>
2021-05-07 08:08:39 -07:00
Neha Ojha
c8e48c5c25 qa/suites/rados/standalone: remove mon_election symlink
The standalone tests need parameters to be passed as ceph_args to
override defaults.

This was just doubling the number of standalone tests being run in each rados
run with no effect!

Signed-off-by: Neha Ojha <nojha@redhat.com>
2021-05-07 00:42:53 +00:00
Sridhar Seshasayee
cc1fc98ea4 qa/suites/rados/mgr/tasks/progress: use high_recovery_ops for faster recovery
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
2021-05-06 17:54:38 +00:00
Kefu Chai
fee3028abc
Merge pull request #41014 from smithfarm/wip-mempool-cacheline-49781
qa: verify the benefits of mempool cacheline optimization

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-04-30 19:36:17 +08:00
Loïc Dachary
7fe0ac7c11 qa: verify the benefits of mempool cacheline optimization
There already is a test to verify the mempool sharding works, in the sense that
it uses at least half of the variables available to count the number of
allocated objects and their total size. This new test verifies that, with
sharding, object counting is at least twice faster than without sharding. It
also collects cacheline contention data with the perf c2c tool. The manual
analysis of this data shows the optimization gain is indeed related to cacheline
contention.

Fixes: https://tracker.ceph.com/issues/49896

Signed-off-by: Loïc Dachary <loic@dachary.org>
2021-04-30 12:11:13 +08:00
Josh Durgin
0e273e6760
Merge pull request #40593 from ideepika/wip-new-testing-params
qa/config/rados: add dispatch delay testing params

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sam Just <sjust@redhat.com>
2021-04-28 11:26:58 -07:00
Sage Weil
75480f52e4 Merge PR #40941 into master
* refs/pull/40941/head:
	qa/suites/rados/cephadm/smoke-roleless: test client-keyring
	qa/tasks/cephadm.py: adjust client.admin key mode; place on all hosts
	cephadm: distribute client.admin keyring+conf to label:_admin on bootstrap
	doc/cephadm: document the default 'admin' label
	mgr/cephadm: 'ceph orch client-keyring ...' commands to manage keyring files
	mgr/cephadm: reimplement ceph.conf pushing
	mgr/cephadm: use _write_remote_file for ceph.conf
	mgr/cephadm: _write_remote_file helper
	mgr/cephadm: add placementspec for which hosts get ceph.conf

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Adam King <adking@redhat.com>
2021-04-28 14:26:35 -04:00
Sage Weil
e41931d042 qa/suites/rados/cephadm/smoke-roleless: test client-keyring
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-27 18:29:50 -04:00
Sage Weil
b0dcaf2cfa qa/tasks/cephadm.py: adjust client.admin key mode; place on all hosts
Except during upgrades, since it is not supported there.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-27 18:29:50 -04:00
Yuri Weinstein
2771cb0887
Merge pull request #41006 from yuriw/wip-yuriw-simlinks-master
qa/tests: changed simlink to upgrade/parallel only

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-04-23 09:49:07 -07:00
Yuri Weinstein
b6c84d5621 qa/tests: changed simlink to upgrade/parallel only
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2021-04-23 08:20:01 -07:00
Neha Ojha
839ce6de15 qa/suites/upgrade/pacific-x/rgw-multisite: fix install version
Before:

2021-04-22T06:20:58.260 INFO:teuthology.task.install:Ceph rpm upgrade from
16.2.1-103.gb54cefd0.el8 to 16.2.1-103.gb54cefd0

After:

2021-04-22T21:08:16.565 INFO:teuthology.task.install:Ceph rpm upgrade from
16.2.1-105.g540c2494.el8 to 17.0.0-3515.ga5b16446

Fixes: https://tracker.ceph.com/issues/50388
Signed-off-by: Neha Ojha <nojha@redhat.com>
2021-04-22 22:33:28 +00:00
Deepika Upadhyay
b2c2a4326c qa/config/rados: add dispatch delay testing params
these parameters have proven to catch some of the uncaught bugs such as:
https://tracker.ceph.com/issues/48417, adopting them will help in
preventing more such hard to debug bugs.

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-04-19 12:28:18 +05:30
Sage Weil
9c1f128885 qa/suites/rados/cephadm/smoke-*: use cephadm.wait_for_service
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 16:00:31 -04:00
Sage Weil
c20323d114 qa/suites/rados/cephadm/smoke-singlehost: test --single-host-defaults
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 16:00:31 -04:00
Sage Weil
16b30f2858 qa/suites/rados/cephadm/smoke-*: use cephadm.wait_for_service
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 09:49:45 -05:00
Sage Weil
3ff3f697b4 qa/suites/rados/cephadm/smoke-roleless: test rgw-ingress
Test this properly by downing each rgw and haproxy in turn and ensuring
that things remain up.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 09:49:45 -05:00
Sage Weil
7e16bf3468 qa/suites/rados/cephadm/smoke-roleless: add rgw-ingress test case
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 07:36:47 -05:00
Kefu Chai
d54605c7e6
Merge pull request #40837 from liewegas/cephadm-qa-shorten-thrash
qa/suites/rados/thrash: shorten radosbench

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-04-16 12:42:05 +08:00
Casey Bodley
dbede38c7e
Merge pull request #40821 from cbodley/wip-50218
qa/rgw: use symlinks to specify distro

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2021-04-14 14:45:00 -04:00
Sage Weil
babbab14f4 qa/suites/rados/thrash: shorten radosbench
This is the longest of the thrash workloads; reducing it will bring
this test in line with the others (<= 45 min).

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-13 17:42:21 -05:00
Casey Bodley
8cb94edcb1 qa/rgw: rgw/crypt uses supported-random-distro$
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-04-13 10:44:05 -04:00
Ilya Dryomov
7eb9c5ddb2 Merge branch 'master' into wip-unauthorized-gids
Sync up with master up to commit 3d8e73b266 ("Merge pull request
#40731 from tchaikov/wip-yamlize-options").  Specifically, bring in
src/common/options.cc yamlization and move new auth-related options
into src/common/options/global.yaml.in.

Conflicts:
	src/common/options.cc
	src/common/options/global.yaml.in

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2021-04-13 15:42:06 +02:00
Sage Weil
dcd90a1c8d Merge PR #40626 into master
* refs/pull/40626/head:
	qa/suites/rados/objectstore: separate store_test tests
	qa/standalone: split osd/ into 2 directories

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-04-12 22:38:49 -04:00
Sage Weil
d4ac4f3b78 qa/suites/rados/objectstore: separate store_test tests
This takes 5 hours currently.

- Separate out filestore and memstore into separate task (~1 hr)
- Split bluestore into -a and -b (a tests exclude SynethicMatrixC,
  b tests include it)

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-12 16:27:29 -05:00
Ilya Dryomov
ca822b9a8d
Merge pull request #40576 from idryomov/wip-no-cephxv2-for-unmap
qa/suites/krbd: don't require CEPHX_V2 for unmap subsuite

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2021-04-12 21:23:06 +02:00
Kefu Chai
8b879ca72c qa/suites/upgrade/pacific-x/rgw-multisite: specify distro
instead of using the default / existing distro, specify the distro for
testing. this change prevents us from using bionic for testing the
builds. this has two consequences:

* so we are one step closer to a non-bionic world.
* avoid building packages with PPA repo which *might* introduce runtime
  dependencies on 3rd party runtimes provided by PPA repo.

see also: https://tracker.ceph.com/issues/50218

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-12 12:56:35 -04:00
Casey Bodley
6262cfa41f qa/rgw: reenable valgrind in rgw/website
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-04-12 12:31:26 -04:00
Casey Bodley
bbefbd3146 qa/rgw: rgw/tempest uses ubuntu_latest.yaml
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-04-12 12:30:01 -04:00
Casey Bodley
0a4437ba6f qa/rgw: rgw/multifs uses ubuntu_latest.yaml
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-04-12 12:28:55 -04:00
Casey Bodley
09c93945a7 qa/rgw: rgw/thrash uses ubuntu_latest.yaml
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-04-12 12:28:55 -04:00