Commit Graph

132 Commits

Author SHA1 Message Date
Christopher Hoffman
4a749891c8 qa/tasks: Include stderr on tasks badness check.
Make sure that first_in_ceph_log() doesn't return None (which is
treated as success/"no badness" by the caller) if the cluster log file
is missing.

Fixes: https://tracker.ceph.com/issues/57864
Co-authored-by: Ilya Dryomov <idryomov@gmail.com>
Signed-off-by: Christopher Hoffman <choffman@redhat.com>
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 5d92965b31)
2025-01-17 19:35:36 +01:00
Ilya Dryomov
0a7172a21b qa/tasks/cephadm: drop --cluster and sudo in create_rbd_pool()
The cluster (name) is already specified in the arguments passed to
_shell() and this command doesn't need privileges.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 26cee25873)
2024-07-11 17:23:50 +02:00
Ilya Dryomov
cdf1f9f4d0 qa/tasks/cephadm: don't wait for OSDs in create_rbd_pool()
This fails because teuthology.wait_until_osds_up() wants to use
adjust-ulimits wrapper which isn't available in "cephadm shell"
environment.  The whole thing is also redundant because cephadm task
is supposed to wait for OSDs to come up earlier, in ceph_osds().

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 0bda782952)
2024-07-11 17:23:45 +02:00
Ilya Dryomov
368e39819f qa/tasks: initialize rbd pool instead of just tagging it
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit fcc440da9a)
2024-07-11 17:23:42 +02:00
Adam King
9c18e882ba qa/cephadm: add ca signed key to cephadm task
To allow bootstrapping a cluster using a CA signed
key instead of the standard pubkey authentication.
Will allow explicit testing of this as we add support
for it

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit bef9617c51)
2023-08-23 17:44:52 -04:00
Adam King
4d0cc762d7
Merge pull request #48428 from zmc/cephadm-smoke-small
qa/suites: Add orch/cephadm/smoke-small suite

Reviewed-by: Adam King <adking@redhat.com>
2023-01-24 09:08:44 -05:00
Adam King
8086a6f3a6 qa/tasks/cephadm.py: fix pulling cephadm from git.ceph.com
The old way of using the git archive doesn't seem
to work anymore so trying another approach

Was getting "fatal: operation not supported by protocol"
when trying the git archive

Fixes: https://tracker.ceph.com/issues/58222

Signed-off-by: Adam King <adking@redhat.com>
2022-12-12 12:40:10 -05:00
Zack Cerza
b72fcf11cd qa/tasks/cephadm: Add no_cgroups_split option
This will be useful for deployments based on rootless containers

Signed-off-by: Zack Cerza <zack@redhat.com>
2022-10-10 16:07:42 -06:00
Zack Cerza
af67a7152c tasks/cephadm: Add 'osd_method' config item
Since the default of None gets interpreted as 'lvm', the only useful
value would be 'raw'. This will enable us to use loop devices inside
containers for OSDs.

Signed-off-by: Zack Cerza <zack@redhat.com>
2022-09-26 15:35:08 -06:00
Zack Cerza
74c3446c25 cephadm: Avoid TypeError in download_cephadm()
Signed-off-by: Zack Cerza <zack@redhat.com>
2022-09-26 15:35:08 -06:00
Michael Fritch
95ead7401f qa/tasks/cephadm: conditionally pull cephadm binary from chacra
When a specific (older) version of cephadm is needed for a test, pull
it from chacra. If we need to bootstrap an older version of ceph, we
pull an older "single-script" binary using the older method.

Co-authored-by: Michael Fritch <mfritch@suse.com>
Co-authored-by: John Mulligan <jmulligan@redhat.com>
Signed-off-by: Michael Fritch <mfritch@suse.com>
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-09-13 12:17:20 -04:00
Sebastian Wagner
1ddd179054 qa/tasks: Drop downloading cephadm from the git source tree
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
2022-09-13 12:17:20 -04:00
Milind Changire
bf83eaa4e7 qa: enhancement for subvol creation and mounting
Fixes: https://tracker.ceph.com/issues/54317
Signed-off-by: Milind Changire <mchangir@redhat.com>
2022-04-07 14:15:56 +05:30
Zack Cerza
0f1118e228 cephadm: Make hostname change idempotent
Inside a container, we may already have a short hostname - and also lack
the privileges to change it.

Signed-off-by: Zack Cerza <zack@redhat.com>
2022-02-16 12:13:05 -07:00
Patrick Donnelly
27c1110129
qa/tasks/cephadm: setup CephManager when OSDs are provisioned
The Filesystem object may use this when configuring EC data pools at
file system creation (via a FuseMount).

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2022-02-02 10:44:34 -05:00
Patrick Donnelly
2436405c5d
qa/tasks/cephadm: setup file system if MDS are provisioned
This is the same behavior/code as what the ceph task does.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2022-02-02 10:44:34 -05:00
Sage Weil
9d50154a93 qa/tasks/cephadm: pull image to all hosts in parallel
This doesn't affect bootstrap, but it does mean we avoid any delay
the first time we cephadm.shell on some non-boostrap host.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-11-25 07:52:56 -06:00
Sage Weil
3a110f6c00 qa/tasks/cephadm: add hosts via mon remote
If we use a new remote for each shell command, we end up waiting
for the image to pull on every host in sequence.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-11-25 07:52:56 -06:00
Sage Weil
0e40064d31 qa/tasks/cephadm: use shortname for remote directory
This aligns with what the ceph and syslog tasks do.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-11-25 07:52:56 -06:00
Sage Weil
689d7ceabd qa/tasks/cephadm: deploy no more than 5 mons in roleless mode
Signed-off-by: Sage Weil <sage@newdream.net>
2021-11-25 07:52:55 -06:00
Deepika Upadhyay
cbd2c71398 qa/tasks: adapt ceph_iscsi.py task to ceph_iscsi_client
* we use setup_iscsi_client.py to deploy iscsi client services,
  configuring intiator and mulitpath this is done by qa task
  ceph_iscsi_client
* qa/cephadm: adds remotes ip addresses to iscsi gateway,
* rename poolname: iscsi >> datapool, which we usually use for tests and
  expresses type of pool more clearly.

Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-10-18 13:21:50 +05:30
Sage Weil
dda15a7924 qa/tasks/cephadm: wait for osds to start explicitly
Signed-off-by: Sage Weil <sage@newdream.net>
2021-10-08 16:06:28 -05:00
Sage Weil
d3c9486ed9 qa/tasks/cephadm: if no osd roles, --all-available-devices
Signed-off-by: Sage Weil <sage@newdream.net>
2021-10-08 16:06:28 -05:00
Sage Weil
9ab9cc26e2 Merge PR #41007 into master
* refs/pull/41007/head:
	qa/tasks/cephfs/test_nfs: fix info test
	doc/cephfs/fs-nfs-exports: document --ingress --virtual-ip
	mgr/nfs: move ingress vs virtual_ip check to cluster interface
	PendingReleaseNotes: clarify deprecated
	PendingReleaseNotes: note breaking CLI changes
	doc/cephadm/nfs: document nfs+ingress
	qa/suites/rados/cephadm/smoke-roleless: test nfs, nfs + ingress
	mgr/nfs: take --ingress argument to 'nfs cluster create'
	mgr/cephadm: adjust debug output for device refresh
	mgr/cephadm: ingress: fix log msg
	mgr/cephadm: fix logging of config/placement errors
	common/options: enable nfs module for new clusters
	cephadm: --stop-signal=SIGTERM
	mgr/orchestrator: default nfs pool, namespaces
	mgr/cephadm: nfs: create pool if it doesn't yet exist
	doc/cephadm/nfs: update
	mgr/nfs: change 'nfs cluster info'
	mgr/nfs: take optional virtual_ip for deploying ingress
	mgr/nfs: remove 'nfs cluster update'
	mgr/nfs: factor out ganesha pool creation
	mgr/nfs: delete -> rm for CLI
	mgr/nfs: add some type annotations
	python-common: fix IngressSpec yaml dump
	mgr/cephadm: ingress: remove eth0 default
	qa/tasks/cephadm: allow mounting volumes in shell
	cephadm: add -v arg to shell
	qa/tasks/vip: add 'vip.exec' task
	mgr/orchestrator: add --port arg to 'orch apply nfs'
	mgr/cephadm: nfs: add purge
	mgr/cephadm: ingress: support nfs
	mgr/cephadm: do not reconfigure daemons on deleted services
	mgr/cephadm: nfs: shell out to rados tool for conf creation
	mgr/cephadm: nfs: add rank to grace file from mgr module
	mgr/cephadm: nfs: bind ganesha to appropriate ip:port
	mgr/cephadm: enable ranked daemons for nfs
	mgr/cephadm: support creation of daemons with ranks
	mgr/cephadm: make _plan show removed daemon names
	mgr/cephadm/schedule: assign/map ranks
	mgr/cephadm: add rank[_generation] properties
	mgr/cephadm/inventory: store optional rank_map along with specs
	mgr/cephadm: include service_name is generated DaemonDescription
	mgr/orchestrator: include service_name in DaemonDescription dump
	mgr/cephadm/inventory: fix deleted check
	mgr/cephadm: simplify
	mgr/cephadm/schedule: make placement shuffle deterministic
	mgr/cephadm: document CephadmService flags

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Varsha Rao <varao@redhat.com>
2021-05-25 16:17:44 -04:00
sunilkumarn417
364fb5899b qa/tasks/cephadm: Include bootstrap registry options for downstream
- registry-url, registry-username and registry-password bootstrap options are
supported now. This is needed to access monitoring service container images.
- usage of RHEL distribution based cephadm in download_cephadm task.

Signed-off-by: sunilkumarn417 <sunnagar@redhat.com>
2021-05-24 12:38:54 +05:30
Sage Weil
b711a75277 qa/tasks/cephadm: allow mounting volumes in shell
Signed-off-by: Sage Weil <sage@newdream.net>
2021-05-19 08:43:14 -04:00
Sage Weil
b6d1536a67 qa/tasks/cephadm: add whitespace between functions
Signed-off-by: Sage Weil <sage@newdream.net>
2021-05-18 10:05:16 -05:00
Sage Weil
267cefc040 qa/tasks/cephadm: clean up ctx.manager setup
Signed-off-by: Sage Weil <sage@newdream.net>
2021-05-14 15:02:13 -05:00
Sage Weil
75480f52e4 Merge PR #40941 into master
* refs/pull/40941/head:
	qa/suites/rados/cephadm/smoke-roleless: test client-keyring
	qa/tasks/cephadm.py: adjust client.admin key mode; place on all hosts
	cephadm: distribute client.admin keyring+conf to label:_admin on bootstrap
	doc/cephadm: document the default 'admin' label
	mgr/cephadm: 'ceph orch client-keyring ...' commands to manage keyring files
	mgr/cephadm: reimplement ceph.conf pushing
	mgr/cephadm: use _write_remote_file for ceph.conf
	mgr/cephadm: _write_remote_file helper
	mgr/cephadm: add placementspec for which hosts get ceph.conf

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Adam King <adking@redhat.com>
2021-04-28 14:26:35 -04:00
Sage Weil
b0dcaf2cfa qa/tasks/cephadm.py: adjust client.admin key mode; place on all hosts
Except during upgrades, since it is not supported there.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-27 18:29:50 -04:00
Sage Weil
a5b1644660 Merge PR #40913 into master
* refs/pull/40913/head:
	qa/tasks/cephadm: fix ctx archive check for teuthology

Reviewed-by: Sage Weil <sage@redhat.com>
2021-04-22 09:50:02 -04:00
Deepika Upadhyay
ec2a402f55 qa/tasks/cephadm: fix ctx archive check for teuthology
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-04-19 19:48:47 +05:30
Sage Weil
c20323d114 qa/suites/rados/cephadm/smoke-singlehost: test --single-host-defaults
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 16:00:31 -04:00
Sage Weil
65f2848e97 qa/tasks/cephadm: ignore --keep-logs failure
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 14:12:49 -05:00
Sage Weil
97c143c344 qa/tasks/cephadm: use yaml.dump_all()
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 10:30:05 -05:00
Sage Weil
deec9074bb qa/tasks/cephadm: tear down clsuter before gathering logs
We dont' always stop all services, because teuthology doesn't know about
things it didn't start.  Use rm-cluster to tear things down, but do not
remove the logs themselves.  After we get logs, we'll clean up completely.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 09:49:45 -05:00
Sage Weil
ced2f7fe4a qa/tasks/cephadm: add wait_for_service
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 09:49:45 -05:00
Sage Weil
bb825157dc qa/tasks/cephadm: allow skip_monitor_stack=true
(Useful for roleless when we want to go faster)

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 09:49:45 -05:00
Sage Weil
0d561e2741 qa/tasks/cephadm: do subst_vip for cephadm.shell and .apply
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 09:49:45 -05:00
Sage Weil
0803957695 qa/tasks/cephadm: shell: take 'all-roles' or 'all-hosts'
'all' is ambiguous

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 07:36:47 -05:00
Sage Weil
779af3da6f qa/tasks/cephadm: let cephadm.shell take string or list
Make it a bit more forgiving.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-16 07:36:47 -05:00
Sage Weil
05b3ce2585 qa/tasks/cephadm: add apply() method/task
Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-05 11:44:22 -05:00
Sage Weil
a17593a177 qa/tasks/cephadm: use 'orch apply mon' to deploy mons
The 'orch daemon add ...' command is not idempotent and can cause
duplicate (and failing) attempts to add the same mon.

Maintain the ability to add mons the old way for the benefit of testing
upgrades from early octopus versions of cephadm.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-03-22 16:28:40 -05:00
Sage Weil
0a139c1ffc qa/tasks/cephadm.py: don't set mgr count to +1
Signed-off-by: Sage Weil <sage@newdream.net>
2021-03-11 14:16:15 -05:00
Sage Weil
1bfb0cd84b qa/tasks/cephadm: drop realm.zone convention for rgw
Note that cephadm.py will no longer do anything with rgw realms and
zones.  That means that the setup of rgw roles here is only useful
for the default zone and a non-multisite config.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-03-09 14:29:43 -05:00
Sage Weil
208407c5c0 qa/tasks/cephadm: remove mirror code
This is being done from ansible now.  Also, it breaks when
the conf file has unqualified-search-registries but not 'registry'
entries.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-03-04 07:39:15 -06:00
Neha Ojha
3ced450cdd Revert "qa/tasks/cephadm: use cluster.sh and remote.sh for better readibility"
This reverts commit 6d7ba1180e.

Fixes: https://tracker.ceph.com/issues/49228
Signed-off-by: Neha Ojha <nojha@redhat.com>
2021-02-09 22:15:37 +00:00
Sage Weil
5842058d12 qa/tasks/cephadm: allow custom git_url for cephadm_branch pull
Normally the git_url is git://git.ceph.com/ceph-ci.git, which mirrors
upstream ceph-ci.git.  However, the release branches aren't present there.
Allow a custom git_url so we can pull these from the main ceph.git.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-01-29 11:45:02 -06:00
Sage Weil
9eef1bcced qa/suites/rados/cephadm/upgrade: avoid allow_ptrace with old cephadm
Older cephadm is not smart enough to not combine --cap-add=SYS_PTRACE
and --privileged, which some version of podman cannot handle.

For upgrades, leave off the allow_ptrace behavior since we may be starting
on one of those old versions.

See also https://tracker.ceph.com/issues/46429

Fixes: https://tracker.ceph.com/issues/48142
Signed-off-by: Sage Weil <sage@newdream.net>
2021-01-20 16:40:17 -06:00
Neha Ojha
f3af610721 qa/tasks/cephadm.py: do not create rbd pool by default
rados/cephadm/smoke* does not use the install task and the adjust-ulimits
dependency is met as a part of it. create_rbd_pool needs adjust-ulimits,
so for now we will disable create_rbd_pool by default and only set it
to true for the upgrade suite.

Signed-off-by: Neha Ojha <nojha@redhat.com>
2021-01-06 17:35:15 +00:00