This was running ceph-volume through the
cephadm shell previously, but as we are trying
to remove mount points from cephadm shell, this
no longer works (specifically without the /dev mount)
Signed-off-by: Adam King <adking@redhat.com>
The cluster (name) is already specified in the arguments passed to
_shell() and this command doesn't need privileges.
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
This fails because teuthology.wait_until_osds_up() wants to use
adjust-ulimits wrapper which isn't available in "cephadm shell"
environment. The whole thing is also redundant because cephadm task
is supposed to wait for OSDs to come up earlier, in ceph_osds().
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
These configs may be set using:
ceph:
cluster-config:
entity:
foo: bar
same as the current:
ceph:
config:
entity:
foo: bar
The configs will be set in parallel using the `ceph config set` command.
The main benefit here is to avoid using the ceph.conf to set configs which
cannot be overriden using subsequent `ceph config` command. The only way to
override is to change the ceph.conf in the test (yuck) or the admin socket
(which gets reset when the daemon restarts).
Finally, we can now exploit the `ceph config reset` command will let us
trivially rollback config changes after a test completes. That is exposed
as the `ctx.config_epoch` variable.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Add an option to skip getting a url from shaman/chacra and just
provide a specific download location.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
The code was duplicated across two of the functions after getting the
cephadm binary download url from shaman/chacra. Deduplicate it.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Log in to a registry before calling cephadm pull.
Fixes: https://tracker.ceph.com/issues/65398 (maybe)
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Add a wait_for_service_not_present task function that will wait until a
given service name is not present in the list of running cephadm
services. This is intended for testing service cleanup operations.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
This is specifically being added with the orch/cephadm suite
in mind, where coming up with a viable ignorelist has proved
difficult. The orch testing does a lot of actions that can
cause thigns like an OSD or MON daemon to be down very
briefly, and I've found the vast majority of the time we
really don't want to fail the test when these pop up as cephadm
testing really only benefits from catching the CEPHADM_ errors/
warnings rather than eveyr possible one. Rather than continuing to
play whack-a-mole with the errors in the logs, this
patch should allow us to limit what we fail on to at
least get the suite in a good spot again. We can always
phase out the uses of this new "log-only_match" option
later in a more controlled way, and adding it shouldn't
affect log scraping for any of the tests that aren't
facing a similar issue.
Signed-off-by: Adam King <adking@redhat.com>
For now, it only works in a roleless fashion as OSD deployment
with roles is a bit more complicated. Unless we want to deploy
a mix of raw and lvm OSDs in one test, having it work in roleless
setups should be enough.
Signed-off-by: Adam King <adking@redhat.com>
To have the standalone (non-AD) server test function similarly to the AD
member server test we need to set a variable for samba client container
command similar to how the AD setup command does it.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Looking at the code that expands `all-roles` and `all-hosts` there's no
proper error checking for when these values appear but there are >1
top-level roles in the task config. If a user does this it'll fail
but in a somewhat unclear manner. Add a new condition that raises a
clear exception in this case hopefully saving someone future debugging
time.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
All `exec`-style function in teuthology appear to have a transformation
block that expands names like `all-roles` and `all-hosts`. With the new
cephadm.exec task that block appeared twice in cephadm.py. This change
removes the duplication by creating an _expand_roles function that
can be called from the command executing functions.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Add a `role_to_remote` template filter function that has the ability to
map a role name to a remote. Attributes of the remote can then be
used to get the actual node ip or name.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Add a new cephadm.exec task that works similarly to the existing
vip.exec but instead of only considering VIP related string replacements
it uses that templating feature that was recently added to the
cephadm module for generalized string templating.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Add a cephadm.exclude role that excludes a test node from cluster setup
and related commands. I need this as I have test node that will be set
up as an AD Domain Controller for testing Samba and do not want that
node to be have *any* other services running on it.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
There are cases where I want to pass some large-ish strings to ceph
commands executed via cephadm shell. Allow items within the commands
list to be dicts containing a command (as before) and an optional
stdin variable. This change also supports possible future extensions as
well.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Add a new task function to cephadm.py that sets up a container running
the Samba based domain controller on a node using podman or docker.
Much of the function actually deals with disabling systemd-resolved
because that service conflicts with the DNS server component of the DC.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Add functions to cephadm.py that will be later used to template
strings within the yaml files in the cephadm suites. This will be used
to replace the specific subst_vip call with generic calls that let
tests access "any" variables stored on the test ctx.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
This is v2 of the rbd/nvmeof test: It deploys 1 gateway and 1 initiator.
Then does basic verification on nvme commands and runs fio.
This commit creates:
1. qa/tasks/nvmeof.py: adds a new 'Nvmeof' task which deploys
the gateway and shares config with the initiator hosts.
Sharing config was previously done by 'nvmeof_gateway_cfg' task
in qa/tasks/cephadm.py (that task is removed in this commit).
2. qa/workunits/rbd/nvmeof_basic_tests.sh:
Runs nvme commands (discovery, connect, connect-all, disconnect-all,
and list-subsys) and does basic verification of the output.
3. qa/workunits/rbd/nvmeof_fio_test.sh:
Runs fio command. Also runs iostat in parallel if IOSTAT_INTERVAL
variable is set. This variable configures the delay between each iostat
print.
nvmeof-cli upgrade from v0.0.6 to v0.0.7 introduced major changes
to all nvmeof commands. This commit changes v0.0.6 commands to
v0.0.7 in qa/workunits/rbd/nvmeof_initiator.sh
Signed-off-by: Vallari Agrawal <val.agl002@gmail.com>
This is to allow us to pull the latest build of
cephadm off of a stable branch (currently the only
valid option for that is reef, although this hopefully
will work with squid, T release, etc. in the future).
This should allow us to bootstrap cliusters based on
those stable branches for use in upgrade testing
Signed-off-by: Adam King <adking@redhat.com>
A basic test for ceph-nvmeof[1] where
nvmeof initiator is created.
It requires use of a new task "nvmeof_gateway_cfg"
under cephadm which shares config information
between two remote hosts.
[1] https://github.com/ceph/ceph-nvmeof/
Signed-off-by: Vallari Agrawal <val.agl002@gmail.com>
To allow bootstrapping a cluster using a CA signed
key instead of the standard pubkey authentication.
Will allow explicit testing of this as we add support
for it
Signed-off-by: Adam King <adking@redhat.com>
The old way of using the git archive doesn't seem
to work anymore so trying another approach
Was getting "fatal: operation not supported by protocol"
when trying the git archive
Fixes: https://tracker.ceph.com/issues/58222
Signed-off-by: Adam King <adking@redhat.com>
Since the default of None gets interpreted as 'lvm', the only useful
value would be 'raw'. This will enable us to use loop devices inside
containers for OSDs.
Signed-off-by: Zack Cerza <zack@redhat.com>
When a specific (older) version of cephadm is needed for a test, pull
it from chacra. If we need to bootstrap an older version of ceph, we
pull an older "single-script" binary using the older method.
Co-authored-by: Michael Fritch <mfritch@suse.com>
Co-authored-by: John Mulligan <jmulligan@redhat.com>
Signed-off-by: Michael Fritch <mfritch@suse.com>
Signed-off-by: John Mulligan <jmulligan@redhat.com>
The Filesystem object may use this when configuring EC data pools at
file system creation (via a FuseMount).
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
This doesn't affect bootstrap, but it does mean we avoid any delay
the first time we cephadm.shell on some non-boostrap host.
Signed-off-by: Sage Weil <sage@newdream.net>
If we use a new remote for each shell command, we end up waiting
for the image to pull on every host in sequence.
Signed-off-by: Sage Weil <sage@newdream.net>
* we use setup_iscsi_client.py to deploy iscsi client services,
configuring intiator and mulitpath this is done by qa task
ceph_iscsi_client
* qa/cephadm: adds remotes ip addresses to iscsi gateway,
* rename poolname: iscsi >> datapool, which we usually use for tests and
expresses type of pool more clearly.
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
