Restore ability to run radosgw_admin.py unit standalone--improved
to use vstart_runner hooks.
Local rgwadmin(...) wrapper suggested as a cleanup in review by Casey.
Fixes: https://tracker.ceph.com/issues/52837
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Without this plenty tests become incompatible with vstart_runner.py.
Ideally, vstart_runner.py should've been updated in commit 7812cfb674.
Fixes: https://tracker.ceph.com/issues/53043
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Stop importing CommandFailedError from teuthology.orchestra.run, it is
actually defined in teuthology.exception.
Fixes: https://tracker.ceph.com/issues/51226
Signed-off-by: Rishabh Dave <ridave@redhat.com>
* refs/pull/38481/head:
qa/vstart_runner: inherit methods instead of duplicating them
qa/ceph_manager: make it possible to reuse few methods
qa/vstart_runner: don't use "shell=False" in run_ceph_w()
qa/ceph_manager: minor refactor
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Inherit methods run_ceph_w(), run_cluster_cmd(), raw_cluster_cmd() and
raw_cluster_cmd_result() from ceph_manager.CephManager in
vstart_runner.LocalCephManager instead of duplicating them.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Instead prepend "exec sudo" to the command arguments of
LocalCephManager.run_ceph_w(). This makes the default parameter
"shell=False" redundant in case of
ceph_manager.CephManager.run_ceph_w(), so get rid of it too and update
calls to run_ceph_w() accordingly.
The reason behind using any of these workarounds is that running "ceph
-w" with "shell" set to True leads to crash for Ceph API CI job. See
this ticket for more details: https://tracker.ceph.com/issues/49644.
The reason behind switching the workaround is that in the following
commits to reduce duplication LocalCephManager.run_ceph_w() will be
deleted and CephManager.run_ceph_w() will be used by LocalCephManager
via inheritance. However, due to the issue described above, Ceph API
test will fail since "shell" is set to "True" for the command issued by
CephManager.run_ceph_w(). Prepending "exec sudo" to the command when it
is used in LocalCephManager makes this duplication unnecessary and also
prevents Ceph API test from failing.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
to silence mypy warnings like:
tasks/vstart_runner.py:691: error: Definition of "_run_python" in base class "LocalCephFSMount" is incompatible with definition in base class "CephFSMount"
tasks/vstart_runner.py:705: error: Definition of "_run_python" in base class "LocalCephFSMount" is incompatible with definition in base class "CephFSMount"
Signed-off-by: Kefu Chai <kchai@redhat.com>
* refs/pull/42029/head:
vstart_runner: use FileNotFoundError when os.stat() fails
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
* refs/pull/42030/head:
vstart_runner: maintain log level when --debug is passed
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
otherwise following error is expected in some cases:
INFO:__main__:Traceback (most recent call last):
INFO:__main__: File "/home/jenkins-build/build/workspace/ceph-api/qa/tasks/mgr/test_dashboard.py", line 18, in setUp
INFO:__main__: self._assign_ports("dashboard", "ssl_server_port")
INFO:__main__: File "/home/jenkins-build/build/workspace/ceph-api/qa/tasks/mgr/mgr_test_case.py", line 197, in _assign_ports
INFO:__main__: cls.mgr_cluster.mgr_stop(mgr_id)
INFO:__main__: File "/home/jenkins-build/build/workspace/ceph-api/qa/tasks/mgr/mgr_test_case.py", line 30, in mgr_stop
INFO:__main__: self.mgr_daemons[mgr_id].stop()
INFO:__main__: File "../qa/tasks/vstart_runner.py", line 558, in stop
INFO:__main__: os.kill(pid, signal.SIGTERM)
INFO:__main__:TypeError: an integer is required (got type NoneType)
Signed-off-by: Kefu Chai <kchai@redhat.com>
Add log messages to indicate start and end of execution of stop.sh and
vstart.sh. Running both of these scripts is a major step in execution of
vstart_runner.py and not passing --debug makes it impossible to know
whether these scripts have started or finished running.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
When --debug and --clear-old-log options are passed to vstart_runner.py,
it ends up resetting log level to the default level (which is
logging.INFO) despite of --debug. Set "log_level" as default parameter
to init_log() so that the log level code for clearing old log can pass
current logging level to init_log() to maintain the log level user
desired for.
Fixes: https://tracker.ceph.com/issues/51344
Signed-off-by: Rishabh Dave <ridave@redhat.com>
* refs/pull/40412/head:
vstart_runner: reuse code in LocalRemoteProcess
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
* refs/pull/40431/head:
qa/cephfs: remove create_keyring_file from cephfs_test_case.py
qa/cephfs: don't use sudo to write files in /tmp
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
Since teuthology.orchestra.remote.mktemp() can write a temporary file
and not just create it, create_keyring_file() is now redundant.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Files in /tmp cannot be written by any user( including the root user)
other than the file owner even if the permission mode on the file is
777.
Fixes: https://tracker.ceph.com/issues/49466
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Pass environment copy with max required mgrs when shell kwarg is True.
Fixes: https://tracker.ceph.com/issues/50077
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
Reduce duplication in LocalRemoteProcess.wait() and
LocalRemoteProcess.finished() by moving the common code to a separate
method and then using that separate method instead.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
* refs/pull/38443/head:
qa: set "shell" to False for run_ceph_w()
vstart_runner: make "shell" a default argument
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
It's not yet possible to completely remove the dependency on
mds_ids/mds_daemons in the CephFS tests but this commit reduces it
enough for most code paths to work with cephadm.
The main change here is use of CephManager.do_rados, with some
improvements.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Setting shell to True in call to run() in LocalCephManager.run_ceph_w()
leads to a crash when self.subproc.communicate() is executed for the
process created by running "ceph -w".
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Add "shell" as a parameter for LocalRemoteProcess._do_run() and set its
default value to True. This is necessary to align the interface that
executes the commands for the teuthology and for vstart_runner.py which
reduces compatibility bugs between vstart_runner and teuthology for
tests.
Also, update the part where LocalRemote._do_run() sets shell to True
based on whether or not list contains an instance of class Raw.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Get rid of try-except block in _create_mntpt() since "mkdir -p" doesn't
raise any error when the directory to be created already exists. Also,
use chmod command instead of mkdir command to set the permission mode on
directory since mkdir command would have no effect on the directory's
permission mode if the directory already exists.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Accepting "cwd" as a parameter in _create_mntpt() methods was a mistake
that happened commit 5e71e9b065.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Modify CephManager.run_cluster_cmd() to accept command arguments as
string as well since typing commands as strings is much lesser effort
than typing as list. This brings the interface a step closer to
teuthology.orchestra.remote.run()'s interface since it too can accept
commands arguments as string.
The change in cephfs_test_case.py is just to allow testing this PR
locally and on teuthology.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
* refs/pull/39780/head:
qa/vstart_runner: dont log "not Ceph bin" msg too often
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
The message is logged everytime a binary not from Ceph repo's build
directory is executed, which it too often.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
In methods raw_cluster_cmd_result() of CephManager and LocalCephManager
and raw_cluster_cmd of LocalCephManager when keyword arguments are
passed instead of positional arguments, the methods run ceph command
with no arguments. This is because the methods do
"kwargs['args'] = args" unconditionally.
Fixes: https://tracker.ceph.com/issues/49486
Signed-off-by: Rishabh Dave <ridave@redhat.com>
