we use `six.StringIO` for stdout when writing output messages from
`self.subproc.communicate()`, and `six.StringIO` accepts strings, so we
need to decode the output before sending them to `six.StringIO`.
Signed-off-by: Kefu Chai <kchai@redhat.com>
* refs/pull/32252/head:
qa/cephfs/begin: libaio-devel on el8
qa/tasks: nosetests -> python -m nose
qa/tasks/rbd_fio: fio 2.21 -> 3.16
src/test/cli-integration/rbd/snap-diff.t: python -> python
qa/workunits: use nose 3
qa/tasks/cbt: install python3 deps
qa/tasks/ceph_manager.py: do not use python to write a file
test/pybind/test_rados: execute takes a bytes (not str) payload
qa/packages/packages: python[3]-ceph is no more
qa: use python3 for venvs etc
packaging: remove python3-ipaddres, as it is part of the stdlib in py3
qa/packages: python-ceph -> python3-ceph
qa/distros: centos7 -> centos8, rhel7 -> rhel8
spec: remove _python_buildid in favor of python3_pkgversion macro
spec: remove python2 packages and conditions
debian: remove python >= 2.7 requirement
debian: add mgr python versions
debian: explicitly set PYTHON2=OFF to prevent picking up python2 interpreter
debian: update control file to use python3 dependency names
debian: remove all python2 overrides and declarations
debian: remove all python2 install files
Reviewed-by: Alfredo Deza <adeza@redhat.com>
To be able to catch problems with python2 *and* python3, run flake8
with both versions. From the flake8 homepage:
It is very important to install Flake8 on the correct version of
Python for your needs. If you want Flake8 to properly parse new
language features in Python 3.5 (for example), you need it to be
installed on 3.5 for Flake8 to understand those features. In many
ways, Flake8 is tied to the version of Python on which it runs.
Also fix the problems with python3 on the way.
Note: This requires now the six module for teuthology. But this is
already an install_require in teuthology itself.
Signed-off-by: Thomas Bechtold <tbechtold@suse.com>
There were a couple of problems found by flake8 in the qa/
directory (most of them fixed now). Enabling flake8 during the usual
check runs hopefully avoids adding new issues in the future.
Signed-off-by: Thomas Bechtold <tbechtold@suse.com>
In the test
test_cephfs_shell.TestMisc.test_issue_cephfs_shell_cmd_at_invocation,
don't set the shell option with in the command to be tested since
CephFS shell can now read it's value from ceph.conf.
Also, add methods to set config options for vstart as well teuthology
testing.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
This shouldn't be an inconvenience since default value for omit_sudo is
already False in teuthology's codebase.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Allow specifying the directory in CephFS that should be FUSE mounted on
the given path when running tests locally.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
The pg count needs to be a power-of-two since
dff5697464.
Also, mon_pg_warn_min_per_osd is disabled by default now (or set to a
low value in vstart/testing) so there's no need to base the pg count on
this value.
Ideally someday we can remove this so that the default cluster value is
used but we need to keep this for deployments of older versions of Ceph.
Fixes: https://tracker.ceph.com/issues/42228
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Add a new command-line switch to allow it to use the kernel client
instead, and add all of the machinery to handle local kcephfs mounts.
Document this in the developer guide, along with the appropriate scary
warnings about using this on a machine that you care about. While we're
in there, also correct a typo about FUSE configuration.
Fixes: https://tracker.ceph.com/issues/41910
Signed-off-by: Jeff Layton <jlayton@redhat.com>
This achieves 2 things: makes it more understandable for the reader that
the boolean variable is an option and reads less like a method name.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Add an option "--clear-old-log" to get rid of log entries before making
entries for current batch of tests.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
stdout of ps command is generally pretty huge which makes it harder to
interpret logs. Don't print it by default and add "--log-ps-output" to
enable printing it.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Override default timeout for execution of vstart.sh and stop.sh since
neither scripts take more than 100 seconds normally.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Add "--teardown" option so that vstart_runner.py tears down Ceph
cluster automatically after the batch of tests has completed running.
Also delete build/dev and build/out directories, after running stop.sh.
Update the doc for vstart_runner.py as well.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
As there is now a jenkins job to run this script
(see https://github.com/ceph/ceph-build/pull/1351),
this refactoring adapt the script to be run in a jenkins job as well as locally.
Signed-off-by: alfonsomthd <almartin@redhat.com>
put_file() in vstart_runner.py should ignore the error when source and
destination paths supplied are same, since source and destination for
put_file() method in teuthology are on different machines, they
represent different locations.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Fix problem in LocalRemote::_perform_checks_and_return_list_of_args() when LocalCephManager::raw_cluster_cmd() is called with a empty command parameter.
Fixes: https://tracker.ceph.com/issues/40729
Signed-off-by: Volker Theile <vtheile@suse.com>
* refs/pull/28561/head:
vstart_runner: upgrade the check for commands to be run as another user
vstart_runner: split unicode arguments into lists
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Rectify the condition that checks if command to be issued as another
user using sudo is issued as a single argument after "-c".
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Right now vstart_runner.py checks if the command to be ran as a
different user using sudo is a single argument by checking if "-c" is
the second last arguemnt. However, this check fails in a case where
"-c" is option for a command being ran as root (e.g. "sudo python -c
some-python-program"). So, add check for this.
Fixes: http://tracker.ceph.com/issues/39385
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Too often in tests it is required to run a command that we want to (or
will probably) fail so that we can check if it failed for expected
reason(s) (eg. a testcase where command must fail with "Permission
denied"). Therefore, add few wrapper methods for vstart_runner.py's
run() with check_status's default value as False so that we don't need
to pass check_status's value at every call to run() or it's other
wrappers.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Since quotes are not a token for Python's shell simulation (unlike
Bash), raise a RuntimeError for them and add an exception for Python
command since it quotes to define a program on commandline.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Till now the run() method in vstart_runner.py accepts commands only as
a (Python) list. It is very inconvenient to write commands as
['setfacl', '-m', 'user:someuser:rwx', 'somedir/somefile'] when they
can be simply be written as
'setfacl -m user:someuser:rwx somedir/somefile'. Also, converting the
latter form to former one is exceedingly simple. So writing commands as
strings should be allowed.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
sudo cannot be omitted from the given command's arguments, when running
passwd, chown and, specially, when sudo is used for running the given
command as different user.
Signed-off-by: Rishabh Dave <ridave@redhat.com>
* When the creation of the cluster is delegated to vstart_runner.py
(--create or --create-target-only) the amount of MGRs required
is calculated by the script so there is no more skipped tests
due to insufficient amount of MGRs.
* Additionally, this issue is not reproducible anymore:
Fixes: https://tracker.ceph.com/issues/37964
* Fixed typo: TEUTHOLOFY_PY_REQS
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
Move it up into CephTestCase so that mgr tests can
use it too, and pick it up in vstart_runner.py so
that these tests will work neatly there.
Signed-off-by: John Spray <john.spray@redhat.com>
Some classes should still be imported directly from collections;
only OrderedDict, Iterable and Callable (in the context of the
ceph codebase) are found in collections.abc.
The current code works due to the fallback support for Python 2.
Signed-off-by: James Page <james.page@ubuntu.com>
Python 3.7 now shows a warning as below.
/usr/bin/ceph:128: DeprecationWarning: Using or importing the ABCs from
'collections' instead of from 'collections.abc' is deprecated, and in
3.8 it will stop working
import rados
This patch addresses the that particular issue.
Signed-off-by: Ganesh Maharaj Mahalingam <ganesh.mahalingam@intel.com>
