Modified remote.py to use the paramiko SFTPClient get
method to extract long files (mostly tar files) from
the remote host. The code formerly saved the data
in a long local string which was very inefficient.
Fixes: 8261
Signed-off-by: Warren Usui <warren.usui@inktank.com>
Fixed method names to be non-redundant (remote_mktemp in remote is
now just mktemp, for example), and made some parameters be more
self descriptive. Added some docstrings. Fixed sudo setting in
get_file method.
Made chmod independent of the actual sftp file getting.
Do not do extra file copy if non-sudo read is needed.
Fixed some parameter names. Made sure temp files are removed.
Paramiko 1.13.0 checks data in the Channel and fails if
invalid UTF-8 characters are sent. The teuthology/misc.py
functions that piped cat output (get_file) and piped tar
output (pull_directory and pull_directory_tarball) formerly
did not work for Paramiko 1.13.0.
Code was changed to use SFTPClient to copy raw data. The
remote_mktemp and function was changed to be a method of the
remote object. Remote object methods to copy files and write
tar files were also added.
In misc.py, get_file() was changed to be a wrapper around the
remote object call. Pull_directory() and pull_directory_tarball
were changed to use the remote object methods as well.
The remote methods used to send the data makes use of the
SFTPClient interface in Paramiko.
The remote_mktemp function was also modified slightly to insure
that new-lines do not appear in temp file names.
Fixes: 8116
Signed-off-by: Warren Usui <warren.usui@inktank.com>
and '(remote,) = ctx.cluster.only(role).remotes.iterkeys()' would fail with
ValueError and no message if there were less than 0 or more than 1 key.
Now a new function, get_single_remote_value() is called which prints out
more understandable messages.
Fixes: 7510
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Signed-off-by: Warren Usui <warren.usui@inktank.com>
I didn't love the way safe_while was encouraged to be used and it didn't
fit right with the new no-raising behavior. Now it's encouraged to be
used like this:
with safe_while() as proceed:
while proceed():
do_things()
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
If we're going to embark on a mission to rid ourselves of
infinitely-looping while loops, it seems smart to start marking the ones
we've fixed in order to make grepping for unfixed loops easier.
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
Symptom was that 'ceph --admin-daemon... config get fsid'
returned zeros, while correct fsid was present in cluster maps.
Fix it by populating FSID in ceph.conf, after extracting it from
monmap.
I was originally attempting a more complicated locking mechanism
but I think its almost as good to just have it attempt the other
machine type if one.
Signed-off-by: Sandon Van Ness <sandon@inktank.com>
Fixed some things that were being done incorrectly.
Some distro kernels have no debug so added | true when disabling
kdb. Also changed what was skipping kernels if non-ubuntu to also
schedule kernel install if a distro kernel.
Signed-off-by: Sandon Van Ness <sandon@inktank.com>
Since the default OS version is different for each distro the
argument default is None instead of explicity set to a value
like with get_distro. Fixed some logic around that and the tests
making the arugment always take precidence.
Signed-off-by: Sandon Van Ness <sandon@inktank.com>
You can use --os-type as an argument when not running teuthology
tests but instead just using teuthology-lock. This adds the ability
to also use --os-version so you can specify the version of the
distro without having to run an actual test with a yaml like you
normally would have had to do setting os_version in the yaml.
Signed-off-by: Sandon Van Ness <sandon@inktank.com>
We should never run with a conflicting testdir in the basedir, and the
code to do this is confusing and buggy. Go back to a single testdir and
simple checks.
Signed-off-by: Sage Weil <sage@inktank.com>
The cd needs to happen at the beginning of the command, not at the end, or
else the funky wrapper scripts don't work right.
This also cleans up the command line construction a bit so that it is more
flexible, explicit, and hopefully less fragile.
Signed-off-by: Sage Weil <sage@inktank.com>
