Commit Graph

2009 Commits

Author SHA1 Message Date
Zack Cerza
36fe6a5830 Remote.hostname doesn't have to be a property
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-05-10 09:10:22 -05:00
Zack Cerza
b2648b21af Fix PEP-8 issues
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-05-10 09:10:22 -05:00
Zack Cerza
8567352360 Pass hostname to execute()
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-05-10 09:10:22 -05:00
Zack Cerza
429553053f Use 'true' instead of 'echo online'
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-05-10 09:10:22 -05:00
Zack Cerza
d0f7a47bf9 Add Remote.ensure_online()
If the connection is alive, do nothing. If not, reconnect. Allow any
exceptions to bubble up to the caller. This is intended to fix unhelpful
'Bad File Descriptor' errors we were seeing when VMs go down.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-05-10 09:10:22 -05:00
Zack Cerza
3adb7d4629 Use Remote.hostname in logs
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-05-10 09:10:22 -05:00
Zack Cerza
29d3299497 Consolidate log file setup into shared function
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-05-10 09:10:21 -05:00
Zack Cerza
5dbce8b68f Use Unicode format strings
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-05-10 09:10:21 -05:00
Zack Cerza
085c508fd4 Revert "Revert "Show hostname instead of IP in errors""
This reverts commit 10fee0e368.

Conflicts:
	teuthology/orchestra/run.py
2014-05-10 09:10:21 -05:00
Zack Cerza
e2adb1fce1 Merge pull request #254 from ceph/wip-7707-wusui
Use master as default for debian upgrade.
2014-05-09 13:17:31 -05:00
Zack Cerza
8460c7a8d9 Force log lines to be interpreted as UTF-8
Any invalid UTF-8 byte will be replaced with a Unicode replacement
character: U+FFFD or '�'

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-05-09 10:45:01 -05:00
Zack Cerza
cd7f268d9d Use binary flag for paramiko ChannelFiles
This works around http://tracker.ceph.com/issues/8313

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-05-09 09:24:22 -05:00
Warren Usui
c0ba105453 Use master as default for debian upgrade.
Make sure that uri is defined for debian upgrades.
Use master as default.

Added _get_uri_() which consolidates checking and assignment of
sha1, tags, and branch values.

Fixes: 7707
Signed-off-by: Warren Usui <warren.usui@inktank.com>
2014-05-08 19:12:16 -07:00
Warren Usui
6fbf98bb0c Further clarify 'too many values to unpack' error.
Many errors in yaml configurations cause ValueError to get thrown
with the message 'too many values to unpack.'  A previously reverted
change tried to handle all these situations and print an appropriate
message.

The current behavior of throwing the ValueError exception and exiting
is probably what we still want in these cases.  So instead of handling
the error, the code now checks for the exception at the top-most call
and displays appropriate messages in log.error and in the ctx.summary
data itself.

Fixes: 7510
Signed-off-by: Warren Usui <warren.usui@inktank.com>
2014-05-08 10:57:06 -07:00
Josh Durgin
bca32ef54b Merge pull request #252 from ceph/wip-fsx-krbd
rbd_fsx: expose krbd and related fsx options

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2014-05-07 18:14:54 -07:00
Zack Cerza
bb170c1bf7 Merge pull request #249 from ceph/wip-8284
rados.py: Add pool_snaps option for ceph_test_rados test command
2014-05-07 13:20:40 -05:00
Ilya Dryomov
b105a07ac6 rbd_fsx: expose krbd and related fsx options
Expose

-K (enable krbd mode) through 'krbd',
-Z (use direct IO) through 'direct_io',
-U (disable randomized striping) through 'randomized_striping',
-H (disable discard ops) through 'punch_holes',

-r readbdy (read alignment) through 'readbdy',
-w writebdy (write alignment) through 'writebdy',
-h holebdy (discard alignment) through 'holebdy'.

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>
2014-05-07 17:55:09 +04:00
Zack Cerza
a1838b2a0f Rewrite most file-retrieval functions
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-05-06 14:59:52 -05:00
Zack Cerza
01cf3671b4 Fix linter errors
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-05-06 14:59:52 -05:00
Warren Usui
8bed6ab625 FIx mktemp dir and redundant Paramiko connecting.
Use previously initialized connection for sftp_get calls.
Use local directory for tarball temp file location.
2014-05-06 14:59:52 -05:00
Warren Usui
36b07b8aee Use SFTPClienti get for long reads/writes
Modified remote.py to use the paramiko SFTPClient get
method to extract long files (mostly tar files) from
the remote host.  The code formerly saved the data
in a long local string which was very inefficient.

Fixes: 8261
Signed-off-by: Warren Usui <warren.usui@inktank.com>
2014-05-06 14:59:52 -05:00
Zack Cerza
091d1fe48d Revert "Revert "Clean up remote.py and misc.py changes.""
This reverts commit 02504c3fd2.
2014-05-06 14:59:52 -05:00
Zack Cerza
0268487057 Revert "Revert "Handle raw data I/O.""
This reverts commit 0c8a3e2334.
2014-05-06 14:59:51 -05:00
Sandon Van Ness
7b1eec94f1 Use longer varchar for locked_by in DB.
Signed-off-by: Sandon Van Ness <sandon@inktank.com>
2014-05-06 12:06:56 -07:00
David Zafman
ea3bef1e1d rados.py: Add pool_snaps option for ceph_test_rados test command
Fixes: #8284

Signed-off-by: David Zafman <david.zafman@inktank.com>
2014-05-06 10:30:36 -07:00
Sage Weil
4ad23dc5f8 rgw: fix indentation for cache_pools
Fixes: #8295
Signed-off-by: Sage Weil <sage@inktank.com>
2014-05-06 09:22:49 -07:00
Sage Weil
33b4bfc764 ceph_manager: reset osd weights to 1 when waiting for clean
If we leave the weights adjusted, we can get PGs stuck in a remapped state
because we are probabilistically rejecting placement.  Avoid this by
setting on in osds to 1.0 before waiting for clean.

Signed-off-by: Sage Weil <sage@inktank.com>
2014-05-05 17:05:10 -07:00
Zack Cerza
0c8a3e2334 Revert "Handle raw data I/O."
This reverts commit 257e1459fa.
2014-05-05 10:03:29 -05:00
Zack Cerza
02504c3fd2 Revert "Clean up remote.py and misc.py changes."
This reverts commit 74eff43ee1.
2014-05-05 10:03:18 -05:00
Sage Weil
794c9465da ceph_manager: fix float stringification
Signed-off-by: Sage Weil <sage@inktank.com>
2014-05-03 07:24:32 -07:00
Sage Weil
a723ddf5db ceph_manager: fix typo
From ce7fa1839f. Tested this time.

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2014-05-02 21:18:38 -07:00
Sage Weil
ce7fa1839f ceph_manager: randomly reweight in osds
Signed-off-by: Sage Weil <sage@inktank.com>
2014-05-02 16:32:53 -07:00
Sage Weil
c5da7b21f7 rgw: option to create a cache pool
64mb for now!

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2014-05-01 14:36:26 -07:00
Alfredo Deza
f74eea7b9e Merge pull request #247 from ceph/requests-sessions
Use a requests.Session object for retries instead of safe_while
2014-05-01 13:17:00 -04:00
Zack Cerza
3fa6271f1a Calculate a timeout to use based on queue size
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-30 14:30:43 -05:00
Zack Cerza
041666b07a Add --runs, to print only run names
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-30 14:30:43 -05:00
Zack Cerza
a9d7aa3514 Refactor teuthology.beanstalk
This architecture will make it easier to add new functionality.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-30 14:30:43 -05:00
Zack Cerza
adc51e1c69 Drop usage of safe_while
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-29 16:41:45 -05:00
Zack Cerza
ea9c034f3c Use the new ResultsReporter.session object
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-29 16:17:08 -05:00
Zack Cerza
ec72137e1b Add a requests.Session object to ResultsReporter
By default it is set up to retry requests 10 times

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-29 16:17:08 -05:00
Zack Cerza
61e469b632 Remove unused timeout arg to ResultsReporter init
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-29 16:17:07 -05:00
Warren Usui
74eff43ee1 Clean up remote.py and misc.py changes.
Fixed method names to be non-redundant (remote_mktemp in remote is
now just mktemp, for example), and made some parameters be more
self descriptive.  Added some docstrings.  Fixed sudo setting in
get_file method.

Made chmod independent of the actual sftp file getting.
Do not do extra file copy if non-sudo read is needed.
Fixed some parameter names. Made sure temp files are removed.
2014-04-29 16:14:11 -05:00
Warren Usui
257e1459fa Handle raw data I/O.
Paramiko 1.13.0 checks data in the Channel and fails if
invalid UTF-8 characters are sent.  The teuthology/misc.py
functions that piped cat output (get_file) and piped tar
output (pull_directory and pull_directory_tarball) formerly
did not work for Paramiko 1.13.0.

Code was changed to use SFTPClient to copy raw data.  The
remote_mktemp and function was changed to be a method of the
remote object.  Remote object methods to copy files and write
tar files were also added.

In misc.py, get_file() was changed to be a wrapper around the
remote object call.  Pull_directory() and pull_directory_tarball
were changed to use the remote object methods as well.

The remote methods used to send the data makes use of the
SFTPClient interface in Paramiko.

The remote_mktemp function was also modified slightly to insure
that new-lines do not appear in temp file names.

Fixes: 8116
Signed-off-by: Warren Usui <warren.usui@inktank.com>
2014-04-29 16:14:11 -05:00
Zack Cerza
3faeb08db2 When deleting all of a run's jobs, delete the run
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-29 15:18:42 -05:00
Gregory Meno
5339c1f2ee Changes so these are not installed and still removed 2014-04-29 15:18:42 -05:00
Gregory Meno
ce778848a5 Changes invocation of serialize_remote_roles to internal task to avoid being run during nuke 2014-04-29 05:09:07 -07:00
Gregory Meno
1532af440e Moves node: remote mapping to the internal task. 2014-04-29 05:09:07 -07:00
Gregory Meno
d71a87452f These will likely go somewhere better before merge 2014-04-29 05:09:07 -07:00
Gregory Meno
063b6a27f2 Fixes #8050 Adds a cluster.yaml that is written by interactive task 2014-04-29 05:09:07 -07:00
Sage Weil
f261687f29 valgrind: fix tcmalloc suppression for trusty
Fixes: #8225
Signed-off-by: Sage Weil <sage@inktank.com>
2014-04-28 15:57:36 -07:00