Commit Graph

2269 Commits

Author SHA1 Message Date
Zack Cerza
ea9c034f3c Use the new ResultsReporter.session object
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-29 16:17:08 -05:00
Zack Cerza
ec72137e1b Add a requests.Session object to ResultsReporter
By default it is set up to retry requests 10 times

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-29 16:17:08 -05:00
Zack Cerza
61e469b632 Remove unused timeout arg to ResultsReporter init
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-29 16:17:07 -05:00
Zack Cerza
3f93d16897 Merge pull request #243 from ceph/wip-8116-wusui
Wip 8116 wusui
2014-04-29 16:16:27 -05:00
Warren Usui
74eff43ee1 Clean up remote.py and misc.py changes.
Fixed method names to be non-redundant (remote_mktemp in remote is
now just mktemp, for example), and made some parameters be more
self descriptive.  Added some docstrings.  Fixed sudo setting in
get_file method.

Made chmod independent of the actual sftp file getting.
Do not do extra file copy if non-sudo read is needed.
Fixed some parameter names. Made sure temp files are removed.
2014-04-29 16:14:11 -05:00
Warren Usui
257e1459fa Handle raw data I/O.
Paramiko 1.13.0 checks data in the Channel and fails if
invalid UTF-8 characters are sent.  The teuthology/misc.py
functions that piped cat output (get_file) and piped tar
output (pull_directory and pull_directory_tarball) formerly
did not work for Paramiko 1.13.0.

Code was changed to use SFTPClient to copy raw data.  The
remote_mktemp and function was changed to be a method of the
remote object.  Remote object methods to copy files and write
tar files were also added.

In misc.py, get_file() was changed to be a wrapper around the
remote object call.  Pull_directory() and pull_directory_tarball
were changed to use the remote object methods as well.

The remote methods used to send the data makes use of the
SFTPClient interface in Paramiko.

The remote_mktemp function was also modified slightly to insure
that new-lines do not appear in temp file names.

Fixes: 8116
Signed-off-by: Warren Usui <warren.usui@inktank.com>
2014-04-29 16:14:11 -05:00
Zack Cerza
3faeb08db2 When deleting all of a run's jobs, delete the run
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-29 15:18:42 -05:00
Gregory Meno
5339c1f2ee Changes so these are not installed and still removed 2014-04-29 15:18:42 -05:00
Gregory Meno
ce778848a5 Changes invocation of serialize_remote_roles to internal task to avoid being run during nuke 2014-04-29 05:09:07 -07:00
Gregory Meno
1532af440e Moves node: remote mapping to the internal task. 2014-04-29 05:09:07 -07:00
Gregory Meno
d71a87452f These will likely go somewhere better before merge 2014-04-29 05:09:07 -07:00
Gregory Meno
063b6a27f2 Fixes #8050 Adds a cluster.yaml that is written by interactive task 2014-04-29 05:09:07 -07:00
Sage Weil
f261687f29 valgrind: fix tcmalloc suppression for trusty
Fixes: #8225
Signed-off-by: Sage Weil <sage@inktank.com>
2014-04-28 15:57:36 -07:00
Sage Weil
bab84d45ab Revert "valgrind.supp: be less picky about library versions"
This reverts commit f895d16c9e.
2014-04-28 15:57:36 -07:00
wusui
2ac27d8f13 Merge pull request #244 from ceph/wip-7199-wusui
Wip 7199 wusui
2014-04-28 11:02:07 -07:00
Warren Usui
694827bc0c Allow scrubbing while thrashing
Added ability to implement scrubbing while thrashing
(scrub_interval in config can be set to an interval
similar to how clean_interval is set).  Defaults to 0,
which indicates that no scrubbing will take place.
Add scrub_interval description to thrashosds docstring.

Fixes: 7199
Signed-off-by: Warren Usui <warren.usui@inktank.com>
2014-04-28 11:00:38 -07:00
Zack Cerza
2cbe1dc0f8 Only attempt to use sudo if necessary 2014-04-28 11:12:29 -05:00
Zack Cerza
f102e4943d Post last_in_suite jobs, but delete when run
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-25 15:22:30 -05:00
Zack Cerza
5de353e73f Update unit test for Cluster.__repr__()
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-25 15:01:41 -05:00
Zack Cerza
e6e287446a Fix Cluster.__repr__()
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-25 11:36:43 -05:00
Zack Cerza
af209851e6 Don't push last_in_suite jobs to paddles
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-24 10:46:11 -05:00
Samuel Just
e64d8314ac task/: add tests for ec and rep mark_unfound_lost delete
Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-04-22 17:39:29 -07:00
Zack Cerza
8350b6e490 Bump psutil version requirement
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-22 17:46:02 -05:00
Sandon Van Ness
9078513c0b Fix for #8115
Increase boot disk size per #8115 where monitors shut down due to
/ being full on vm machines.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>
2014-04-22 12:35:11 -07:00
Zack Cerza
1448cdf5fc Work around #8166
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-21 10:11:15 -05:00
Zack Cerza
007d975290 Require requests >= 1.0 2014-04-18 10:49:51 -05:00
Zack Cerza
0e90c69fd4 watch_tube() belongs to the beanstalk module
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-17 16:06:32 -05:00
Zack Cerza
e9a1c778f1 Update requests version
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-17 15:33:26 -05:00
Zack Cerza
5dbc64264e s/wait-for-package/wait_for_package/
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-17 15:03:37 -05:00
Alfredo Deza
5580ffb8ab Merge pull request #242 from ceph/wip-7773
Mirror beanstalkd queue in paddles
2014-04-17 15:40:59 -04:00
Zack Cerza
03b8cdac38 Refactor try_delete_jobs()
Also tweak its error message
2014-04-17 12:27:42 -05:00
Zack Cerza
d12e6f4e00 Be slightly less verbose about logging
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-17 12:27:41 -05:00
Zack Cerza
741c773be6 Look for archive_base in config
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-17 12:27:41 -05:00
Zack Cerza
ee33192f70 When deleting jobs, also delete them from paddles
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-17 12:27:41 -05:00
Zack Cerza
8fdea4d132 Submit queued jobs to paddles
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-17 12:27:41 -05:00
Zack Cerza
8a4de411a6 Rename teuthology.queue to teuthology.worker
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-17 12:27:41 -05:00
Zack Cerza
1449e753ab Use shared methods to connect to beanstalkd
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-17 12:27:41 -05:00
Zack Cerza
165f5d5345 When killing a run, delete paddles jobs
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-17 12:27:41 -05:00
Zack Cerza
66a2742211 Add methods for querying and deleting jobs
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-17 12:27:41 -05:00
Zack Cerza
fc94879498 safe_while: Don't sleep() on the first attempt
This was causing unnecessary delays in several places

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-17 11:08:36 -05:00
Zack Cerza
e323392778 Pass -D flag to teuthology report
Fixes an issue where tests run on old teuthology branches that died for
uncommon reasons were not being marked as dead

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-17 09:43:07 -05:00
Zack Cerza
4b9202bc2f Update to use psutil 2.x API
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-16 15:21:11 -05:00
Zack Cerza
8b93c03fc9 Generate subtasks instead of copy/pasting them
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-16 09:45:07 -05:00
Zack Cerza
761d76937d Don't run apache functions if not using apache
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-16 09:45:07 -05:00
Zack Cerza
089dda1599 Optionally use civetweb instead of apache
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-16 09:45:07 -05:00
Zack Cerza
823219bb27 Don't pass apache's config to radosgw
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-15 14:28:04 -05:00
Zack Cerza
12af2abb4c Rename some functions and variables
This is to make the refactoring a little smoother and easier to read.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-15 13:12:00 -05:00
Zack Cerza
f82f663782 Fix all but one of the PEP-8 issues
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-15 10:58:36 -05:00
Warren Usui
4c0151367e Improve unlock error messages.
Added messages if the hostname is invalid, and if
the user is not the owner of the lock.

Fixes: 6295
Signed-off-by: Warren Usui <warren.usui@inktank.com>
2014-04-14 17:21:37 -07:00
Warren Usui
eef2bf672b Fix kvm issues for Trusty
Change kvm reference to qemu-system-x86_64 and use raw format
in kvm/qemu command.  Tested on both Trusty and Precise.

Fixes: 7825
Signed-off-by: Warren Usui <warren.usui@inktank.com>
2014-04-14 16:07:27 -07:00