Commit Graph

2451 Commits

Author SHA1 Message Date
Zack Cerza
bb170c1bf7 Merge pull request #249 from ceph/wip-8284
rados.py: Add pool_snaps option for ceph_test_rados test command
2014-05-07 13:20:40 -05:00
Ilya Dryomov
b105a07ac6 rbd_fsx: expose krbd and related fsx options
Expose

-K (enable krbd mode) through 'krbd',
-Z (use direct IO) through 'direct_io',
-U (disable randomized striping) through 'randomized_striping',
-H (disable discard ops) through 'punch_holes',

-r readbdy (read alignment) through 'readbdy',
-w writebdy (write alignment) through 'writebdy',
-h holebdy (discard alignment) through 'holebdy'.

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>
2014-05-07 17:55:09 +04:00
Zack Cerza
a1838b2a0f Rewrite most file-retrieval functions
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-05-06 14:59:52 -05:00
Zack Cerza
01cf3671b4 Fix linter errors
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-05-06 14:59:52 -05:00
Warren Usui
8bed6ab625 FIx mktemp dir and redundant Paramiko connecting.
Use previously initialized connection for sftp_get calls.
Use local directory for tarball temp file location.
2014-05-06 14:59:52 -05:00
Warren Usui
36b07b8aee Use SFTPClienti get for long reads/writes
Modified remote.py to use the paramiko SFTPClient get
method to extract long files (mostly tar files) from
the remote host.  The code formerly saved the data
in a long local string which was very inefficient.

Fixes: 8261
Signed-off-by: Warren Usui <warren.usui@inktank.com>
2014-05-06 14:59:52 -05:00
Zack Cerza
091d1fe48d Revert "Revert "Clean up remote.py and misc.py changes.""
This reverts commit 02504c3fd2.
2014-05-06 14:59:52 -05:00
Zack Cerza
0268487057 Revert "Revert "Handle raw data I/O.""
This reverts commit 0c8a3e2334.
2014-05-06 14:59:51 -05:00
Zack Cerza
38578b8034 Revert "Restrict paramiko to old versions for now"
This reverts commit c86df77aa6.
2014-05-06 14:59:51 -05:00
Sandon Van Ness
7b1eec94f1 Use longer varchar for locked_by in DB.
Signed-off-by: Sandon Van Ness <sandon@inktank.com>
2014-05-06 12:06:56 -07:00
David Zafman
ea3bef1e1d rados.py: Add pool_snaps option for ceph_test_rados test command
Fixes: #8284

Signed-off-by: David Zafman <david.zafman@inktank.com>
2014-05-06 10:30:36 -07:00
Zack Cerza
bdd1b5acf2 Merge pull request #251 from ceph/wip-8295
rgw: fix indentation for cache_pools
2014-05-06 11:34:04 -05:00
Sage Weil
4ad23dc5f8 rgw: fix indentation for cache_pools
Fixes: #8295
Signed-off-by: Sage Weil <sage@inktank.com>
2014-05-06 09:22:49 -07:00
Josh Durgin
d158c156ff Merge pull request #250 from ceph/wip-fix-thrasher
ceph_manager: reset osd weights to 1 when waiting for clean

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2014-05-05 17:20:19 -07:00
Sage Weil
33b4bfc764 ceph_manager: reset osd weights to 1 when waiting for clean
If we leave the weights adjusted, we can get PGs stuck in a remapped state
because we are probabilistically rejecting placement.  Avoid this by
setting on in osds to 1.0 before waiting for clean.

Signed-off-by: Sage Weil <sage@inktank.com>
2014-05-05 17:05:10 -07:00
Zack Cerza
c86df77aa6 Restrict paramiko to old versions for now
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-05-05 13:10:42 -05:00
Zack Cerza
0c8a3e2334 Revert "Handle raw data I/O."
This reverts commit 257e1459fa.
2014-05-05 10:03:29 -05:00
Zack Cerza
02504c3fd2 Revert "Clean up remote.py and misc.py changes."
This reverts commit 74eff43ee1.
2014-05-05 10:03:18 -05:00
Sage Weil
794c9465da ceph_manager: fix float stringification
Signed-off-by: Sage Weil <sage@inktank.com>
2014-05-03 07:24:32 -07:00
Sage Weil
a723ddf5db ceph_manager: fix typo
From ce7fa1839f. Tested this time.

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2014-05-02 21:18:38 -07:00
Josh Durgin
7a46469fc4 Merge pull request #248 from ceph/wip-thrash-osd-weights
ceph_manager: randomly reweight in osds

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2014-05-02 17:06:54 -07:00
Sage Weil
ce7fa1839f ceph_manager: randomly reweight in osds
Signed-off-by: Sage Weil <sage@inktank.com>
2014-05-02 16:32:53 -07:00
Zack Cerza
5844c23eb1 Bump paramiko to 1.12.0
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-05-02 11:48:08 -05:00
Sage Weil
c5da7b21f7 rgw: option to create a cache pool
64mb for now!

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2014-05-01 14:36:26 -07:00
Alfredo Deza
f74eea7b9e Merge pull request #247 from ceph/requests-sessions
Use a requests.Session object for retries instead of safe_while
2014-05-01 13:17:00 -04:00
Zack Cerza
d1b93530b3 Add branch name to job config
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-05-01 11:48:54 -05:00
Zack Cerza
ab9645f97b Add suite name to job config
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-05-01 11:25:36 -05:00
Zack Cerza
ba66c6ba8a Add /build and /*.yaml to gitignore
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-05-01 11:25:19 -05:00
Zack Cerza
3fa6271f1a Calculate a timeout to use based on queue size
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-30 14:30:43 -05:00
Zack Cerza
041666b07a Add --runs, to print only run names
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-30 14:30:43 -05:00
Zack Cerza
a9d7aa3514 Refactor teuthology.beanstalk
This architecture will make it easier to add new functionality.

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-30 14:30:43 -05:00
Zack Cerza
adc51e1c69 Drop usage of safe_while
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-29 16:41:45 -05:00
Zack Cerza
ea9c034f3c Use the new ResultsReporter.session object
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-29 16:17:08 -05:00
Zack Cerza
ec72137e1b Add a requests.Session object to ResultsReporter
By default it is set up to retry requests 10 times

Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-29 16:17:08 -05:00
Zack Cerza
61e469b632 Remove unused timeout arg to ResultsReporter init
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-29 16:17:07 -05:00
Zack Cerza
3f93d16897 Merge pull request #243 from ceph/wip-8116-wusui
Wip 8116 wusui
2014-04-29 16:16:27 -05:00
Warren Usui
74eff43ee1 Clean up remote.py and misc.py changes.
Fixed method names to be non-redundant (remote_mktemp in remote is
now just mktemp, for example), and made some parameters be more
self descriptive.  Added some docstrings.  Fixed sudo setting in
get_file method.

Made chmod independent of the actual sftp file getting.
Do not do extra file copy if non-sudo read is needed.
Fixed some parameter names. Made sure temp files are removed.
2014-04-29 16:14:11 -05:00
Warren Usui
257e1459fa Handle raw data I/O.
Paramiko 1.13.0 checks data in the Channel and fails if
invalid UTF-8 characters are sent.  The teuthology/misc.py
functions that piped cat output (get_file) and piped tar
output (pull_directory and pull_directory_tarball) formerly
did not work for Paramiko 1.13.0.

Code was changed to use SFTPClient to copy raw data.  The
remote_mktemp and function was changed to be a method of the
remote object.  Remote object methods to copy files and write
tar files were also added.

In misc.py, get_file() was changed to be a wrapper around the
remote object call.  Pull_directory() and pull_directory_tarball
were changed to use the remote object methods as well.

The remote methods used to send the data makes use of the
SFTPClient interface in Paramiko.

The remote_mktemp function was also modified slightly to insure
that new-lines do not appear in temp file names.

Fixes: 8116
Signed-off-by: Warren Usui <warren.usui@inktank.com>
2014-04-29 16:14:11 -05:00
Zack Cerza
3faeb08db2 When deleting all of a run's jobs, delete the run
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-29 15:18:42 -05:00
Gregory Meno
5339c1f2ee Changes so these are not installed and still removed 2014-04-29 15:18:42 -05:00
Gregory Meno
ce778848a5 Changes invocation of serialize_remote_roles to internal task to avoid being run during nuke 2014-04-29 05:09:07 -07:00
Gregory Meno
1532af440e Moves node: remote mapping to the internal task. 2014-04-29 05:09:07 -07:00
Gregory Meno
d71a87452f These will likely go somewhere better before merge 2014-04-29 05:09:07 -07:00
Gregory Meno
063b6a27f2 Fixes #8050 Adds a cluster.yaml that is written by interactive task 2014-04-29 05:09:07 -07:00
Sage Weil
f261687f29 valgrind: fix tcmalloc suppression for trusty
Fixes: #8225
Signed-off-by: Sage Weil <sage@inktank.com>
2014-04-28 15:57:36 -07:00
Sage Weil
bab84d45ab Revert "valgrind.supp: be less picky about library versions"
This reverts commit f895d16c9e.
2014-04-28 15:57:36 -07:00
wusui
2ac27d8f13 Merge pull request #244 from ceph/wip-7199-wusui
Wip 7199 wusui
2014-04-28 11:02:07 -07:00
Warren Usui
694827bc0c Allow scrubbing while thrashing
Added ability to implement scrubbing while thrashing
(scrub_interval in config can be set to an interval
similar to how clean_interval is set).  Defaults to 0,
which indicates that no scrubbing will take place.
Add scrub_interval description to thrashosds docstring.

Fixes: 7199
Signed-off-by: Warren Usui <warren.usui@inktank.com>
2014-04-28 11:00:38 -07:00
Zack Cerza
2cbe1dc0f8 Only attempt to use sudo if necessary 2014-04-28 11:12:29 -05:00
Zack Cerza
f102e4943d Post last_in_suite jobs, but delete when run
Signed-off-by: Zack Cerza <zack.cerza@inktank.com>
2014-04-25 15:22:30 -05:00