Commit Graph

901 Commits

Author SHA1 Message Date
Sage Weil
c6a427be55 ceph: sudo grep /var/log/ceph/ceph.log
Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-10 17:08:55 -07:00
Sage Weil
d9a233c9bc mds_thrash: requery mds status inside the wait loop
Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-09 22:17:15 -08:00
Sage Weil
f1d66d2dd5 schedule_suite.sh: check, but do not sync, clock for each run
Trust NTP to actually sync things.  But check the clock skew so we can
verify it if something seems off.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-09 21:34:24 -08:00
Sage Weil
5df43e64ab mpi: substitute $TESTDIR
Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-09 21:20:43 -08:00
Sage Weil
b4d2234376 exec/pexec: substitute $TESTDIR into command
It might be better to make this unescaped, but that's trickier.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-09 18:07:06 -08:00
Sage Weil
36b877add3 Revert "ceph.conf: osd debug op order = true"
This reverts commit 4f68e3ee12.

Until we fix 4392... the kernel is sending ops out of order.
2013-03-08 09:33:51 -08:00
Samuel Just
e7c079f790 rados.py: add option to do many short runs
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2013-03-07 18:06:55 -08:00
tamil
09e14bf474 ceph-deploy task
Signed-off-by: tamil <tamil.muthamizhan@inktank.com>
2013-03-07 17:56:29 -08:00
tamil
e0ad6faa5e added 'extras' to install extra packages excluding ceph
Signed-off-by: tamil <tamil.muthamizhan@inktank.com>
2013-03-07 17:43:40 -08:00
Josh Durgin
cef6e5d5e9 lock: remove description when unlocking
This will hopefully make it easier to track down what's leaving nodes unlocked.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2013-03-07 09:40:29 -08:00
Sage Weil
4f68e3ee12 ceph.conf: osd debug op order = true
Debug the osd op ordering by default.  Most of the runs have a small number
of clients, which makes the STL maps cheap.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-06 21:35:41 -08:00
Joe Buck
da1e63bddd testing: fix logic in determining Hadoop master
Signed-off-by: Joe Buck <jbbuck@gmail.com>
2013-03-05 18:48:09 -08:00
Joe Buck
6b0a8d3db7 testing: specify ceph.conf path in core-site.xml
Signed-off-by: Joe Buck <jbbuck@gmail.com>
2013-03-05 18:47:55 -08:00
Sage Weil
1660d896b0 install: install libcephfs-java
Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-05 15:12:02 -08:00
Sage Weil
daad53fc4f ceph-fuse: install via install.py
Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-05 14:14:12 -08:00
Warren Usui
09979541ca Implement email task.
Email.py was added so that the emailto attribute could be passed,
and to prevent 'module object has no attribute: email' errors from
happening.  Run.py actual performs the email operation and calls
suite.email_results to do the actual send mail operation.  The
information passed right now is the summary and config information.

Signed-off-by: Warren Usui <warren.usui@inktank.com>
2013-02-27 12:28:59 -08:00
Warren Usui
c5b55f9b76 Fix pass/fail display on exit.
Signed-off-by: Warren Usui <warren.usui@inktank.com>
2013-02-27 12:28:59 -08:00
Sage Weil
1e6e8aeb7f ceph-fuse: fix install_debs call with new syntax
Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-27 10:40:50 -08:00
Sage Weil
c0c33834cf install: poll for packages if wait-for-package: true
If the yaml has

 wait-for-package: true

then block and poll for the packages to appear if they are not already
there.  This is only useful for new branches or explicit sha1's, obviously.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-26 21:02:14 -08:00
Sage Weil
454e161829 install: fix branch/tag/sha1 selection
We were only allowing selection of packages by branch!

Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-26 21:01:11 -08:00
Sage Weil
d908c30d60 ceph_manager: use an exception type
Otherwise this obscures the error

Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-26 16:47:39 -08:00
Joe Buck
e00bff2278 teuthology: remove CEPH_JAVA_PATH
Since teuthology now uses debian packages,
we do not need to set this in workunit.
The one test that uses this now tests for
it locally.

Signed-off-by: Joe Buck <jbbuck@gmail.com>
2013-02-26 10:18:25 -08:00
Joe Buck
9af61cb617 teuthology: add an extra_packages flag to install
Some tests require additional packages
(e.g., java bindings, hadoop bindings).
Extend the install task to allow for those
packages to be specified in the yaml files.

Signed-off-by: Joe Buck <jbbuck@gmail.com>
Reviewed-by: Sam Lang <sam.lang@inktank.com>
2013-02-26 10:15:45 -08:00
Sage Weil
51fa5fb2c1 nuke: blow away /home/ubuntu/cephtest too
(along with /tmp/cephtest)

Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-25 19:15:15 -08:00
Warren Usui
3ebabb346f Add timer.py and display summary info in run.py.
Signed-off-by: Warren Usui <warren.usui@inktank.com>
2013-02-25 19:15:15 -08:00
caleb miles
b744f42361 radosgw-admin: Bug fixes for issue 4251.
Fixes errors introduced by d44fb14762.

Signed-off-by caleb.miles <caleb.miles@inktank.com>
2013-02-25 19:14:21 -08:00
Sage Weil
24eeb0da95 rbd: drop udev
The packages have these rules now.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-25 15:12:56 -06:00
Sage Weil
d7337364a3 exec, pexec: set TESTDIR
Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-25 15:12:56 -06:00
Sage Weil
31ef4a1276 ceph.conf: debug ms = 1 for monitors
Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-25 15:12:56 -06:00
Sage Weil
a60e7b599a rbd: set TESTDIR when running xfstests
[elder@inktank.com: fixed typo]

Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-25 15:12:41 -06:00
Sage Weil
d8021a1aa0 nuke: sudo for killall
Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-22 10:51:51 -08:00
Joao Eduardo Luis
045a86680d task: ceph: create monitor data directories prior to --mkfs
The new monitor store does not create the data directory on --mkfs. We
must create it instead, much like what happens with the osds.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-02-22 01:04:06 +00:00
Joao Eduardo Luis
c1b75c6b0c task: mon_thrash: Thrash multiple monitors and 'maintain-quorum' option
We now add a new option 'thrash-many' that by being set to true will break
the default behaviour of killing only one monitor at a time.  Instead,
this option will select up to the maximum number of killable monitors to
kill in each round.

We also add a new 'maintain-quorum' option that will limit the amount of
monitors that can be killed in each thrashing round.  If set to true, this
option will limit the amount of killable monitors up to (n/2-1).  This
means that if we are running a configuration that only has up to two
configured monitors, if 'maintain-quorum' is set to true, this task won't
run as there are no killable monitors -- in such a scenario, this option
should be set to false.

Furthermore, if 'store-thrash' is set to true, then 'maintain-quorum' must
also be set to true, as we cannot let the task to thrash all the monitor
stores, or we wouldn't be able to sync from other monitors, nor can we
let quorum be dropped, or we won't be able to resync our way into quorum.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-02-22 00:44:49 +00:00
Joao Eduardo Luis
d28bb05a65 task: mon_thrash: Add 'seed' and 'store-thrash' options
This patch introduces an option to thrash a monitor store when we thrash
the monitors, as well as a 'store-thrash-probability' option (defaulting
to 50%).

We also took this opportunity to introduce a new 'seed' option, that ought
to allow a given run of this task to be reproducible.  This might come in
hand when attempting to reproduce a given behavior that would otherwise
be randomly triggered.

You should note that while the 'seed' option will indeed mimic past
behaviors, this only applies to a past behavior of this task: other tasks
are not affected by this value, nor are any workunits or even ceph daemons.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2013-02-22 00:44:25 +00:00
Sage Weil
278be217b6 ceph.conf: log file, not log dir
Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-21 16:21:33 -08:00
Sage Weil
c85ba56397 install: ignore apt-get update failures on package removal
At that point we don't really care.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-21 15:08:22 -08:00
Sage Weil
17be13b2b4 ceph: fix log, /var/run/ceph stupid
Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-21 15:08:14 -08:00
Josh Durgin
a862d8bf77 Fix unused vars, unused imports, and aliasing
Found by pyflakes

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2013-02-21 14:47:00 -08:00
Josh Durgin
23669be9af radosgw-admin: fix errors found by pyflakes
Using the wrong variable, and true/false are capitalized in python.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2013-02-21 14:37:46 -08:00
Sage Weil
8b0eef29c5 ceph_manager: fix asok string formatting
Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-21 12:52:30 -08:00
Sage Weil
5c0a2f40de ceph: make /var/run/ceph writeable by non-root too
Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-21 12:45:26 -08:00
Sage Weil
1d62baf107 ceph: fix /var/log/ceph chown/cmod typo
Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-21 12:35:55 -08:00
Sage Weil
e1573ea181 ceph.conf: use default locations for admin socket; fix client log file
Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-21 11:35:36 -08:00
Sage Weil
2574d87562 make /var/log/ceph writeable by non-root; make clients log to it
Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-21 11:05:30 -08:00
Sage Weil
4431e1efab install: be slightly more efficient
Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-20 21:49:22 -08:00
Sage Weil
00986ce080 install: be more careful about package removal
- call apt separately for each package; it will error out annoyingly if
there is one in the list not in the APT sources.
- use dpkg with appropriate force to clean up broken half-installs.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-20 21:46:37 -08:00
Sage Weil
7d8a72a379 ceph-fuse: install -dbg package, too
Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-20 21:36:47 -08:00
Sage Weil
c64cd20103 radosgw-admin: fix sleep syntax error
Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-20 21:02:16 -08:00
Sage Weil
ecb563b221 install: install libcephfs1[-dbg]
Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-20 16:27:52 -08:00
Sage Weil
28f11d037f ceph_manager: drop extra line
Ooops, from 0c990f3add.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-20 16:21:42 -08:00
Sage Weil
9996bdbe6e run: print pass/FAIL as final line
Makes it easy to tell at a glance if your last test passed or not.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-20 15:27:23 -08:00
Yehuda Sadeh
6c2f4a2bbe Merge remote-tracking branch 'origin/wip-3634' 2013-02-20 14:10:50 -08:00
Sage Weil
0c990f3add ceph_manager: eventually time out while waiting for admin socket
Otherwise we get stuck in a loop if an osd crashes unexpectedly, the
task never fails, and we don't collect all the evidence.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-19 22:24:27 -08:00
Sage Weil
9617dcc6b2 Merge branch 'unstable'
Conflicts:
	teuthology/task/workunit.py
2013-02-19 21:04:24 -08:00
Sage Weil
1a0e201d65 ceph: fix valgrind log check
Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-18 16:43:04 -08:00
Sage Weil
2653b5a07b install: clean up flavor, distro, arch detection
Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-18 15:40:43 -08:00
Joe Buck
84d7f37fdb testing: export TESTDIR in workunit task.
Some command-line tools need to reference the path
to the test directory, which is created at run-time.
We export this as TESTDIR

Signed-off-by: Joe Buck <jbbuck@gmail.com>
Reviewed-by:  Sam Lang <sam.lang@inktank.com>
2013-02-18 15:22:52 -08:00
Sage Weil
3f7c9bcaa4 move the install to a separate task.
Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-18 15:06:52 -08:00
Sage Weil
176a3408c9 testrados -> ceph_test_rados
Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-18 13:39:20 -08:00
Sage Weil
4869b494c9 lock: allow filtering by description, description substring
Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-18 13:39:06 -08:00
Sage Weil
2a344c1079 rgw: sudo
Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-18 13:39:06 -08:00
Sage Weil
ed82d87566 fix a few archive/log stragglers
Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-18 13:39:06 -08:00
Sage Weil
38b30c646f ceph: make gitbuilder host configurable
Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-18 13:39:06 -08:00
Sage Weil
b1c9864a7a ceph: install -dbg packages, too
Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-18 13:39:06 -08:00
Sage Weil
d8b6bf94ec ceph: create /var/run/ceph
Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-18 13:39:05 -08:00
Sage Weil
969a6def3b ceph-fuse: sudo
Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-18 13:39:05 -08:00
Sage Weil
4eb047b4ea sudo for admin socket commands
Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-18 13:39:05 -08:00
Sage Weil
0ad55b3604 cfuse -> ceph-fuse 2013-02-18 13:39:05 -08:00
Sage Weil
88ebc27f16 ceph: store logs in normal location
We need to switch around how these are compressed and pulled, since they
aren't in the regular archive dir anymore.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-18 13:39:05 -08:00
Sage Weil
d1d36241b7 ceph: use default data, keyring locations
This required reordering the cluster setup so that we do the ceph-osd
--mkfs --mkkey prior to gathering keys and initializing the monitors.

Also, run daemons as root.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-18 13:39:05 -08:00
Sage Weil
7a61b237d6 ceph: don't uninstall librados, librbd
This forces uninstall of kvm too, which is expensive.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-18 13:39:05 -08:00
Sage Weil
bc9ecf7813 ceph: pass package version to apt-get install
This avoids problems when a different or newer version of the package is
already installed.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-18 13:39:05 -08:00
Sage Weil
c9d3dea320 avoid secretfile, except for kclient
Only mount.ceph needs the secret in a standalone file.  Remove other users,
and simplify that one.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-18 13:39:05 -08:00
Sage Weil
45ddbe636e rgw: specify keyring location
Otherwise we look at the default /var/lib/ceph/radosgw/ceph-$id/keyring.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-18 13:39:05 -08:00
Sage Weil
a54200d444 nuke: tolerate failed dpkg --configure -a/apt-get -f install
Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-18 13:39:05 -08:00
Sage Weil
4dc6c8b275 install radosgw
Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-18 13:39:05 -08:00
Sage Weil
333c7260bc rbd: remove merge cruft
Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-18 13:39:05 -08:00
Sage Weil
8b971611f2 ceph: simplify apt-key management
Run apt-key as root. No need to initialize ubuntu user's gpg.  Fix
whitespace.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-18 13:39:04 -08:00
Sage Weil
eadefec8ff ceph: put client keyrings in /etc/ceph/ceph.$name.keyring
And make it world readable, for ubuntu's sake.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-18 13:39:04 -08:00
Sage Weil
149be93639 nuke: dpkg --configure -a and apt-get -f install
Installing debs means we are more likely to hit a case where we interrupt
apt/dpkg.  Try to mop up as best we can in nuke.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-18 13:39:04 -08:00
Sage Weil
3400ea39ba nuke: whitespace
Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-18 13:39:04 -08:00
Sage Weil
27fec2377f ceph: simpilfy package removal
apt-get doesn't have a nice way to tell if the package is not install and
we don't need to purge it.  Well, not one I found in 5 minutes.  Just
do a big purge and assume it works, or failed because there was nothing to
be done.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-18 13:39:04 -08:00
Sage Weil
28116db6a0 nuke: remove librados, librbd
Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-18 13:39:04 -08:00
Sage Weil
a529bb725f ceph: install ceph-mds, ceph-common
Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-18 13:39:04 -08:00
Sage Weil
5235fc18a0 ceph: fix purge
Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-18 13:39:03 -08:00
Sander Pool
c525e1061b Install ceph debs and use installed debs
The ceph task installs ceph using the debian
packages now, and all invocations of binaries installed
in {tmpdir}/binary/usr/local/bin/ are replace with
the use of the binaries installed in standard locations
by the debs.

Author:    Sander Pool <sander.pool@inktank.com>
Signed-off-by: Sam Lang <sam.lang@inktank.com>
2013-02-18 13:39:03 -08:00
Sage Weil
d790eeb451 nuke: testrados -> ceph_test_rados
Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-18 13:38:54 -08:00
Sage Weil
7a5fd05edd misc: replace : with - in testdir name
The :'s break the list in $PATH.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-17 22:13:45 -08:00
Sage Weil
9513f2f206 rbd_fsx: binary name now has ceph_ prefix
Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-15 09:12:25 -08:00
Sage Weil
5d6d6884fe rados: testrados -> ceph_test_rados
Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-13 14:10:33 -08:00
Sage Weil
6e3c2d93fb peer: add recovery delay to make test behave
Otherwise it was (very) racy!
2013-02-11 06:59:17 -08:00
Sandon Van Ness
a56eb88c16 Merge to include --machine-type and changes to --summary
Added the ability to support multiple types of machines with
--machine-type added to teuthology-lock when used with --lock-many
or --machine-type with teuthology --lock (automated tests). It
defaults to 'plana' and the 'vps' type is currently unused but
should be in the future.

Also updated teutholoy-lock --summary to be machine type aware
and sort things in a nice output.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2013-02-07 16:34:14 -08:00
Sandon Van Ness
75d86e47fd Made teuthology-lock --summary machine type aware.
Signed-off-by: Sandon Van Ness <sandon@van-ness.com>
2013-02-07 16:06:21 -08:00
Sandon Van Ness
030bc7c23d Added support for multiple types of machines.
Added the ability to support multiple types of machines with
--machine-type added to teuthology-lock when used with --lock-many
or --machine-type with teuthology --lock (automated tests). It
defaults to 'plana' and the 'vps' type is currently unused but
should be in the future.

Signed-off-by: Sandon Van Ness <sandon@van-ness.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2013-02-07 13:26:37 -08:00
Sage Weil
ed9103aad5 rgw: parse testdir into apache.conf
Also fix up the template to use {{field}} for stuff we don't want to parse.
There is probably a better way...

Signed-off-by: Sage Weil <sage@inktank.com>
2013-02-06 22:02:10 -08:00
Sage Weil
67bbb9c77b osd_recovery: add missing testdir arg 2013-02-06 21:44:10 -08:00
Sage Weil
561ea14c6e ceph_manager: take int or string to osd_admin_socket
This fixes a failure on dump_stuck.
2013-02-06 17:14:24 -08:00
Sage Weil
3fbb552240 radosbench: fix missing format value
tdir is substituted in at the end.  There is probably a better way to do
this.
2013-02-06 09:06:35 -08:00
Sage Weil
936f314a63 rgw: fix testdir format on f
Format the path, not filehandle
2013-02-06 09:04:37 -08:00
Josh Durgin
ed3c3615c3 nuke: don't try unmount if we're rebooting everything anyway
This can cause issues when unmount hangs. Our automatic runs reboot
everything unconditionally, so this caused a bunch of unecessary hangs
when an fs was accidentally rendered un-unmountable.
2013-02-05 23:31:39 -08:00