Commit Graph

1139 Commits

Author SHA1 Message Date
Sam Lang
3b0d91533e misc: Use tempfile.mkstemp() instead of tempnam
tempnam() is considered an unsafe security risk because the filename
generated is easy to guess and can be symlinked in advance.  Use
mkstemp() instead.

Signed-off-by: Sam Lang <sam.lang@inktank.com>
Reviewed-by: Joe Buck <jbbuck@gmail.com>
2013-04-15 13:43:20 -05:00
Dan Mick
f69ddafde7 Revert "Revert "Install.py: Prevent prompts from breaking apt""
This reverts commit 67a616a979.

Sigh.  As it turns out, /etc/default/grub being hacked also
causes the same problem.  I think there's a way to fix that cleanly
as well, but until then, replacing the "accept installed version"
hack here so jobs can run.
2013-04-15 11:24:31 -07:00
Sam Lang
50aaece463 misc: Use pythonic 'is not None' for jobid case
The conditional 'if global_jobid:' evaluates to true
in some cases even when global_jobid is None.

Signed-off-by: Sam Lang <sam.lang@inktank.com>
2013-04-13 10:12:45 -05:00
Sam Lang
715245e100 misc: Fix name parsing
Use last two digits of year.

Signed-off-by: Sam Lang <sam.lang@inktank.com>
2013-04-12 19:04:19 -05:00
Dan Mick
67a616a979 Revert "Install.py: Prevent prompts from breaking apt"
This reverts commit 5995ae7e78.

With the changes to ceph-qa-chef and the teuthology kernel task,
we're no longer touching packaged file /etc/grub.d/10_linux, which
was the reason for this apt forcing.  Remove so that we find other
package problems that might be masked by this; we can always
put it back if there are such problems until we can fix those as well.

Signed-off-by: Dan Mick <dan.mick@inktank.com>
(cherry picked from commit c2b0828b19)
2013-04-12 15:49:24 -07:00
Dan Mick
52cdaae683 kernel.py: put submenu name in 01_ceph_kernel if necessary
We had been writing 01_ceph_kernel with the kernel title, and
relying on the fact that grub.cfg would never have submenus in it
(implemented by a hack to /etc/grub.d/10_linux which neutered its
submenu creation).  However, that hack was modifying a package file,
and got in the way of later apt commands.  Rather than doing it
that way, this divines the title of the submenu and sets the
default variable to "submenu>kernel", which works to select the
desired kernel.

It depends on there being only one level of submenu, and on the
format of the menuentry and submenu commands, dictated by grub2.
None of this is likely to work at all outside Ubuntu.

Fixes: #4496
Signed-off-by: Dan Mick <dan.mick@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>
(cherry picked from commit 52aec32a7d)
2013-04-12 15:49:15 -07:00
Samuel Just
2c7b1f3d62 peer.py: we can't assume pg query state will match mon pg state
The pg state could easily have changed in the mean time,
for example, from recovery_wait to recovering.

Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
2013-04-12 15:02:16 -07:00
Sam Lang
35e6db72a1 Merge pull request #10 from ceph/wip-fix-importcycle
lock: Fix import cycle breakage

Reviewed-by: Warren Usui <warren.usui@inktank.com>
2013-04-12 13:06:55 -07:00
Dan Mick
c2b0828b19 Revert "Install.py: Prevent prompts from breaking apt"
This reverts commit 5995ae7e78.

With the changes to ceph-qa-chef and the teuthology kernel task,
we're no longer touching packaged file /etc/grub.d/10_linux, which
was the reason for this apt forcing.  Remove so that we find other
package problems that might be masked by this; we can always
put it back if there are such problems until we can fix those as well.

Signed-off-by: Dan Mick <dan.mick@inktank.com>
2013-04-12 10:57:41 -07:00
Sam Lang
a6b84a5613 lock: Fix import cycle breakage
fa2049f caused an import cycle between lock.py and misc.py.  Move the
needed functions from lock.py to lockstatus.py so that we can avoid the
import cycle.

Signed-off-by: Sam Lang <sam.lang@inktank.com>
2013-04-12 12:57:33 -05:00
wusui
a246f1e8ad Merge pull request #9 from ceph/wip-short-testdir
misc: Use job id and make short path for testdir

Reviewed-by: Warren Usui <warren.usui@inktank.com>
2013-04-12 10:22:15 -07:00
Dan Mick
52aec32a7d kernel.py: put submenu name in 01_ceph_kernel if necessary
We had been writing 01_ceph_kernel with the kernel title, and
relying on the fact that grub.cfg would never have submenus in it
(implemented by a hack to /etc/grub.d/10_linux which neutered its
submenu creation).  However, that hack was modifying a package file,
and got in the way of later apt commands.  Rather than doing it
that way, this divines the title of the submenu and sets the
default variable to "submenu>kernel", which works to select the
desired kernel.

It depends on there being only one level of submenu, and on the
format of the menuentry and submenu commands, dictated by grub2.
None of this is likely to work at all outside Ubuntu.

Fixes: #4496
Signed-off-by: Dan Mick <dan.mick@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>
2013-04-11 13:41:45 -07:00
Sam Lang
fa2049ffbb misc: Use job id and make short path for testdir
Nightlies run on teuthology currently use a testdir of
/home/ubuntu/cephtest, but this causes stale job errors occasionally
from the previous tests not getting properly cleaned up, which prevents
the nightlies from running successfully.

The misc.py get_testdir() function can specify a testdir that is
specific to the job, but previously the path was too long and would
cause separate job failures.

This patch does two things to resolve that.  First, it uses the job id
from the teuthology run if one exists.  This should be a relatively
short number that will identify the job run effectively.  Second,
if the job id isn't available, it creates a shortened form of the
job's name, for example the job name:

teuthology-2013-04-09_23:51:49-rgw-next-testing-basic

becomes:

te1304092351rntb

Signed-off-by: Sam Lang <sam.lang@inktank.com>
2013-04-11 13:53:38 -05:00
Sandon Van Ness
e2938f4026 Fix for kdb: doesn't work on mira nodes
This is a fix for issue #4677 which was caused by kdb output being
hard-coded to ttyS1 which is fine for all our hardware except mira
machines. This change just checks to see if mira is in the host's
name and uses ttyS2 instead (simple fix).
2013-04-09 13:12:58 -07:00
Sandon Van Ness
9c9baef680 Fix: kdb: doesn't work on mira nodes
Change kernel.py to use ttyS2 for kdb output instead of ttyS1 when
the node is a mira machine. This is a fix for issue #4677
2013-04-09 13:09:39 -07:00
Joe Buck
826ad0d816 teuthology: fix for ssh-keys-task
Resolves an issue where we
were not properly escaping the generated
public key when doing matches against it.

Signed-off-by: Joe Buck <jbbuck@gmail.com>
Reviewd-by: Sam Lang <sam.lang@inktank.com>
2013-04-08 11:54:55 -07:00
caleb miles
fa70eb8f67 radosgw-admin: Test bucket list for bucket starting with underscore.
Signed-off-by: caleb miles <caleb.miles@inktank.com>
2013-04-05 10:18:49 -07:00
Sandon Van Ness
41028847f8 Install.py: Prevent prompts from breaking apt
Change apt commands to prevent prompts from coming up (forcing
non-interactive mode) so things like grub or other stuff doesn't
break teuthology runs.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2013-04-04 19:40:21 -07:00
Sandon Van Ness
5995ae7e78 Install.py: Prevent prompts from breaking apt
Change apt commands to prevent prompts from coming up (forcing
non-interactive mode) so things like grub or other stuff doesn't
break teuthology runs.

Signed-off-by: Sandon Van Ness <sandon@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2013-04-04 19:15:14 -07:00
Joe Buck
4824dafb3c teuthology: extend Hadoop task to support branches
Modify the Hadoop task to support branches
being specified for both the Apache and Inktank
Hadoop branches.

Signed-off-by: Joe Buck <jbbuck@gmail.com>
Reviewd-by: Sam Lang <sam.lang@inktank.com>
2013-04-04 15:57:31 -07:00
Joe Buck
4258807bfc teuthology: remove previous test ssh keys
Updated the ssh-keys task to cleanup
any left-over keys from previous tasks
(indicated by the user being 'ssh-keys-user').
Also, some of the functions in the ssh_keys task seem
like they could be useful in general.
This patch refactors them into misc.py.

Signed-off-by: Joe Buck <jbbuck@gmail.com>
Reviewd-by: Sam Lang <sam.lang@inktank.com>
2013-04-04 15:55:54 -07:00
Sage Weil
ad8a346ddd Merge pull request #8 from ceph/wip_4510
repair_test: add test for repairing read errs and truncations
2013-04-04 08:50:27 -07:00
Sage Weil
939223aa4f workunit: sudo rm -rf ...
Signed-off-by: Sage Weil <sage@inktank.com>
2013-04-03 22:01:01 -07:00
Sage Weil
5c80201ec4 scheduled_suite.sh: check clock skew at start and end of run
Signed-off-by: Sage Weil <sage@inktank.com>
2013-04-03 14:00:36 -07:00
Warren Usui
ba536a8353 Merge branch 'wip-teuthologyvminstall-wusui' 2013-04-03 12:34:01 -07:00
Warren Usui
8129bffb17 Implement full reinstallation of a VM system.
Downburst create is used to reinstall a VM when it is locked.
Downburst destroy is used to remove a VM when it is unlocked.
Host keys are regenerated on each vm instantiation, so the keys
need to be checked prior to use.
If needed, qa-ceph-chef is run on newly installed systems to insure that
they are fully functional.

Signed-off-by: Warren Usui <warren.usui@inktank.com>
2013-04-03 12:29:47 -07:00
Sage Weil
cf4bf09b2c ceph.conf: lower mon disk avail warning threshold
Only wanr when we hit 90% instead of default 70%

Signed-off-by: Sage Weil <sage@inktank.com>
2013-04-03 10:03:09 -07:00
caleb miles
982e1ed18f radosgw-admin: Add test of duplicate user email specification.
Signed-off-by: caleb miles <caleb.miles@inktank.com>
2013-04-03 08:12:56 -07:00
caleb miles
fb34c38332 radosgw-admin: Test subuser mask durability when creating new key.
Signed-off-by: caleb miles <caleb.miles@inktank.com>
2013-04-03 06:04:01 -07:00
caleb miles
7b3973fff2 radosgw-admin: cluster info -> zone info
Signed-off-by caleb.miles <caleb.miles@inktank.com>
2013-04-01 20:46:30 -07:00
Samuel Just
d81babffe5 repair_test: add test for repairing read errs and truncations
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
2013-04-01 16:38:33 -07:00
Samuel Just
502e43978d repair_test: add test for repairing read errs and truncations
Signed-off-by: Samuel Just <sam.just@inktank.com>
2013-04-01 16:37:12 -07:00
Josh Durgin
2a1cdda90d locker: try to make up for apache timeouts
If the lock request succeeds in updating the db, but the client gets a
timeout from apache, they can now try again and get back the machines
they just locked.

Only automatic runs have a description set when locking several
machines, so this does not affect users of teuthology-lock
--lock-many, where no description can be set in the same request.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2013-03-29 16:34:15 -07:00
Sage Weil
aeb1bbe414 do not archive on pass if 'archive-on-error: True'
Optional flag makes us suck down the archive (mostly, the logs, which
might be huge for some debugging tests) unless the test has failed.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-29 14:27:20 -07:00
Sage Weil
a40b850eb3 locker: log desc too
Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-29 14:27:13 -07:00
Sage Weil
9f46f47b6b run: clean up machine_type thing
Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-29 12:19:05 -07:00
Sage Weil
e8afa454d8 ceph_manager: retry set_pool_property on EAGAIN
Retry indefinitely, for now.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-28 15:25:10 -07:00
Sage Weil
b815268b58 run: machine-type: foo, not machine_type: foo
Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-28 15:25:10 -07:00
Sage Weil
7dca4aee9e Merge pull request #6 from ceph/wip-mds-thrasher-logging
task/mds_thrash: Log mds dump after long delay
2013-03-27 08:56:04 -07:00
Sam Lang
6fd7ebd44d task/mds_thrash: Log mds dump after long delay
In cases where the mds thrasher continuously loops
waiting for an mds to be removed from the map, or
for a new mds to become active, we want to start logging
the mds state for debugging.

Signed-off-by: Sam Lang <sam.lang@inktank.com>
2013-03-27 08:48:45 -05:00
Sage Weil
bc54a8bfaa locker: make desc optional
Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-26 13:27:53 -07:00
Sage Weil
07e324ae65 ceph.conf: osd debug op order = true
Debug the osd op ordering by default.  Most of the runs have a small number
of clients, which makes the STL maps cheap.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-26 12:21:13 -07:00
Sage Weil
8eabe1bb4a locker/api: fix DELETE
Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-26 11:40:13 -07:00
Sage Weil
86230f15cf Merge branch 'wip-lock'
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2013-03-26 11:34:33 -07:00
Sage Weil
0b72c8dbd2 lock: pass desc to lock operation; leave on unlock
Pass the desc to the lock operation.

The unlock operation now clears desc for us; no need to do it outselves.

Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-25 16:47:02 -07:00
Sage Weil
56820b3bad locker: set desc on lock
Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-25 16:47:02 -07:00
Sage Weil
6208af93ba locker: clear desc on unlock
Signed-off-by: Sage Weil <sage@inktank.com>
2013-03-25 16:47:02 -07:00
Samuel Just
c50b143e92 thrashosds: add test_backfill_full
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>
2013-03-25 15:39:12 -07:00
Samuel Just
97a5c05141 thrashosds.py: fix line length
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>
2013-03-25 15:39:11 -07:00
Josh Durgin
e594fcb30f locker: log updates
Note whenever locks are acquired/released, or a machine's description is updated.
Under apache, these will go to error.log.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
2013-03-25 15:42:08 -07:00