Commit Graph

47 Commits

Author SHA1 Message Date
Alan Somers
3aae5ca6fd scripts: fix bash path in shebangs
/bin/bash is a Linuxism.  Other operating systems install bash to
different paths.  Use /usr/bin/env in shebangs to find bash.

Signed-off-by: Alan Somers <asomers@gmail.com>
2017-07-27 13:24:26 -06:00
Sage Weil
fd9582f085 Merge pull request #15432 from dachary/wip-osd-new
ceph-disk: support osd new

Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-07-18 13:12:51 -05:00
Kefu Chai
73c0740b08 tests: ceph-disk: use communicate() instead of wait() for output
to avoid possible deadlock. quote from doc of Popen.wait()

> This will deadlock when using stdout=PIPE and/or stderr=PIPE and the
child process generates enough output to a pipe such that it blocks
waiting for the OS pipe buffer to accept more data. Use communicate() to
avoid that.

and print out the stdout and stderr using LOG.warn() if the command
fails.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-07-15 11:27:02 +08:00
Loic Dachary
d199cedc8f tests: ceph-disk destroy needs --purge
The former semantic of ceph-disk destroy is now implemented with the
--purge flag. Use that for the ceph-disk suite.

Signed-off-by: Loic Dachary <loic@dachary.org>
2017-07-14 19:47:01 +02:00
Sage Weil
5cfe4cfa13 ceph-disk: add --filestore argument, default to --bluestore
Signed-off-by: Sage Weil <sage@redhat.com>
Signed-off-by: Loic Dachary <loic@dachary.org>
2017-06-06 19:45:24 +02:00
Loic Dachary
c83d030d30 ceph-disk: do not create bluestore wal/db partitions by default
Fixes: http://tracker.ceph.com/issues/18291

Signed-off-by: Loic Dachary <loic@dachary.org>
2016-12-17 20:55:36 +01:00
Jayashree Candadai
db917d50eb ceph-disk: allow using a regular file as a journal
Because of a missing return, ceph-disk prepare would fail if given a
regular file as a journal. If the journal file does not exist, ceph-disk
will create it but fail to ensure that the ceph user owns it. The
symlink to the journal file is not set when the journal file is
specified on the command line and the journal file does not exist at
all. The ceph-osd daemon will silently create it as a file but it will
not be the file given in argument.

Add a test case to verify using a regular file as a journal works as
expected.

Fixes: http://tracker.ceph.com/issues/17662

Signed-off-by: Jayashree Candadai <jayaajay@indiana.edu>
Signed-off-by: Loic Dachary <ldachary@redhat.com>
2016-10-24 15:58:31 +02:00
Loic Dachary
469a53a4ad tests: ceph-disk: force debug monc = 0
The sh function will collect both stderr and stdout and debug
will mess the json parsing.

Fixes: http://tracker.ceph.com/issues/17607

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2016-10-18 17:33:23 +02:00
Loic Dachary
eb968f886e Merge pull request #10135 from david-z/wip-enhance-ceph-disk-bluestore
ceph-disk: support creating block.db and block.wal with customized size for bluestore

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2016-09-22 18:13:00 +02:00
Zhi Zhang
67b11b0140 ceph-disk: update/add ceph-disk test cases for bluestore
Signed-off-by: Zhi Zhang <zhangz.david@outlook.com>
2016-09-20 11:43:26 +08:00
Kefu Chai
440c360d18 Merge pull request #10815 from ceph/wip-qa-workunits-py3
Python 3 compatibility for workunits

Reviewed-by: Kefu Chai <kchai@redhat.com>
2016-09-19 18:45:34 +08:00
Loic Dachary
7cbf1f0a5e tests: populate /dev/disk/by-partuuid for scsi_debug
The scsi_debug SCSI devices do not have a symlink in /dev/disk/by-partuuid
because they are filtered out by 60-persistent-storage.rules. That was
worked around by 60-ceph-partuuid-workaround-rules which has been
removed by 9f76b9ff31.

Add create rules targetting this specific case, only for tests since the
problem does not show in real use cases.

Fixes: http://tracker.ceph.com/issues/17100

Signed-off-by: Loic Dachary <loic@dachary.org>
2016-08-24 10:10:30 +02:00
Anirudha Bose
07b15e7086 qa/workunits: Allow setting $PYTHON in ceph-disk/ceph-disk.sh
$PYTHON defaults to "python"

Signed-off-by: Anirudha Bose <ani07nov@gmail.com>
2016-08-20 01:18:33 +05:30
Anirudha Bose
92dbeabfdd qa/workunits: Python 3 compat fixes for ceph-disk/ceph-disk-test.py
Signed-off-by: Anirudha Bose <ani07nov@gmail.com>
2016-08-20 01:18:32 +05:30
Li Peng
88ae8c38d0 Fix typos, change prefered to preferred 2016-04-22 15:18:44 +08:00
Loic Dachary
94c5df4567 tests: verify ceph-disk lockbox activation scenarii
Simulate the cases where the activation (via udev running trigger)
sequences are:

* journal then lockbox
* data then lockbox
* lockbox

All of them must end with the OSD verfied to be up.

Signed-off-by: Loic Dachary <loic@dachary.org>
2016-03-04 09:13:35 +07:00
Loic Dachary
ed56cef1f1 tests: make ceph-disk workunit resilient to non ascii
Signed-off-by: Loic Dachary <loic@dachary.org>
2016-03-04 09:13:35 +07:00
Loic Dachary
869eb3af80 tests: verify ceph-disk activate-lockbox
Signed-off-by: Loic Dachary <loic@dachary.org>
2016-03-04 09:13:35 +07:00
Loic Dachary
1ec58fcfc8 ceph-disk: implement lockbox key management
Instead of storing the dmcrypt keys in the /etc/ceph/dmcrypt-keys
directory, they are stored in the monitor. If a machine with
OSDs created with ceph-disk prepare --dmcrypt is lost, it does
not contain the key that would allow to decrypt their content.

The dmcrypt key is retrieved from the monitor using a different keyring
for each OSD. It is stored in a small partition called the lockbox. At
boot time the lockbox is mounted

    /var/lib/ceph/osd-lockbox/$uuid

and used when the $uuid partition is detected by udev to map it with
cryptsetup.

The OSDs that were prepared prior to the lockbox implementation are
supported by looking up the key found in /etc/ceph/dmcrypt-keys before
looking in /var/lib/ceph/osd-lockbox/$uuid.

http://tracker.ceph.com/issues/14669 Fixes: #14669

Signed-off-by: Loic Dachary <loic@dachary.org>
2016-03-04 09:13:35 +07:00
Loic Dachary
36656c8dc9 tests: ceph-disk tests pid files must exist
http://tracker.ceph.com/issues/13422 made it so ceph-osd won't start
unless the pidfile can be created successfully. The default location
being the current directory, ceph-osd must explicitly be told to write
in a directory where it has write permissions.

Signed-off-by: Loic Dachary <loic@dachary.org>
2016-02-04 17:01:46 +07:00
Loic Dachary
c50f3b9f48 tests: ceph-disk qa workunit for bluestore
Signed-off-by: Loic Dachary <loic@dachary.org>
2016-02-04 17:01:46 +07:00
Loic Dachary
65bc36e610 ceph-disk: bluestore prepare
Only support the block file for now. It is handled the same as the
journal, only with a different name (block) and it's own set of ptypes
depending on multipath or dmcrypt.

Signed-off-by: Loic Dachary <loic@dachary.org>
2016-02-04 17:01:46 +07:00
Loic Dachary
ec4a28885a tests: generic ceph-disk qa check_osd_status
The helper function no longer has journal hardcoded.

Signed-off-by: Loic Dachary <loic@dachary.org>
2016-02-04 14:38:08 +07:00
Loic Dachary
c7fbe2124b tests: instructions to collect ceph-disk coverage
Signed-off-by: Loic Dachary <loic@dachary.org>
2016-02-04 14:38:08 +07:00
Loic Dachary
5151063fda ceph-disk: refactor into a proper python module
Signed-off-by: Loic Dachary <loic@dachary.org>
2016-02-04 14:36:37 +07:00
Loic Dachary
756b8e6015 ceph-disk: deactivate must unmap dmcrypted journal
If the journal is not unmapped, ceph-disk destroy will fail to zap the
corresponding devices because it is still held by devicemapper.

A consequence of this modification is that

   ceph-disk activate --dmcrypt --reactivate

no longer works from the command line, because it does not map the
dmcrypted journal. The --reactivate option is added to activate-journal
which will map both the journal and the data devices, if necessary.

http://tracker.ceph.com/issues/14233 Fixes: #14233

Signed-off-by: Loic Dachary <loic@dachary.org>
2016-01-06 11:13:16 +01:00
Loic Dachary
798ae1a944 tests: update ceph-disk workunit debug reminders
Signed-off-by: Loic Dachary <loic@dachary.org>
2016-01-06 11:13:16 +01:00
Loic Dachary
fd7fe8c497 tests: ceph-disk workunit increase verbosity
So that reading the teuthology log is enough in most cases to figure out
the cause of the error.

Signed-off-by: Loic Dachary <loic@dachary.org>
2015-12-21 11:31:25 +01:00
Joe Julian
9cbe1320bd pep8 changes
Signed-off-by: Joe Julian <jjulian@io.com>
2015-11-18 12:14:30 -08:00
Joe Julian
cb18a103bb Add test cases to validate symlinks pointing to devs
Signed-off-by: Joe Julian <jjulian@io.com>
2015-11-18 12:13:52 -08:00
Vicente Cheng
b954c519e8 tests: ceph-disk: add wait_for_osd_down() in ceph-disk-test.py of qa
- add wait_for_osd_down() to avoid the side effect of deactivate

Signed-off-by: Vicente Cheng <freeze.bilsted@gmail.com>
2015-11-17 09:24:43 +08:00
Vicente Cheng
0f892e65a4 tests: ceph-disk: modify the ceph-disk qa test cases
- minor correct for the latest ceph-disk qa test cases

Signed-off-by: Vicente Cheng <freeze.bilsted@gmail.com>
2015-11-17 09:24:43 +08:00
Vicente Cheng
7e88cf005f tests: ceph-disk: add deactivate/reactivate/destroy test cases.
- using the deactivate/destroy feature to destroy osd
  - test reactivate option when the osd goes deactive
  - add check_osd_status to check osd status when osd goes up

Signed-off-by: Vicente Cheng <freeze.bilsted@gmail.com>
2015-11-17 09:24:43 +08:00
Vicente Cheng
06aeec9e5e tests: ceph-disk: modified the destroy_osd test function.
- use the new implementation (ceph-disk deactivate/destroy) instead of step by step remove

Signed-off-by: Vicente Cheng <freeze.bilsted@gmail.com>
2015-11-17 09:24:43 +08:00
Loic Dachary
f4906a124c tests: ceph-disk workunit uses configobj
Instead of using augtool to modify the configuration file, use
configobj. It is also used by the install teuthology task. The .ini
lens (puppet lens really) is unable to read ini files created by
configobj.

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-10-23 00:44:23 +02:00
Loic Dachary
163de5b0f8 tests: ceph-disk workunit uses the ceph task
The ceph-disk workunit deploy keys that are not deployed by default by
the ceph teuthology task.

The OSD created by the ceph task are removed from the default
bucket (via osd rm) so they do not interfere with the tests.

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-10-23 00:43:30 +02:00
Loic Dachary
182676d6bf tests: ceph-disk: workunit must fail when test fail
Signed-off-by: Joe Julian <me@joejulian.name>
2015-09-22 08:46:57 +02:00
Loic Dachary
0cf0e88e29 tests: ceph-disk: only install multipath on CentOS
If installed on Ubuntu where multipath does not activate properly, it
interferes with the other tests.

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-09-22 08:46:57 +02:00
Loic Dachary
fb4dd7d4f2 tests: ceph-disk: inline run_osd
Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-09-22 08:46:57 +02:00
Loic Dachary
24b3a800e6 tests: ceph-disk: wait for the OSD to be up
After preparing an OSD, wait for the corresponding OSD to be up
according to ceph osd dump before asserting the devices are in the
expected state. Otherwise the test races with ceph-disk activate which
is run asynchronously via udev / upstart / system.

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-09-22 08:46:57 +02:00
Loic Dachary
46a3c93ab3 tests: ceph-disk: race condition is fixed
It turns out it was not CentOS 7 specific. There is no excuse to skip
the tests anymore.

http://tracker.ceph.com/issues/12787 Refs: #12787

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-09-22 08:46:57 +02:00
Loic Dachary
d24f1f10aa tests: ceph-disk: multipath now auto activates
http://tracker.ceph.com/issues/12786 Refs: #12786

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-09-22 08:46:57 +02:00
Loic Dachary
d4d5153cd4 tests: ceph-disk: use sh() instead of helper() when possible
Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-09-22 08:46:57 +02:00
Loic Dachary
b6470382f8 tests: doc: ceph-disk workunit reminders
Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-09-22 08:46:57 +02:00
Loic Dachary
32446ffb00 tests: ceph-disk: dmcrypt simplification
* Get rid of the cryptsetup calls that are redundant with what ceph
  prepare already does
* Do not use the --dmcrypt-key-dir option. This is less coverage but it
  interferes with the udev logic and is expected to be refactored soon.

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-09-01 19:04:19 +02:00
Loic Dachary
5ce7ed1bdd ceph-disk: integration tests for multipath
Add integration tests for multipath to the ceph-disk workunit, with the
following caveats:

A workaround is added (explicit call to ceph-disk activate) until the
CentOS activation bug http://tracker.ceph.com/issues/12786 is fixed.

The tests do not run on Ubuntu because of the multipath / device mapper
bug
https://bugs.launchpad.net/ubuntu/+source/multipath-tools/+bug/1488688
and it has not been tested on Debian.

http://tracker.ceph.com/issues/11881 Refs: #11881

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-08-29 02:37:52 +02:00
Loic Dachary
d447098cfc ceph-disk: implement workunit
This new ceph-disk workunit re-implements the tests that previously were
in the src/test/ceph-disk.sh src/test/ceph-disk-root.sh scripts and is
meant to run in a virtual machine instead of docker.

Signed-off-by: Loic Dachary <ldachary@redhat.com>
2015-08-29 02:37:52 +02:00