Commit Graph

630 Commits

Author SHA1 Message Date
Loic Dachary
df3c70a954 qa: scripts to help test ceph-qa-suite
This workunit will be used by tests as a placeholder that always return
true. This is helpful in tests when a script from the qa/workunits
directory is mandatory but we do not care about testing anything.  For
an example of how it can be used, check
https://github.com/ceph/ceph-qa-suite/pull/120

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
2014-09-05 17:49:40 +02:00
Loic Dachary
e13ddc7dae tests: qa/workunits/cephtool/test.sh early fail
Add tests to fail as soon as an unexpected condition is met in
test_mon_osd. Otherwise the actual error will be more difficult find in
the logs.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
2014-08-31 22:59:04 +02:00
John Spray
9fc26840cf tools: use cout instead of cerr in journal tool
Aside from being a bit odd to begin with, using stderr
was causing tests to fail because the output was polluted
by log output which is also on stderr.

Fixes: 9281

Signed-off-by: John Spray <john.spray@redhat.com>
2014-08-29 18:34:39 +01:00
John Spray
e0b19e3023 qa: fix+cleanup hadoop wordcount test
The glob for the examples jar was wrong.

Fixes: #9260

Signed-off-by: John Spray <john.spray@redhat.com>
2014-08-29 13:29:22 +01:00
Sage Weil
16d946bfc7 qa/workunits/hadoop/wordcount: remove bogus rm -r
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-25 11:03:39 -07:00
Gregory Farnum
d9cf299134 Merge pull request #2279 from ceph/wip-hadoop
fix and reorg hadoop workunits

Reviewed-by: Greg Farnum <greg@inktank.com>
2014-08-19 11:47:07 -07:00
Sage Weil
977f85279f qa/workunits/cephtool/test.sh: test reweight-by-pg
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-19 08:16:41 -07:00
Sage Weil
1dc1fb8a60 qa/workunits/hadoop: move all hadoop tests into a hadoop/ dir
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-18 08:39:14 -07:00
Sage Weil
3d3fcc98be qa/workunits/hadoop-wordcount: fix/use -rmr command
-rm -r -f ... doesn't seem to work; use -rmr instead.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-18 08:38:10 -07:00
Sage Weil
adaf5a6a88 qa/workunits/hadoop-wordcount: use -x
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-18 08:37:38 -07:00
Sage Weil
055be68cf8 qa/workunits/rbd/qemu-iotests: touch common.env
This seems to be necessary on trusty.

Backport: firefly, dumpling
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-17 20:54:28 -07:00
Sage Weil
5045c5cb4c qa/workunits/rest/test.py: use rbd instead of data pool for size tests
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-16 22:07:56 -07:00
Sage Weil
3279f3e737 qa/workunits/rest/test.py: do snap test on our data2/3 pool
This way it works when a 'data' pool doesn't already exist.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-16 22:07:56 -07:00
Sage Weil
6d7a229c14 qa/workunits/rest/test.py: fix rd_kb -> rd_bytes
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-16 22:07:56 -07:00
Sage Weil
34fe7a8214 Merge pull request #2217 from ceph/wip-problem-osds
mon: 'ceph osd blocked-by' for histogram of peers OSDs are waiting for

Reviewed-by: Samuel Just <sam.just@inktank.com>
2014-08-16 13:15:10 -07:00
Sage Weil
14614e013f qa/workunits/rest/test.py: fix 'df' test to use total_used_bytes
This changed back in ee2dbdb0f5

Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-16 13:06:02 -07:00
Sage Weil
cec40dae17 qa/workunits/cephtool: verify setmaxosd doesn't let you clobber osds
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-14 13:18:07 -07:00
Sage Weil
bee79ec480 qa/workunits/cephtool: add trivial 'ceph osd blocked-by' test
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-08 18:12:31 -07:00
Sage Weil
6380f47868 qa/workunits/cephtool: add simple 'ceph osd perf' test
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-08 18:12:31 -07:00
Sage Weil
e3819b6216 qa/workunits/rados/test_rados_tool: add a few xattr tests
Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-06 17:06:26 -07:00
Sage Weil
ea731ae142 qa/workunits/cephtool/test.sh: fix 'ceph df ...' tests
Broken by ee2dbdb0f5 and friends.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-08-06 13:16:49 -07:00
Loic Dachary
8363a94a60 erasure-code: HTML display of benchmark results
The ceph_erasure_code_benchmark output is converted into a JSON series
suitable to display in HTML with the http://www.flotcharts.org/
library. A self contained copy of the HTML,JS,CSS files is included for
durability and can be used from the source tree with:

    CEPH_ERASURE_CODE_BENCHMARK=src/ceph_erasure_code_benchmark  \
    PLUGIN_DIRECTORY=src/.libs \
        qa/workunits/erasure-code/bench.sh fplot jerasure |
        tee qa/workunits/erasure-code/bench.js

and display with:

    firefox qa/workunits/erasure-code/bench.html

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-08-04 11:42:05 +02:00
Loic Dachary
e11c3fcc3b erasure-code: rework benchmark suite
Expand the default suite to enumerate all cases that are relevant to the
current code base so that it is easier to consume. Namely it means

 * iterating over object sizes of 4KB (what is used by default) and
   1MB (what was previous benchmarked)
 * grouping results in series that would make sense to plot to get the
   behavior of a given technique for a series of K/M values and all
   possible erasures.

Instead of specifying the iterations to run, set the size of the total
data set to be exercised and compute the iterations by dividing it by
the object size. Since the object size varies, it is impractical to
preset the number of iterations and get meaningful results.

The PARAMETERS environment variable is added to enable the caller to
inject --parameter jerasure-variant=generic, for instance.

The packets size is calculated based on the other parameters. The
options are limited when packets are small (4KB) and it would not make a
real difference to give control over it. The packet size is capped to
a maximum of 3100 bytes which is roughly what has been found to be an
optimal value for large packets (1MB).

Signed-off-by: Loic Dachary <loic@dachary.org>
2014-08-04 11:42:01 +02:00
Dan Mick
aa9ae1f270 qa/workunits/cephtool/test_daemon.sh: verify ceph -c works with daemon
Signed-off-by: Dan Mick <dan.mick@inktank.com>
2014-07-30 14:52:48 -07:00
Dan Mick
22d20f39b7 qa/workunits/cephtool/test_daemon.sh: typo
Signed-off-by: Dan Mick <dan.mick@inktank.com>
2014-07-30 14:52:48 -07:00
Dan Mick
97a8d5a9fd qa/workunits/cephtool/test_daemon.sh: allow local ceph command 2014-07-30 14:52:44 -07:00
Sage Weil
f7112c5beb Merge pull request #2105 from rootfs/wip-qa-hadoop-wordcount
update hadoop-wordcount test to be able to run on hadoop 2.x. 

Reviewed-by: Sage Weil <sage@redhat.com>
2014-07-22 08:42:03 -07:00
rootfs
e311a085a8 uncomment cleanup command 2014-07-22 11:31:37 -04:00
John Spray
0cd0268421 qa: generalise cephtool for vstart+MDS
Previously this test assumed no pre-existing
filesystem and no MDS running.  Generalize it
to nuke any existing filesystems found before
running, so that you can use it inside a vstart
cluster that had MDS>0.

Signed-off-by: John Spray <john.spray@redhat.com>
2014-07-18 16:53:58 +01:00
Sage Weil
4d6899c756 qa/workunits/cephtool/test.sh: fix erasure_code_profile get test
I broke this in ce9f12d7a2 (the pool isn't
type erasure).

Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-17 10:14:35 -07:00
Sage Weil
ce9f12d7a2 qa/workunits/cephtool/test.sh: test osd pool get erasure_code_profile
Signed-off-by: Sage Weil <sage@inktank.com>
2014-07-16 17:55:36 -07:00
rootfs
d915ceb0a9 update hadoop-wordcount test to be able to run on hadoop 2.x. The hadoop and mapreduce library are no longer hard coded so they can be specified to point to the right path. The relative paths hdfs are changed to absolute paths.
A sample command to run the test on hadoop 2.x is
TESTDIR=/home/test HADOOP_HOME=/usr/lib/hadoop HADOOP_MR_HOME=/usr/lib/hadoop-mapreduce sh workunits/hadoop-wordcount/test.sh starting hadoop-wordcount test

Signed-off-by: rootfs <hchen@redhat.com>
2014-07-15 07:44:01 -04:00
John Spray
cceab2b7cb qa: retire kclient-specific layout test
`cephfs set_layout` was broken and is now deprecated
in favour of using xattrs for layout.  Retire the
kclient-specific test.

Fixes: #8773

Signed-off-by: John Spray <john.spray@redhat.com>
2014-07-15 12:23:39 +01:00
Gregory Farnum
d24fa335e9 Merge pull request #2096 from ceph/wip-test-post-file
qa/workunits/post-file.sh

Reviewed-by: Greg Farnum <greg@inktank.com>
2014-07-11 16:39:02 -07:00
Sage Weil
29f20b7e82 qa/workunits/post-file.sh
Test ceph-post-file.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-11 15:33:35 -07:00
Joao Eduardo Luis
50e93c2138 qa/workunits: cephtool: adjust pool name where missing as it has changed
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-07-11 15:22:43 +01:00
Joao Eduardo Luis
6cd345732b qa/workunits: cephtool: cleanup after pool creation
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-07-11 01:02:04 +01:00
Joao Eduardo Luis
704b0a33f2 qa/workunits: cephtool: pool needs to be a tier to be used as such
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-07-11 00:17:47 +01:00
Joao Eduardo Luis
49db676715 qa/workunits: cephtool: test erroneous 'tier remove'
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-07-11 00:17:47 +01:00
Joao Eduardo Luis
9fea033f30 qa/workunits: cephtool: test get/set on both tier and non-tier pools
Make sure gets and sets of tiering-specific variables succeed on tier
pools and fail on non-tier pools.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-07-11 00:17:47 +01:00
Joao Eduardo Luis
df5944955d qa/workunits: cephtool: split get/set on tier pools from get/set tests
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-07-11 00:17:47 +01:00
Greg Farnum
c15e524169 Revert "qa: add an fsx run which turns on kernel debugging"
This reverts commit 29c33f0c05.
We don't need the debugging any more, and having two separate fsx runners
already caused one update-in-the-wrong-place issue.

Signed-off-by: Greg Farnum <greg@inktank.com>
2014-07-09 10:49:59 -07:00
Joao Eduardo Luis
b927c0de7d qa/workunits: cephtool: test for 'osd pool {get,set}-quota'
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-07-08 19:33:14 +01:00
Sage Weil
dd5b60b4f1 Merge remote-tracking branch 'gh/next' 2014-07-04 21:18:55 -07:00
Loic Dachary
2226b91e7d qa: support running under non privileged user
If the test is run against a cluster started with vstart.sh (which is
the case for make check), the --asok-does-not-need-root disables the use
of sudo and allows the test to run without requiring privileged user
permissions.

Signed-off-by: Loic Dachary <loic@dachary.org>
(cherry picked from commit 522174b066)
2014-07-02 20:31:31 -07:00
Sage Weil
55e297ad9a Merge pull request #2046 from ceph/wip-8670
mon: OSDMonitor: 'osd pool' - if we can set it, we must be able to get it

Reviewed-by: Loic Dachary <loic@dachary.org>
Reviewed-by: Sage Weil <sage@redhat.com>
2014-07-02 17:44:56 -07:00
Joao Eduardo Luis
ddc04c83ff mon: OSDMonitor: 'osd pool' - if we can set it, we must be able to get it
Add support to get the values for the following variables:
 - target_max_objects
 - target_max_bytes
 - cache_target_dirty_ratio
 - cache_target_full_ratio
 - cache_min_flush_age
 - cache_min_evict_age

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-07-02 19:35:31 +01:00
Sage Weil
bb3e1c92b6 qa/workunits/rest/test.py: make osd create test idempotent
Avoid possibility that we create multiple OSDs do to retries by passing in
the optional uuid arg.  (A stray osd id will make the osd tell tests a
few lines down fail.)

Fixes: #8728
Signed-off-by: Sage Weil <sage@inktank.com>
2014-07-02 10:39:17 -07:00
John Spray
e2b151d009 mds: Update default FS name
From 'default' to 'cephfs' so that it looks
at least vaguely meaningful.

Signed-off-by: John Spray <john.spray@redhat.com>
2014-07-01 18:40:26 +01:00
John Spray
f62f7f57e0 qa: update cephtool EC pools test to respect IDs
...of data and metadata pools which are no longer
constant.

Signed-off-by: John Spray <john.spray@redhat.com>
2014-07-01 18:40:03 +01:00