When run in a shared environment ( as opposed as a machine created for
the purpose of running this test only ), it is important to cleanup
leftovers to avoid poluting the /tmp space. Create a common temporary
directory for all tmp files.
Signed-off-by: Loic Dachary <loic@dachary.org>
First, add the ability to modify max_file_size. While we are at it, move
to a more sensible interface for adjusting max_mds too.
Signed-off-by: Sage Weil <sage@inktank.com>
The three rules created by build_simple are identical. They are replaced
by a single rule named replicated_rule which is set to be used by the
data, rbd and metadata pools.
Instead of hardcoding the ruleset number to zero, it is read from
osd_pool_default_crush_ruleset which defaults to zero.
The CEPH_DEFAULT_CRUSH_REPLICATED_RULESET enum is moved from osd_type.h to
config.h because it may be needed when osd_type.h is not included.
Signed-off-by: Loic Dachary <loic@dachary.org>
Creating an erasure pool will crash the OSD because OSD::_make_pg
asserts if the type is not replicated. The tests related to erasure
coded pool creation are removed from qa/workunits/cephtool/test.sh.
The osd-create-pool.sh unit test covers the cases removed from test.sh
more extensively. The intent is to check the interactions with the MON
only, therefore it does not run an OSD and the absence of erasure code
placement group backend implementation is not an issue.
Signed-off-by: Loic Dachary <loic@dachary.org>
When osd create pool is called twice on the same pool, it will succeed
because the pool already exists. However, if a different type is
specified, it must fail.
Signed-off-by: Loic Dachary <loic@dachary.org>
It looked like it worked because the wrapper hide the error. The failing
tests are commented out so that the other tests can be used.
Signed-off-by: Loic Dachary <loic@dachary.org>
Display benchmark results for the default erasure code plugins, in a tab
separated CSV file. The first two column contain the amount of KB
that were coded or decoded, for a given combination of parameters
displayed in the following fields.
seconds KB plugin k m work. iter. size eras.
1.2 10 example 2 1 encode 10 1024 0
0.5 10 example 2 1 decode 10 1024 1
It can be used as input for a human readable report. It is also intented
to be used to show if a given version of an erasure code plugin performs
better than another.
The last column ( not shown above for brievety ) is the exact command
that was run to produce the result so it can be copy / pasted to
reproduce them or to profile.
Only the jerasure techniques mentionned in
https://www.usenix.org/legacy/events/fast09/tech/full_papers/plank/plank_html/
are benchmarked, the others are assumed to be less interesting.
Signed-off-by: Loic Dachary <loic@dachary.org>
crush: make set_chooseleaf_tries work with firstn chooseleaf, too
Reviewed-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Loic Dachary <loic@dachary.com>
instead of assuming the pool size is 2, query it and increment it to
test for pool set data size. It allows to run the test from vstart.sh
without knowing what the required pool size is in advance:
rm -fr dev out ; mkdir -p dev ; \
MON=1 OSD=3 ./vstart.sh -n -X -l mon osd
LC_ALL=C PATH=:$PATH CEPH_CONF=ceph.conf \
../qa/workunits/cephtool/test.sh
Signed-off-by: Loic Dachary <loic@dachary.org>
The file removal installed to be triggered when the script stops must
not fail if the file does not exist.
Signed-off-by: Loic Dachary <loic@dachary.org>
We ran into problems before when we made this a string because a mixed
cluster of mons might forward a client request with the wrong schema.
To make this work, we make the new code understand both the new and
old schema, and also backport a change to emperor and dumpling to
handle the new schema.
For the previous attempt to do this, see:
337195f04653eed8e8f153a5b074f3bd48408998
2fe0d0d97af95c22db80800f5b9da51f672d9407
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
feed osd info about os, kernel, memory, arch to the mons
Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>
Reviewed-by: Loic Dachary <loic@dachary.org>
Basic testing by forcing each monitor out of quorum at a time and making
sure they still reply to ping requests.
Fixes: #6705
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
Although fsstress was being called with a static path the directory
it was writing to was in the current directory so doing a cd to the
source directory that is made in /tmp and then removing it later
caused it to be unable to write the files in a non-existent dir.
This change gets the current path first and cd's back into it after
it is done compiling fsstress.
Issue #6479.
Signed-off-by: Sandon Van Ness <sandon@inktank.com>
Reviewed-by: Alfredo Deza <alfredo.deza@inktank.com>
It's legal to give a CephEntityAddr to osd blacklist with no nonce,
so allow it in the valid() method; also add validation of any nonce
given that it's a long >= 0.
Also fix comment on CephEntityAddr type description in MonCommands.h,
and add tests for invalid nonces (while fixing the existing tests to remove
the () around expect_false args).
Fixes: #6425
Signed-off-by: Dan Mick <dan.mick@inktank.com>
Test that it works in snaptest-0.sh, and set the flag in
all the snap workunits so they continue to function.
Signed-off-by: Greg Farnum <greg@inktank.com>
Reviewed-by: Loic Dachary <loic@dachary.org>
The adjust method returns a count of adjusted items.
Add a test.
Fixes: #6382
Backport: dumpling
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>
When using the properties key=value only, the test was inverted
and an attempt to obtain a substring at index string::npos throws
an exception.
Add variations of osd pool create to qa/workunits/mon/pool_ops.sh
to assert the problem has been fixed and all code paths are used.
http://tracker.ceph.com/issues/6357fixes#6357
Signed-off-by: Loic Dachary <loic@dachary.org>
This may need to change since it exploits some of the loose
consistency we currently have with caching pools, but for now
it checks that the Objecter does what we want.
Signed-off-by: Greg Farnum <greg@inktank.com>
Some distro's have a lack of ltp-kernel packages and all we need is
fstress. This just modified the shell script to download/compile
fstress from source and copy it to the right location if it doesn't
currently exist where it is expected. It is a very small/quick
compile and currently only SLES and debian do not have it already.
Reviewed-by: Sage Weil <sage@inktank.com>
Signed-off-by: Sandon Van Ness <sandon@inktank.com>
Some distro's have a lack of ltp-kernel packages and all we need is
fstress. This just modified the shell script to download/compile
fstress from source and copy it to the right location if it doesn't
currently exist where it is expected. It is a very small/quick
compile and currently only SLES and debian do not have it already.
Reviewed-by: Sage Weil <sage@inktank.com>
Signed-off-by: Sandon Van Ness <sandon@inktank.com>
This patch change the fsx.sh to pull better fsx.c from xfstests site
to support hole punching test.
Signed-off-by: Yunchuan Wen <yunchuanwen@ubuntukylin.com>
Signed-off-by: Li Wang <liwang@ubuntukylin.com>
[Also move into a separatate test script; validate result -sage]
Signed-off-by: Dan Mick <dan.mick@inktank.com>
Signed-off-by: Sage Weil <sage@inktank.com>
requests 0.12.1 handles queryparams in the URL with embedded
spaces; requests 0.8.2 does not. Avoid the issue by quoting the
URL into expect().
Signed-off-by: Dan Mick <dan.mick@inktank.com>
dump_info() got a new field outside the mdsmap section; it's ok for
the overall "report", but not for "mds stat". Add an enclosing section
in "mds stat". Fix test to expect new level.
Fixes: #5718
Signed-off-by: Dan Mick <dan.mick@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
set env var TEST_EXIT_ON_ERROR=0 to obtain all errors instead of exiting
with return 1 on first error found.
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
A monc/mon connection fault or the dup command test flag may mean an extra
osd id is created that we isn't actually up; reorder so that doesn't screw
up 'osd ls'.
Signed-off-by: Sage Weil <sage@inktank.com>
da0aff28ab478bcc3136715f92bc1af8d4b403c1 removed 'ceph sync status';
test script needs to stop trying it
Signed-off-by: Dan Mick <dan.mick@inktank.com>
Patterned after cephtool/test.sh, with some deeper validation of
output format and contents (because structured output is easier
to validate).
Signed-off-by: Dan Mick <dan.mick@inktank.com>
From: Yan, Zheng <yan.zheng@intel.com>
Simple reproducer for #5453, modified to run for a finite number of
iterations.
Signed-off-by: Sage Weil <sage@inktank.com>
Test case for failure in #5467. Supplying new auth info overwrites.
Signed-off-by: Dan Mick <dan.mick@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
We can get random messages to stderror from socket reconnects and such;
discard those if we are looking at stderr in the test.
Signed-off-by: Sage Weil <sage@inktank.com>
If we add an item that already exists in particular position, we should
update instead of inserting it; the CrushWrapper methods are not
idempotent.
Signed-off-by: Sage Weil <sage@inktank.com>
'ceph-conf ...' doesn't give you final/default values, only what is in the
conf file. Use -w output to test this instead.
Fixes: #5327
Signed-off-by: Sage Weil <sage@inktank.com>
Trying to track down this failure:
2013-06-12T06:11:13.430 INFO:teuthology.task.workunit.client.0.err:+ rsync -auv --exclude local/ /usr/ usr.2
2013-06-12T06:11:13.430 INFO:teuthology.task.workunit.client.0.err:+ tee a
2013-06-12T06:11:13.527 INFO:teuthology.task.workunit.client.0.out:sending incremental file list
2013-06-12T06:11:46.206 INFO:teuthology.task.workunit.client.0.out:
2013-06-12T06:11:46.208 INFO:teuthology.task.workunit.client.0.out:sent 1689627 bytes received 8302 bytes 50684.45 bytes/sec
2013-06-12T06:11:46.208 INFO:teuthology.task.workunit.client.0.out:total size is 3274130495 speedup is 1928.31
2013-06-12T06:11:46.209 INFO:teuthology.task.workunit.client.0.err:+ wc -l a
2013-06-12T06:11:46.209 INFO:teuthology.task.workunit.client.0.err:+ grep 4
2013-06-12T06:11:46.211 INFO:teuthology.task.workunit:Stopping misc on client.0...
...and am perplexed!
Signed-off-by: Sage Weil <sage@inktank.com>
The sysfs entries for snapshots went away a while ago, and this
script used them to verify sizes matched what was expected.
Instead, look at the mapped size of the snapshot in the places
that used to look for the image's snapshot sysfs files.
Also, switch over to using "udevadm settle" rather than a delay to
wait for udev to do its thing. Insert them at more appropriate
places--right after "rmd map" commands and before and after the
"rbd unmap" calls.
Stop doing the manual refresh calls as well. The osd will trigger
refreshes whenever the image size or shapshot context changes.
Finally, the cleanup routine is called initially, when there really
isn't expected to be anything to clean up. Change the rbd commands
to run there conditionally, only if the target of the command
already exists.
Signed-off-by: Alex Elder <elder@inktank.com>
There's no guarantee the rbd module is loaded when this script is
run, so add a line that loads it if necessary.
Signed-off-by: Alex Elder <elder@inktank.com>
An rbd clone image can be created with an object order that differs
from that of its parent. This patch adds testing for that in
qa/workunits/rbd/image_read.sh. By default, clone images will be
created with an object size twice as big as that of its parent.
For simplicity, when a clone's object order differs from its parent
the order will be either one more than or one less than that of its
parent image, meaning its object size is either double or half of
the size of objects used in the parent.
Signed-off-by: Alex Elder <elder@inktank.com>
Add testing to verify that a snapshot of a clone and a clone of
that snapshot both produce the correct results when read.
Signed-off-by: Alex Elder <elder@inktank.com>
Move the dd command that touches the last byte in a local file
into create_image() where it belongs (out of fill_original()).
Signed-off-by: Alex Elder <elder@inktank.com>
The function boolean_toggle() in qa/workunits/rbd/image_read.sh is
defined but never used. My intentions were good though. Fix it and
use it for argument parsing.
Change the minimum supported object order so it matches what the
command line interface enforces.
Assign the initial value of TEST_CLONES from the environment if it's
available.
Change defaults to use format 2 and test clones.
Output details about the parameters of the run even if not being
verbose.
Make the order of assignment of argument variables consistent.
And fix a typo unmap_image().
Signed-off-by: Alex Elder <elder@inktank.com>
Since snapshots never change, it's safe to read from replicas for them.
A common use for this would be reading from a parent snapshot shared by
many clones.
Convert LibrbdWriteback and AioRead to use the ObjectOperation api
so we can set flags. Fortunately the external wrapper holds no data,
so its lifecycle doesn't need to be managed.
Include a simple workunit that sets the flags in various combinations
and looks for their presence in the logs from 'rbd export'.
Fixes: #3064
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
Some plana have non-world-readable crap in /usr/local/samba. Avoid
/usr/local entirely for that and any similar landmines.
Signed-off-by: Sage Weil <sage@inktank.com>
This uses the old stand-alone qemu-iotests repo so it works with the
version of qemu in Ubuntu 12.04. The tests depend tightly on qemu
version, so to use later tests we'd need to install corresponding
versions of qemu.
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
This adds a new test script for validating data reads from a mapped
rbd image is what it's expected to be.
See the content of the file for a bit more explanation.
Signed-off-by: Alex Elder <elder@inktank.com>
unit tests are added in test/filestore/store_test.cc for the
FileStore::_detect_fs method, when using ext4. It tests the following
situations:
* without user_xattr, ext4 fails
* mounted with user_xattr, ext4 fails if filestore_xattr_use_omap is false
* mounted with user_xattr, ext4 succeeds if filestore_xattr_use_omap is true
The tests are grouped in an ext4 dedicated class
TEST(EXT4StoreTest, _detect_fs)
The qa/workunits/filestore/filestore.sh script is added to prepare the
environment required for the unit tests ( create an image file,
formats it with ext4 etc.). It runs ceph_test_filestore with a sudo
to allow it to mount(2) and umount(2) the ext4 file system. It is
called with
ceph_test_filestore --gtest_filter=EXT4StoreTest.*
to only run the ext4 dependent tests.
The filestore.sh script is meant to be used as part of teuthology
in order to increase the code coverage for src/os/FileStore.cc. It is
self tested and can be checked from the source directory with
CEPH_TEST_FILESTORE=../../../src/ceph_test_filestore filestore.sh TEST
http://tracker.ceph.com/issues/4321 refs #4321
Signed-off-by: Loic Dachary <loic@dachary.org>
Stress test that does io on an image while we are mirroring a diff from
earlier snaps to a second copy. At the end, verify that all snaps have
matching content.
Signed-off-by: Sage Weil <sage@inktank.com>
Export a diff of an image from a previous snapshot to a file (or stdout).
Import a diff and apply it to an image, and then create the ending
snapshot.
Signed-off-by: Sage Weil <sage@inktank.com>
Remove now superfluous directory changes
that are causing tests to fail.
This code should have been removed when we transitioned
from running tests with Ant to using Java to run the tests.
Signed-off-by: Joe Buck <jbbuck@gmail.com>
Reviewed-by: Noah Watkins <noahwatkins@gmail.com>
This script uses the python bindings to libcephfs and rados
to create files and check the correctness of the backtrace
written to the 'parent' xattr on the first object (if its
a file) or inode (if its a dir). The script includes test cases
that kill the mds at specific kill points and restart it through
teuthology using the teuthology restart task.
Signed-off-by: Sam Lang <sam.lang@inktank.com>
The hadoop-internal-tests workunit needs
to be updated in light of our moving to
using stock Hadoop with our hadoop-cephfs
jars.
Signed-off-by: Joe Buck <jbbuck@gmail.com>
Small tweaks to the hadoop-internal test
to better use existing environment varaibles
and in response to the recent teuthology
changes.
Signed-off-by: Joe Buck <jbbuck@gmail.com>
Reviewed-by: Noah Watkins <noahwatkins@gmail.com>
qa/workunits/rbd/test_lock_fence.sh runs using test/rbdrw.py
rbdrw.py creates an image, locks it, and runs an I/O loop;
test_lock_fence.sh runs it, waits, and then blacklists that client,
which causes rbdrw.py to get ESHUTDOWN on operations thereafter.
Currently doesn't work with rbd caching enabled.
rbd.py gets new exception type for ESHUTDOWN
Fixes: #3190
Signed-off-by: Dan Mick <dan.mick@inktank.com>
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
So we don't have to figure out which test is running from the output,
which can be difficult with the system tests.
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
Use qi to parse a strictly formatted set of key/value pairs. Be picky
about whitespace. Any subset of recognized keys is allowed. Parse the
same set of keys as the ceph.*.layout.* vxattrs.
Signed-off-by: Sage Weil <sage@inktank.com>
The '! command' doesn't fail properly, even with -e, in bash (wtf!).
Also, the last pool deletion command succeeds because the pool
'--yes-i-really-really-mean-it' doesn't exist. So drop that test.
Signed-off-by: Sage Weil <sage@inktank.com>
