OSDs that for some reason get behind on processing their op queue break
expect_alloc_hint_eq(), as it pokes the FS and not the journal. Fix it
by flushing the journal before proceeding with anything else.
Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>
The qa and functional tests are adapted to the new command prototype
requiring a profile instead of a list of properties. When possible the
implicit ruleset creation is used to simplify the test setup.
Signed-off-by: Loic Dachary <loic@dachary.org>
A new module, s3_utilities.pm has been created. It contains
subroutines common to at least two of the workunits in this
directory. Code was moved here from the other pl files, and
some minor changes (paramers and scope changes) were needed.
Fixes: 7472
Signed-off-by: Warren Usui <warren.usui@inktank.com>
If I have to touch this again I will remove it. Ugh. This time,
ubuntu@teuthology:/var/lib/teuthworker/archive/teuthology-2014-03-11_02:30:01-rados-firefly-distro-basic-plana/125922
hit NXIO a few lines down because one of the OSDs was still down.
Signed-off-by: Sage Weil <sage@inktank.com>
Added port (fixed value for right now in teuthology) to hostname.
Fixes: 7374
Reviewed-by: Yehuda Sadeh <yehuda@inktank.com>
Signed-off-by: Warren Usui <warren.usui@inktank.com>
(cherry picked from commit 8200b8a02511e367370d33cb74c3d45ef85fca31)
- fix the wait check for osds to come back up
- make sure they get marked back in, too
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>
'rados cppool' copies the contents but that doesn't make the destination
pool an unmanaged snaps pool. Therefore, we must get an ENOTSUP when
we try to remove an unmanaged snap from a not-unmanaged pool.
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
This wreaks havoc on our QA because it marks osds up and down and then
immediately after that we try to scrub and some osds are still down.
Adjust the CLI test to wait for all OSDs to come back up after thrashing.
Signed-off-by: Sage Weil <sage@inktank.com>
Prevent creation of buckets of type 0 ('osd', 'device', etc.), as they
will confusing the mapping algorithm.
Signed-off-by: Sage Weil <sage@inktank.com>
The cache pools will throttle when they reach the target max size, so it
is important to make the administrator aware when they approach that point.
Unfortunately it is not particularly easy to efficiently keep track of
which PGs have hit their limit and use that for reporting. However, it
is easy to raise a flag when we start to approach the target for the
entire pool, and that sort of early warning is arguably more useful
anyway.
Trigger the warning based on the target full ratio. Not when we hit the
target, but when we are 2/3 between it and completely full.
Implements: #7442
Signed-off-by: Sage Weil <sage@inktank.com>
This is a friendlier interface for setting up a cache tier with some
reasonable defaults (defined via config options). This will simplify
the user experience and documentation.
Signed-off-by: Sage Weil <sage@inktank.com>
In general, users should not use non-empty pools as new tiers or else
things can behave strangely:
- the data sets are unrelated behavior will be... strange.
- if the cache pool is not "new" and does not do the OMAP flag, the OSD
will not know not to flush omap objects to an EC base tier
- probably other random stuff I'm forgetting
Allow a user to shoot themselves in the foot with --force-nonempty.
Implements: #7457
Signed-off-by: Sage Weil <sage@inktank.com>
We would like to get the hit set parameters: hit_set_type |
hit_set_period | hit_set_count | hit_set_fpp via OSDMonitor
Signed-off-by: Kai Zhang <zakir.exe@gmail.com>
comment out erasure pool related tests when an OSD is involved because
it does not work yet. See http://tracker.ceph.com/issues/7360.
Signed-off-by: Loic Dachary <loic@dachary.org>
Now ObjectStore support three backend types, so we need to make each backend
share unit test to avoid duplicate codes.
This patch mainly make workload_generator workable for objectstore.
Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
On Linux ENOTSUP is remapped
/usr/include/x86_64-linux-gnu/bits/errno.h
/* Linux has no ENOTSUP error code. */
# define ENOTSUP EOPNOTSUPP
and should have different values on other operating systems. On Ubuntu
precise the string returned when translating the error value of ENOTSUP
or EOPNOTSUPP is always EOPNOTSUPP but on Ubuntu saucy it is always
ENOTSUP.
Replace ENOSYS with ENOTSUP because the expected semantic is very
similar and modify the test to not check on the string translation of
the error.
Rework the check_response shell function to optionaly check the return
code. The erasure coded pool size change test verifies the error message
only but not the error code.
Signed-off-by: Loic Dachary <loic@dachary.org>
We had already added this as a flag (set/unset) when I generalized the
'mds set_max_mds' to be 'ceph mds set <var> <val>'. Switch the snaps
flag to be a key/value to with true/false (similar to the hashpspool
pool flag) since there are fewer users and the var/val approach is more
general.
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
When run in a shared environment ( as opposed as a machine created for
the purpose of running this test only ), it is important to cleanup
leftovers to avoid poluting the /tmp space. Create a common temporary
directory for all tmp files.
Signed-off-by: Loic Dachary <loic@dachary.org>
First, add the ability to modify max_file_size. While we are at it, move
to a more sensible interface for adjusting max_mds too.
Signed-off-by: Sage Weil <sage@inktank.com>
This will show up on the command line and logs, making it more
clear than EINVAL.
Fixes#6851 and #4045
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
The three rules created by build_simple are identical. They are replaced
by a single rule named replicated_rule which is set to be used by the
data, rbd and metadata pools.
Instead of hardcoding the ruleset number to zero, it is read from
osd_pool_default_crush_ruleset which defaults to zero.
The CEPH_DEFAULT_CRUSH_REPLICATED_RULESET enum is moved from osd_type.h to
config.h because it may be needed when osd_type.h is not included.
Signed-off-by: Loic Dachary <loic@dachary.org>
Creating an erasure pool will crash the OSD because OSD::_make_pg
asserts if the type is not replicated. The tests related to erasure
coded pool creation are removed from qa/workunits/cephtool/test.sh.
The osd-create-pool.sh unit test covers the cases removed from test.sh
more extensively. The intent is to check the interactions with the MON
only, therefore it does not run an OSD and the absence of erasure code
placement group backend implementation is not an issue.
Signed-off-by: Loic Dachary <loic@dachary.org>
When osd create pool is called twice on the same pool, it will succeed
because the pool already exists. However, if a different type is
specified, it must fail.
Signed-off-by: Loic Dachary <loic@dachary.org>
It looked like it worked because the wrapper hide the error. The failing
tests are commented out so that the other tests can be used.
Signed-off-by: Loic Dachary <loic@dachary.org>
Display benchmark results for the default erasure code plugins, in a tab
separated CSV file. The first two column contain the amount of KB
that were coded or decoded, for a given combination of parameters
displayed in the following fields.
seconds KB plugin k m work. iter. size eras.
1.2 10 example 2 1 encode 10 1024 0
0.5 10 example 2 1 decode 10 1024 1
It can be used as input for a human readable report. It is also intented
to be used to show if a given version of an erasure code plugin performs
better than another.
The last column ( not shown above for brievety ) is the exact command
that was run to produce the result so it can be copy / pasted to
reproduce them or to profile.
Only the jerasure techniques mentionned in
https://www.usenix.org/legacy/events/fast09/tech/full_papers/plank/plank_html/
are benchmarked, the others are assumed to be less interesting.
Signed-off-by: Loic Dachary <loic@dachary.org>
crush: make set_chooseleaf_tries work with firstn chooseleaf, too
Reviewed-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Loic Dachary <loic@dachary.com>
