We are relying on connection features to track OSD supported
features. However, we were not forwarding connection features
when we forwarded a message from a peon to the leader. That
was breaking the OSD feature tracking.
Fixes: 7051
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
It is inconvenient to run such tests in the
qa/workunits/cephtool/test.sh because they require that the mon is
restarted to test errors in the format of the default erasure code
properties and check the appropriate error message is output.
osd-pool-create.sh runs a single mon from sources using command
line options and a temporary directory, the same way vstart.sh does but
lightweight.
Signed-off-by: Loic Dachary <loic@dachary.org>
If no properties are set when creating an erasure coded pool, default to
using the jerasure plugin with the cauchy_good technique which is the
fastest.
The defaults are set with osd_pool_default_erasure_code_properties.
The erasure code plugins are loaded from the directory specified in the
erasure-code-directory property. Contrary to the other properties it
will most commonly be the same throughout the cluster. The default is
set to /usr/lib/ceph/erasure-code with
osd_pool_default_erasure_code_directory
Signed-off-by: Loic Dachary <loic@dachary.org>
Add a stringstream argument to prepare_new_pool for the purpose of
recording human readable error message.
Signed-off-by: Loic Dachary <loic@dachary.org>
It is capable of parsing json or key=value pairs. The prototype is made
to look like get_str_list. The implementation is in common + include and
use .h. It will probably be moved to common and use .hpp instead, along
with str_list.{cc,h}.
Signed-off-by: Loic Dachary <loic@dachary.org>
They must be dumped with open_object_section instead of
open_array_section otherwise only the values are displayed.
Signed-off-by: Loic Dachary <loic@dachary.org>
When osd create pool is called twice on the same pool, it will succeed
because the pool already exists. However, if a different type is
specified, it must fail.
Signed-off-by: Loic Dachary <loic@dachary.org>
Install the plugins in /usr/lib/ceph/erasure-code instead of
/usr/lib/erasure-code to comply with FHS : "Applications may use a
single subdirectory under /usr/lib."
http://refspecs.linuxfoundation.org/FHS_2.3/fhs-2.3.html
The debian package is modified to install the plugins as part of the
ceph package which also ships rados-classes.
Signed-off-by: Loic Dachary <loic@dachary.org>
The test is updated to remove unecessary asserts. Since all combinations
of properties and pool type are allowed, there is no way to statically
check the validity of the arguments.
Signed-off-by: Loic Dachary <loic@dachary.org>
/usr/bin/ld: mongoose/mongoose.o: undefined reference to symbol 'dlsym@@GLIBC_2.2.5'
/lib/x86_64-linux-gnu/libdl.so.2: error adding symbols: DSO missing from command line
error: collect2: ld returned 1 exit status
Signed-off-by: Sage Weil <sage@inktank.com>
Saw this test fail due to ill-timed thrashing:
/a/teuthology-2013-12-20_23:00:02-rados-master-testing-basic-plana/10941
Signed-off-by: Sage Weil <sage@inktank.com>
This linux version check is used in FileJournal to check about write
caching behavior. This is a temporary fix that will result in the
failure path and a warning about writing caching being turned on until
methods for OSX/FreeBSD/Windows can be found to find the same
information.
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
On OSX without linking in libcommon at the end of these make targets
there is a missing reference to pipe_cloexec, even though the dependency
is present indirectly through libglobal.
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
valloc conflicts with an existing call, and none of these macros are
actually used in buffer.h. The DARWIN check isn't valid either since
this is an installed header and that depends on acconfig.h
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
It looked like it worked because the wrapper hide the error. The failing
tests are commented out so that the other tests can be used.
Signed-off-by: Loic Dachary <loic@dachary.org>
Display benchmark results for the default erasure code plugins, in a tab
separated CSV file. The first two column contain the amount of KB
that were coded or decoded, for a given combination of parameters
displayed in the following fields.
seconds KB plugin k m work. iter. size eras.
1.2 10 example 2 1 encode 10 1024 0
0.5 10 example 2 1 decode 10 1024 1
It can be used as input for a human readable report. It is also intented
to be used to show if a given version of an erasure code plugin performs
better than another.
The last column ( not shown above for brievety ) is the exact command
that was run to produce the result so it can be copy / pasted to
reproduce them or to profile.
Only the jerasure techniques mentionned in
https://www.usenix.org/legacy/events/fast09/tech/full_papers/plank/plank_html/
are benchmarked, the others are assumed to be less interesting.
Signed-off-by: Loic Dachary <loic@dachary.org>
Implement the ceph_erasure_code_benchmark utility to:
* load an erasure code plugin
* loop over the encode/decode function using the parameters from the
command line
* print the number of bytes encoded/decoded and the time to process
When decoding, random chunks ( as set with --erasures ) are lost on each
run.
For instance:
$ ceph_erasure_code_benchmark \
--plugin jerasure \
--parameter erasure-code-directory=.libs \
--parameter erasure-code-technique=reed_sol_van \
--parameter erasure-code-k=2 \
--parameter erasure-code-m=2 \
--workload decode \
--erasures 2 \
--iterations 1000
0.964759 1048576
shows 1GB is decoded in 1second.
It is intended to be used by other scripts to present a human readable
output or detect performance regressions.
Signed-off-by: Loic Dachary <loic@dachary.org>
The XOR based example is ten times slower than it could because it uses
the buffer::ptr[] operator. Use a temporary char * instead. It performs
as well as jerasure Reed Solomon when decoding with a single erasure:
$ ceph_erasure_code_benchmark \
--plugin example --parameter erasure-code-directory=.libs \
--parameter erasure-code-technique=example \
--parameter erasure-code-k=2 --parameter erasure-code-m=1 \
--erasure 1 --workload decode --iterations 5000
8.095007 5GB
$ ceph_erasure_code_benchmark \
--plugin jerasure --parameter erasure-code-directory=.libs \
--parameter erasure-code-technique=reed_sol_van \
--parameter erasure-code-k=10 --parameter erasure-code-m=6 \
--erasure 1 --workload decode --iterations 5000
7.870990 5GB
Signed-off-by: Loic Dachary <loic@dachary.org>
When profiling, tools such as valgrind --tool=callgrind require that the
dynamically loaded libraries are not dlclosed so they can collect usage
information.
The public ErasureCodePluginRegistry::disable_dlclose boolean is introduced
for this purpose.
Signed-off-by: Loic Dachary <loic@dachary.org>
