Display benchmark results for the default erasure code plugins, in a tab
separated CSV file. The first two column contain the amount of KB
that were coded or decoded, for a given combination of parameters
displayed in the following fields.
seconds KB plugin k m work. iter. size eras.
1.2 10 example 2 1 encode 10 1024 0
0.5 10 example 2 1 decode 10 1024 1
It can be used as input for a human readable report. It is also intented
to be used to show if a given version of an erasure code plugin performs
better than another.
The last column ( not shown above for brievety ) is the exact command
that was run to produce the result so it can be copy / pasted to
reproduce them or to profile.
Only the jerasure techniques mentionned in
https://www.usenix.org/legacy/events/fast09/tech/full_papers/plank/plank_html/
are benchmarked, the others are assumed to be less interesting.
Signed-off-by: Loic Dachary <loic@dachary.org>
Implement the ceph_erasure_code_benchmark utility to:
* load an erasure code plugin
* loop over the encode/decode function using the parameters from the
command line
* print the number of bytes encoded/decoded and the time to process
When decoding, random chunks ( as set with --erasures ) are lost on each
run.
For instance:
$ ceph_erasure_code_benchmark \
--plugin jerasure \
--parameter erasure-code-directory=.libs \
--parameter erasure-code-technique=reed_sol_van \
--parameter erasure-code-k=2 \
--parameter erasure-code-m=2 \
--workload decode \
--erasures 2 \
--iterations 1000
0.964759 1048576
shows 1GB is decoded in 1second.
It is intended to be used by other scripts to present a human readable
output or detect performance regressions.
Signed-off-by: Loic Dachary <loic@dachary.org>
The XOR based example is ten times slower than it could because it uses
the buffer::ptr[] operator. Use a temporary char * instead. It performs
as well as jerasure Reed Solomon when decoding with a single erasure:
$ ceph_erasure_code_benchmark \
--plugin example --parameter erasure-code-directory=.libs \
--parameter erasure-code-technique=example \
--parameter erasure-code-k=2 --parameter erasure-code-m=1 \
--erasure 1 --workload decode --iterations 5000
8.095007 5GB
$ ceph_erasure_code_benchmark \
--plugin jerasure --parameter erasure-code-directory=.libs \
--parameter erasure-code-technique=reed_sol_van \
--parameter erasure-code-k=10 --parameter erasure-code-m=6 \
--erasure 1 --workload decode --iterations 5000
7.870990 5GB
Signed-off-by: Loic Dachary <loic@dachary.org>
When profiling, tools such as valgrind --tool=callgrind require that the
dynamically loaded libraries are not dlclosed so they can collect usage
information.
The public ErasureCodePluginRegistry::disable_dlclose boolean is introduced
for this purpose.
Signed-off-by: Loic Dachary <loic@dachary.org>
Add configuration variable to override compatible acting set handling.
Later we'll check the osdmap that all OSDs are updated to use new acting sets.
Fixes: #6990
Signed-off-by: David Zafman <david.zafman@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>
(cherry picked from commit 19cff890eb)
We kick the blocked contexts in the completion path of process_copy_chunk(),
after we have take the RWWRITE obc lock. There is no need to delay the
unblocking until the RepGather finishes.
This also fixes a leak: the ondone wasn't getting cleaned up if a peering
interval change happens and the repgather is applied early in on_change().
Signed-off-by: Sage Weil <sage@inktank.com>
These counts will be useful (even necessary!) for the cache agent, and are
generally interesting to the admin as well.
Signed-off-by: Sage Weil <sage@inktank.com>
Linger operations will follow the object to the cache pool when the pool
overlay process is set. If we evict the object, the object_info_t will
go away along with the watch state and confusing things will happen.
Prevent that from happening by returning EBUSY when you try to evict a
watched object.
Note that you *can* flush a watched object, and the dirty flag will be
cleared. But you still can't evict it.
Signed-off-by: Sage Weil <sage@inktank.com>
After we get the copy-from data and unblock the obc, we still need to take
the RWWRITE lock on the object for the duration of the repop while we
actually apply the change locally.
Signed-off-by: Sage Weil <sage@inktank.com>
In the process of fixing this for flush, we break promote, so we need to
adjust them both here. Basic strategy: do not set user_modify, but handle
the user_version explicitly in the callbacks.
For copy_from, we don't have a clean way to pass the result through to
finish_copyfrom in do_osd_ops; do so by putting it in user_at_version. (If
we were to call finish_copyfrom directly from the callback this might
be simpler, but let's not go there right now.)
For promote, it is a trivial fix.
Signed-off-by: Sage Weil <sage@inktank.com>
Implement a rados operation that will flush a dirty object in the cache
tier by writing it back to the base tier.
Signed-off-by: Sage Weil <sage@inktank.com>
ceph-mds --reset-journal didn't work; it would deadlock waiting for
the osdmap. Comparing the init code in the Dumper (that worked) with
that in the Resetter (that didn't), I noticed the lock had to be
released before waiting for the osdmap.
Now the resetter works. However, both the resetter and the dumper
fail an assertion after they've performed their task; I didn't look
into it:
../../src/msg/SimpleMessenger.cc: In function 'void SimpleMessenger::reaper()' t
hread 7fdc188d27c0 time 2013-12-19 04:48:16.930895
../../src/msg/SimpleMessenger.cc: 230: FAILED assert(!cleared)
ceph version 0.72.1-6-g6bca44e (6bca44ec129d11f1c4f38357db8ae435616f2c7c)
1: (SimpleMessenger::reaper()+0x706) [0x880da6]
2: (SimpleMessenger::wait()+0x36f) [0x88180f]
3: (Resetter::reset()+0x714) [0x56e664]
4: (main()+0x1359) [0x562769]
5: (__libc_start_main()+0xf5) [0x3632e21b45]
6: /l/tmp/build/ceph/build/src/ceph-mds() [0x564e49]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to int
erpret this.
2013-12-19 04:48:16.934093 7fdc188d27c0 -1 ../../src/msg/SimpleMessenger.cc: In
function 'void SimpleMessenger::reaper()' thread 7fdc188d27c0 time 2013-12-19 04
:48:16.930895
../../src/msg/SimpleMessenger.cc: 230: FAILED assert(!cleared)
Signed-off-by: Alexandre Oliva <oliva@gnu.org>
Because the gitbuilder build environment does not read the build depends
from the packages and needs to be updated independantly.
Signed-off-by: Loic Dachary <loic@dachary.org>
make check runs vstart.sh to setup a cluster from source and it misses
the python-argparse library as well as uuidgen otherwise it will fail
with:
./vstart.sh: 460: uuidgen: not found
...
Error: Import No module named argparse
Signed-off-by: Loic Dachary <loic@dachary.org>
