common/LogEntry.cc: In member function 'void LogEntry::log_to_syslog(std::string, std::string)':
warning: common/LogEntry.cc:161:20: format '%lu' expects argument of type 'long unsigned int', but argument 4 has type 'uint64_t {aka long long unsigned int}' [-Wformat=]
msg.c_str());
^
Signed-off-by: Sage Weil <sage@redhat.com>
ECRecPred assumes recovering any number of chunk is possible as long as
at least K chunks are available. It builds the want() set accordingly in
the constructor and arbitrarily set the first K chunks. But it would be
the same if it set the last K chunks.
While this is correct for jerasure and isa plugins, it is not true in
general. The predicate should assume that all chunks are going to be
recovered and return true if they can all be recovered. Otherwise, the
following can happen:
* a PG has chunks 0,1,2,3 for K=2, M=2
* ECRecPred is initialized a set want to 0,1 because K=2
* ECRecPred claims the plugin can recover when provided 0,1
* the plugin is then required to recover 0,1 using 2,3 and fails
This can happen for the LRC plugin with k=4,m=2,l=3 which is
01234567
DDc_DDc_
DDDc____
____DDDc
and if chunks 0,1,2,3 are missing there is no way to recover chunks
4,5,6,7.
Signed-off-by: Loic Dachary <loic-201408@dachary.org>
Using the command line to create an LRC pool validates that the plugin
can be loaded and the ruleset created immediately after the crush table
is populated with the default rulesets.
Add a test that takes out the first OSD holding data for an object and
checks the recovery is effective by retrieving the object content and
checking it is as expected.
Signed-off-by: Loic Dachary <loic@dachary.org>
Add a configuration mode to the LRC plugin, suitable for most use cases
and simpler to explain and understand. It uses k,m parameters similar to
the jerasure and isa plugins and adds l for locality. For instance:
k=4 m=2 l=3
ruleset_locality=rack
ruleset_failure_domain=host
Will translate into
local_group_count = (k+m)/l
ruleset-steps = "["
" [\"choose\", \"rack\", " + local_group_count + "], "
" [\"chooseleaf\", \"host\", " + (l + 1) + "], "
"]";
layers = "[ "
" [ \"DDc_DDc_\", \"\" ],"
" [ \"DDDc____\", \"\" ],"
" [ \"____DDDc\", \"\" ],"
"]";
It is less flexible because k+m must be a multiple of l and other
similar relationships and also because it cannot express recursive
configurations with locality in the datacenter and locality in a rack
within the datacenter.
Signed-off-by: Loic Dachary <loic@dachary.org>
Recursively apply erasure code techniques so that recovering from the
loss of some chunks only require a subset of the available chunks, most
of the time.
http://tracker.ceph.com/issues/7238Fixes: #7238
Signed-off-by: Loic Dachary <loic@dachary.org>
Remapping the chunks after they have been encoded by the plugin is fine
if remapping is transparent to the plugin. However, the goal is to
delegate the remapping of the chunks to the plugin, for LRC in
particular. The chunks are therefore reordered according to the
directions found in the mapping array before they are passed to
decode_chunks.
Signed-off-by: Loic Dachary <loic-201408@dachary.org>
When setting a new erasure coded profile, the key/value of the default
profile are re-used so that ruleset-failure-domain=osd can be overridden
like so:
ceph osd erasure-code-profile set ruleset-failure-domain=osd
This is however not appropriate if the plugin is different from the
defaut plugin (for instance LRC instead of jerasure) because the
parameters are not the same.
If the plugin is changed, the default profile is not reused.
Signed-off-by: Loic Dachary <loic-201408@dachary.org>
Do not restrict the character set of the key=value pairs of osd
erasure-code-profile set to allow for things like:
layers='[
[ "_aAA_aAA", "" ],
[ "bBBB____", "" ],
[ "____cCCC", "" ],
]'
Signed-off-by: Loic Dachary <loic@dachary.org>
Aside from being a bit odd to begin with, using stderr
was causing tests to fail because the output was polluted
by log output which is also on stderr.
Fixes: 9281
Signed-off-by: John Spray <john.spray@redhat.com>
When we prioritize an item move it into a second priority list/set, but
do not start immediately, so that we still obey the max. When we go to
start an item, pull items first off the priority list, then off the regular
list.
Signed-off-by: Sage Weil <sage@redhat.com>
The commit 9b802701f7 preloads the
supported plugins in the OSD. They must also be preloaded in the mon for
the same reasons.
http://tracker.ceph.com/issues/9273Fixes: #9273
Signed-off-by: Loic Dachary <loic-201408@dachary.org>
For the compiler's sake:
tools/ceph_objectstore_tool.cc:2547:15: warning: ‘r’ may be used
uninitialized in this function [-Wmaybe-uninitialized]
Signed-off-by: Zhiqiang Wang <wonzhq@hotmail.com>
evicting
Commit 13b9dc70 introduces local_mtime to fix an issue when doing
flushing. Should also use this to calculate the object age when doing
evicting.
Signed-off-by: Zhiqiang Wang <wonzhq@hotmail.com>
Objecter never saw any OSD maps because of 1e1ee480 and
the dispatchers being in the wrong order -- ignoring map
in Client was hiding it from Objecter.
Fixes: #9266
Signed-off-by: John Spray <john.spray@redhat.com>
ceph_objectgstore_tool:
Fix bugs in the way collection_list_partial() was being called
which caused objects to be seen over and over again.
Unit test:
Fix get_objs() to walk pg tree for pg with sub-directories
Create more objects to test object listing code
Limit number of larger objects
Limit number of objects which get attributes and omaps
Signed-off-by: David Zafman <dzafman@redhat.com>
Add namespaces to testing
Increase filesize so export will have multiple chunks
Put json for each object into the db dict
Signed-off-by: David Zafman <david.zafman@inktank.com>
