Previously, we did not actually persist the osd compatibility
mask. Without persisting the current compat mask, a previous,
incompatible version of the OSD would not be prevented from
starting on the same store.
Signed-off-by: Samuel Just <sam.just@inktank.com>
CompatSet users number the Feature objects rather than
providing masks. Thus, we should do
mask |= (1 << f.id) rather than mask |= f.id.
In order to detect old, broken encodings, the lowest
bit will be set in memory but not set in the encoding.
We can reconstruct the correct mask from the names map.
This bug can cause an incompat bit to not be detected
since 1|2 == 1|2|3.
fixes: #2748
Signed-off-by: Samuel Just <sam.just@inktank.com>
Greg points out that we could have a situation like:
- mon recovers..
- goes through osdmaps, notes an osd was removed and removes from
full/nearfull
- goes through pgmaps, and re-adds it when it encounters some osd_stat_ts.
Fix this by removing the osd from the full/nearfull set when we remove
the osd_stat_t from the pgmap. Any osd removal is always followed by
an osd_stat_rm[] record when the primary processes the new osdmap and
proposed the appropriate pgmap updates.
Signed-off-by: Sage Weil <sage@inktank.com>
It is possible for a .new file to already exist, potentially with a
larger size. This would happen if:
- we were proposing a different value
- we crashed (or were stopped) before it got renamed into place
- after restarting, a different value was proposed and accepted.
This isn't so unlikely for the log state machine, where we're
aggregating random messages. O_TRUNC ensure we avoid getting the tail
end of some previous junk.
I observed #2593 and found that a logm state value had a larger size on
one mon (after slurping) than the others, pointing to put_bl_sn_map().
While we are at it, O_TRUNC put_int() too; the same type of bug is
possible there, too.
Fixes: #2593
Signed-off-by: Sage Weil <sage@inktank.com>
Use a helper to dump /proc/self/fd when we hit EMFILE in the filestore.
Ideally, we should trigger this in other appropriate places, but it is
not immediately clear that there is a sane way to do that.
Fixes: #2330
Signed-off-by: Sage Weil <sage@inktank.com>
Users probably want get_pg_acting_rank(). If they don't, they can probably
have the mapping and can calculate the rank themselves. Having this here
is asking for bugs like #2022.
Signed-off-by: Sage Weil <sage@inktank.com>
We want to look at the acting set here, nothing else. This was causing us
to erroneously queue ops for later (wasting memory) and to erroneously
print out a 'misdrected op' message in the cluster log (confusion and
incorrect [but ignored] -ENXIO reply).
Fixes: #2022
Signed-off-by: Sage Weil <sage@inktank.com>
Make the helper exclusively for the PG != NULL cases, and open-code the
one PG == NULL caller. This is simpler, and lets us include more useful
information in the log message.
Signed-off-by: Sage Weil <sage@inktank.com>
Stores absolute path to the generated keyring so that tests running in
other directories (e.g. src/java/test) can simply reference the
generated ceph.conf.
Signed-off-by: Noah Watkins <jawhawk@cs.ucsc.edu>
In order to test monitor and osd failure detection and false
positive correction, this patch adds the following options:
1. osd_debug_drop_ping_probability: probability of dropping
a string of pings from a client upon ping recipt.
2. osd_debug_drop_ping_duration: number of pings to drop in
a row.
This should help with replicating some wrongly-marked-down
thrashing cases.
Signed-off-by: Samuel Just <sam.just@inktank.com>
When we get a ping reply, remove the peer from the failure_queue
and send a still alive message if the peer is in the failure_pending
map.
Otherwise, the monitor could slowly accumulate sporadic failure reports
leading to an osd being incorrectly marked out.
This bug may have been contributing to the wrongly-marked-down
thrashing observed on some systems.
Signed-off-by: Samuel Just <sam.just@inktank.com>
If the osd recieving the log has divergent entries, it will
also have a "divergent" stat structure. In general, it suffices
to simply trust the stat structure shipped with the authoritative
log and info since merge_log is only used to merge an authoritative
log.
Probably fixes#2769.
In cases like #2769, this bug can result in a primary with a stat
structure which double counts an operation: once for the
divergent operation, and once for the replay. It turned up
in a regression suite run as a scrub stat mismatch.
Signed-off-by: Samuel Just <sam.just@inktank.com>
Use a single helper to add/remove osds from the [near]full sets. This
keeps the logic in a single place, and simplifies the code somewhat.
Signed-off-by: Sage Weil <sage@inktank.com>
The [near]full sets are volatile state. Remove removed (or created)
osds from the set when we process a map.
Fixes: #2779
Signed-off-by: Sage Weil <sage@inktank.com>
Consider the following sequence:
1. issue, apply repop
2. replicas and primary commit
Here, repop->waitfor_(ack|disk) are empty, so we mark
repop->done and remove_repop.
3. interval change, repops still in queue are marked aborted
4. activate, last_update_applied = last_update
5. the repop from one enters apply_repop, is not aborted,
and finds that last_update_applied has passed it by.
Fixes#2749
Signed-off-by: Samuel Just <sam.just@inktank.com>
test/test_librbd.cc: In member function ‘virtual void LibRBD_TestClone_Test::TestBody()’:
warning: test/test_librbd.cc:1040:111: format ‘%ld’ expects argument of type ‘long int’, but argument 2 has type ‘uint64_t {aka long long unsigned int}’ [-Wformat]
warning: test/test_librbd.cc:1040:111: format ‘%ld’ expects argument of type ‘long int’, but argument 3 has type ‘uint64_t {aka long long unsigned int}’ [-Wformat]
warning: test/test_librbd.cc:1040:111: format ‘%ld’ expects argument of type ‘long int’, but argument 4 has type ‘int64_t {aka long long int}’ [-Wformat]
Signed-off-by: Sage Weil <sage@inktank.com>
This used to be conditional on config having osd_crush_location set,
but with that, minimal configuration left the OSD completely out of
the crush map, and prevented the OSD from starting properly.
Note: Ceph does not currently let this mechanism automatically move
hosts to another location in the CRUSH hierarchy. This means if you
let this run with defaults, setting osd_crush_location later will not
take effect. Set up your config file (or Chef environment) fully
before starting the OSDs the first time.
Signed-off-by: Tommi Virtanen <tv@inktank.com>
Issue #2776. Allow the removal of multiple objects in a single
rados tool command:
# rados -p pool rm obj1 [obj2 [...]]
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
Bad linebreaks, wrapping, stringification, missing doc for bench args
Signed-off-by: Dan Mick <dan.mick@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>
Bug #2772. This fixes an issue that was introduced when we
added the 'rados cp' command. The -t param was already used
for rados bench. With this change the only way to specify
a target pool is using --target-pool.
Though this problem is post argonaut, the 'rados cp' command
has been backported, so we need this fix there too.
Backport: argonaut
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
Sum the quantized weights for each bucket, and check that for overflow.
This could change the results of a compile marginally if the map is using
non-divisible weight values that quantize funny. The old code might
calculate a bucket sum that is not the actual sum of the quantized weights.
Signed-off-by: Sage Weil <sage@inktank.com>
Disallow setting OSD weights to a value over 10,000 and cap bucket weight
at 10,000,000 in a CRUSH map. Addresses issue #2101.
Signed-off-by: caleb miles <caleb.miles@inktank.com>
Bad linebreaks, wrapping, stringification, missing doc for bench args
Signed-off-by: Dan Mick <dan.mick@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>
This is leftover from when we built a libcrush.so. We can re-add when we
start doing that again.
Reported-by: Laszlo Boszormenyi <gcs@debian.hu>
Signed-off-by: Sage Weil <sage@inktank.com>
* simple helper to translate name to id
* verify sub type is valid in caller
* assert sub type is valid in method
* simplify iterator usage
Among other things, this gets rid of this noise in the logs:
2012-07-10 20:51:42.617152 7facb23f1700 1 mon.a@1(peon).log v310 check_sub sub monmap not log type
Signed-off-by: Sage Weil <sage@inktank.com>
