Commit Graph

19109 Commits

Author SHA1 Message Date
Sage Weil
ba1d3b1da0 mon: use can_mark_*() helpers
So we can generalize beyond NO* flags.  We'll soon be adding other reasons
to not mark things up/down/in/out.  This lets us keep all though checks in
one place.

The helper methods will tell us why we can't do the thing (e.g., "NODOWN
flag is set").  The callers will generally tell us exactly what didn't
happen (e.g., "failure report of X ignored").

Signed-off-by: Sage Weil <sage@newdream.net>
2012-04-25 11:15:34 -07:00
Sage Weil
59957da257 mon: do not mark osds out if NOOUT flag is set
Do not mark down osds out when NOOUT flag is set.  This is more or less
equivalent to setting a very long 'mon osd down out interval', but
reversible and less annoying.

Signed-off-by: Sage Weil <sage@newdream.net>
2012-04-24 15:47:40 -07:00
Sage Weil
2673875f0c mon: do not mark booting osds in if NOIN flag is set
If the NOIN osdmap flag is set, do not mark booting osds in.  Normally
we would for a range of reasons (always, new, auto-marked-out), but block
them all.

Do not limit manual 'ceph osd in N' commands.

Signed-off-by: Sage Weil <sage@newdream.net>
2012-04-24 15:47:40 -07:00
Sage Weil
9ff535ad2c mon: always remove booting osds from down_pending_out
The down_pending_out tracks OSDs that are down that we may want to
auto-mark out.  If an osd boots, it should be removed from this list
because it is no longer down; it doesn't matter whether it is marked in
or not.

Signed-off-by: Sage Weil <sage@newdream.net>
2012-04-24 15:28:36 -07:00
Sage Weil
addfb2c670 mon: prevent osd mark-down with NODOWN flag
If the NODOWN osdmap flag is set,

 - ignore osd failure reports
 - do not mark osds down due to lack of osd/pg stats

We *do* still allow explicit admin 'ceph osd down N' commands, and a
booting OSD to mark the previous instance of itself down.

Signed-off-by: Sage Weil <sage@newdream.net>
2012-04-24 14:31:42 -07:00
Sage Weil
f0773863ce osd: do not attempt to boot if NOUP
If NOUP is set, do not send the boot message.

We already send onetime subscriptions to the osdmap, so we will find out
about osdmap flag changes.  If it is cleared later, we'll pass into
start_boot() and _got_boot_version() again and send it then.

Signed-off-by: Sage Weil <sage@newdream.net>
2012-04-24 14:31:42 -07:00
Sage Weil
d84255ecba mon: prevent osd from booting if NOUP
Do not add an osd attempting to boot to the map if NOUP is sent.  Instead,
send it the latest osdmap so it knows that it's not allowed to boot.

Signed-off-by: Sage Weil <sage@newdream.net>
2012-04-24 14:16:42 -07:00
Sage Weil
d3f55dd917 mon: 'osd {set,unset} {noin,noout,noup,nodown}'
Move the set/unset flag code into a helper, and also use that for the
pause/unpause commands.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-04-24 14:10:47 -07:00
Sage Weil
6003325d24 osdmap: add NOUP, NODOWN, NOIN, NOOUT flags
These prevent OSDs from being marked up, down, in, or out, respectively.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-04-24 14:10:46 -07:00
Sage Weil
e51772ca19 librbd: pass errors removing head back to user
In particular, the OSD may return EBUSY if there are still watchers.
Ignore ENOENT, as that may indicate we are cleaning up a previously
aborted removal.

Fixes: #2311
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-04-24 13:39:38 -07:00
Sage Weil
ccf7d9309e mon: clean up handle_osd_timeouts a bit
Signed-off-by: Sage Weil <sage@newdream.net>
2012-04-24 10:55:22 -07:00
Sage Weil
36ffed45e8 mon: fix pg stats timeout
We clear out the osd entry when an osd goes up or down.  Thus, if we find
it missing from an up osd, we should start the timer.  Otherwise we get
behavior like this

2012-04-24 13:22:47.888291 7fa5bc587700 mon.peon5752@0(leader).osd e21633 OSDMonitor::handle_osd_timeouts: never got MOSDPGStat info from osd 521. Marking down!
2012-04-24 13:22:50.076394 7fa5bcd88700 log [INF] : osd.521 [2607:f298:4:2243::7088]:6806/53217 boot
2012-04-24 13:22:52.903558 7fa5bc587700 mon.peon5752@0(leader).osd e21638 OSDMonitor::handle_osd_timeouts: never got MOSDPGStat info from osd 521. Marking down!
2012-04-24 13:23:15.144532 7fa5bcd88700 log [INF] : osd.521 [2607:f298:4:2243::7088]:6806/53217 boot
2012-04-24 13:23:17.967118 7fa5bc587700 mon.peon5752@0(leader).osd e21663 OSDMonitor::handle_osd_timeouts: never got MOSDPGStat info from osd 521. Marking down!
2012-04-24 13:23:22.173778 7fa5bcd88700 log [INF] : osd.521 [2607:f298:4:2243::7088]:6806/53217 boot
2012-04-24 13:23:22.981556 7fa5bc587700 mon.peon5752@0(leader).osd e21668 OSDMonitor::handle_osd_timeouts: never got MOSDPGStat info from osd 521. Marking down!
2012-04-24 13:23:45.245380 7fa5bcd88700 log [INF] : osd.521 [2607:f298:4:2243::7088]:6806/53217 boot

when the pg stats message doesn't arrive quickly enough.

Fixes: #2341
Signed-off-by: Sage Weil <sage@newdream.net>
Reviewed-by: Greg Farnum <gregory.farnum@dreamhost.com>
2012-04-24 10:55:18 -07:00
Sage Weil
7b832f4266 mon: fix whitespace
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-04-24 10:49:30 -07:00
Greg Farnum
2b302015e7 mon: fix pgmonitor ratio commands
The indices were set incorrectly when I whipped thi sup. That's what
you get for not testing nor being careful enough in review.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>
2012-04-24 10:35:32 -07:00
Sage Weil
2bdaba4f01 run_seed_to.sh: rework the script, make it more flexible and broaden the tests.
Allow for '-h' and other options such as disabling the journal sync tests,
defining it is to be run on a btrfs FS, enabling exit on error (default is
now 'off'), and allow certain env variables to specify additional options
to each store.

Signed-off-by: Joao Eduardo Luis <jecluis@gmail.com>
2012-04-23 20:31:02 -07:00
Sage Weil
e65b797164 librbd: rev version for discard addition
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-04-23 19:43:25 -07:00
Sage Weil
0777613654 Merge remote-tracking branch 'gh/wip-discard' 2012-04-23 13:58:34 -07:00
Sage Weil
43d1a9201c run_seed_to.sh: remove stray arg
This crept in in commit d1740bd586.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-04-23 09:30:50 -07:00
Sage Weil
e9ecd1b384 perfcounters: tolerate multiple loggers with the same name
Make them unique by appending -<ptr>, so that the json we dump will remain
valid.

We may also want to allow people to share counters of the same type.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-04-22 14:23:52 -07:00
Sage Weil
be438d6e4e Merge branch 'master' into wip-discard 2012-04-21 21:01:49 -07:00
Sage Weil
97f507ffb7 Makefile: disable format-security warning
The prt() varargs function generates this warning

test/rbd/fsx.c: In function ‘prt’:
warning: test/rbd/fsx.c:203:2: format not a string literal and no format arguments [-Wformat-security]
warning: test/rbd/fsx.c:205:3: format not a string literal and no format arguments [-Wformat-security]

Disable that check for the fsx build only.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-04-21 20:28:45 -07:00
Sage Weil
c8377e466c filestore: verify that fiemap works
Check for a bug present in older versions of ext4.  If present, disable
FIEMAP.  See #2328.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-04-21 14:31:42 -07:00
Sage Weil
7471a9b1e2 rados: fix error printout for mapext
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-04-21 13:32:46 -07:00
Sage Weil
07ddff4271 librbd: instrument with perfcounters
Track IO operations on a per-image basis.

Implements: #1451
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-04-21 12:50:05 -07:00
Sage Weil
fb9fdf45b0 librbd: fix ictx_check pointer weirdness by using std::string
I was seeing failures of LibRBD.TestIOToSnapshot where we would fail to
refresh after rollback, even though the snap existed.  I assume it is
because the std::string whose c_str() we were pointing to was reallocated.

Use a std::string here instead.

This code is weird.

Signed-off-by: Sage Weil <sage@newdream.net>
2012-04-20 17:13:08 -07:00
Samuel Just
888a082f23 FileJournal: don't wait flusher until completions are queued
Fixes: #2324
Signed-off-by: Samuel Just <samuel.just@dreamhost.com>
2012-04-20 17:01:21 -07:00
Sage Weil
d1740bd586 filestore: fix collection_add journal replay problem
In collection_add we have a two-phase guard set on the linked object via
the old name.  During replay, we might see that the dest name is missing
and replay the operation, and in the process overwrite a newer guard with
an older one.

Avoid this by checking the source name too, and skipping the operation
entirely if a new guard exists.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-04-20 16:56:57 -07:00
Sage Weil
92b299afc5 FileStoreDiff: flip sense of diff*() methods around
true means diff, false means same.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-04-20 16:56:06 -07:00
Sage Weil
ca369c98a3 test_idempotent_sequence: Use FileStoreDiff class instead.
Use FileStoreDiff instead of having the diff code embedded in the test,
allowing for more tests and people to use the code in case it comes in
hand.

Signed-off-by: Joao Eduardo Luis <jecluis@gmail.com>
2012-04-20 16:55:55 -07:00
Joao Eduardo Luis
5466ebc776 test_idempotent_sequence: Output missing options on "usage".
Signed-off-by: Joao Eduardo Luis <jecluis@gmail.com>
2012-04-20 16:51:52 -07:00
Joao Eduardo Luis
474612918e FileStoreDiff: check if two FileStores match.
This code should be on a stand-alone class, instead of being embedded on
a single test, in case someone or something find it useful somewhere down
the line.

Signed-off-by: Joao Eduardo Luis <jecluis@gmail.com>
2012-04-20 16:51:40 -07:00
Sage Weil
4ddbbf5467 librbd: allow image resize to non-block boundaries
The caller is still invalidating the entire cache, so we don't need to
deal with discard at this level.  That might be worth cleaning up
later, though.

Fixes: #2296
Signed-off-by: Sage Weil <sage@newdream.net>
2012-04-20 16:51:01 -07:00
Sage Weil
165038d589 objectcacher: rename truncate_set -> discard_set, and use discard
Do not assume the object extents are at the trailing edge of objects.
Instead, discard arbitrary extents.  Fix callers.

Signed-off-by: Sage Weil <sage@newdream.net>
2012-04-20 16:51:01 -07:00
Sage Weil
7232396598 objectcacher: implement Object::discard()
Discard a range of bytes from an object.

Signed-off-by: Sage Weil <sage@newdream.net>
2012-04-20 16:51:01 -07:00
Sage Weil
5b60343710 librbd: fix debug output
objects is misleading here, these are byte offsets

Signed-off-by: Sage Weil <sage@newdream.net>
2012-04-20 16:51:01 -07:00
Sage Weil
81af4e8618 librbd: make discard invalidate the range in cache
Fed this to test_librbd_fsx and it was happy.

Signed-off-by: Sage Weil <sage@newdream.net>
2012-04-20 16:51:01 -07:00
Sage Weil
b94d6a6cf4 librbd: fix zeroing of trailing bits on short reads that span objects
handle_sparse_read() was taking buf_ofs and buf_len, but buf_len was being
interpreted as the total size of the buffer, not the length of the extent
in the buffer start at buf_ofs.  Both callers pass in an extent length, so
fix the zero code to do the right thing.

Specifically, the behavior I saw was:

 - read range spanning 2 objects, trailing 20k and leading 50k
 - first object didn't exist, zeroed first 20k of buffer
 - second object didn't exist, zeroed next 30k (50k-20k) of buffer
   - the last 20k of buffer was unzeroed.

Signed-off-by: Sage Weil <sage@newdream.net>
2012-04-20 16:51:01 -07:00
Sage Weil
379c5e721e librbd: fix debug output for image resize
Print old -> new, not new -> old.

Signed-off-by: Sage Weil <sage@newdream.net>
2012-04-20 16:51:00 -07:00
Sage Weil
dd78c3df65 test_librbd_fsx: port newer xfsprogs version
Builds and runs... for a few ops at least.

Signed-off-by: Sage Weil <sage@newdream.net>
2012-04-20 16:51:00 -07:00
Sage Weil
e861e20a1a revert to xfstests' fsx, which has discard support 2012-04-20 16:51:00 -07:00
Sage Weil
a8aaa93b3e admin_socket: add a separate debug subsys/level
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2012-04-20 16:50:43 -07:00
Sage Weil
f7699cc993 log: prefix dump with line numbers
This makes it easier to interpret the dump, and makes it obvious what is
dump (and potentially a dup of something that was already logged) and what
is not.

Signed-off-by: Sage Weil <sage@newdream.net>
2012-04-20 16:36:54 -07:00
Sage Weil
a68b3069de test_filestore_workloadgen: track pg log size in memory
Avoid a stat!

Signed-off-by: Sage Weil <sage@newdream.net>
2012-04-20 15:35:44 -07:00
Sage Weil
6b28f55cd3 test_filestore_workloadgen: whitespace
Signed-off-by: Sage Weil <sage@newdream.net>
2012-04-20 15:30:45 -07:00
Sage Weil
2834f601ef organize test_filestore_idempotent*
Signed-off-by: Sage Weil <sage@newdream.net>
2012-04-20 15:25:32 -07:00
Sage Weil
8bb58bc2b4 remove hopelessly flawed test_filestore_idempotent
Signed-off-by: Sage Weil <sage@newdream.net>
2012-04-20 15:23:17 -07:00
Sage Weil
94ebe8d951 test_store -> test_filestore
and organize source

Signed-off-by: Sage Weil <sage@newdream.net>
2012-04-20 15:22:36 -07:00
Sage Weil
77c645a199 test_wrkldgen -> test_filestore_workloadgen
Also put the code in src/test/filestore/.

Signed-off-by: Sage Weil <sage@newdream.net>
2012-04-20 15:20:16 -07:00
Sage Weil
f00dab6f4a ceph-authtool: rename cli test dir
Signed-off-by: Sage Weil <sage@newdream.net>
2012-04-20 13:28:23 -07:00
Sage Weil
613d81794b ceph-conf: rename cli test dir
Signed-off-by: Sage Weil <sage@newdream.net>
2012-04-20 13:28:03 -07:00