Commit Graph

11779 Commits

Author SHA1 Message Date
Sage Weil
deb9ef760e v0.24~rc 2010-11-07 09:44:04 -08:00
Sage Weil
0b19092098 Merge remote branch 'origin/testing' into unstable 2010-11-07 09:42:51 -08:00
Sage Weil
a4674af5db mds: eval: put scatter in MIX if replicated, otherwise LOCK
Signed-off-by: Sage Weil <sage@newdream.net>
2010-11-07 07:49:59 -08:00
Sage Weil
33c6e230a2 mds: do not scatter_writebehind in MIX state
Replicas might come in while we're flushing and get a MIX state with
the old state.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-11-07 07:45:52 -08:00
Sage Weil
1bf8e73299 Merge branch 'unstable' into mix_stale 2010-11-06 21:05:11 -07:00
Sage Weil
bdc2fa5b34 mds: remove MIX_STALE
Yay, we don't need it!

If we can't update the frag on scatter, fine.  The staleness of the frag
is implicit in the frag's scatter stat version not matching the inode's.
If/when we do want to update it, the frag will clearly be writable, and
we can bring it back in sync then.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-11-06 21:01:20 -07:00
Sage Weil
c1ee560e42 mds: don't fuss with versions when taking frag/rstat from frag; it's never stale here
Signed-off-by: Sage Weil <sage@newdream.net>
2010-11-06 21:01:20 -07:00
Sage Weil
1eb94da2f6 mds: introduce/use helpers to resync stale fragstat/rstat; update version
Simplifies code.

Also, update the version when we resync!

Signed-off-by: Sage Weil <sage@newdream.net>
2010-11-06 21:01:20 -07:00
Sage Weil
c203482954 mds: ignore done_locking on slave requests' acquire_locks()
Slave requests ask for each xlock one at a time.  Don't bail out based on
the done_locking flag.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-11-06 21:00:55 -07:00
Sage Weil
51b6a863a8 mds: don't use helper for rename srcdn
The rdlock_path_xlock_dentry helper works for _auth_ dentries that we
create locally in an auth dirfrag.  For the srcdn, we need to discover an
_existing_ dentry that is not necessarily auth.

Call path_traverse ourselves, but be careful to take the appropriate locks
on the resulting dn, dir, and ancestors.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-11-06 21:00:51 -07:00
Sage Weil
eb0a60d024 mds: never complete a gather on a flushing lock
The scatter_writebehind() takes a wrlock, but that may still allow the lock
to complete a gather to LOCK and even move to say MIX before the data is
committed.  Bad news!

Signed-off-by: Sage Weil <sage@newdream.net>
2010-11-06 21:00:46 -07:00
Sage Weil
bdf3bc5ef2 mds: update version when bring stale rstat back up to date
Signed-off-by: Sage Weil <sage@newdream.net>
2010-11-06 09:38:15 -07:00
Sage Weil
a74054d11d mds: simplify stale semantics a bit
is_stale() => next MIX is MIX_STALE. Stale flag is then cleared.  Then we
special case the import to preserve stale-ness.

TODO: add_replica_inode likely has this same problem.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-11-06 07:58:32 -07:00
Sage Weil
e27f111f31 mds: preserve stale state on import; some cleanup
Our new invariant is that MIX_STALE always implies is_stale().  And on
import, if is_stale(), MIX becomes MIX_STALE.  This ensures that a replica
that we put into MIX_STALE doesn't turn back into MIX if we import it
and take the auth's state in CInode::decode_import().

Signed-off-by: Sage Weil <sage@newdream.net>
2010-11-05 21:52:28 -07:00
Sage Weil
a582345c57 Merge branch 'mix_stale' into unstable 2010-11-05 17:08:10 -07:00
Sage Weil
4126d1ce36 mds: add more verify_scatter asserts
For catchings fragstat errors sooner.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-11-05 17:06:10 -07:00
Sage Weil
ae670c3387 mds: fix version check on resyncing stale rstat in predirty_journal_parents
We're resyncing rstat, so check the rstat version (not fragstat!)

Signed-off-by: Sage Weil <sage@newdream.net>
2010-11-05 15:24:53 -07:00
Greg Farnum
4cee6ead7b mds: Fix bad inode deref.
Accidentally trying to print out the CInode after removing it in trim_non_auth!
Move the print to before it's been unlinked/removed/etc.
2010-11-05 12:45:45 -07:00
Colin Patrick McCabe
93344fb28e Revisit std::multimap decoder
Previously I changed the std::multimap decoder to minimize the number of
constructor invocations. However, it could be much more expensive to
copy an initialized (decoded) val_t than to copy an empty one. For
example, if we are decoding std::multimap < int, std::set <int> >. So
change the code to insert a non-decoded val_t again.

However, this still saves two constructor invocations over the original.

Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-11-05 12:20:38 -07:00
Colin Patrick McCabe
f015c9894a autogen.sh: check for pkg-config
To avoid seeing confusing errors later in the configure process, in
autogen.sh, check to make sure the pkg-config program is installed.

Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-11-05 11:34:11 -07:00
Sage Weil
989fa67d43 mds: preserve version when recovering rstat from dirfrag in predirty_journal_parents
We don't want to screw up the version here.  This aligns the code with
other instances of this check.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-11-05 10:38:35 -07:00
Sage Weil
9586e9052c mds: restructure finish_scatter_gather_update()
Separate behavior into two dimensions: whether or not we are updating
the dirfrag, and whether or not the dirfrag is stale.

Change the various helpers to NOT implicitly update accounted_*, as the
caller doesn't always want that, notably when we are non-stale but frozen.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-11-04 23:20:33 -07:00
Sage Weil
669a8afaa4 mds: do not bump scatter stat lock in predirty_journal_parents
If we're in the MIX state, we clearly can't touch this without screwing up
the delicate scatter/gather behavior.  If we're in, say, LOCK, there is
still no reason to update it.  One frag at least is local and auth if we
are in this code, but there may be other frags on other nodes.  This would
just make them appear stale when they are not.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-11-04 23:15:06 -07:00
Sage Weil
663b470f91 mds: mark scatterlock stale on import of stale frag scatter stat
When the lock scattered, if we didn't have an auth frag that was frozen,
we go into MIX state.  Later, we may import a stale dirfrag.  We need to
move to MIX_STALE at that point, and/or mark the lock stale so that any
subsequent transition does so.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-11-04 22:48:09 -07:00
Sage Weil
63c1ad84d2 mds: match bottom half of assilate_dirty_rstat_inodes with a dir flag
We only do the assimilate_dirty_rstat_inodes if we do an update AND the
frag rstat was non-stale, but the bottom half (_finish) doesn't have the
same info to know whether we did it because the top half updates the
fragstat version.  Use a flag to indicate we've updated the dirfrag so
the bottom half will only run when needed.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-11-04 22:44:01 -07:00
Sage Weil
9b6d96e9aa mds: fix inode version used for inest in decode_lock_state
We need to pass the inode rstat's version into finish_scatter_update, not
the shadowed local variable.  Otherwise we don't update the dirfrag when
we should.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-11-04 22:19:53 -07:00
Colin Patrick McCabe
62716aa7c9 PGMonitor::update_from_paxos: check for bad input
Be more robust against bad data coming in from the network.

Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-11-04 16:22:21 -07:00
Colin Patrick McCabe
8f3672dc48 Replace sprintf with snprintf
Replace sprintf with snprintf. This is especially critical when the
format string includes "%s".

Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-11-04 14:33:48 -07:00
Colin Patrick McCabe
56179d12f8 start_profiler/enable_profiler_options:fix memleak
Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-11-04 14:26:08 -07:00
Colin Patrick McCabe
e6a751bda5 Set HEAP_PROFILE_INUSE_INTERVAL based on conf
Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-11-04 14:11:41 -07:00
Colin Patrick McCabe
8c8bfdb3e2 CInode::make_path_string: don't coerce ino
CInode::make_path_string: don't coerce the inode number to 32-bits.
Everyone else is treating it as 64 bits; this function should too.

Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-11-04 14:09:32 -07:00
Sage Weil
53f6ed163b mds: mds debug scatterstat to print out projected rstat/fragstat
Signed-off-by: Sage Weil <sage@newdream.net>
2010-11-04 13:17:48 -07:00
Sage Weil
f23ba003d7 mds: verify single frag rstat on projection too
Currently we do a sanity check on gather; do the same check in
project_rstat_frag_to_inode().

Signed-off-by: Sage Weil <sage@newdream.net>
2010-11-04 13:17:48 -07:00
Greg Farnum
4df92adea4 Merge branch 'dumpjournal' into unstable 2010-11-04 11:58:30 -07:00
Greg Farnum
d3c2b9cb2b cmds: Include journal dumper functionality. 2010-11-04 11:41:26 -07:00
Greg Farnum
e0a5de25c1 dumper: Add new Dumper class.
This lets you dump an MDS journal to a file.
2010-11-04 11:41:21 -07:00
Sage Weil
28f956ae31 mds: fix optional frag asserts
We want these to trigger when mds_verify_scatter is true.  Only one !.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-11-04 11:33:49 -07:00
Greg Farnum
86d6e51e8b objecter: add new wait_for_osd_map function. 2010-11-04 11:28:52 -07:00
Sage Weil
8a41d096c7 osd: clean up active <-> booting state transitions
Among other things, get rid of the 'wrongly marked down' log message on
normal startup.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-11-04 11:13:31 -07:00
Colin Patrick McCabe
f917df7942 TestEncoding: count number of ctor invocations
Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-11-04 10:24:52 -07:00
Sage Weil
e90a3b623b mds: dump corrupt events; optionally skip them
If we encounter a bad event in the journal, dump it to the log.

Optionally skip it, if 'mds log skip corrupt events = true'.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-11-03 22:28:57 -07:00
Sage Weil
1c934ebd6f mds: wait for last_failure_osd_epoch before starting journal replay
This is extremely important, and it forces the MDS to get the osdmap that
includes the blacklist entry for its predecessor.  This in turn means that
any OSD we contact trying to read the journal will be forced to get that
osdmap (or newer) before handling our read request, which means that
anything we read cannot be overwritten by a racing request from our
predecessor.  This prevents two MDSs writing to the journal at the same
time.

This change fixes potential (and observed!) journal corruption.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-11-03 22:28:57 -07:00
Sage Weil
f511286634 mon: blacklist and update last_failure_osd_epoch in all failure paths
This includes the pure failure in do_stop(), and the explicit admin
fail command.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-11-03 22:28:56 -07:00
Sage Weil
6345fcdafa mon: update mdsmap.last_failure_osd_epoch when blacklisting
We need to note the osdmap epoch the taking-over mds needs in the mdsmap.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-11-03 22:28:54 -07:00
Sage Weil
0fb2297481 mds: add last_failure_osd_epoch to extended section of mdsmap
Signed-off-by: Sage Weil <sage@newdream.net>
2010-11-03 22:28:47 -07:00
Sage Weil
0d1bfe06b3 client: print useful max_size waiting message
Signed-off-by: Sage Weil <sage@newdream.net>
2010-11-03 16:41:29 -07:00
Sage Weil
fc9059e527 Merge branch 'mix_stale' into unstable 2010-11-03 16:40:19 -07:00
Sage Weil
4f24fcbc8a debian: add gtk build-depends
For ceph -g.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-11-03 16:40:13 -07:00
Sage Weil
d0c29d7d2c mds: add 'mds verify scatter' and re-add some scatter asserts
Check on ifile and inest gather that stats match single-frag dirs.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-11-03 14:47:32 -07:00
Sage Weil
fd57f4de21 mds: fix put_xlock() assert for slave masters
If we are a master of a slave, the state will be LOCK.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-11-03 14:47:32 -07:00