Commit Graph

11373 Commits

Author SHA1 Message Date
Colin P. McCabe
e305ea017d osd: cache coll_t in PG
Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-10-01 10:27:51 -07:00
Colin P. McCabe
aaa58f5df4 gitignore: Ignore cscope and vim temporary files
Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-09-30 18:13:44 -07:00
Colin Patrick McCabe
a4138c9050 osd: generalize coll_t to a string
coll_t is now a string. META_COLL and TEMP_COLL are just constants now.

Now there is a constructor that takes pgid_t and snapid_t, rather than
factory methods. It's clear what that constructor does, so wrapping it
in factory methods should be unecessary.

Bump coll_t serialization version to 3. Implement decoding for the old
versions.

Signed-off-by: Colin McCabe <colinm@hq.newdream.net>
2010-09-30 17:52:53 -07:00
Sage Weil
0e67718a36 mds: drop bad assert
Introduced by f1921c3a95.  Should probably
audit this code.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-09-30 10:54:58 -07:00
Sage Weil
ea6286ac24 Makefile: add missing include
Signed-off-by: Sage Weil <sage@newdream.net>
2010-09-30 10:54:58 -07:00
Colin McCabe
7657a6d5b3 interval_set: hide data members
This change makes interval_set::m and interval_set::_size private data
members in interval_set, instead of public. This change also creates a
non-const iterator. Using this iterator, users can modify the length of
an interval. So now, all users can use the iterators rather than
interacting with the class internals directly.
2010-09-29 19:03:36 -07:00
Greg Farnum
548df8ec13 mon: Fix issue first addressed in 2c5a3d99aa.
We were properly falling out of the while loop when we reached end(), but
not checking for it in the following if-else. Now we do!
Reported-by: Henry C Chang <henry_c_chang@tcloudcomputing.com>
2010-09-29 11:32:01 -07:00
Sage Weil
0523ce102f osd: do not request backlog from peers with empty pg
This avoids stalling out peering, because the peer just responds with
another 'empty' PG::Info in response (which we already have).

Signed-off-by: Sage Weil <sage@newdream.net>
2010-09-29 08:45:06 -07:00
Sage Weil
aa04c8fbbd osd: try to object from other replica(s) on EOF
If during recovery we are unable to pull from a replica due to reaching
EOF (e.g., zeroed out object), pull from the next available replica (if
any).

Eventually this should be extended to do the same when a checksum fails.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-09-29 08:45:06 -07:00
Colin Patrick McCabe
3bab6ac16a Add the setup-chroot.sh script
The setup-chroot.sh script is very handy for building the server in a
chroot environment. I thought I would share it here in case anyone else
finds it useful.
2010-09-28 19:25:10 -07:00
Sage Weil
2223b22dfc osd: clarify comment in recovery code
Signed-off-by: Sage Weil <sage@newdream.net>
2010-09-28 12:31:09 -07:00
Greg Farnum
ab62aabf1f msgr: Don't take over old pipes if they're lossy.
Fixes bug #443.
2010-09-28 11:45:43 -07:00
Colin Patrick McCabe
d8f595e62f Implement interval_set::const_iterator 2010-09-27 14:47:18 -07:00
Colin Patrick McCabe
fb41b8f823 Rename interval_set::begin and end
Rename interval_set::begin and end to interval_set::range_begin and
interval_set::range_end, respectively.
2010-09-27 12:48:57 -07:00
Yehuda Sadeh
5a9a5e74df rgw: send 100-continue response only if requested 2010-09-27 10:01:47 -07:00
Sage Weil
da41f66fa8 mds: set PREXLOCK next state to LOCK
This really shouldn't happen (!), but if it does, at least avoid getting
the primary state out of sync with the replicas.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-09-27 08:33:27 -07:00
Sage Weil
f1921c3a95 mds: don't block request on freezing if we're already auth_pinned.
If we already auth_pinned, we're past the gates; don't stop on freezable.

This screws up xlock: the lock moves to PREXLOCK state, but the request
that would normally xlock it gets deferred because of a racing freezing
of the tree.  Then the PREXLOCK gather kicks in and badness happens.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-09-27 08:31:34 -07:00
Sage Weil
ead92c4fbe mds: block request is freezing
This prevents a deadlock where:

 - client request releases caps
 - caps release deferred (freezing)
 - request proceeds (freezing)
 - can't revoke caps because they're released (but deferred)
 deadlock!

Signed-off-by: Sage Weil <sage@newdream.net>
2010-09-25 14:44:50 -07:00
Sage Weil
2ae0172b06 osd: add coll_t::is_pg() method
This makes the interface a bit more adaptable for a situation where it has
a simple string representation instead of the strict structure it has now.
Eventually this function can simply attempt a pg_t parse.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-09-24 20:10:08 -07:00
Sage Weil
afa2d2fea1 mds: fix ESessions event type
Using the singular event type meant trying to decode as an ESession (and
failing!).

Signed-off-by: Sage Weil <sage@newdream.net>
2010-09-24 16:27:45 -07:00
Sage Weil
44587aaf5c mds: fix xlock state asserts for LocalLock
The LocalLock (versionlocks) allow xlocking but have only a single state
(LOCK_LOCK).

Signed-off-by: Sage Weil <sage@newdream.net>
2010-09-24 16:27:45 -07:00
Sage Weil
1d7eb8a082 mds: fix locallock rule (missing column)
The fwr column was missing, leading to a 0 for xlock, which broke slave
xlocks.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-09-24 15:22:18 -07:00
Sage Weil
f4114cf27e mds: add rename failure hooks 2010-09-24 15:15:12 -07:00
Sage Weil
87d5df1b79 osd: fix pull completion tests, again
op->complete==false is inconclusive.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-09-24 14:50:05 -07:00
Sage Weil
0a719d515e ceph: make version in backtrace look nice
match debug log
include .h, not .c
2010-09-24 14:01:52 -07:00
Sage Weil
27f712a5e2 osd: clean out redundant (and wrong) complete calculation
Signed-off-by: Sage Weil <sage@newdream.net>
2010-09-24 13:57:00 -07:00
Sage Weil
eb0a3fa679 osd: make sparse data/clone push behave with partial object push
We can't error out if we don't get everything we want in one go now that
we support pushing objects in pieces.  Remove this check entirely, since
we don't have a good error handling case anyway.
2010-09-24 11:44:23 -07:00
Sage Weil
7ca8e38c5d mds: defer MExportDirDiscover until we have root inode open
Otherwise we can't traverse or do anything useful.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-09-24 11:44:22 -07:00
Sage Weil
ea49ea65e2 mds: alloc auth xlock on versionlock/LocalLock
This is done when we do a slave xlock in order do avoid pipelining updates
to the inode, making rollback of complex operations like rename/link
safe.
2010-09-24 11:44:22 -07:00
Sage Weil
63bc150fe9 mds: defer cap release and update consistently when frozen
We need to preserve the order of processing of cap release and writeback
messages across handle_client_caps() and process_request_cap_release().
Use a helper with the appropriate condition, and defer the release
processing as needed.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-09-24 11:44:22 -07:00
Sage Weil
7f329dfe46 mds: refactor process_cap_update a bit
Fewer args
2010-09-24 11:44:22 -07:00
Sage Weil
82e52b4c45 mds: drop old/incorrect comment 2010-09-24 11:44:22 -07:00
Sage Weil
f0c89bab41 mds: always mark parent scatterlock when marking dirty rstat
Note that this will let the parent nestlock 'dirty' state get out of
sync with the lock state, as the whole point of the dirty rstat lists is
that it can happen any time.  It does, however, queue us up.
2010-09-24 11:44:22 -07:00
Sage Weil
416470da27 mds: mark dirty rstat inodes during recovery 2010-09-24 11:44:22 -07:00
Sage Weil
c503d362b8 mds: error to log when inode/dirfrag rbytes get out of sync
Signed-off-by: Sage Weil <sage@newdream.net>
2010-09-24 11:44:22 -07:00
Sage Weil
143438aa5c mds: stubs for printing projected fragstat/rstat
Disabled for now, since it is so freaking verbose.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-09-24 11:44:22 -07:00
Sage Weil
28a4c34095 mds: assimilate dirty rstat inodes during scatter_writeback
We put some of the predirty_journal_parents() code that calls the
project_rstat_inode_to_frag() into a common helper and use that.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-09-24 11:44:22 -07:00
Sage Weil
c0d7e8f3c8 mds: maintain dirty_rstat list
Add on fetch or import of dirty_rstat; clear on export of dirty_rstat.
2010-09-24 11:44:22 -07:00
Sage Weil
29b1e84843 mds: add dirty_rstat CInode elist, state, pins
We need to track inodes with unpropagated rstat data on a per-dirfrag
basis so that we can propagate it when the nestlock becomes writeable.
2010-09-24 11:44:22 -07:00
Yehuda Sadeh
628e28e2b2 qa: improved rgw tests 2010-09-24 10:16:14 -07:00
Sage Weil
f7f32b24b1 makefile: drop quotes on tcmalloc CXXFLAGS 2010-09-23 21:20:31 -07:00
Sage Weil
043c9c8bed mds: scatter pin frozen tree on importer too
The importer also needs to scatter pin.  This avoids scatterlock gather
races like so:

A: start exporting to B
A: freeze, scatter pin tree
C: initiate gather
A: delay replay to gather
B: reply to gather, do not include (non-auth) dirfrag
A,B: finish migration
A: reply to gather, do not include (now non-auth) dirfrag
C: gets no info about the dirfrag!

By pinning on the importer, we ensure that at least one MDS will respond
to the gather with auth dirfrag info.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-09-23 16:44:47 -07:00
Sage Weil
c82bc1ccf6 mds: drop dead Renamer code 2010-09-23 16:44:47 -07:00
Sage Weil
2fbd843f3e mds: clarify inode dirstat/rstat locking
The accounted_rstat must always remain consistent with the parent dirfrag,
which in turn means it is governed by the parent's nestlock.

The rstat is protected by _this_ inode's nestlock, and is updated by
scatter_writebehind() or predirty_journal_parents().

Signed-off-by: Sage Weil <sage@newdream.net>
2010-09-23 16:44:47 -07:00
Sage Weil
b108b6a713 mds: fix bounding frag rstat/fragstat update during import
Be careful about when we update bounding dirfrag info during an import.  If
the lock is in a MIX state, we do NOT want to update, since the inode
auth doesn't know jack (unless they are also dirfrag auth, in which case
we'll find out when we unscatter anyway).

Fixes fix 9d81f9d6.
2010-09-23 16:44:47 -07:00
Sage Weil
1c09263467 mds: do not scatter_writebehind on nudge if replicated
This can cause the inode rstat etc to become out of sync with dirfrag
accounted_rstat when the scatterlock is not in a gathered state: the
local values will get updated but those on other nodes will not, and the
inode will drift out of sync with the dirfrags.

Other callers to scatter_writebehind() are all in contexts where we have
_just_ gathered dirfrag state, or there is no remote dirfrag state to
gather.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-09-23 16:44:47 -07:00
Sage Weil
d715338110 mds: use scatter pins for migration instead of rd/wrlocks
This is simpler (for the migrator), and wrlocks allow scatter_writebehind,
which is a no-no for a frozen tree.  By pinning the frozen dir's parent
inode, we prevent any scatter or unscatter operations from implicitly
updating metadata within the frozen root dirfrag.
2010-09-23 16:44:47 -07:00
Sage Weil
961e186d47 mds: add scatterpins 2010-09-23 16:44:47 -07:00
Greg Farnum
690607cb1f backtrace: include ceph version 2010-09-23 10:57:22 -07:00
Yehuda Sadeh
1eaec17943 qa: add simple rgw test 2010-09-22 22:40:28 -07:00