Commit Graph

13924 Commits

Author SHA1 Message Date
Sage Weil
05c281bfa9 Revert "autoconf: Complain if tcmalloc is not found."
This reverts commit a9afdca18e.

This breaks debian build on lenny because tcmalloc isn't part of libgoogle-perftools-dev.
2011-04-02 22:18:59 -07:00
Josh Durgin
d941422211 cclass: add bindir option
Ceph binaries don't live in /usr/local on autotest machines.

Signed-off-by: Josh Durgin <josh.durgin@dreamhost.com>
2011-04-01 16:59:59 -07:00
Sage Weil
3ce9534d78 client: close snaprealm on last cap removal during reconnect
The invariant is is_any_caps() IFF in->snaprealm, where is_any_caps() is
!caps.empty || exporting_mds >= 0.  If we clear exporting_mds and caps can
be non-empty, we need to drop the snaprealm reference.

Fixes crash like

client/Client.cc: In function 'void Client::add_update_cap(Inode*, int, uint64_t, unsigned int, unsigned int, unsigned int, inodeno_t, int)', in thread '0x7f213df10700'
client/Client.cc: 2244: FAILED assert(in->snaprealm == 0)
client/Client.cc: In function 'void Client::add_update_cap(Inode*, int, uint64_t, unsigned int, unsigned int, unsigned int, inodeno_t, int)', in thread '0x7f213df10700'
client/Client.cc: 2244: FAILED assert(in->snaprealm == 0)
 ceph version 0.25-570-g8ea9380 (commit:8ea938000624b5d67237b52cceee83b04d310d3c)
 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x53) [0x7bd566]
 2: (Client::add_update_cap(Inode*, int, unsigned long, unsigned int, unsigned int, unsigned int, inodeno_t, int)+0xe6) [0x64af82]
 3: (Client::add_update_inode(InodeStat*, utime_t, int)+0xca2) [0x63e1ec]
 4: (Client::insert_trace(MetaRequest*, utime_t, int)+0x5cf) [0x63f4cb]
 5: (Client::handle_client_reply(MClientReply*)+0x915) [0x6448c7]
 6: (Client::ms_dispatch(Message*)+0x167) [0x644d5b]
 7: (Messenger::ms_deliver_dispatch(Message*)+0x63) [0x62e911]
 8: (SimpleMessenger::dispatch_entry()+0x6fd) [0x61ace1]
 9: (SimpleMessenger::DispatchThread::entry()+0x2c) [0x610b90]
 10: (Thread::_entry_func(void*)+0x23) [0x62d7f3]
 11: (()+0x68ba) [0x7f21418e98ba]
 12: (clone()+0x6d) [0x7f21407e502d]
 ceph version 0.25-570-g8ea9380 (commit:8ea938000624b5d67237b52cceee83b04d310d3c)

triggered by an MDS reconnect on a 3-node cluster.  (Workload in this case
was just csyn --syn makedirs 7 7 7.)

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-04-01 16:21:45 -07:00
Sage Weil
553da45c7d mds: fix trim_non_auth assert for base inodes
No parent dentry for base dirfrags (e.g., root).

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-04-01 16:21:45 -07:00
Samuel Just
9ac44d4ad1 MDS: change messenger name for replay mdses
This will cause read operations from standby mdses to be distinguishable
from those from the normal by changing the node name in the messenger.
Previously, the replay node would have the same name as the node it's
following.

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>
2011-04-01 16:28:52 -07:00
Sage Weil
b21fbab19d journaler: adjusted requested/received_pos when dropping journal tail
Signed-off-by: Sage Weil <sage@newdream.net>
2011-04-01 15:58:34 -07:00
Colin Patrick McCabe
64afb9d740 confutils: more parsing tests
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-04-01 15:58:29 -07:00
Sage Weil
d4c4b4a0b5 journaler: fix partial tail entry correction
If we encounter a partial tail entry, we drop it by moving the write_pos
(end of journal) back to read_pos.  We also need to reset the read
state (read_buf, requested/received_pos) so that subsequent replay attempts
won't be horribly confused.

Signed-off-by: Sage Weil <sage@newdream.net>
2011-04-01 15:57:46 -07:00
Colin Patrick McCabe
2630ecc618 config: Add confutils unit test
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-04-01 15:34:47 -07:00
Sage Weil
8ea9380006 journaler: fix signedness
Signed-off-by: Sage Weil <sage@newdream.net>
2011-04-01 15:11:21 -07:00
Sage Weil
bb814758f6 journaler: fix warning
warning: osdc/Journaler.cc:682: comparison between signed and unsigned integer expressions

Signed-off-by: Sage Weil <sage@newdream.net>
2011-04-01 15:09:46 -07:00
Sage Weil
bafff04248 mds: fix bounds on import
The add_ambiguous_import() call was clobbering the bounds field for
EImportStart::replay(), screwing up the subtree auth adjustment.  Make the
argument const.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-04-01 14:34:32 -07:00
Sage Weil
03c9365cce mds: rejoin dirfrags explicitly
We need to rejoin dirfrag replicas explicitly.  We were doing only dentries,
but that won't necessarily include every dirfrag.

Signed-off-by: Sage Weil <sage@newdream.net>
2011-04-01 14:34:32 -07:00
Sage Weil
a01fba175b mds: set dir_auth between EImport{Start,Finish}
This ensures that try_trim_non_auth_subtree() doesn't throw out a subtree
we're in the midst of importing (during journal replay).

Signed-off-by: Sage Weil <sage@newdream.net>
2011-04-01 14:34:32 -07:00
Sage Weil
a47c225062 mds: use try_trim_non_auth_subtree helper
This helper captures the logic of keeping subtrees when necessary but
dropping them when possible, and cleaning up as appropriate.

Signed-off-by: Sage Weil <sage@newdream.net>
2011-04-01 14:34:32 -07:00
Sage Weil
6fcecbc129 mds: close exported dirfrag
We have to keep export bounds open for auth subtrees.  After we export a
subtree, though, there are two opportunities to drop empty dirfrags from
our cache:

 - The children of the exported subtree may now be trimmable, if they are
   also non-auth and empty.
 - The exported subtree may be trimmable if it is empty and the parent is
   also non-auth.  This may be true for ancestors further up the hierarchy
   as well.

This helps ensure that when we get to rejoin, the only non-auth subtrees we
have are there because they are non-empty or because they are bounds on our
own subtrees.

Signed-off-by: Sage Weil <sage@newdream.net>
2011-04-01 14:34:32 -07:00
Samuel Just
5d998a7a0d MDSMonitor: MDS do not standby replay until other is active
Previously, an mds could go into standby replay before the mds which it
is replaying has finished creating.

Signed-off-by: Samuel Just <samuel.just@dreamhost.com>
2011-04-01 14:24:41 -07:00
Sage Weil
ff2c2e3197 journaler: fix requested_pos badness in _issue_read
We want to advance requested_pos *only* as far as we actually want to read,
i.e., the previously-probed end of the journal.

This, among other things, will screw us up later when we reprobe and try to
read more because requested_pos is already past read_pos.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-04-01 14:01:40 -07:00
Sage Weil
65714c59ee mds: show correct incarnation on mdsmap update
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-04-01 12:19:42 -07:00
Sage Weil
dec5b6b8f8 mds: fix discover_path
If we have the base dirfrag, do not request it.  Otherwise we can get a
reply that contains only that (partial progress), and we will then fail
to wake up our dentry waiter.

This was broken with the rewrite in b58b8d098e.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-04-01 11:37:26 -07:00
Sage Weil
0fb02b25da Merge branch 'mds_ino' 2011-04-01 11:35:19 -07:00
Sage Weil
1696388e7a Merge remote branch 'origin/rgw' 2011-04-01 11:48:18 -07:00
Colin Patrick McCabe
b60a3c9988 boto_tool: often use get_bucket rather than lookup
get_bucket throws an exception when the bucket doesn't exist, whereas
lookup just returns None. Sometimes we want the former behavior.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-04-01 11:37:35 -07:00
Sage Weil
06ed898ddd mds: fix find_ino_dir completion
Pass the path back in so that we can retraverse the path on retry.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-04-01 11:35:09 -07:00
Sage Weil
c736d7f688 mds: find_ino_dir debugging
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-04-01 11:24:14 -07:00
Sage Weil
2d926968d7 mds: fix dir fetch on lookup_hash fix
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-04-01 11:24:14 -07:00
Sage Weil
9bed4e3a61 mds: fix discover_path
If we have the base dirfrag, do not request it.  Otherwise we can get a
reply that contains only that (partial progress), and we will then fail
to wake up our dentry waiter.

This was broken with the rewrite in b58b8d098e.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-04-01 11:24:14 -07:00
Sage Weil
3bfc468d78 mds: don't query self from find_ino_peers
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-04-01 11:24:14 -07:00
Sage Weil
8408f83cee client: lookup_ino command
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-04-01 11:24:14 -07:00
Sage Weil
2f82b39400 client: fix lookup_hash request hash value
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-04-01 11:24:14 -07:00
Colin Patrick McCabe
a603acc1f2 ceph_argparse: fix silly usage message
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-04-01 11:00:06 -07:00
Colin Patrick McCabe
760bc01f24 config: typecheck initialization macros
When initializing the config_options array, complain if the size of the
option field we're trying to initialize doesn't match the size of our
type. This will prevent careless type annotations from overwriting
neighboring option fields.

Also create a header called "static assert" which implements a
compile-time assert.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-04-01 10:27:40 -07:00
Colin Patrick McCabe
ee483259b4 config: introduce OPT_U64
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-04-01 10:27:39 -07:00
Sage Weil
80d5873c8c client: fix lookuphash to use rjenkins
This is the default.  Not ideal, but avoids explicitly specifying the hash
function.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-04-01 09:28:12 -07:00
Sage Weil
6b4fb92a9a mds: add LOOKUPINO op
This just looks up a single ino (instead of the ino + dirino/hash
LOOKUPHASH business).

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-04-01 09:09:55 -07:00
Sage Weil
1ff3ce1fd7 mds: rename chained _lookup_hash_? methods
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-04-01 09:06:23 -07:00
Sage Weil
80b621ba2a mds: also try to look up ino directly
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-04-01 08:59:33 -07:00
Sage Weil
5e4839b062 makefile: include MMDSFindIno[Reply].h
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-03-31 21:32:56 -07:00
Sage Weil
ddc06f1124 journaler: fix comments
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-03-31 21:10:58 -07:00
Sage Weil
d5fa16a476 Merge branch 'mds_ino' 2011-03-31 18:09:04 -07:00
Sage Weil
30bffabdb5 mds: try find_ino_dir if find_ino_peers fails for lookuphash
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-03-31 18:07:18 -07:00
Sage Weil
d31b76a662 mds: find_ino_dir
Search for a dir ino by checking the ondisk directory object path xattr.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-03-31 18:07:04 -07:00
Colin Patrick McCabe
ce2fabc383 common: fix cmdline of indashes_to_underscores
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-03-31 17:12:34 -07:00
Colin Patrick McCabe
0a63567ea2 rgw: Properly support truncating on S3 PUT
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-03-31 17:02:47 -07:00
Sage Weil
3f1e9b0f7b mds: use helper to generate dir inode object names
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-03-31 16:51:43 -07:00
Sage Weil
26392894cf mds: allow explicit finisher context for path_traverse
Previously we could only path_traverse and retry a request or message.
This just allows an explicit context to be used as well.  It's the caller's
job to clean it up if we return <= 0.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-03-31 16:42:43 -07:00
Sage Weil
34b0813d64 mds: fix possible null dereference
Fix null deref on forward and req (!mdr) caller.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-03-31 16:38:59 -07:00
Sage Weil
705161b11d mds: add mds failure/recovery support for find_ino_peers
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-03-31 16:07:10 -07:00
Sage Weil
3970340567 mds: try find_ino_peers for lookuphash requests
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-03-31 16:07:10 -07:00
Sage Weil
c2c333d8cb mds: find_ino_peer
Search for an ino on peer MDSs.  Still need to handle mds failure/recovery
handling.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-03-31 16:07:10 -07:00