Commit Graph

14657 Commits

Author SHA1 Message Date
Sage Weil
5b43419c72 mds: try_trim_non_auth_subtree if we rename a dir away from a non-auth subtree
It's possible we have non-auth metadata only because we have a subtree
nested beneath. If we rename a directory out of a non-auth subtree, we
should try to trim any non-auth content from that subtree that may now
be possible due to the child subtrees being linked elsewhere.

Fixes: #1146
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-06-08 13:29:21 -07:00
Sage Weil
072e80ed1c mds: use CDIR_AUTH_UNDEF where possible
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-06-08 13:27:41 -07:00
Sage Weil
811dcae71a mds: remove unlinked metadata from cache on replay
If we replay a metablob that unlinks something, throw it out immediately.
Recursively.  This comes up when:
 - we rename a file from one mds to another, and we replay the event on
   the source mds.  the inode gets thrown out.
 - we rename a directory from one mds to another, and when journaled, the
   source mds had no nested metadata.  same thing: we throw it out.  we
   may have something in our cache nested beneath that, though, that was
   since committed and such, but the fact that we didn't journal it being
   reattached elsewhere implies that it was clean and gone when our event
   was journaled, and we can throw it all out.  recursively.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-06-08 13:18:07 -07:00
Sage Weil
a1a71471c2 qa: test witness subtree adjustment
This exercises the mds at least; we don't have a way to verify mds
behavior currently.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-06-08 11:22:26 -07:00
Sage Weil
21795aaf5d qa: add dir rename tests
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-06-07 20:49:45 -07:00
Sage Weil
00ec86a204 mds: open renamed import child frags during journal replay
Open up any child frags of the imported renamed inode that are noted in
the journal event.  (Note we blindly open up that list here; it's up to the
journaler to only populate it when appropriate.)  If the listed frags are
not already open, open them up and set the dir_auth to unknown; presumably
they belong to the rename source/exporter.  If we already had them open,
then the adjust_subtree_after_rename call above will have caught them and
already done the necessary subtree adjustment.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-06-07 20:48:52 -07:00
Sage Weil
44d19304a9 mds: journal open srci frags on srci import (master)
If we are importing the renamed inode, and it is a directory, journal a
list of all open dirfrags (currently, this is actually all frags) so that
we can open them up during journal replay.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-06-07 20:46:42 -07:00
Sage Weil
a7c083ecbf mds: journal renames on witnesses if we have nested subtrees
If a rename witness has any subtrees that are nested beneath the renamed
directory, we need to journal the rename event so that our cache is
properly updated on journal replay.

Further, if we are exporting srci, we also need to journal the dest
(even if we aren't auth for destdn) if we have any open dirfrags because
those will turn into nested subtrees shortly.

We still need to ensure that the cache is properly trimmed during replay.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-06-07 20:43:29 -07:00
Sage Weil
5c870c60ec mds: CDir::contains() should use projected hierarchy
The only current callers are in resolve stage, when nothing is projected.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-06-07 20:40:39 -07:00
Sage Weil
3a2ce92b2c mds: fix check for no-op rename of two links to the same inode
touch a
ln a b
mv a b

is a no-op, according to posix.  Move the check earlier.


Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-06-07 15:13:04 -07:00
Sage Weil
2969161b9e qa: add rename dir checks
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-06-07 13:34:57 -07:00
Sage Weil
2cb0c110b5 qa: rename tests
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-06-07 13:27:31 -07:00
Sage Weil
3e0352d37c mds: rename: add missing pop_and_dirty_projected_inode calls
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-06-07 13:24:38 -07:00
Sage Weil
569baab64b mds: rename: use temp *in in _rename_apply
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-06-07 13:24:35 -07:00
Sage Weil
bdfcc0df34 mds: rename: move unlink out of every possible branch
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-06-07 13:24:11 -07:00
Sage Weil
6b2bf234b8 mds: rename: fix oldin pop
- make the pop match position with the project in prepare
- don't pop on linkmerge, since we don't project in that case

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-06-07 11:25:39 -07:00
Sage Weil
bf7d7fa938 mds: rename: some cleanup
- add oldin temp
- make if conditions match between _prepare and _apply

no functional changes.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-06-07 11:24:50 -07:00
Sage Weil
b794aeb394 client: fix rename request target selection
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-06-07 11:23:54 -07:00
Sage Weil
084a1c747d client: fix mds selection for unlink
It was broken when a remote link target was on another mds.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-06-07 11:11:07 -07:00
Sage Weil
a2288574aa client: make debug prints for choose_target_mds more helpful
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-06-07 11:08:02 -07:00
Sage Weil
5e679d3ae6 client: allow caps to be dropped on another inode for requests
We need the ability to drop caps on another inode that isn't req->inode
or req->old_inode in the request struct.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-06-07 11:07:44 -07:00
Sage Weil
0fb7973745 mds: fix unlink new snaprealm check
The bool was flipped.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-06-07 10:59:09 -07:00
Sage Weil
dbe41a30f2 client: fix rename debug print
Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-06-07 10:50:50 -07:00
Sage Weil
e36a4b261b mds: rename: fix handling of target that is remote dentry
If the target is a remote dentry, we need to consider that the destdn
and desti may have different auths.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-06-07 10:04:09 -07:00
Sage Weil
a5662293a2 client: fix mds routing for link requests
We want to send link requests to the auth for the new name, not the
target inode.  We also want to drop FILE_SHARED caps on new name's
directory.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-06-07 09:58:29 -07:00
Sage Weil
7770e1c626 mds: rename: only add target to stray if destdn is a primary
Only add target into the stray dir if we are renaming over a primary
dentry.  (Otherwise we aren't moving the target.)

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-06-07 09:49:04 -07:00
Sage Weil
df7f8954c3 mds: fix/clean up xlock import/export
- create xlock import/export helpers
- fix/simplify checks: we want to export/import only xlocks on the inode
that is being migrated, unless they are locallock.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-06-07 09:41:56 -07:00
Sage Weil
b5011e2116 mds: adjust subtree roots on rename
If we replay a dir rename operation, we need to adjust the subtree map
accordingly.

This covers the case where the metablob contains both the src and dest
dentries.  Remaining cases will follow.

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
2011-06-06 13:30:25 -07:00
Sage Weil
10750f803f common: add module_type to CephContext; use to initialize rotating keys
This fixes rotating keys.

Signed-off-by: Sage Weil <sage@newdream.net>
2011-06-06 13:06:41 -07:00
Sage Weil
73257a0d4b Merge remote branch 'origin/wip_librbd_locking' 2011-06-06 10:58:32 -07:00
Sage Weil
e2e60147ec monclient: use default mon port if not specified in conf
Signed-off-by: Sage Weil <sage@newdream.net>
2011-06-06 09:25:12 -07:00
Sage Weil
ea56a558ce monmaptool: use default mon port if not specified
Signed-off-by: Sage Weil <sage@newdream.net>
2011-06-06 09:25:00 -07:00
Colin Patrick McCabe
69f9087432 dumpjnl: call msgr->register_entity before start
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-06-03 18:45:22 -07:00
Colin Patrick McCabe
4abdf6fda4 simple_spin: use file-scope global not function
function-scoped globals are protected by a mutex, and taking a mutex
inside a spin lock implementation kind of defeats the point...

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-06-03 18:15:11 -07:00
Sage Weil
ec18be530a simple_spin: fix linker error
For some stupid reason the linker is pulling this in twice, resulting in

/bin/sh ../libtool --tag=CXX   --mode=link g++  -Wall -D__CEPH__ -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D_THREAD_SAFE -rdynamic -Wtype-limits -Wignored-qualifiers -Winit-self -Wpointer-arith -fno-strict-aliasing -Wnon-virtual-dtor -Wno-invalid-offsetof -Wstrict-null-sentinel  -g -Wextra -Wno-missing-field-initializers -Wno-missing-declarations -Wno-unused-parameter -Wl,--as-needed -latomic_ops  -o radosgw_admin radosgw_admin-rgw_fs.o radosgw_admin-rgw_rados.o radosgw_admin-rgw_user.o radosgw_admin-rgw_access.o radosgw_admin-rgw_op.o radosgw_admin-rgw_rest.o radosgw_admin-rgw_rest_os.o radosgw_admin-rgw_rest_s3.o radosgw_admin-rgw_common.o radosgw_admin-rgw_cache.o radosgw_admin-rgw_os.o radosgw_admin-rgw_os_auth.o radosgw_admin-rgw_formats.o radosgw_admin-rgw_log.o rgw_escape.o radosgw_admin-rgw_admin.o librgw.a librados.a -lfcgi -lcurl -lexpat -lpthread -lm -lcrypto++
libcommon.a(libcommon_a-simple_spin.o):(.data+0x8): multiple definition of `foo'
test_trans.o:/home/sage/ceph/src/./common/debug.h:51: first defined here

...which points to some larger linking issue.  However, this fixes it for
now.

Signed-off-by: Sage Weil <sage@newdream.net>
2011-06-03 16:36:13 -07:00
Greg Farnum
53adde037c fix the MonClient problems for --with-debug programs.
Still doesn't compile, though.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>
2011-06-03 17:13:47 -07:00
Colin Patrick McCabe
740eea1ee8 Refactor MonClient, KeyRing
MonClient should contain a KeyRing and a RotatingKeyRing. All the
MonClient users, except possibly csyn, don't want to manage those
objects themselves.

Don't chdir until after we have opened the KeyRing. If the KeyRing is at
a relative path, a chdir may make it inaccessible. Separate the chdir
function from the daemonize function.

Refactor the cmds argument parsing a little bit. Separate the special
actions from the normal operations of the daemon.

This should allow librados and libceph to support CephX finally! yay!

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-06-03 14:57:43 -07:00
Colin Patrick McCabe
ed41f29ac9 remove g_keyring
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-06-03 14:47:31 -07:00
Colin Patrick McCabe
90b53543ea dout:remove stream from dout_emerg_streams earlier
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-06-03 14:47:31 -07:00
Colin Patrick McCabe
98226c2226 DoutStreambuf: de-globalize dout lock
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-06-03 14:47:29 -07:00
Colin Patrick McCabe
5b7049c8a4 DoutStreambuf: de-globalize emergency logging
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-06-03 14:47:00 -07:00
Colin Patrick McCabe
6ed9a58331 Add simple_spin
Add simple spinlock implementation that is safe to use from anywhere.

Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-06-03 14:47:00 -07:00
Greg Farnum
70d77095b8 Revert "cfuse.cc: use safe_write"
This reverts commit e8ac5aa2a4.

This commit is just erroneous. It adds checks on a pipe write
for the result and an abort if the write failed. But that's broken
in the desired case where we succeed, block on ceph_fuse_ll_main(),
and the parent process is long-gone by the time we get to this code!

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>
2011-06-03 14:29:35 -07:00
Greg Farnum
b4eb5efaf8 rados_bencher: re-add written objects constraint to read benchmark.
Somehow, in the last major change, the constraints that kept the
bencher from trying to read non-existent objects got removed. Put
a check back in the main bench loop to fix that.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>
2011-06-03 11:53:10 -07:00
Greg Farnum
d1d3e26c7d mds: remove now-erroneous comment
Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>
2011-06-03 10:49:44 -07:00
Greg Farnum
4ef743088c Merge branch 'next'
Conflicts:
	src/mds/Server.cc

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>
2011-06-03 10:12:10 -07:00
Greg Farnum
19949f6d60 mds: Clean up _rename_prepare journaling
This has been broken for a while in terms of journaling
things the MDS isn't auth for. This patch should fix that, and
adds a few asserts to that effect.

Also adds a new not_journaling flag to _rename_prepare
for those cases which call the function and then discard
the bufferlist results.
Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>
2011-06-03 09:53:20 -07:00
Greg Farnum
4689073c99 mds: _rename_prepaer should only journal dest if auth for it
Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>
2011-06-03 09:48:58 -07:00
Greg Farnum
cd5049dc8b uclient: reset flushing_caps on (mds) cap import.
Previously, we could get stuck thinking that we'd flushed caps
(that went to the original MDS, waited on freeze for export,
and then were dropped) without ever telling the auth MDS that we
wanted to do so. This caused hung shutdowns:
1) during shutdown we drop all our caps
2) we get stuck and notice that we have a flushing cap
3) we send cap flush
4) MDS ignores it (I think because actual data already got updated?
and now we don't have the proper caps either)

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>
2011-06-02 18:35:45 -07:00
Greg Farnum
c28b749bce uclient: don't use racy check for uncommitted data.
Previously we used a check for if there were CEPH_CAP_FILE_BUFFER refs,
but that was racy if we had other threads (they could hold caps for
sync writes or something). Instead, see if we have any in-flight
writes or uncommitted objects.

Signed-off-by: Greg Farnum <gregory.farnum@dreamhost.com>
2011-06-02 18:35:45 -07:00