Commit Graph

34614 Commits

Author SHA1 Message Date
Sage Weil
36aaab9eee osd/ReplicatedPG: fix cache full -> not full requeueing when !active
We only want to do this if is_active().  Otherwise, the normal
requeueing code will do its thing, taking care to get the queue orders
correct.

Backport: firefly
Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-25 14:50:51 -07:00
Sage Weil
c5f766bb16 ceph_test_rados_api_tier: do fewer writes in HitSetWrite
We don't need to do quite so many writes.  It can be slow when we are
thrashing and aren't doing anything in parallel.

Fixes: #8932
Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-25 13:53:03 -07:00
Dan Mick
d0b98bcb35 Merge pull request #2145 from ceph/wip-ref-put
common/RefCountedObject: fix use-after-free in debug print

Reviewed-by: Dan Mick <dan.mick@inktank.com>
2014-07-25 13:19:42 -07:00
Sage Weil
f3609205e7 common/RefCountedObject: fix use-after-free in debug print
We could race with another thread that deletes this right after we call
dec().  Our access of cct would then become a use-after-free.  Valgrind
managed to turn this up.

Copy it into a local variable before the dec() to be safe, and move the
dout line below to make this possibility explicit and obvious in the code.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-25 13:17:32 -07:00
Josh Durgin
d8eb656069 Merge pull request #2143 from ceph/wip-rgw-align
Wip rgw align

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2014-07-25 11:36:29 -07:00
Yehuda Sadeh
14cad5ece7 rgw: object write should not exceed part size
Fixes: #8928

This can happen if the stripe size is not a multiple of the chunk size.

Backport: firefly

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2014-07-25 11:35:10 -07:00
Yehuda Sadeh
fc83e197ab rgw: align object chunk size with pool alignment
Fixes: #8442
Backport: firefly
Data pools might have strict write alignment requirements. Use pool
alignment info when setting the max_chunk_size for the write.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2014-07-25 11:35:01 -07:00
Sage Weil
c91b22c0b4 Merge pull request #2141 from ceph/wip-8882
osd: set pg flag INCOMPLETE_CLONES when turning off cache pool

Reviewed-by: Greg Farnum <greg@inktank.com>

First patch Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>
2014-07-25 10:34:33 -07:00
John Wilkins
1f9c7324a0 doc: Add additional hyperlink to Cache Tiering defaults.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2014-07-25 09:55:52 -07:00
John Wilkins
4047660ce4 doc: Update doc from user feedback.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2014-07-25 09:55:28 -07:00
Sage Weil
d1dfb9b607 osd: fix bad Message* defer in C_SendMap and send_map_on_destruct
We were carrying a bare Message*, which could get freed if the op was
canceled (or possibly completed).  Instead, just stash the entity_name_t,
the only piece we need.  The Connection is properly ref counted so no
worries there.

Fixes: #8926
Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-25 09:20:50 -07:00
Sage Weil
3e7ed42379 Merge pull request #2142 from ceph/wip-data-pool
test: catch a straggler still using 'data' pool

Reviewed-by: Sage Weil <sage@redhat.com>
2014-07-25 09:03:34 -07:00
John Spray
5740266096 test: catch a straggler still using 'data' pool
Used rbd pool instead, which is still created by default.

Signed-off-by: John Spray <john.spray@redhat.com>
2014-07-25 17:01:39 +01:00
Ma Jianpeng
4eb18dd487 os/FileJournal: Update the journal header when closing journal
When closing journal, it should check must_write_header and update
journal header if must_write_header alreay set.
It can reduce the nosense journal-replay after restarting osd.

Signed-off-by: Ma Jianpeng <jianpeng.ma@intel.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2014-07-24 18:54:33 -07:00
Sage Weil
63c1711a9e msg/SimpleMessenger: drop local_conneciton priv link on shutdwon
This breaks ref cycles between the local_connection and session, and let's
us drop the explicit set_priv() calls in OSD::shutdown().

Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-24 18:22:22 -07:00
John Wilkins
4fe07925e4 doc: Updated mon doc per feedback. Fixed hyperlinks.
Signed-off-by: John Wilkins <john.wilkins@inktank.com>
2014-07-24 16:00:52 -07:00
Gregory Farnum
2a6b5309e5 Merge pull request #2079 from nereocystis/seq_read_bench-args
Make the declaration argument names match those in the implementation (as used by callers).

Reviewed-by: Greg Farnum <greg@inktank.com>
2014-07-24 14:36:21 -07:00
Abhishek Lekshmanan
c51182257e doc: update radosgw man page with available opts
Fixes:#8112

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
2014-07-24 13:21:25 -07:00
Abhishek Lekshmanan
e259aca55a rgw: list all available options during help()
Adding the available help arguments from the man page

Fixes: #8112

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
2014-07-24 13:17:26 -07:00
Abhishek Lekshmanan
99e80a5f62 rgw: format help options to align with the rest
Whitespace removal to make all help options align in a similar fashion

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Signed-off-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
2014-07-24 13:15:53 -07:00
Sage Weil
63abf11390 osd/ReplicatedPG: observe INCOMPLETE_CLONES in is_present_clone()
We cannot assume that just because cache_mode is NONE that we will have
all clones present; check for the absense of the INCOMPLETE_CLONES flag
here too.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-24 10:07:33 -07:00
Sage Weil
41364711a6 osd/ReplicatedPG: observed INCOMPLETE_CLONES when doing clone subsets
During recovery, we can clone subsets if we know that all clones will be
present.  We skip this on caching pools because they may not be; do the
same when INCOMPLETE_CLONES is set.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-24 10:07:33 -07:00
Sage Weil
956f28721d osd/ReplicatedPG: do not complain about missing clones when INCOMPLETE_CLONES is set
When scrubbing, do not complain about missing cloens when we are in a
caching mode *or* when the INCOMPLETE_CLONES flag is set.  Both are
indicators that we may be missing clones and that that is okay.

Fixes: #8882
Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-24 10:07:33 -07:00
Sage Weil
54bf055c5d osd/osd_types: add pg_pool_t FLAG_COMPLETE_CLONES
Set a flag on the pg_pool_t when we change cache_mode NONE.  This
is because object promotion may promote heads without all of the clones,
and when we switch the cache_mode back those objects may remain.  Do
this on any cache_mode change (to or from NONE) to capture legacy
pools that were set up before this flag existed.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-24 10:06:55 -07:00
Sage Weil
67d13d76f5 mon/OSDMonitor: improve no-op cache_mode set check
If we have a pending pool value but the cache_mode hasn't changed, this is
still a no-op (and we don't need to block).

Backport: firefly
Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-24 10:06:54 -07:00
Sage Weil
7d137430aa Merge remote-tracking branch 'gh/next' 2014-07-23 19:14:52 -07:00
Sage Weil
f9e885b990 Merge pull request #2127 from ceph/wip-8701
filestore: fix collection_move behavior

Reviewed-by: Greg Farnum <greg@inktank.com>
2014-07-23 19:13:55 -07:00
Sage Weil
c757fbdd7b Merge pull request #2140 from ceph/wip-8889
osd: greedily get obc write lock in some cases

Reviewed-by: Greg Farnum <greg@inktank.com>
2014-07-23 19:13:11 -07:00
Sage Weil
d4faf747b7 ceph_test_objectstore: clean up on finish of MoveRename
Otherwise, we leave collections around, and the next test fails.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-23 14:48:14 -07:00
Sage Weil
3ec9a42b47 os/LFNIndex: use FDCloser for fsync_dir
This prevents an fd leak when maybe_inject_failure() throws an exception.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-23 14:48:14 -07:00
Sage Weil
6fb3260d59 os/LFNIndex: only consider alt xattr if nlink > 1
If we are doing a lookup, the main xattr fails, we'll check if there is an
alt xattr.  If it exists, but the nlink on the inode is only 1, we will
kill the xattr.  This cleans up the mess left over by an incomplete
lfn_unlink operation.

This resolves the problem with an lfn_link to a second long name that
hashes to the same short_name: we will ignore the old name the moment the
old link goes away.

Fixes: #8701
Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-23 14:48:14 -07:00
Sage Weil
ec36f0a130 os/LFNIndex: remove alt xattr after unlink
After we unlink, if the nlink on the inode is still non-zero, remove the
alt xattr.  We can *only* do this after the rename or unlink operation
because we don't want to leave a file system link in place without the
matching xattr; hence the fsync_dir() call.

Note that this might leak an alt xattr if we happen to fail after the
rename/unlink but before the removexattr is committed.  We'll fix that
next.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-23 14:48:14 -07:00
Sage Weil
a320c260a9 os/LFNIndex: FDCloser helper
Add a helper to close fd's when we leave scope.  This is important when
injecting failures by throwing exceptions.

Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-23 14:48:14 -07:00
Sage Weil
b2cdfce646 os/LFNIndex: handle long object names with multiple links (i.e., rename)
When we rename an object (collection_move_rename) to a different name, and
the name is long, we run into problems because the lfn xattr can only track
a single long name linking to the inode.  For example, suppose we have

foobar -> foo_123_0 (attr: foobar) where foobar hashes to 123.

At first, collection_add could only link a file to another file in a
different collection with the same name. Allowing collection_move_rename
to rename the file, however, means that we have to convert:

col1/foobar -> foo_123_0 (attr: foobar)

to

col1/foobaz -> foo_234_0 (attr: foobaz)

This is a problem because if we link, reset xattr, unlink we end up with

col1/foobar -> foo_123_0 (attr: foobaz)

if we restart after we reset the attr.  This will cause the initial foobar
lookup to since the attr doesn't match, and the file won't be able to be
looked up.

Fix this by allow *two* (long) names to link to the same inode.  If we
lfn_link a second (different) name, move the previous name to the "alt"
xattr and set the new name.  (This works because link is always followed
by unlink.)  On lookup, check either xattr.

Don't even bother to remove the alt xattr on unlink.  This works as long
as the old name and new name don't hash to the same shortname and end up
in the same LFN chain.  (Don't worry, we'll fix that next.)

Fixes part of #8701
Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-23 14:48:14 -07:00
Sage Weil
cf98805c09 ceph_test_objectstore: fix warning
Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-23 14:48:14 -07:00
Samuel Just
6aa48a485e store_test: add long name collection_move_rename tests
Currently fails.

Signed-off-by: Samuel Just <sam.just@inktank.com>
2014-07-23 14:48:14 -07:00
Dan Mick
c0cb56f6c8 ceph.spec.in: add bash completion file for radosgw-admin
Signed-off-by: Sandon Van Ness <sandon@inktank.com>
(cherry picked from commit b700963071)
2014-07-23 13:32:00 -07:00
Dan Mick
1ad4cd386e ceph.spec.in: rhel7-related changes:
udev rules: /lib -> /usr/lib
/sbin binaries move to /usr/sbin or %{_sbindir}

Signed-off-by: Sandon Van Ness <sandon@inktank.com>
(cherry picked from commit 235e4c7de8)
2014-07-23 13:31:53 -07:00
Dan Mick
c57811fc4b Fix/add missing dependencies:
- rbd-fuse depends on librados2/librbd1
- ceph-devel depends on specific releases of libs and libcephfs_jni1
- librbd1 depends on librados2
- python-ceph does not depend on libcephfs1

Signed-off-by: Sandon Van Ness <sandon@inktank.com>
(cherry picked from commit 7cf8132239)
2014-07-23 13:31:47 -07:00
Dan Mick
793e05a27a ceph.spec.in: whitespace fixes
Signed-off-by: Sandon Van Ness <sandon@inktank.com>
(cherry picked from commit ec8af52a5e)
2014-07-23 13:31:39 -07:00
Dan Mick
dae6ecbc31 ceph.spec.in: split out ceph-common as in Debian
Move files, postun scriptlet, and add dependencies on ceph-common
where appropriate

Signed-off-by: Sandon Van Ness <sandon@inktank.com>
(cherry picked from commit e131b9d5a5)
2014-07-23 13:31:22 -07:00
Sage Weil
a05a0da3b1 common/random_cache: fix typo
Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-23 10:11:59 -07:00
Sage Weil
54ea5c13ac Merge pull request #2136 from yuyuyu101/fix-randomcache
common/RandomCache: Fix inconsistence between contents and count

Reviewed-by: Sage Weil <sage@redhat.com>
2014-07-23 09:57:59 -07:00
Haomai Wang
5efdc6236c common/RandomCache: Fix inconsistence between contents and count
The add/clear method may cause count inconsistent with the real size of
contents.

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
2014-07-23 11:31:46 +08:00
Sage Weil
356af4bf46 osd/ReplicatedPG: debug obc locks
Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-22 18:01:14 -07:00
Sage Weil
6fe27823b8 osd/ReplicatedPG: greedily take write_lock for copyfrom finish, snapdir
In the cases where we are taking a write lock and are careful
enough that we know we should succeed (i.e, we assert(got)),
use the get_write_greedy() variant that skips the checks for
waiters (be they ops or backfill) that are normally necessary
to avoid starvation.  We don't care about staration here
because our op is already in-progress and can't easily be
aborted, and new ops won't start because they do make those
checks.

Fixes: #8889
Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-22 18:00:34 -07:00
Sage Weil
09626501d7 osd: allow greedy get_write() for ObjectContext locks
There are several lockers that need to take a write lock
because there is an operation that is already in progress and
know it is safe to do so.  In particular, they need to skip
the starvation checks (op waiters, backfill waiting).

Signed-off-by: Sage Weil <sage@redhat.com>
2014-07-22 18:00:09 -07:00
Josh Durgin
422218a3b3 Merge pull request #2120 from ceph/wip-8858
Wip 8858

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
2014-07-22 16:58:25 -07:00
Gregory Farnum
29401c7e77 Merge pull request #2133 from ceph/wip-8897
os: fix build warnings with name/attr len checks (fixes 8889)

Reviewed-by: Greg Farnum <greg@inktank.com>
2014-07-22 15:36:40 -07:00
João Eduardo Luís
d37b2ac1e3 Merge pull request #2128 from ceph/wip-8851
mon: AuthMonitor: always encode full regardless of keyserver having keys

Reviewed-by: Gregory Farnum <greg@inktank.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2014-07-22 22:10:17 +01:00