Commit Graph

39558 Commits

Author SHA1 Message Date
Josh Durgin
7fed0a366d librbd: hold snap_lock while reading parent info in diff_iterate
Caught be the re-added assertions in ImageCtx::get_parent_info()

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2015-02-25 17:27:34 -08:00
Josh Durgin
df42fd3833 test_librbd: close ioctx after imagectx
There's no need to explicitly close the ioctx. Doing so may cause
problems when the Images using it are destroyed afterwards.  Just let
normal cleanup at the end of the block take care of it in the correct
order.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2015-02-25 17:27:33 -08:00
Josh Durgin
06e5a3979f rbd: fix --image-feature parsing
Need to use _witharg(), not _flag()

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2015-02-25 17:27:33 -08:00
Josh Durgin
eef7466a5c librbd: apply flag read failure to all snaps
Don't check just the features of head, since it may be possible to
disable object map in the future.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2015-02-25 17:27:33 -08:00
Josh Durgin
6ac8139b25 librbd: make ImageCtx->object_map always present
This simplifies locking by obviating the NULL checks.  We no longer
need md_lock to protect these acceses. We can use object_map_lock
instead, to make sure no one reads an object map while its being
updated.

Keep track of whether the object map is enabled for a given snapshot
internally. In each public method, check this state, and automatically
set it correctly when refreshing the object map. During snapshot
removal, unconditionally try to remove the object map object, to
protect against bugs leaking objects, and to be consistent with image
removal.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2015-02-25 17:27:33 -08:00
Josh Durgin
04d360a4c1 librbd: take ImageCtx->snap_lock for write in add_snap()
add_snap() updates the ImageCtx snapshot metadata in memory, as well
as reading the flags as part of the object map snapshot. Both of these
require holding snap_lock.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2015-02-25 15:42:32 -08:00
Josh Durgin
40c2abb08a librbd: use snap_lock to protect ImageCtx->flags
This is another step towards eliminating md_lock from the writeback
path. Almost all the places that use ImageCtx->flags already use
snap_lock, so there's no need to create a new lock. For the others,
add a helper, test_flags() that acquires the lock, similar to
test_features().

This also makes sure we look up flags of the snapshot we're operating
on, instead of those for head.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2015-02-25 15:42:31 -08:00
Josh Durgin
bb4041fc95 librbd: add locking asserts to ImageCtx
A bunch of these used to be here, but were removed when converting to
RWLocks, before RWLocks had is_[w]locked() methods.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2015-02-25 15:41:54 -08:00
Josh Durgin
4bcbdbfd0f librbd: fix ImageWatcher::is_lock_supported() locking
Take snap_lock while reading ImageCtx->snap_id, and
look up the features by snap_id as well.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2015-02-25 15:41:54 -08:00
Josh Durgin
a94ceb6088 librbd: add and use a test_features() helper
This gets the appropriate locks, and checks the currently open
snapshot instead of head.  Looking up features by snap_id prepares us
for future addition or removal of e.g. an object map throughout the
life of an image.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2015-02-25 15:41:54 -08:00
Josh Durgin
cffd93a32f librbd: use ImageCtx->snap_lock for ImageCtx->features
This was being protected by md_lock, but that has become too coarse
since it is used to prevent writes from proceeding while flushing
caches for a snapshot. With the addition of ObjectMap and
ImageWatcher, writeback could try to acquire md_lock again, leading to
a deadlock.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2015-02-25 15:41:50 -08:00
Josh Durgin
e1f2c97256 Merge pull request #3777 from ceph/wip-librbd-snap-create-race
librbd: fixed snap create race conditions

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2015-02-23 10:00:01 -08:00
Jason Dillaman
63f6c9bac9 librbd: fixed snap create race conditions
Since the post-snap create header update runs asynchrously
in a finalizer callback, it's possible that the snapshot
is not immediately visible.  Also, if a proxied snap create
message is replayed, it's possible for the client to receive
a EEXISTS error.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-02-23 12:46:33 -05:00
Josh Durgin
da1ed53987 Merge pull request #3758 from ceph/wip-10898
librbd: improved ImageWatcher duplicate message detection

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2015-02-23 09:08:11 -08:00
Josh Durgin
53c300a7d0 Merge pull request #3770 from ceph/wip-librbd-exclusive-lock-config
rbd: disable RBD exclusive locking by default

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2015-02-23 08:37:04 -08:00
Jason Dillaman
69b3857924 librbd: improved ImageWatcher duplicate message detection
Added a unique client id to announcement messages so that duplicate
lock release / acquired / requested messages can be detected and
ignored by the client.  Also fixed an issue processing the result
code for async operations.

Fixes: #10898
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-02-23 11:07:53 -05:00
Jason Dillaman
942f875135 librbd: add test instances for watch/notify messages
Ensure that the librbd watch/notify messages are tested
for backwards compatibility.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-02-23 11:07:53 -05:00
Jason Dillaman
10d86863f2 librbd: minor cleanup of ImageWatcher messages
Moved all RPC messages to their own classes to facilitate cleaner
version control and backward compatibility.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-02-23 11:07:52 -05:00
Jason Dillaman
0ed296b1e3 rbd: disable RBD exclusive locking by default
Utilize the existing rbd_default_features config option to
control whether or not to enable RBD exclusive locking and
object map features by default.  Also added a new option to
the rbd cli to specify the image features when creating images.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-02-23 10:47:05 -05:00
Kefu Chai
f6530474c0 Merge pull request #3741 from ceph/wip-cmake-vstart
cmake fixes and enable vstart with cmake build 

Reviewed-by: Kefu Chai <kchai@redhat.com>
2015-02-22 10:10:40 +08:00
Sage Weil
cbb2e53d61 Merge pull request #3729 from guangyy/wip-4254-hammer
osd: number of degraded objects in EC pool is wrong when there is OSD down(in).

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2015-02-21 10:10:12 -08:00
Sage Weil
a07a956d77 Merge pull request #3722 from ceph/wip-10787
mon: fix osd_epoch cache bug 10787

Reviewed-by: Joao Eduardo Luis <joao@redhat.com>
2015-02-21 10:08:59 -08:00
Sage Weil
571f60b88c Merge branch 'osd-fix' of git://github.com/wonzhq/ceph into hammer
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2015-02-21 10:07:20 -08:00
Sage Weil
87096125ac Merge branch 'wip-5639' of git://github.com/rzarzynski/ceph into hammer
Reviewed-by: Sage Weil <sage@redhat.com>
2015-02-21 10:05:55 -08:00
Sage Weil
79cbcfbab1 Merge pull request #3760 from ceph/wip-10883
osd: Fix FileJournal wrap to get header out first

Reviewed-by: Sage Weil <sage@redhat.com>
2015-02-20 15:18:41 -08:00
Sage Weil
1ad76622bf Merge pull request #3767 from athanatos/wip-10881
Wip 10881

Reviewed-by: Sage Weil <sage@redhat.com>
2015-02-20 15:14:03 -08:00
Sage Weil
6586541b0d Merge pull request #3768 from athanatos/wip-10780
Wip 10780

Reviewed-by: Sage Weil <sage@redhat.com>
2015-02-20 15:10:35 -08:00
Sage Weil
f6098bd9a9 Merge pull request #3769 from athanatos/wip-10908
Wip 10908

Reviewed-by: Sage Weil <sage@redhat.com>
2015-02-20 15:08:20 -08:00
Yehuda Sadeh
170c88d413 cmake: radosgw, radosgw-admin related fixes
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2015-02-20 14:28:13 -08:00
Yehuda Sadeh
4feb171bf4 vstart.sh: can use binaries outside of ceph/src
If setting CEPH_BUILD_ROOT, will use that path, otherwise runs
everything from current directory as before.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2015-02-20 14:28:12 -08:00
Ken Dreyer
f85f7adc95 Merge pull request #3744 from ceph/wip-10884-hammer-rpm-devel-split
ceph.spec: split ceph-devel to appropriate *-devel packages

Reviewed-by: Sandon Van Ness <sandon@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>
2015-02-20 10:44:45 -07:00
Josh Durgin
c0e897e9fd Merge pull request #3764 from ceph/wip-10919
cls_rbd: invalidate bufferlist CRC when updating object map

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2015-02-20 09:00:26 -08:00
Jason Dillaman
55d3db979a cls_rbd: invalidate bufferlist CRC when updating object map
The bit vector was not invalidating the bufferlist's CRC, resulting
in peer OSDs rejecting write op due to a mismatched CRC on the
message.

Fixes: #10919
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-02-20 10:37:59 -05:00
Josh Durgin
54dd411aea Merge pull request #3759 from ceph/wip-10914
osdc: pass fadvise op flags to WritebackHandler read requests

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2015-02-19 18:47:16 -08:00
David Zafman
970bb4901f osd: Fix FileJournal wrap to get header out first
Correct and restore assert that was removed

Cause by f46b1b473f
Fixes: #10883
Backport: dumpling, firefly, giant

Signed-off-by: David Zafman <dzafman@redhat.com>
2015-02-19 13:29:14 -08:00
Jason Dillaman
01f04cb932 osdc: pass fadvise op flags to WritebackHandler read requests
librbd was previously attempting to cast the provided Context to
retrieve the fadvise flags.  To eliminate the unsafe cast, now
the fadvise flags are directly passed to the WritebackHandler::read
callback.

Fixes: #10914
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-02-19 15:50:36 -05:00
Sage Weil
c4632427e2 osd/OSDMap: include pg_temp count in summary
It is useful to know how big the pg_temp map is.  Strictly speaking
this is part of the OSDMap so I'm including it here.  It looks like
this:

     osdmap e25: 3 osds: 3 up, 3 in; 1 remapped pgs

It might be more user-friendly to put it in a line with the pgmap
somewhere (where other pg counts are included), but it doesn't quite
fit there either.  So sticking with where it lives in the data
structure!

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit db06582a06)
2015-02-19 11:10:12 -08:00
Sage Weil
d999ccb263 Merge pull request #3663 from ceph/wip-10765
librados: close watch/notify race

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2015-02-19 11:02:26 -08:00
Sage Weil
2a2d8cb2f0 Merge pull request #3693 from ceph/wip-objecter-linger-locking
objecter: clean up linger op locking

Reviewed-by: Sage Weil <sage@redhat.com>
2015-02-19 11:02:08 -08:00
Sage Weil
ebdfb127d2 Merge pull request #3710 from ceph/wip-10844
mon: MonCap: take EntityName instead when expanding profiles

Reviewed-by: Sage Weil <sage@redhat.com>
2015-02-19 11:01:07 -08:00
Samuel Just
d6e268906a PG: compensate for bug 10780 on older peers
Previously, there was a harmless bug where we didn't fill in the
last_epoch_started field for a peer which we are resetting the
last_backfill line for.  It's no longer harmless since we use that
as the activation epoch, so if the peer is missing the MIN_SIZE
feature bit, we fill in the last_epoch_started it meant to fill in.

Signed-off-by: Samuel Just <sjust@redhat.com>
2015-02-18 15:09:58 -08:00
Samuel Just
1d0d3dc8db PG: add a config option to enable (by default) recovery below min_size
This may be useful at some point.

Signed-off-by: Samuel Just <sjust@redhat.com>
2015-02-18 15:09:56 -08:00
Josh Durgin
a0ec53e882 Merge pull request #3756 from ceph/wip-10899
librbd: fixed ImageWatcher recursive locking issues

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2015-02-18 08:55:13 -08:00
Jason Dillaman
08eb584f58 librbd: fixed ImageWatcher recursive locking issues
It was possible for ImageWatcher to attempt to re-acquire held locks
via context callbacks.  This issue affected resizing/flattening when
no work was required and rescheduling a watch upon two successive
failures.

Fixes: #10899
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2015-02-18 11:10:57 -05:00
Samuel Just
48466f8d7c ReplicatedPG: only allow a degraded write if we have at least min_size copies
Also, add a config option to disable the feature.

Signed-off-by: Samuel Just <sjust@redhat.com>
2015-02-17 22:40:12 -08:00
Samuel Just
d05539e13a ECBackend: use tbl for empty transaction as well if necessary
Signed-off-by: Samuel Just <sjust@redhat.com>
2015-02-17 15:30:49 -08:00
Samuel Just
e059d586c6 ReplicatedBackend: use tbl for empty transaction to backfill peer also
Fixes: 10908
Signed-off-by: Samuel Just <sjust@redhat.com>
2015-02-17 15:15:27 -08:00
Samuel Just
25f9363b90 osd/: s/is_degraded_object/is_degraded_or_backfilling_object
Currently, this method also returns true if the object is backfilling.

Signed-off-by: Samuel Just <sjust@redhat.com>
2015-02-17 15:09:54 -08:00
Josh Durgin
1fa74bee3f Merge pull request #3749 from ceph/wip-10900
test/librbd/fsx.c: disable RBD object map for krbd

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2015-02-16 21:58:56 -08:00
Sage Weil
9264d4eaf7 Merge pull request #3747 from ceph/wip-10892
osd: clear obc cache on_shutdown

Reviewed-by: Samuel Just <sjust@redhat.com>
2015-02-16 17:50:22 -08:00