Commit Graph

83325 Commits

Author SHA1 Message Date
Yehuda Sadeh
c598e53c95 rgw: parse old rgw_obj with namespace correctly
Fixes: http://tracker.ceph.com/issues/22982

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2018-02-13 10:18:08 -08:00
Patrick Donnelly
7bf6eb0198
qa: ignore io pause warnings in mds-full test
Fixes: http://tracker.ceph.com/issues/22990

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-02-13 09:54:12 -08:00
Gregory Farnum
240b688177
Merge pull request #20147 from smithfarm/wip-15904
tools: rados: make -f be --format for consistency with ceph tool

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2018-02-13 09:07:24 -08:00
Kefu Chai
9a0d1517a8
Merge pull request #20422 from cbodley/wip-check-cls-rgw
cmake: 'make check' builds radosgw and its cls dependencies

Reviewed-by: Ali Maredia <amaredia@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-02-14 00:35:14 +08:00
Casey Bodley
a029d20b86 cmake: 'make check' builds radosgw and its cls dependencies
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2018-02-13 10:32:11 -05:00
Jason Dillaman
bcfcbcfe0b librbd: flush journal commit position after stopping replay
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-02-13 10:05:29 -05:00
Jason Dillaman
487963cbf0 test/librbd: disable clone v2 for negative test cases
Fixes: http://tracker.ceph.com/issues/22979
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-02-13 09:14:18 -05:00
Jason Dillaman
e57a735bfe cls/rbd: don't clear CLONE_CHILD op feature if still a child
After removing the last snapshot linked to a parent image,
don't clear the CLONE_CHILD op feature bit if the image HEAD
is still linked to the parent.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2018-02-13 09:14:16 -05:00
Mykola Golub
f5c02adfdb librbd: potential race in RewatchRequest when resetting watch_handle
Signed-off-by: Mykola Golub <mgolub@suse.com>
2018-02-13 14:20:09 +02:00
Jos Collin
884b936f40 Handle dir->dentries.empty()
Handle dir->dentries.empty() (during stale file handle error) situation appropriately.

Fixes: http://tracker.ceph.com/issues/21995
Signed-off-by: Jos Collin <jcollin@redhat.com>
2018-02-13 09:12:53 +05:30
David Zafman
ba5a53ceeb osd: Fix assert when checking missing version
Fixes: http://tracker.ceph.com/issues/21218

Signed-off-by: David Zafman <dzafman@redhat.com>
2018-02-12 16:57:02 -08:00
Brad Hubbard
96125fe2b2 test/admin_socket_output: better error reporting
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2018-02-13 10:22:53 +10:00
Sage Weil
e48d236053 osd/PG: keep PG alive until the final onreadable context under old ch is done
The onreadable completions go through a finisher; add a final event
in that stream that keeps the PG alive while prior events flush.

flush() isn't quite sufficient since it doesn't wait for the finisher
events to flush too--only for the actual apply to have happened.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-02-12 14:35:28 -06:00
Sage Weil
98b4e900df os/filestore: wait_for_apply on get_omap_iterator
Note that we don't need to worry about the internal get_omap_iterator
callrs (e.g., omap_rmkeyrange) because the apply thread does these
ops sequentially and in order.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-02-12 14:35:28 -06:00
Sage Weil
766092e333 osd/PG: flush ch before pg delete
Make sure SnapMapper's ContainerContexts don't outlive the mapper
itself.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-02-12 14:35:28 -06:00
Sage Weil
0c71bbab4e ceph-objectstore-tool: fix SnapMapper vs import
- keep mapper around for duration of import
- flush in-flight requests before tearing it down.  This is necessary
because the mapper still uses onreadable.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-02-12 14:35:28 -06:00
Sage Weil
c1c495f097 osd/PG: remove last_updated_applied, op_applied()
No longer used or needed, yay!

Signed-off-by: Sage Weil <sage@redhat.com>
2018-02-12 14:35:28 -06:00
Sage Weil
01b68f969c osd/PGBackend: drop on_applied callback for submit_transaction
This removes a ton of tracking for ReplicatedBackend.  ECBackend needs
to keep most of it so that it can track in-flight applies on legacy
peer OSDs.  We can remove this post-nautilus.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-02-12 14:35:28 -06:00
Sage Weil
ac055b224a osd/PGBackend: remove on_applied() from interface
PrimaryLogPG calls it synchronously, on its own, after
submit_transaction.  That means the backends no longer need to
track it or call back to it.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-02-12 14:35:27 -06:00
Sage Weil
c489112a1d osd/PG: drop scrub machinery to wait for last_updated_applied
This is no longer needed--queued writes are immediately readable.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-02-12 14:35:27 -06:00
Sage Weil
f3d1b2574e osd/PrimaryLogPG: clean out applied members of RepGather
on_applied
rep_done (same as all_committed)
applies_with_commit (no longer relevant)

Signed-off-by: Sage Weil <sage@redhat.com>
2018-02-12 14:35:27 -06:00
Sage Weil
b21dbb3601 osd/PrimaryLogPG: drop repop_all_applied()
We don't need to eval_repop() since we know nothing has committed yet.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-02-12 14:35:27 -06:00
Sage Weil
17dd5eea5a osd/PG: remove obsolete comment in _delete_some()
Signed-off-by: Sage Weil <sage@redhat.com>
2018-02-12 14:35:27 -06:00
Sage Weil
ee314b9351 osd/ECBackend: unify commit+applied msg; drop applied callback
Signed-off-by: Sage Weil <sage@redhat.com>
2018-02-12 14:35:27 -06:00
Sage Weil
60776b6e18 osd/ReplicatedBackend: do applied work synchronously
Signed-off-by: Sage Weil <sage@redhat.com>
2018-02-12 14:35:27 -06:00
Sage Weil
2faf2626bb osd/PrimaryLogPG: call op_applied directly after queueing txn
There is still more cleanup to be done, here!  Later.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-02-12 14:35:27 -06:00
Sage Weil
08007acb15 osd/PG: stop holding PG ref until on_applied on PG removal
Not necessary--the backend should manage the lifecycle.  We just own
one ref anyway.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-02-12 14:35:27 -06:00
Sage Weil
a841443046 osd/PG: make C_DeleteMore called only once (on safe, not applied)
Signed-off-by: Sage Weil <sage@redhat.com>
2018-02-12 14:35:27 -06:00
Sage Weil
ad74b3d1c5 osd: drop C_UpdateLastRollbackInfoTrimmedToApplied
The variable can probably be removed as well, but leaving that for another
time.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-02-12 14:35:27 -06:00
Sage Weil
26e450753c osd/PGBackend: drop on_local_applied_async
Unused!

Signed-off-by: Sage Weil <sage@redhat.com>
2018-02-12 14:35:27 -06:00
Sage Weil
52dd99e301 osd: drop unnecessary OSDMap pinning
Signed-off-by: Sage Weil <sage@redhat.com>
2018-02-12 14:35:26 -06:00
Sage Weil
c244300ef3 osd/PrimaryLogPG: remove ondisk_{read,write}_lock infra
This is no longer needed.  FileStore was the only backend doing async
applies, and it now blocks until apply all on its own.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-02-12 14:35:26 -06:00
Sage Weil
fbb5a9dd6e os/ObjectStore: drop most queue_transactions() variants
Register completions on the Transaction(s) in the caller!

Signed-off-by: Sage Weil <sage@redhat.com>
2018-02-12 14:35:26 -06:00
Sage Weil
4044b4574c osd: synchronously open collection handles
bluestore and memstore are the only backends to implement
open_collection, and both of them can issue a handle immediately
after queue_transaction.  Do that!

Signed-off-by: Sage Weil <sage@redhat.com>
2018-02-12 14:35:26 -06:00
Sage Weil
907b6281e9 os: apply_transaction -> queue_transaction
Signed-off-by: Sage Weil <sage@redhat.com>
2018-02-12 14:35:26 -06:00
Sage Weil
33665bfe9f msg/DispatchQueue: prevent queueing of new messages after shutdown
The other queue_* methods check stop; enqueue() did not.

Fixes: http://tracker.ceph.com/issues/18351
Signed-off-by: Sage Weil <sage@redhat.com>
2018-02-12 14:21:01 -06:00
Sage Weil
754577379a os/filestore: keep OpSequencers alive
Prevent a collection delete + recreate sequence from allowing two
conflicting OpSequencers for the same collection to exist as this
can lead to racing async apply threads.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-02-12 13:56:43 -06:00
Sage Weil
03213f4ff6 os/filestore: more efficient waiter tracking
Signed-off-by: Sage Weil <sage@redhat.com>
2018-02-12 13:56:43 -06:00
Sage Weil
3fd1634fe9 os/filestore: wait_for_apply on read ops
On any read, wait for any updates to the object to apply first.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-02-12 13:56:43 -06:00
Sage Weil
28d63a4d8e os/filestore: track objects with writes being applied
Note that this is *slight* overkill in that a *source* object of a clone
will also appear in the applying map, even though it is not being
modified.  Given that those clone operations are normally coupled with
another transaction that does write (which is why we are cloning in the
first place) this should not make any difference.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-02-12 13:56:43 -06:00
Sage Weil
13738cd752
Merge pull request #19939 from liewegas/wip-osd-new-class
mon,osd: do not use crush_device_class file to initalize class for new osds

Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Andrew Schoen <aschoen@redhat.com>
2018-02-12 11:17:14 -06:00
Gregory Farnum
72ad76402c
Merge pull request #20363 from dongbula/e-rados-get-put-command
make rados get/put/append command help txt clear

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-02-12 08:56:56 -08:00
Kefu Chai
3a1357450c doc/dev/macos: add doc for building on MacOS
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-02-13 00:05:47 +08:00
Jos Collin
f8b751d1bc client: lookup_ino when i1 becomes NULL
Also set FUSE_CAP_EXPORT_SUPPORT

Fixes: http://tracker.ceph.com/issues/21995
Signed-off-by: Jos Collin <jcollin@redhat.com>
2018-02-12 20:28:46 +05:30
Jeff Layton
4143c6e972 client: clamp I/O sizes to INT_MAX when we can't return larger values
We have several API functions that allow the caller to request I/Os
larger than INT_MAX bytes, but that return an int. Ensure that we don't
try to do more I/O than we can represent in the return value.

Tracker: http://tracker.ceph.com/issues/22948
Signed-off-by: Jeff Layton <jlayton@redhat.com>
2018-02-12 07:17:56 -05:00
Jeff Layton
b431d2879a test: new testcase for ceph_ll_readv and ceph_ll_writev
Just copy the PreadvPwritev test and alter accordingly.

Tracker: http://tracker.ceph.com/issues/22948
Signed-off-by: Jeff Layton <jlayton@redhat.com>
2018-02-12 07:17:56 -05:00
Jeff Layton
4e1f059222 client: hook up ceph_ll_readv and ceph_ll_writev
Break the core of _preadv_pwritev out into a function that takes a Fh.
Make _preadv_pwritev into a wrapper around that.

Then add in plumbing for ceph_ll_readv and ceph_ll_writev.

Tracker: http://tracker.ceph.com/issues/22948
Signed-off-by: Jeff Layton <jlayton@redhat.com>
2018-02-12 07:17:56 -05:00
Jeff Layton
5579035cfd client: type safety cleanup for _read and _write codepaths
In the read codepath, bl->length() returns an unsigned value, and that
could end up looking negative when cast to int. On the write side,
totalwritten is a uint64_t, which could look negative when cast to int.

Have the underlying layers pass back an int64_t and convert them to
int at a higher level. This prepares the underlying infrastructure for
ceph_ll_readv and ceph_ll_writev support.

Tracker: http://tracker.ceph.com/issues/22948
Signed-off-by: Jeff Layton <jlayton@redhat.com>
2018-02-12 07:17:51 -05:00
lvshuhua
1bf0696f7d tools/rados: make rados get/put/append command help txt clearer
Fixes: http://tracker.ceph.com/issues/22958

Signed-off-by: lvshuhua <lvshuhua@cmss.chinamobile.com>
2018-02-12 15:30:15 +08:00
Kefu Chai
87e3aa6427
Merge pull request #20392 from adamemerson/wip-clangtastic-voyage
cmake, test/fs, client: fix build with clang

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-02-12 11:19:39 +08:00