Commit Graph

113257 Commits

Author SHA1 Message Date
Mykola Golub
c20868af9d
Merge pull request #35629 from dillaman/wip-librbd-asio-2
librbd: switch IO path to use new librados asio API

Reviewed-by: Mykola Golub <mgolub@suse.com>
2020-07-18 19:42:00 +03:00
Patrick Donnelly
4fef026da0
Merge PR #36134 into master
* refs/pull/36134/head:
	mgr/nfs: Remove NParts and Cache_Size from MDCACHE block

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
2020-07-17 20:30:42 -07:00
Patrick Donnelly
046b80f7dd
Merge PR #36107 into master
* refs/pull/36107/head:
	mds: fix mds forwarding request 'no_available_op_found'

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-07-17 20:29:44 -07:00
Patrick Donnelly
a43122b716
Merge PR #36097 into master
* refs/pull/36097/head:
	mds: fix nullptr dereference in MDCache::finish_rollback

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-07-17 20:28:50 -07:00
Patrick Donnelly
66b9bf0cd8
Merge PR #35969 into master
* refs/pull/35969/head:
	mds: optimize random threshold lookup for dentry load

Reviewed-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>
2020-07-17 20:28:03 -07:00
Patrick Donnelly
c049d31540
Merge PR #35953 into master
* refs/pull/35953/head:
	qa: Fix name collisions

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
2020-07-17 20:20:47 -07:00
Patrick Donnelly
9908dc3e31
Merge PR #35902 into master
* refs/pull/35902/head:
	ceph-dedup-tool: fix warning
	mds: slave -> peer
	mds: master -> leader

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-07-17 20:19:33 -07:00
Patrick Donnelly
c950df44f3
Merge PR #36139 into master
* refs/pull/36139/head:
	mon: refactor ServiceMap::Service::get_task_summary()
	mon: only dump non-empty "task status"
	mgr/ServiceMap: move ServiceMap::Daemon implementation into .cc file

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2020-07-17 20:17:30 -07:00
Patrick Donnelly
edb1ad8ee5
Merge PR #36140 into master
* refs/pull/36140/head:
	mgr/volumes: fix "ceph nfs export" help messages

Reviewed-by: Varsha Rao <varao@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-07-17 20:15:29 -07:00
Daniel Gryniewicz
e51d0e092d
Merge pull request #35851 from dang/wip-dang-zipper-6
Wip dang zipper 6
2020-07-17 15:35:47 -04:00
Daniel Gryniewicz
f2bd2fb876 Clean up some leaks in rgw_admin
Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2020-07-17 12:05:49 -04:00
Daniel Gryniewicz
ede7561f4f NULL out g_ceph_context when deleting it
This avoids a use-after-free when there is a crash on shutdown (or when
something like ASAN triggers an ABORT on leak).

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2020-07-17 12:05:49 -04:00
Daniel Gryniewicz
99f7c4aa12 Zipper Object
This pass modifies RGW to use RGWObject and RGWRadosObject.
Also improves RGWBucket usage.  This converts many of the APIs to
unique_ptr.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2020-07-17 12:05:49 -04:00
Jason Dillaman
d9f3cc75c3
Merge pull request #35553 from zdover23/wip-doc-iscsi-target-cli-daemon
doc/rbd: add rbd-target-gw enable and start

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Alexandra Settle <asettle@suse.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-07-17 07:16:31 -04:00
Sebastian Wagner
b77b4346d4
Merge pull request #35846 from sebastian-philipp/podman-pull-retry
cephadm: Retry pull on transient error

Reviewed-by: Ricardo Marques <rimarques@suse.com>
2020-07-17 13:02:27 +02:00
Laura Paduano
8cd0490777
Merge pull request #34696 from tspmelo/wip-locl
mgr/dashboard: Use $localize and locl for TS translations

Reviewed-by: Ernesto Puertat <epuertat@redhat.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Sebastian Krah <skrah@suse.com>
2020-07-17 11:20:28 +02:00
Laura Paduano
e7a102f338
Merge pull request #36000 from tspmelo/wip-favicon
mgr/dashboard: Display cluster status in favicon

Reviewed-by: Ernesto Puertat <epuertat@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2020-07-17 09:55:09 +02:00
Sebastian Wagner
82acfe90b7
Merge pull request #35456 from sebastian-philipp/cephadm-events
mgr/cephadm: Add k8s-style event system

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2020-07-17 09:49:14 +02:00
Laura Paduano
e9816a7868
Merge pull request #35985 from bk201/wip-46413
mgr/dashboard: fix regression in delete OSD modal

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
2020-07-17 09:33:15 +02:00
Varsha Rao
90c0ebc45e mgr/nfs: Remove NParts and Cache_Size from MDCACHE block
As setting them to small value affects the performance and they are not related
to metadata caching. https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/495185

Fixes: https://tracker.ceph.com/issues/46579
Signed-off-by: Varsha Rao <varao@redhat.com>
2020-07-17 11:36:32 +05:30
Patrick Donnelly
1ebb2958c5
Merge PR #36124 into master
* refs/pull/36124/head:
	script/ptl-tool: add flag to turn off credits

Reviewed-by: Jos Collin <jcollin@redhat.com>
2020-07-16 21:10:07 -07:00
Jason Dillaman
5b2f82003b test/librbd: drop test_internal.ShrinkFlushCase test case
It's a racey test that is better exercised by the existing resize
state machine mock tests.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-07-16 17:19:04 -04:00
Jason Dillaman
44a0d7c839 librbd: utilize neorados to issue async blacklist request
The librados API does not currently offer an async 'mon_command'
API method. Instead of adding one just to support this effort,
re-use the neorados API to issue an asynchronous 'mon_command'
for blacklisting a client.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-07-16 16:02:55 -04:00
Jason Dillaman
c87c669a78 librbd: managed_lock::BreakRequest needs a reference to AsioEngine
The current usage of the asio::ContextWQ to similate an asynchronous blocklist
API call is resulting in deadlock in the rbd-mirror HA tests when multiple
blocklists are occurring concurrently. The next commit will switch to use the
neorados async MON command API (since librados doesn't offer one).

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-07-16 16:02:55 -04:00
Jason Dillaman
50694f7902 librbd: fix race condition with AIO completion callbacks
Now that librbd utilizes multiple threads for the IO path, it's
possible for a race condition to occur if a client app is waiting
on a completion to fire and the actual invokation of the
completion.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-07-16 16:02:55 -04:00
Jason Dillaman
cfbd188646 librbd: directly post completions to the asio strand
Avoid the unnecessary wrapping of the strand completion in a
bind_executor by just directly posting the completion to the
strand.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-07-16 16:02:55 -04:00
Jason Dillaman
9b6a2040b7 librbd: allocate the asio strands directly on the heap
This will assist with potential race condition debugging since the
stand pointer will be invalidated by the time the strand has been
destructed and shut down.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-07-16 16:02:31 -04:00
Jason Dillaman
fed9f94300 librbd: ensure all asio completions are complete at ImageCtx destruction
With multiple threads of execution possible, we need to ensure that
all completions have fired prior to the destruction of the AsioEngine.
We also need to ensure that the AsioEngine is destoyed outside the
context of its owned stands.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-07-16 16:01:28 -04:00
Jason Dillaman
5a4c61a781 librbd: increased logging for asio::ContextWQ
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-07-16 16:01:28 -04:00
Jason Dillaman
15ec517c59 librbd: switch the IO path to utilize the neorados API
IO operations to the cluster are now dispatched via the neorados
API which allows multiple threads to be utilized for processing
incoming and outgoing IO.

This also involves switching from a map for tracking sparse extents
to a vector of pairs since that matches the new API for sparse
read operations.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-07-16 15:59:31 -04:00
Jason Dillaman
bb14993873 librbd: switch all queued contexts in IO path to use asio post
The post operation is similar to the asio::ContextWQ::queue wrapper
but execution is not limited to a single thread strand.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-07-16 15:59:31 -04:00
Jason Dillaman
11ae84df8b librbd: switch external API callbacks to use dedicated asio strand
This ensures that the API callers will not receive concurrent
callbacks and allows internal AioCompletion users to be able to
use all available asio dispatch threads.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-07-16 15:59:31 -04:00
Jason Dillaman
09347f6d5c librbd: integrate neorados into ImageCtx
Also create an up-to-date data_io_context that mimics the function
of ImageCtx::data_ctx. The data_io_context will eventually be passed
via the IO dispatch specs to replace the passing of the snapshot
id vectors.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-07-16 15:59:31 -04:00
Jason Dillaman
f5a169bd4e librbd: replace ImageCtx::get_work_queue with direct AsioEngine usage
The ImageCtx will contain a shared_ptr to its AsioEngine and previous
usage for get_work_queue now directly instantiate their own AsioEngine
instance.

Additionally, AsioEngine is now lighter weight by re-using the
io_context exposed via a neorados-wrapped librados API.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-07-16 15:59:31 -04:00
Jason Dillaman
8581bd27e2 cls/rbd: add support for neorados-style IO requests
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-07-16 15:59:31 -04:00
Jason Dillaman
769391ac43 test/librados_test_stub: decode "osd blacklist" MON command
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-07-16 15:59:31 -04:00
Jason Dillaman
4192e91cab test/librados_test_stub: add support for basic neorados ops
Internally it re-uses the existing librados-style architecture adapted
to fit the new interface.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-07-16 15:59:31 -04:00
Jason Dillaman
ab1b152414 test/librados_test_stub: pass read snap id to read operation hooks
The neorados API does not require the creation of heavy IoCtx-like
objects with static read snap_ids pre-assigned. Therefore, we will
need to pass the read snap_id to all affected functions and adjust
all dependent unittests to expect a new parameter.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-07-16 15:59:31 -04:00
Jason Dillaman
e25b57e018 test/librbd: fixed gmock warnings for get_parent_overlap calls
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-07-16 15:59:31 -04:00
Jason Dillaman
24bd5fb294 neorados: allow a client to wait for the latest OSD map
This is required when issuing a client blacklist to ensure that
they cannot overwrite any OSD data before their map updates.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-07-16 15:59:31 -04:00
Jason Dillaman
96c0c255cc neorados: fixed missing handling of read flags
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-07-16 15:59:31 -04:00
Jason Dillaman
71fc3a3eed neorados: support blkin trace passing on execute calls
librbd passes blkin traces from the user API down through to
Objecter and back. Add these missing hooks to the neorados API
since they weren't included in the intial revision.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-07-16 15:59:31 -04:00
Jason Dillaman
7f36144f83 common/Timer: fixed invalid read from deleted object
The std::conditional_variable will keep the provided reference and
repeatedly dereference it even after the lock was dropped and
re-acquired. This can lead to an invalid read if the associated
schedule entry has been removed while waiting.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-07-16 15:59:31 -04:00
Mykola Golub
8b3735fe0d
Merge pull request #36077 from dillaman/wip-librbd-parent-cache
librbd: fix parent cache initialization failures

Reviewed-by: Mykola Golub <mgolub@suse.com>
2020-07-16 22:16:50 +03:00
Tiago Melo
d3436f068b mgr/dashboard: Use $localize and locl for TS translations
Fixes: https://tracker.ceph.com/issues/45950

Signed-off-by: Tiago Melo <tmelo@suse.com>
2020-07-16 16:01:02 +00:00
Kefu Chai
2298dd16b2
Merge pull request #36128 from xxhdx1985126/wip-crimson-fix-osd-shutdown
crimson/osd: fix osd shutdown problem

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-07-16 23:46:01 +08:00
Kefu Chai
688db8d604
Merge pull request #36142 from tchaikov/wip-crimson-cleanup
crimson: do not capture unused variables

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2020-07-16 23:44:47 +08:00
Laura Paduano
62eb03f7b1
Merge pull request #35987 from tspmelo/wip-update-lodash
mgr/dashboard: Fix some npm audit warnings

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2020-07-16 17:43:43 +02:00
Patrick Donnelly
757ff29229
Merge PR #36138 into master
* refs/pull/36138/head:
	doc/cephfs/nfs: Update about deploying nfs-ganesha with cephadm in vstart

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-07-16 07:24:16 -07:00
Laura Paduano
0d44516540
Merge pull request #35960 from votdev/issue_46395_required_field_message
mgr/dashboard: Use same required field message accross the UI

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2020-07-16 16:20:21 +02:00