Commit Graph

101984 Commits

Author SHA1 Message Date
Kefu Chai
8dfb9dbae4
Merge pull request #30258 from tchaikov/wip-41721
test/cls_rbd/test_cls_rbd: update TestClsRbd.sparsify

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-09-10 16:27:13 +08:00
Sebastian Wagner
0ad89da657 doc/orchestrator: Sync status with reality
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2019-09-10 08:48:59 +02:00
xie xingguo
1aabf7aa0a os/bluestore: fix space balancing overflow
When gifting, bluestore should not gift more than it actually has.
same for bluefs when reclaming.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-09-10 07:55:45 +08:00
Nathan Cutler
0589dbb1ca
Merge pull request #30273 from neha-ojha/wip-documenting-ceph
doc/start/documenting-ceph.rst: make better doc recommendations

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
2019-09-10 00:52:00 +02:00
Neha Ojha
07aef79693 doc/start/documenting-ceph.rst: make better doc recommendations
I do not think "Editors" is relevant anymore.

Signed-off-by: Neha Ojha <nojha@redhat.com>
2019-09-09 15:40:45 -07:00
J. Eric Ivancich
38e02a7157 rgw: fix minimum of unordered bucket listing
A recent PR made sure that a bucket listing could not request too many
entries at once. It also did a minimum computation for number of
entries. For ordered listing the minimum was 0, as required to pass
all unit tests. However the minimum for unordered listing was left at
1. In order to make ordered and unordered listing behave the same --
with the exception of ordering -- the minimum for unordered listing is
modified to 0.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2019-09-09 15:17:18 -04:00
Patrick Donnelly
e1e962d28a
Merge PR #30187 into master
* refs/pull/30187/head:
	script/backport-resolve-issue: better error message

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-09-09 11:33:40 -07:00
Alfredo Deza
4612ab3ae6 ceph-volume create a new tox.ini for shell-based tests
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2019-09-09 13:40:06 -04:00
Neha Ojha
009057e154
Merge pull request #30226 from neha-ojha/wip-41385-2
osd/MissingLoc.cc: do not rely on missing_loc_sources only

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed By: Xie Xingguo <xie.xingguo@zte.com.cn>
2019-09-09 09:48:45 -07:00
Sage Weil
efff7776f1 osd/PeeringState: send new message types
Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-09 11:22:12 -05:00
Sage Weil
77ecc32788 osd/PeeringState: give require_osd_release to BufferedRecoveryMessages
They need to know what type of message(s) to queue.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-09 11:22:12 -05:00
Sage Weil
ce05c17299 osd: add new notify, query, and info messages
These are streamlined to only include the information we need for the
peering events, as reflected by the old handle_fast_pg_{notify,query,info}
messages.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-09 11:22:11 -05:00
Sage Weil
91d123cded osd/PeeringState: use send_info() for replica activation ack
Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-09 11:22:11 -05:00
Sage Weil
7744fc6920 osd/PeeringState: remove old info_map member
Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-09 11:22:11 -05:00
Sage Weil
56c542ccb3 osd/PeeringState: send infos via message_map (not info_map)
Queue up separate message for each.

Note that even though we are currenting sending this via a pg_notify_t,
not all of those fields are needed or consumed on the other end (notably,
PastIntervals is not used by handle_fast_pg_info).  Only pass the fields
we need to send_info() so that we can improve the transport to be less
weird later.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-09 11:22:11 -05:00
Sage Weil
362421c2f9 osd/PeeringState: remove old query_map member
Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-09 11:22:11 -05:00
Sage Weil
5b42225b03 osd/PeeringState: send queries via message_map (not query_map)
Queue up separate messages for each query.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-09 11:22:11 -05:00
Sage Weil
76b5a0022d osd/PeeringState: use return value from discover_all_missing
Right now we tell whether discover_all_missing did something by looking
at whether query_map is empty. Instead, use an explicit return value.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-09 11:22:11 -05:00
Sage Weil
61bf8f409a osd/PeeringState: give PeeringCtxWrapper a BufferedRecoveryMessages ref
The ref can be either a raw BufferedRecoveryMessages (buffering) or
a PeeringCtx (child class, no buffering).  This avoids the need for a
member ref, and also means that we can (later) make methods take the
BufferedRecoveryMessages ref as a general interface to consume for both
buffered and non-buffered work that is outside of PeeringState.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-09 11:22:11 -05:00
Sage Weil
49a0573aa8 osd/PeeringState: send notifies via message_map (not notify_list)
Queue up a separate message for each notify, and drop the old notify_list
plumbing.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-09 11:22:11 -05:00
Sage Weil
a888ebe4b4 osd/PeeringState: add message_map to PeeringCtx/BufferedRecoveryMessages
Have a generalized map of messages that are queued to be sent out.  Send
them when we dispatch the PeeringCtx.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-09-09 11:22:11 -05:00
Sage Weil
c3d21f119a Merge PR #30196 into master
* refs/pull/30196/head:
	qa/tasks/ceph: restart: stop osd, mark down, then start

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-09-09 11:19:32 -05:00
Casey Bodley
781c7e71b7
Merge pull request #30268 from cbodley/wip-41719
cls/rgw: fix bilog trim tests in ceph_test_cls_rgw

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
2019-09-09 12:18:39 -04:00
Sage Weil
9b082434af Merge PR #30220 into master
* refs/pull/30220/head:
	test: Allow fractional milliseconds to make test possible

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-09-09 11:09:04 -05:00
Tiago Melo
02b4fc0a38 mgr/dashboard: Fix error message not displayed in pool form
Since the parent form-group was not assigned to any field and was never invalid,
the invalid-feedback element was never displayed.

Signed-off-by: Tiago Melo <tmelo@suse.com>
2019-09-09 14:29:43 +00:00
Casey Bodley
faf9cb384d cls/rgw: fix bilog trim tests in ceph_test_cls_rgw
the bucket index prepare step no longer writes a bilog entry, so the
tests were expecting more entries

Fixes: https://tracker.ceph.com/issues/41719

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-09-09 10:14:51 -04:00
Kefu Chai
583b09f30a
Merge pull request #29127 from bk201/wip-40337
mgr/dashboard: orchestrator integration initial works

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2019-09-09 22:01:58 +08:00
Kefu Chai
855d84d283 test/cls_rbd/test_cls_rbd: update TestClsRbd.sparsify
after #30061, extents returned by sparse read will always be
an empty map as long as the extents to be read is empty or the extent(s)
in it are empty, even of the objectstorage does not support sparse read.

Fixes: https://tracker.ceph.com/issues/41721
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-09-09 21:44:53 +08:00
Alfredo Deza
68f061401d
Merge pull request #30115 from alfredodeza/wip-rm41378-2
ceph-volume: pre-install python-apt and its variants before test runs

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
2019-09-09 09:32:03 -04:00
Kefu Chai
670a9a564f
Merge pull request #30142 from votdev/issue_41648
mgr/dashboard: Handle always-on Ceph Manager modules correctly

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-09-09 20:32:30 +08:00
Kefu Chai
e584866216
Merge pull request #30079 from uweigand/issue-41605-fixes
Fix big-endian handling

Reviewed-by: Sage Weil <sage@redhat.com>
2019-09-09 20:29:39 +08:00
Kefu Chai
8b88ab0f2c
Merge pull request #30087 from egggHang/ec_benchmark
test/erasure-code: add exception handling to k & m

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-09-09 20:24:20 +08:00
Kefu Chai
72bd45ae65
Merge pull request #29892 from Yan-waller/wip-yanj-fixreed_sol_r6_op
osd: fix the missing default value m=2 of reed_sol_r6_op in profile

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-09-09 20:22:08 +08:00
Kefu Chai
45392a202b
Merge pull request #30125 from majianpeng/msg-protocolv1-reduece-swap
msg/async/ProtocolV1: avoid unnecessary bufferlist::swap.

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2019-09-09 20:11:46 +08:00
Yan Jun
aed94cf203 PendingReleaseNotes: note about changes in ceph osd erasure-code-profile set
Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>
2019-09-09 20:02:08 +08:00
Yan Jun
1032b981da osd: add sanity check for invalid m/w of liber8tion
Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>
2019-09-09 20:00:20 +08:00
Yan Jun
87c20c39b2 ec/jerasure: save the default m=2 of reed_sol_r6_op in profile
save the default value of m to profile so that it could be dumped
in command 'ceph osd erasure-code-profile get xxx', which is more
useful and friendly to user.

Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>
2019-09-09 20:00:20 +08:00
Kefu Chai
a8c710ef19
Merge pull request #30109 from rzarzynski/wip-bug-25098-bluefs_layout_t
os/bluestore: verify disk layout of BlueFS

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2019-09-09 19:53:53 +08:00
Kefu Chai
3c0400e518
Merge pull request #30197 from bk201/wip-41675
mgr/orchestrator: fix refs property of progresses

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2019-09-09 19:19:44 +08:00
Volker Theile
8ac6559637 mgr/dashboard: Handle always-on Ceph Manager modules correctly
Currently always-on modules are not marked as enabled in the WebUI and can be disabled. This PR will fix that.
Note, this PR will NOT implement code that will prevent a developer from trying to disable an always-on module through the REST API. The Mgr Python extension will throw an adequate exception.

This PR will also do:
* Remove old code fragments from a previous Mgr Module management UI that is obsolete now.
* Cleanup code in BaseMgrModule code.

Fixes: https://tracker.ceph.com/issues/41648

Signed-off-by: Volker Theile <vtheile@suse.com>
2019-09-09 11:12:55 +02:00
xie xingguo
dbcde2daf4 os/bluestore: add bluestore_bluefs_max_free option
This prevents bluefs from trying to acquire more space than
it actually needs, e.g., due to batch deletion of user data.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-09-09 08:06:36 +08:00
Kefu Chai
9e7ecf708a
Merge pull request #30199 from IlsooByun/cmake_ptyhon_var
build/ops: fix build fail related to PYTHON_EXECUTABLE variable

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-09-09 00:02:20 +08:00
Sage Weil
cdb8df13cb Merge PR #29986 into master
* refs/pull/29986/head:
	mgr/pg_autoscaler: use 'stored' for pool_logical_used

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-09-07 15:09:25 -05:00
Sage Weil
8d94dc7f0f Merge PR #30139 into master
* refs/pull/30139/head:
	mgr/DaemonServer: skip adjusting pgp_num when merging is in-progress

Reviewed-by: Sage Weil <sage@redhat.com>
2019-09-07 15:08:57 -05:00
Xie Xingguo
8c99f815f3
Merge pull request #30061 from xiexingguo/wip-sparse-read
os, osd: readv

Reviewed-by: Sage Weil <sage@redhat.com>
2019-09-07 10:34:09 +08:00
Xie Xingguo
8971bc7ba9
Merge pull request #30059 from xiexingguo/wip-clean-vs-repair
osd/PG: fix _finish_recovery vs repair race

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-09-07 10:31:28 +08:00
Kefu Chai
dd9c4e0b2a
Merge pull request #30190 from tchaikov/wip-python-rpm-macro
install-deps.sh: install `python*-devel` for python*rpm-macros

Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-09-07 08:57:11 +08:00
Neha Ojha
5906a57320 osd/MissingLoc.cc: do not rely on missing_loc_sources only
In 624ade487e, we relied on missing_loc_sources
to check for strays and remove an OSD from missing_loc. However, it is
possible that missing_loc_sources is empty while there are still OSDs
present in missing_loc. Since the aim is to just remove a stray OSD from
missing_loc, we do not need to rely on missing_loc_sources. We still
clean missing_loc_sources if any stray is present in it.

Signed-off-by: Neha Ojha <nojha@redhat.com>
2019-09-06 15:49:36 -07:00
Casey Bodley
60b2dbe67a rgw: fix dns name comparison for virtual hosting
Fixes: https://tracker.ceph.com/issues/41692

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-09-06 15:23:27 -04:00
David Zafman
6d2e4cb109 test: Allow fractional milliseconds to make test possible
Fixes: https://tracker.ceph.com/issues/41689

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-09-06 11:23:52 -07:00