Commit Graph

91184 Commits

Author SHA1 Message Date
Sage Weil
d6022dd87b mgr/DaemonServer: block pg_num decrease(merge) until pgp_num is reduced
We can't merge until the PGs are stored together.  (The mon would stop
us if we tried, but let's not waste time trying.)

Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-16 07:22:48 -05:00
Sage Weil
155d9a2c00 mgr/DaemonServer: adjust_pgs(): cosmetic change to debug output
Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-16 07:22:48 -05:00
Sage Weil
128437c579 mon/PGMap: add get_recovery_stats()
This is some of the same info we get in the json dump from
print_summary -> overall_recovery_summary -> recovery_summary.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-16 07:22:48 -05:00
Sage Weil
edbd592ee4 mgr/balancer: mgr/balancer/max_misplaced -> pg_max_misplaced
Make this a compiled-in option that is consumed by other components as
well as the balancer.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-16 07:22:48 -05:00
Sage Weil
4fc2af95ce pybind/mgr/mgr_module: add get_option()
get_config() gets a mgr-style config option.  Add get_option which
fetches a compiled-in config option.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-16 07:22:48 -05:00
Sage Weil
b4c17cca15 mgr/DaemonServer: allow pg_num increases that abort pending merges
If we are waiting for a PG to merge we can't decrease more, but if we
were in the process of merging and an increase is requested, we can
abort the merge by increase pg_num_actual whenever we want.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-16 07:22:48 -05:00
Sage Weil
b84fa91368 mon/OSDMonitor: resent pre-nautilus client ops on aborted merge
If we are in premerge (pg_num_pending == pg_num - 1) and abort by
increasing pg_num, we the last_force_op_resend_prenautilus since it will
be an interval change for nautlius+.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-16 07:22:48 -05:00
Sage Weil
48412ce4e4 mon/OSDMonitor: make pgp_num track pg_num more consistently
Previously we were automatically adjusting pgp_num_target on a
pg_num_target change *only* when decreasing pg_num.  Instead, make
pgp_num (continue to) track pg_num if it currently matches.  If it ever
is set differently than pg_num, leave it different (unless/until it
matches again).

This is still slightly weird, but I think in practice it is good enough.
In the rare case that the admin manually sets pgp_num to something
different than pg_num, they probably won't also be using automagic
pg_num adjustment that might make them match and start tracking again.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-16 07:22:48 -05:00
Sage Weil
747c9dcf0a Merge PR #24095 into master
* refs/pull/24095/head:
	osd: do not authenticate heartbeat connections until nautilus
	msg,osd: enable unauthenticated Dispatcher for pre-nautilus OSD compat
	osd: add missing space in heartbeat debug output
	mgr/DaemonServer: add missing return
	msg/async: drop verify_authorizer wrapper
	osd: authenticate ping sessions
	msg/simple: remove verify_authorizer wrapper
	msg: remove unused ms_verify_authorizer
	msg/async: remove get_authorizer wrapper
	msg/simple: remove get_authorizer wrapper
	msg/Messenger: pull authenticator validation into Messenger
	msg/Messenger: uninline ms_deliver_verify_authorizer
	mgr/DaemonServer: expose keyring for authenticator verification
	mon: expose keyring for msgr1 authentication
	mds: expose keyring for authenticater verification
	osd: expose keyring for authenticater verification
	msg/Dispatcher: add ms_get_auth1_authorizer_keystore
	mon: fix ref cycle breakage in handle_forward
	mon: use MonOpRequest get_session() instead of PaxosServiceMessage's
	mon: get session from MonOpRequest in handle_command
	messages/MForward: drop unused ctor
	mon: use ms_handle_authentication to parse caps
	mon: kill Session::global_id and use Connection member instead
	mgr/DaemonServer: move session setup into ms_handle_authentication
	mds: move session setup into ms_handle_authentication
	osd: move session setup into ms_handle_authentication
	msg: new ms_handle_authentication, add fields to Connection

Reviewed-by: Ricardo Dias <rdias@suse.com>
2018-10-16 07:20:17 -05:00
Sage Weil
05faeb4d12 Merge PR #24579 into master
* refs/pull/24579/head:
	qa/osd: fixup osd-rep-recov-eio.sh fails to parse pg dump

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2018-10-16 07:17:59 -05:00
John Spray
e6a26aeff7
Merge pull request #24597 from batrick/i36450
qa: fix run call args

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
2018-10-16 13:09:08 +01:00
Kefu Chai
67777bbe7a
Merge pull request #23983 from theanalyst/vstart-rgw-asok
vstart: set admin socket for RGW in conf

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2018-10-16 19:15:35 +08:00
Kefu Chai
52706830d6
Merge pull request #24576 from cyx1231st/wip-seastar-msgr-refactor
crimson/net: seastar-msgr refactoring

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-10-16 16:44:16 +08:00
Kefu Chai
e1fd799b75
Merge pull request #24255 from gregsfortytwo/wip-crush-doc
doc: explain 'firstn v indep' in the CRUSH docs

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2018-10-16 13:49:19 +08:00
Patrick Donnelly
2760ddde89
Merge PR #24286 into master
* refs/pull/24286/head:
	client: fix fuse client can't read or write data due its caps is invalid

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-10-15 21:34:31 -07:00
Patrick Donnelly
6a4cc58a9d
Merge PR #24292 into master
* refs/pull/24292/head:
	qa: add test for rctime on root inode
	mds: set rctime on new system inode
	mds: small refactor

Reviewed-by: Zheng Yan <zyan@redhat.com>
2018-10-15 21:31:04 -07:00
Patrick Donnelly
35843a31c8
Merge PR #24486 into master
* refs/pull/24486/head:
	client: explicitly show blacklisted state via asok status command

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-10-15 21:23:19 -07:00
Patrick Donnelly
14f23d602b
Merge PR #24508 into master
* refs/pull/24508/head:
	cephfs-shell: fixup 'str' object has no attribute 'decode'

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-10-15 21:12:56 -07:00
Sage Weil
585d21df65 Merge PR #24596 into master
* refs/pull/24596/head:
	ptl-tool.py: move githubmap update into merge commit

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2018-10-15 18:41:53 -05:00
Patrick Donnelly
d491227956
qa: fix run call args
Fixes: http://tracker.ceph.com/issues/36450
Introduced-by: 95746ecce9
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-10-15 14:45:18 -07:00
Patrick Donnelly
28b9b4ea2b
Merge PR #24562 into master
* refs/pull/24562/head:
	removed warning for resolved errata

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-10-15 14:34:22 -07:00
Gregory Farnum
929a9201c8
Merge pull request #24186 from libingyang-zte/master
doc: fix typos in doc/releases
2018-10-15 14:17:36 -07:00
Gregory Farnum
9b099583ca
Merge pull request #24238 from LenzGr/doc-leads-update
doc/dev: Updated component leads table
2018-10-15 14:15:43 -07:00
Yingxin
265fcdd2ed crimson/net: clean seastar-msgr class dependencies
Remove protocol-specific interfaces from Messenger/Connection classes,
and let SocketMessenger manage SocketConnection instead of Connection.

Signed-off-by: Yingxin <yingxin.cheng@intel.com>
2018-10-16 04:43:35 +08:00
Sage Weil
2152d8ffb7 osd: do not authenticate heartbeat connections until nautilus
Some (currently) pre-nautilus OSDs will crash if you try to authenticate
a heartbeat connection but they are not expecting it:

src/auth/Crypto.h: 109: FAILED assert(ckh)
 ceph version 12.2.8-457-gccd69ef (ccd69ef36a) luminous (stable)
 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x110) [0x5618b04aaea0]
 2: (()+0x41cbec) [0x5618afe2cbec]
 3: (CephxSessionHandler::_calc_signature(Message*, unsigned long*)+0x8c5) [0x5618b0777ba5]
 4: (CephxSessionHandler::check_message_signature(Message*)+0x7d) [0x5618b077800d]
 5: (AsyncConnection::process()+0x1b44) [0x5618b0761a04]
 6: (EventCenter::process_events(int, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> >*)+0x359) [0x5618b0546079]
 7: (()+0xb38c3e) [0x5618b0548c3e]
 8: (()+0xb5070) [0x7ff04faf5070]
 9: (()+0x7dd5) [0x7ff050168dd5]
 10: (clone()+0x6d) [0x7ff04f259b3d]

See http://tracker.ceph.com/issues/36443

It won't be fixed in all clusters before upgrade to nautilus, though, so
we also need to work around it here.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-15 15:01:40 -05:00
Sage Weil
b8d1c80370 msg,osd: enable unauthenticated Dispatcher for pre-nautilus OSD compat
Before nautilus, osd heartbeats are sent over an unauthenticated channel.
We need support here to allow these connections when they are necessary
for upgrade compatibility.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-15 15:01:40 -05:00
Sage Weil
4b2f82ca5e osd: add missing space in heartbeat debug output
Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-15 15:01:40 -05:00
Sage Weil
28debec0f3 mgr/DaemonServer: add missing return
Fixes: http://tracker.ceph.com/issues/36110
Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-15 15:01:40 -05:00
Sage Weil
a5f9295537 msg/async: drop verify_authorizer wrapper
Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-15 15:01:40 -05:00
Sage Weil
a3c40b345f osd: authenticate ping sessions
Do not set up a Session object, though--nobody cares (currently!).

This avoids having to special-case the generic authorizer validation
code in msg/* to have to handle non-authenticated sessions.  Also, it
seems like a good idea to authenticate these sessions!

Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-15 15:01:40 -05:00
Sage Weil
e91fd88b92 msg/simple: remove verify_authorizer wrapper
Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-15 15:01:40 -05:00
Sage Weil
1e3517a31e msg: remove unused ms_verify_authorizer
Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-15 15:01:40 -05:00
Sage Weil
31f82c95d6 msg/async: remove get_authorizer wrapper
Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-15 15:01:40 -05:00
Sage Weil
955017ea4d msg/simple: remove get_authorizer wrapper
Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-15 15:01:40 -05:00
Sage Weil
b362ee2120 msg/Messenger: pull authenticator validation into Messenger
This code is essentially identical across the OSD and MDS.  The
monitor is annoyingly different, but in a msgr1 specific way that
we can handle carrying here until msgr1 gets ripped out in a
couple years.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-15 15:01:40 -05:00
huanwen ren
f1219d716d qa/osd: fixup osd-rep-recov-eio.sh fails to parse pg dump
Fixes: http://tracker.ceph.com/issues/36418
Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>
2018-10-16 02:18:22 +08:00
Lenz Grimmer
4decc6b3a4
Merge pull request #24577 from p-na/fix-saves-invalid-config
dashboard/mgr: Save button doesn't prevent saving an invalid form

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2018-10-15 18:01:41 +02:00
Scoots Hamilton
9352899618 removed warning for resolved errata
Signed-off-by: Scoots Hamilton <scoots@redhat.com>
2018-10-15 11:23:31 -04:00
Sage Weil
f22e55ad7a Merge PR #24473 into master
* refs/pull/24473/head:
	common: drop get_contiguous() from ceph::bufferlist.

Reviewed-by: Sage Weil <sage@redhat.com>
2018-10-15 08:42:50 -05:00
Sage Weil
36c3bce219 Merge PR #24493 into master
* refs/pull/24493/head:
	mgr/DaemonState: clean up device life_expectancy values
	mgr/devicehealth: warn based on life_expectancy_max
	mgr/devicehealth: warn on failing devices at 6 weeks

Reviewed-by: John Spray <john.spray@redhat.com>
2018-10-15 08:36:18 -05:00
Lenz Grimmer
66db97eb6b
Merge pull request #24523 from s0nea/wip-dashboard-configs-table-cleanup
mgr/dashboard: config options table cleanup

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2018-10-15 15:14:52 +02:00
Tatjana Dehler
f4b2f4d91d mgr/dashboard: config options table cleanup
Remove columns 'tags', 'enum_values', 'long_desc', 'type', 'flags',
'daemon_default', 'desc', 'level', 'can_update_at_runtime', 'services',
'max', 'see_also', 'min' and 'source' from table view and add them to
the details.
The table contains 'name', 'value' and 'default' only.

Fixes: http://tracker.ceph.com/issues/34533

Signed-off-by: Tatjana Dehler <tdehler@suse.com>
2018-10-15 11:36:43 +02:00
John Spray
cf31e6f8a4
Merge pull request #24560 from sebastian-philipp/orchestrator-fix-rook-cluster-in-name
mgr/rook: Fix Rook cluster name detection

Reviewed-by: John Spray <john.spray@redhat.com>
2018-10-15 09:23:39 +01:00
Patrick Nawracay
486f3f279d mgr/dashboard: Save button doesn't prevent saving an invalid form
Fixes: http://tracker.ceph.com/issues/36426

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>
2018-10-15 08:40:51 +02:00
Sage Weil
7ac6ab4b2f Merge PR #24494 into master
* refs/pull/24494/head:
	ceph-kvstore-tool: rename repair -> destructive-repair

Reviewed-by: Neha Ojha <nojha@redhat.com>
2018-10-14 13:11:11 -05:00
Sage Weil
b462cf2b22 msg/Messenger: uninline ms_deliver_verify_authorizer
Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-14 12:01:10 -05:00
Sage Weil
f2f4493b26 mgr/DaemonServer: expose keyring for authenticator verification
Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-14 12:01:10 -05:00
Sage Weil
cdb5b9da3f mon: expose keyring for msgr1 authentication
Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-14 12:01:10 -05:00
Sage Weil
6296573ef1 mds: expose keyring for authenticater verification
Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-14 12:01:10 -05:00
Sage Weil
14e3cb1555 osd: expose keyring for authenticater verification
Signed-off-by: Sage Weil <sage@redhat.com>
2018-10-14 12:01:09 -05:00