Commit Graph

97825 Commits

Author SHA1 Message Date
Samuel Just
5e36acdb6f osd/: mechanically rename RecoveryMachine/State/Ctx to Peering*
I'm going to extract this logic and reuse it in crimson.  Recovery* has
always been a confusing name as it implements neither log-based recovery
nor backfill.  Rather, it's mainly the buisiness logic for agreeing on
an authoritative log and some ancillary things such as scrub/backfill
reservation.

$ for i in $(git grep -l 'RecoveryMachine'); do sed -i 's/RecoveryMachine/PeeringMachine/g' $i; done
$ for i in $(git grep -l 'RecoveryState'); do sed -i 's/RecoveryState/PeeringState/g' $i; done
$ for i in $(git grep -l 'RecoveryCtx'); do sed -i 's/RecoveryCtx/PeeringCtx/g' $i; done

Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:11 -07:00
Gregory Farnum
8cd1128157
Merge pull request #27867 from tchaikov/wip-ceph-object-corpus
ceph-object-corpus: pick up 15.0.0-539-g191ab33faf

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2019-04-29 13:45:16 -07:00
Casey Bodley
f0a9719d2c
Merge pull request #27730 from cbodley/wip-rgw-boost-170
rgw: fix compile errors with boost 1.70

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2019-04-29 13:06:47 -04:00
Kefu Chai
c99f7e1682
Merge pull request #27861 from tchaikov/wip-utime-cleanup
include/utime: do not cast sec to time_t

Reviewed-by: Samuel Just <sjust@redhat.com>
2019-04-30 00:17:08 +08:00
Kefu Chai
b1ed98a617 ceph-object-corpus: pick up 15.0.0-539-g191ab33faf
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-30 00:13:41 +08:00
Lenz Grimmer
f92638fe3d
Merge pull request #27784 from Devp00l/wip-39483
mgr/dashboard: Make all columns sortable
2019-04-29 18:05:26 +02:00
Sage Weil
f3f2490620 Merge PR #27590 into master
* refs/pull/27590/head:
	messages: #include necessary header

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-04-29 11:04:31 -05:00
Sage Weil
5b1bbdfec1 Merge PR #27804 into master
* refs/pull/27804/head:
	common, osd: remove erroneous 'section' in dump functions
	dencoder: include some missed types
	common: make utime_t dencoder-compatible
	common: make uuid_d dencoder-compatible
	osd: make watch_item_t dencoder-compatible

Reviewed-by: Sage Weil <sage@redhat.com>
2019-04-29 11:04:15 -05:00
Sage Weil
0191a27579 Merge PR #27832 into master
* refs/pull/27832/head:
	qa/suites/upgrade/mimic-x-singleton: suppress legacy statfs warning
	osd: reenable authorizer on hb messenger for nautilus+
	mon/MonClient: do not crash on empty auth request buffer
	msg: set_require_authorizer on messenger, not dispatcher
	mon/Monitor: allow probe if MMonProbe::mon_release == 0
	mon: fix min_mon_release print

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-04-29 11:03:53 -05:00
Kefu Chai
519f417ef5
Merge pull request #27788 from cyx1231st/wip-seastar-msgr-batch-send
crimson/net: improve batching in the write path

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-04-29 23:38:16 +08:00
Kefu Chai
41747f7d9e
Merge pull request #27776 from JunHe77/install-deps
install-deps.sh: add option to skip prebuilt boost-* pkgs installation

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-04-29 22:34:10 +08:00
Stephan Müller
ba6d0f96db mgr/dashboard: Make all columns sortable
Fixes: https://tracker.ceph.com/issues/39483
Signed-off-by: Stephan Müller <smueller@suse.com>
2019-04-29 15:48:37 +02:00
Kefu Chai
50226ef63c include/utime: do not cast sec to time_t
strictly speaking, time_t is an opaque time, and in this context, we
need a uint32_t, so it is not necessary and wrong. let's remove it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 21:11:52 +08:00
Lenz Grimmer
34b4cce61e
mgr/dashboard: auth: fix user lastUpdate timestamp generation (#27604)
mgr/dashboard: auth: fix user lastUpdate timestamp generation

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2019-04-29 14:23:17 +02:00
Lenz Grimmer
106388be8f
monitoring: Prometheus default alerts (#27596)
monitoring: Prometheus default alerts

Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2019-04-29 13:36:05 +02:00
Kefu Chai
1e8183e01b
Merge pull request #27837 from cyx1231st/wip-seastar-msgr-fix-read
crimson/net: misc fixes in v1 read path

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-04-29 18:51:09 +08:00
Yingxin Cheng
5ca97cd358 crimson/net: misc fixes in v1 read path
* abort when decoding/signature-check failed;
* change log-level to warn;
* free the decoded message when failed;
* add unlikely to read path;

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-29 23:13:23 +08:00
Yingxin Cheng
c9aae7a99f crimson/net: increase out buffer to 65536
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-29 15:58:03 +08:00
Yingxin Cheng
4df5bb5860 crimson/net: group write-related general implementations
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-29 15:58:03 +08:00
Yingxin Cheng
4091f76a68 crimson/net: use optional keepalive_ack to notify writes
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-29 15:58:03 +08:00
Yingxin Cheng
b5e9c89065 crimson/net: gather message buffers when send
Gather buffers from pending messages/keepalive and send them together.

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-29 15:58:03 +08:00
Kefu Chai
f2bafe4b03
Merge pull request #27809 from tchaikov/wip-crimson-auth
crimson,auth: fix FTBFS of crimson-osd and fix v1/v2 auth

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-04-29 09:10:16 +08:00
Kefu Chai
378b057e59
Merge pull request #27831 from tchaikov/wip-crimson-silence-warning
crimson/net: prefer <fmt/chrono.h> over <fmt/time.h>

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2019-04-29 09:09:15 +08:00
Sage Weil
fdf6f8b26b qa/suites/upgrade/mimic-x-singleton: suppress legacy statfs warning
Signed-off-by: Sage Weil <sage@redhat.com>
2019-04-28 19:21:52 -05:00
Mykola Golub
24ef73f4cf
Merge pull request #27756 from dillaman/wip-39025
librbd: initial consolidation of internal locks

Reviewed-by: Mykola Golub <mgolub@suse.com>
2019-04-28 20:03:02 +03:00
Kefu Chai
535aef6082 crimson/osd: silence -Wunused-variable warning
silence warning like

/home/kchai/ceph/src/crimson/osd/heartbeat.cc:314:24: warning: unused
variable ‘peer’ [-Wunused-variable]
       auto& [peer, info] = item;
                        ^

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
2e28ea3f30 crimson/mon: log when a monitor is fully connected
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
870937b979 crimson/mon: hold rotating_keyring using a pointer
we could have kept rotating_keyring as a plain member variable, but
`AuthClientHandler` keeps a weak reference to it, we are not able to
update it to point it to the new rotating_keyring in the newly created
active_con created from the connected pending connection -- the move
ctor of ceph::mon::Connection does create a new RotatingKeyring. so this
has two consequences:

- the raw pointer held by AuthClientHandler is not valid anymore, after
  the pending connection is destroyed when it is promoted to active_con.
  so we are writing to freed memory when renewing the rotating keyring.
- we won't have access to the updated keyring.

if we use a std::vector<unique_ptr<Connection>> for pending_conns, this
change won't be needed. we could make this change in a future change
though.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
7dfca035ac auth,mon,crimson: pass KeyStore by const reference
AuthAuthorizeHandler::verify_authorizer() neither changes the keystore,
nor expects a nullptr. so we should pass the keystore by const reference
for better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
0a8dbddf88 test/crimson: do not link against ceph-common
crimson-common and ceph-common have subtle difference which is
conditionalized by WITH_SEASTAR macro. and the different implementations
share the same name, so we should not link crimson executables against ceph-common .

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
adc65bcbe4 auth: do not use cct unless i have to
crimson messenger's tests does not use ConfigProxy at the time of writing,
and ldout() cannot tell if a log message should be written to the
logfile or not without querying config subsystem, so, to avoid involving
config system in crimson messenger's tests, the log line is disabled if
WITH_SEASTAR, we will kill this workaround once we ditch crimson/net/Config.h.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
66b1f5537d crimson/mon: renew rotating secret if necessary
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
1070a1f788 auth/Auth: pass utime_t by const reference
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
8dbb9a1c91 auth/cephx: do not require a cct for accessing config
crimson/msgr does not depend on ConfigProxy at the time of writing. we
want to decouple it from config subsystem in hope to have less cross
subsystem dependencies and speed up the compiling speed.

but, to introduce cephx to msgr, we need to enable msgr to read from
ConfigProxy at runtime. so, to avoid keeping a CephContext instance
simply for accessing the ConfigProxy reference in it, it would be more
efficient to define a free function `conf()` which hides the difference
between crimson and classic user of auth/cephx.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
2283f9580a crimson/net: learn my address from peer
OSD always relies on its server to tell its address, without this
feature, it will advertise a blank ip address to monitor in MOSDBoot.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
e9dc950a16 crimson/mon: pass cct to verify_authorizer()
CephxAuthorizeHandler::verify_authorizer() uses it for querying config
and for a RNG, so we have to pass it a CephContext which has `_conf` and
`random()`.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
4cef78aa91 crimson/osd: pass Connection* to Dispatch::ms_dispatch()
it's a regression introduced by c69bf0c002

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
04786235c2 crimson/net: sign message and check it if authorizer is available
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
1d30bb4e79 crimson/mon: set entity_name before trying to connect to mon
otherwise Monitor.cc or AuthMonitor.cc won't be able to find it in the
keyring.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
ad5fe6a70d crimson/mgr: connect mgr with v2 protocol
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
5f61282e8e crimson: remove ms_get_authorizer() and friends
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
2eb4ee45d5 test/crimson: disable authorizer for tests
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
1a3a14628c crimson/osd: sync msgr's require_authorizer with osdmap
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
0a1095968a crimson/mon: tolerate lack of authorizer for some messengers
also, we should not crash on empty auth request buffer.

see also 64eddc4d and b99daab1

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Yingxin Cheng
88588cf70a crimson/net: migrate protocol v1 to auth-client/server
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Sage Weil
ab565beba6 osd: reenable authorizer on hb messenger for nautilus+
Signed-off-by: Sage Weil <sage@redhat.com>
2019-04-28 10:37:45 -05:00
Kefu Chai
176db063be crimson: add Messenger::{set,get}_require_authorizer()
This flag is used for compatibility with pre-nautilus OSDs, which do not
send authorizers on the OSD heartbeat connections.  However, because the
AuthServer is implemented by MonClient, which is shared across all
OSD messengers, we can't set this to false for the OSD without disabling
all auth.  Instead, make it a Messenger property, and set it only on the
heartbeat server messengers.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-28 23:23:49 +08:00
Sage Weil
b99daab134 mon/MonClient: do not crash on empty auth request buffer
Signed-off-by: Sage Weil <sage@redhat.com>
2019-04-28 08:45:03 -05:00
Sage Weil
f10660e84f msg: set_require_authorizer on messenger, not dispatcher
This flag is used for compatibility with pre-nautilus OSDs, which do not
send authorizers on the OSD heartbeat connections.  However, because the
AuthServer is implemented by MonClient, which is shared across all
OSD messengers, we can't set this to false for the OSD without disabling
all auth.  Instead, make it a Messenger property, and set it only on the
heartbeat server messengers.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-04-28 08:43:28 -05:00
Jason Dillaman
cb40cc2594 librbd: removed 'ImageCtx::parent_lock'
The ImageCtx parent-related data structures are now protected
by the image_lock.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-04-28 09:15:25 -04:00