Commit Graph

97840 Commits

Author SHA1 Message Date
Samuel Just
10842f7c38 PeeringState: route recoverystate_perf calls through PeeringListener
Signed-off-by: Samuel Just <rexludorum@gmail.com>
2019-05-01 11:22:13 -07:00
Samuel Just
fbf06885fb osd/: move flush state into PeeringState
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:13 -07:00
Samuel Just
482eccd13e osd/: move check_recovery_sources and missing_loc into PeeringState
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:13 -07:00
Samuel Just
a22acd6032 osd/: extract MissingLoc into its own file
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:13 -07:00
Samuel Just
e42ca4204b osd/: move update_history and purge_strays into PeeringState
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:12 -07:00
Samuel Just
d995596433 osd/: move proc_replica_info and some callees into PeeringState
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:12 -07:00
Samuel Just
3b158e6308 osd/: move the is_* and acting/up/backfill helpers into PeeringState
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:12 -07:00
sjust@redhat.com
8f3b4336a4 osd/: Move osdmap handling into PeeringState
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:12 -07:00
sjust@redhat.com
e3fa38b984 osd/: move most peering state to PeeringState
This patch moves the 40 something peering state variables over into
PeeringState while leaving references to them in PG.  The following
patches will move over the users until all users are in PeeringState.
Then, the PG references will be removed.  A subsequent patch will also
move the recovery_state member to be the last initialize and first
destructed.

Signed-off-by: sjust@redhat.com <sjust@redhat.com>
2019-05-01 11:22:12 -07:00
Samuel Just
365fe8638b PeeringState: begin abstracting out PG, add dpp for logging
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:12 -07:00
Samuel Just
d96c9072b7 PGStateUtils: remove PG*, move into PGStateHistory into PeeringState
I don't think there's any need to worry about the pg locking from
PGStateHistory.  NamedState::begin/exit and dump_pgstate_history are the
only users, so the only calls should be under the peering event handler,
the asok command, or the PG constructor and destructor.  The first two
already have the lock.  The last should be safe as well as long as
the state machine states are constructed and destructed after and
before the PGStateHistory instance. As such, this patch removes most
of that state leaving the epoch generation as an interface
implemented by PG.

The snap trimming state machine was already excluded, so that this
patch leaves it disabled.

Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:12 -07:00
Samuel Just
1e4166a049 osd/: pull NamedState and PGStateHistory out into PGStateUtils
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:11 -07:00
Samuel Just
442bfd80a1 PeeringState: fix pg->info->pgid.pgid references
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:11 -07:00
Samuel Just
5f202e82c2 PeeringState: adjust logging to avoid using pg
Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:11 -07:00
Samuel Just
12b60ed53a osd/: mechanically rip PeeringState and friends out into their own file
The next step is to rationalize the PeeringState<->PG relationship.

Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:11 -07:00
Samuel Just
5e36acdb6f osd/: mechanically rename RecoveryMachine/State/Ctx to Peering*
I'm going to extract this logic and reuse it in crimson.  Recovery* has
always been a confusing name as it implements neither log-based recovery
nor backfill.  Rather, it's mainly the buisiness logic for agreeing on
an authoritative log and some ancillary things such as scrub/backfill
reservation.

$ for i in $(git grep -l 'RecoveryMachine'); do sed -i 's/RecoveryMachine/PeeringMachine/g' $i; done
$ for i in $(git grep -l 'RecoveryState'); do sed -i 's/RecoveryState/PeeringState/g' $i; done
$ for i in $(git grep -l 'RecoveryCtx'); do sed -i 's/RecoveryCtx/PeeringCtx/g' $i; done

Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:11 -07:00
Gregory Farnum
8cd1128157
Merge pull request #27867 from tchaikov/wip-ceph-object-corpus
ceph-object-corpus: pick up 15.0.0-539-g191ab33faf

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2019-04-29 13:45:16 -07:00
Casey Bodley
f0a9719d2c
Merge pull request #27730 from cbodley/wip-rgw-boost-170
rgw: fix compile errors with boost 1.70

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2019-04-29 13:06:47 -04:00
Kefu Chai
c99f7e1682
Merge pull request #27861 from tchaikov/wip-utime-cleanup
include/utime: do not cast sec to time_t

Reviewed-by: Samuel Just <sjust@redhat.com>
2019-04-30 00:17:08 +08:00
Kefu Chai
b1ed98a617 ceph-object-corpus: pick up 15.0.0-539-g191ab33faf
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-30 00:13:41 +08:00
Lenz Grimmer
f92638fe3d
Merge pull request #27784 from Devp00l/wip-39483
mgr/dashboard: Make all columns sortable
2019-04-29 18:05:26 +02:00
Sage Weil
f3f2490620 Merge PR #27590 into master
* refs/pull/27590/head:
	messages: #include necessary header

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-04-29 11:04:31 -05:00
Sage Weil
5b1bbdfec1 Merge PR #27804 into master
* refs/pull/27804/head:
	common, osd: remove erroneous 'section' in dump functions
	dencoder: include some missed types
	common: make utime_t dencoder-compatible
	common: make uuid_d dencoder-compatible
	osd: make watch_item_t dencoder-compatible

Reviewed-by: Sage Weil <sage@redhat.com>
2019-04-29 11:04:15 -05:00
Sage Weil
0191a27579 Merge PR #27832 into master
* refs/pull/27832/head:
	qa/suites/upgrade/mimic-x-singleton: suppress legacy statfs warning
	osd: reenable authorizer on hb messenger for nautilus+
	mon/MonClient: do not crash on empty auth request buffer
	msg: set_require_authorizer on messenger, not dispatcher
	mon/Monitor: allow probe if MMonProbe::mon_release == 0
	mon: fix min_mon_release print

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-04-29 11:03:53 -05:00
Kefu Chai
519f417ef5
Merge pull request #27788 from cyx1231st/wip-seastar-msgr-batch-send
crimson/net: improve batching in the write path

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-04-29 23:38:16 +08:00
Kefu Chai
41747f7d9e
Merge pull request #27776 from JunHe77/install-deps
install-deps.sh: add option to skip prebuilt boost-* pkgs installation

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-04-29 22:34:10 +08:00
Stephan Müller
ba6d0f96db mgr/dashboard: Make all columns sortable
Fixes: https://tracker.ceph.com/issues/39483
Signed-off-by: Stephan Müller <smueller@suse.com>
2019-04-29 15:48:37 +02:00
Kefu Chai
50226ef63c include/utime: do not cast sec to time_t
strictly speaking, time_t is an opaque time, and in this context, we
need a uint32_t, so it is not necessary and wrong. let's remove it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 21:11:52 +08:00
Lenz Grimmer
34b4cce61e
mgr/dashboard: auth: fix user lastUpdate timestamp generation (#27604)
mgr/dashboard: auth: fix user lastUpdate timestamp generation

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2019-04-29 14:23:17 +02:00
Lenz Grimmer
106388be8f
monitoring: Prometheus default alerts (#27596)
monitoring: Prometheus default alerts

Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2019-04-29 13:36:05 +02:00
Kefu Chai
1e8183e01b
Merge pull request #27837 from cyx1231st/wip-seastar-msgr-fix-read
crimson/net: misc fixes in v1 read path

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-04-29 18:51:09 +08:00
Yingxin Cheng
5ca97cd358 crimson/net: misc fixes in v1 read path
* abort when decoding/signature-check failed;
* change log-level to warn;
* free the decoded message when failed;
* add unlikely to read path;

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-29 23:13:23 +08:00
Yingxin Cheng
c9aae7a99f crimson/net: increase out buffer to 65536
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-29 15:58:03 +08:00
Yingxin Cheng
4df5bb5860 crimson/net: group write-related general implementations
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-29 15:58:03 +08:00
Yingxin Cheng
4091f76a68 crimson/net: use optional keepalive_ack to notify writes
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-29 15:58:03 +08:00
Yingxin Cheng
b5e9c89065 crimson/net: gather message buffers when send
Gather buffers from pending messages/keepalive and send them together.

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-29 15:58:03 +08:00
Kefu Chai
f2bafe4b03
Merge pull request #27809 from tchaikov/wip-crimson-auth
crimson,auth: fix FTBFS of crimson-osd and fix v1/v2 auth

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-04-29 09:10:16 +08:00
Kefu Chai
378b057e59
Merge pull request #27831 from tchaikov/wip-crimson-silence-warning
crimson/net: prefer <fmt/chrono.h> over <fmt/time.h>

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2019-04-29 09:09:15 +08:00
Sage Weil
fdf6f8b26b qa/suites/upgrade/mimic-x-singleton: suppress legacy statfs warning
Signed-off-by: Sage Weil <sage@redhat.com>
2019-04-28 19:21:52 -05:00
Mykola Golub
24ef73f4cf
Merge pull request #27756 from dillaman/wip-39025
librbd: initial consolidation of internal locks

Reviewed-by: Mykola Golub <mgolub@suse.com>
2019-04-28 20:03:02 +03:00
Kefu Chai
535aef6082 crimson/osd: silence -Wunused-variable warning
silence warning like

/home/kchai/ceph/src/crimson/osd/heartbeat.cc:314:24: warning: unused
variable ‘peer’ [-Wunused-variable]
       auto& [peer, info] = item;
                        ^

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
2e28ea3f30 crimson/mon: log when a monitor is fully connected
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
870937b979 crimson/mon: hold rotating_keyring using a pointer
we could have kept rotating_keyring as a plain member variable, but
`AuthClientHandler` keeps a weak reference to it, we are not able to
update it to point it to the new rotating_keyring in the newly created
active_con created from the connected pending connection -- the move
ctor of ceph::mon::Connection does create a new RotatingKeyring. so this
has two consequences:

- the raw pointer held by AuthClientHandler is not valid anymore, after
  the pending connection is destroyed when it is promoted to active_con.
  so we are writing to freed memory when renewing the rotating keyring.
- we won't have access to the updated keyring.

if we use a std::vector<unique_ptr<Connection>> for pending_conns, this
change won't be needed. we could make this change in a future change
though.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
7dfca035ac auth,mon,crimson: pass KeyStore by const reference
AuthAuthorizeHandler::verify_authorizer() neither changes the keystore,
nor expects a nullptr. so we should pass the keystore by const reference
for better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
0a8dbddf88 test/crimson: do not link against ceph-common
crimson-common and ceph-common have subtle difference which is
conditionalized by WITH_SEASTAR macro. and the different implementations
share the same name, so we should not link crimson executables against ceph-common .

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
adc65bcbe4 auth: do not use cct unless i have to
crimson messenger's tests does not use ConfigProxy at the time of writing,
and ldout() cannot tell if a log message should be written to the
logfile or not without querying config subsystem, so, to avoid involving
config system in crimson messenger's tests, the log line is disabled if
WITH_SEASTAR, we will kill this workaround once we ditch crimson/net/Config.h.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
66b1f5537d crimson/mon: renew rotating secret if necessary
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
1070a1f788 auth/Auth: pass utime_t by const reference
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
8dbb9a1c91 auth/cephx: do not require a cct for accessing config
crimson/msgr does not depend on ConfigProxy at the time of writing. we
want to decouple it from config subsystem in hope to have less cross
subsystem dependencies and speed up the compiling speed.

but, to introduce cephx to msgr, we need to enable msgr to read from
ConfigProxy at runtime. so, to avoid keeping a CephContext instance
simply for accessing the ConfigProxy reference in it, it would be more
efficient to define a free function `conf()` which hides the difference
between crimson and classic user of auth/cephx.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
2283f9580a crimson/net: learn my address from peer
OSD always relies on its server to tell its address, without this
feature, it will advertise a blank ip address to monitor in MOSDBoot.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00