Commit Graph

97796 Commits

Author SHA1 Message Date
Stephan Müller
a23ec08795 mgr/dashboard: RBD snapshot name suggestion with local time suffix
The new suffix is still a time that follows the ISO standard as it can
be converted into any other time zone through the time zone prefix of
itself.

Fixes: https://tracker.ceph.com/issues/23858
Signed-off-by: Stephan Müller <smueller@suse.com>
2019-04-29 16:49:28 +02:00
Kefu Chai
41747f7d9e
Merge pull request #27776 from JunHe77/install-deps
install-deps.sh: add option to skip prebuilt boost-* pkgs installation

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-04-29 22:34:10 +08:00
Lenz Grimmer
34b4cce61e
mgr/dashboard: auth: fix user lastUpdate timestamp generation (#27604)
mgr/dashboard: auth: fix user lastUpdate timestamp generation

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2019-04-29 14:23:17 +02:00
Lenz Grimmer
106388be8f
monitoring: Prometheus default alerts (#27596)
monitoring: Prometheus default alerts

Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2019-04-29 13:36:05 +02:00
Kefu Chai
1e8183e01b
Merge pull request #27837 from cyx1231st/wip-seastar-msgr-fix-read
crimson/net: misc fixes in v1 read path

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-04-29 18:51:09 +08:00
Yingxin Cheng
5ca97cd358 crimson/net: misc fixes in v1 read path
* abort when decoding/signature-check failed;
* change log-level to warn;
* free the decoded message when failed;
* add unlikely to read path;

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-29 23:13:23 +08:00
Kefu Chai
f2bafe4b03
Merge pull request #27809 from tchaikov/wip-crimson-auth
crimson,auth: fix FTBFS of crimson-osd and fix v1/v2 auth

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-04-29 09:10:16 +08:00
Kefu Chai
378b057e59
Merge pull request #27831 from tchaikov/wip-crimson-silence-warning
crimson/net: prefer <fmt/chrono.h> over <fmt/time.h>

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2019-04-29 09:09:15 +08:00
Mykola Golub
24ef73f4cf
Merge pull request #27756 from dillaman/wip-39025
librbd: initial consolidation of internal locks

Reviewed-by: Mykola Golub <mgolub@suse.com>
2019-04-28 20:03:02 +03:00
Kefu Chai
535aef6082 crimson/osd: silence -Wunused-variable warning
silence warning like

/home/kchai/ceph/src/crimson/osd/heartbeat.cc:314:24: warning: unused
variable ‘peer’ [-Wunused-variable]
       auto& [peer, info] = item;
                        ^

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
2e28ea3f30 crimson/mon: log when a monitor is fully connected
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
870937b979 crimson/mon: hold rotating_keyring using a pointer
we could have kept rotating_keyring as a plain member variable, but
`AuthClientHandler` keeps a weak reference to it, we are not able to
update it to point it to the new rotating_keyring in the newly created
active_con created from the connected pending connection -- the move
ctor of ceph::mon::Connection does create a new RotatingKeyring. so this
has two consequences:

- the raw pointer held by AuthClientHandler is not valid anymore, after
  the pending connection is destroyed when it is promoted to active_con.
  so we are writing to freed memory when renewing the rotating keyring.
- we won't have access to the updated keyring.

if we use a std::vector<unique_ptr<Connection>> for pending_conns, this
change won't be needed. we could make this change in a future change
though.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
7dfca035ac auth,mon,crimson: pass KeyStore by const reference
AuthAuthorizeHandler::verify_authorizer() neither changes the keystore,
nor expects a nullptr. so we should pass the keystore by const reference
for better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
0a8dbddf88 test/crimson: do not link against ceph-common
crimson-common and ceph-common have subtle difference which is
conditionalized by WITH_SEASTAR macro. and the different implementations
share the same name, so we should not link crimson executables against ceph-common .

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
adc65bcbe4 auth: do not use cct unless i have to
crimson messenger's tests does not use ConfigProxy at the time of writing,
and ldout() cannot tell if a log message should be written to the
logfile or not without querying config subsystem, so, to avoid involving
config system in crimson messenger's tests, the log line is disabled if
WITH_SEASTAR, we will kill this workaround once we ditch crimson/net/Config.h.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
66b1f5537d crimson/mon: renew rotating secret if necessary
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
1070a1f788 auth/Auth: pass utime_t by const reference
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
8dbb9a1c91 auth/cephx: do not require a cct for accessing config
crimson/msgr does not depend on ConfigProxy at the time of writing. we
want to decouple it from config subsystem in hope to have less cross
subsystem dependencies and speed up the compiling speed.

but, to introduce cephx to msgr, we need to enable msgr to read from
ConfigProxy at runtime. so, to avoid keeping a CephContext instance
simply for accessing the ConfigProxy reference in it, it would be more
efficient to define a free function `conf()` which hides the difference
between crimson and classic user of auth/cephx.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
2283f9580a crimson/net: learn my address from peer
OSD always relies on its server to tell its address, without this
feature, it will advertise a blank ip address to monitor in MOSDBoot.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
e9dc950a16 crimson/mon: pass cct to verify_authorizer()
CephxAuthorizeHandler::verify_authorizer() uses it for querying config
and for a RNG, so we have to pass it a CephContext which has `_conf` and
`random()`.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
4cef78aa91 crimson/osd: pass Connection* to Dispatch::ms_dispatch()
it's a regression introduced by c69bf0c002

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
04786235c2 crimson/net: sign message and check it if authorizer is available
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
1d30bb4e79 crimson/mon: set entity_name before trying to connect to mon
otherwise Monitor.cc or AuthMonitor.cc won't be able to find it in the
keyring.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
ad5fe6a70d crimson/mgr: connect mgr with v2 protocol
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
5f61282e8e crimson: remove ms_get_authorizer() and friends
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
2eb4ee45d5 test/crimson: disable authorizer for tests
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
1a3a14628c crimson/osd: sync msgr's require_authorizer with osdmap
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
0a1095968a crimson/mon: tolerate lack of authorizer for some messengers
also, we should not crash on empty auth request buffer.

see also 64eddc4d and b99daab1

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Yingxin Cheng
88588cf70a crimson/net: migrate protocol v1 to auth-client/server
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-29 00:12:57 +08:00
Kefu Chai
176db063be crimson: add Messenger::{set,get}_require_authorizer()
This flag is used for compatibility with pre-nautilus OSDs, which do not
send authorizers on the OSD heartbeat connections.  However, because the
AuthServer is implemented by MonClient, which is shared across all
OSD messengers, we can't set this to false for the OSD without disabling
all auth.  Instead, make it a Messenger property, and set it only on the
heartbeat server messengers.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-28 23:23:49 +08:00
Jason Dillaman
cb40cc2594 librbd: removed 'ImageCtx::parent_lock'
The ImageCtx parent-related data structures are now protected
by the image_lock.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-04-28 09:15:25 -04:00
Jason Dillaman
e1fe6d5a6a librbd: removed 'ImageCtx::object_map_lock'
The 'image_lock' now protects the 'object_map' pointer and
the contents of the object-map itself are protected by a
private lock.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-04-28 09:15:20 -04:00
Jason Dillaman
a8882932e7 librbd: protect object map with a private member lock
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-04-28 09:15:19 -04:00
Jason Dillaman
d1eb013fc9 librbd: removed 'ImageCtx::md_lock'
This lock used to protect the IO pathway to prevent writes but
that is now handled by the io::ImageRequestWQ. Additional
historical uses have been temporarily moved to the
'ImageCtx::image_lock'

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-04-28 09:15:19 -04:00
Jason Dillaman
236ffa3181 librbd: renamed 'ImageCtx::snap_lock' to 'image_lock'
This is the first step in consolidating multiple locks under a single
reader/writer lock for simplicity.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-04-28 09:15:14 -04:00
Kefu Chai
2ff7c1328b
Merge pull request #27766 from majianpeng/msg-noneed-set-connection
msg/async: no-need set connection for Message.

Reviewed-by: Ricardo Dias <rdias@suse.com>
2019-04-28 19:20:49 +08:00
Kefu Chai
994de982c2
Merge pull request #27767 from tchaikov/wip-buf-denc-const
buffer, denc: more constness

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-04-28 19:18:38 +08:00
Yuval Lifshitz
0e8ac0618c
Merge pull request #27671 from yuvalif/fix_arn_issue
rgw/pubsub: fix topic arn. tenant support to multisite tests
2019-04-28 11:43:02 +03:00
Kefu Chai
62c6cb4451 crimson/net: prefer <fmt/chrono.h> over <fmt/time.h>
in latest libfmt, <fmt/time.h> is deprecated.

to silence warnings like

/home/kchai/ceph/src/fmt/include/fmt/time.h:13:2: warning: #warning
fmt/time.h is deprecated, use fmt/chrono.h instead [-Wcpp]
 #warning fmt/time.h is deprecated, use fmt/chrono.h instead
  ^~~~~~~
In file included from
/home/kchai/ceph/src/seastar/include/seastar/core/reactor.hh:72:0,
                 from
/home/kchai/ceph/src/seastar/include/seastar/core/sharded.hh:24,
                 from /home/kchai/ceph/src/crimson/net/Fwd.h:18,
                 from /home/kchai/ceph/src/crimson/net/Protocol.h:9,
                 from /home/kchai/ceph/src/crimson/net/ProtocolV2.h:6,
                 from /home/kchai/ceph/src/crimson/net/ProtocolV2.cc:4:

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-28 16:02:25 +08:00
Mykola Golub
ab1c8043b0
Merge pull request #27757 from dillaman/wip-39021-2
librbd: copyup read stats were incorrectly tied to child

Reviewed-by: Mykola Golub <mgolub@suse.com>
2019-04-27 18:58:53 +03:00
Sage Weil
cf45294686 Merge PR #18694 into master
* refs/pull/18694/head:
	Merge branch 'master' into sk-add_current_num_in_quota_info
	mon/OSDMonitor.cc: Add current numbers of objects and bytes

Reviewed-by: Sage Weil <sage@redhat.com>
2019-04-27 10:41:18 -05:00
Sage Weil
f5314cfb33 Merge PR #22411 into master
* refs/pull/22411/head:
	common/WorkQueue: narrow ThreadPool::_lock in func worker.

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2019-04-27 10:41:01 -05:00
Sage Weil
320acd45a0 Merge PR #27821 into master
* refs/pull/27821/head:
	common/options: allow osd_pool_default_pg_autoscale_mode to update a runtime

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-04-27 10:40:44 -05:00
Sage Weil
f4c4e112b0 Merge PR #27823 into master
* refs/pull/27823/head:
	msg/async/ProtocolV1: generate authorizer for mon to mon connections
	mon/Monitor: require authorizer from peer monitors

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-04-27 10:40:29 -05:00
Neha Ojha
f4684eb864
Merge pull request #27798 from neha-ojha/wip-39441
osd/PG: do not use approx_missing_objects pre-nautilus

Reviewed By: Xie Xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-04-26 16:00:53 -07:00
David Zafman
a8b8ce5c75
Merge pull request #27209 from dzafman/wip-38840
osd: Don't evict after a flush if intersecting scrub range

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-04-26 15:23:37 -07:00
David Zafman
66b041fa4a
Merge pull request #27769 from dzafman/wip-39333
osd-backfill-space.sh test failed in TEST_backfill_multi_partial()

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-04-26 11:55:04 -07:00
Sage Weil
d1b8093174 Merge PR #27732 into master
* refs/pull/27732/head:
	common/assert: include ceph_abort_msg(arg) arg in log output

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2019-04-26 13:24:01 -05:00
Sage Weil
d2520707e0 Merge PR #27729 into master
* refs/pull/27729/head:
	osd: take heartbeat_lock when calling heartbeat()

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-04-26 13:23:36 -05:00
Sage Weil
fe3ead1802 Merge PR #26176 into master
* refs/pull/26176/head:
	os/bluestore: do not check osd_max_object_size in _open_path()

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Sage Weil <sage@redhat.com>
2019-04-26 13:23:21 -05:00