Commit Graph

94289 Commits

Author SHA1 Message Date
Shyukri Shyukriev
28f0749eae qa:rbd/workunits : Replace rbd bench-write with rbd bench --io-type write
Signed-off-by: Shyukri Shyukriev <shshyukriev@suse.com>
2019-01-28 14:24:54 +02:00
Sage Weil
40259dd3eb Merge PR #24546 into master
* refs/pull/24546/head:
	msg/async/ProtocolV2: clear dispatch throttle on connection stop
	msg/async: fix should_use_msgr2 behavior (including monc)
	msg/async/AsyncMessenger: clear need_addr *after* we set our new addr
	msg/async/ProtocolV2: fix handling for v2 client connection with v1 addr
	ceph_test_msgr: do not connect_to on the client side
	msg/async: do not connect from server
	msg/async: do not use peer to addr detection; use getsockname()
	msg/async/ProtocolV2: always send non-empty addrvec for self
	msg/async: never fill out port in myaddr if we didn't bind
	ceph_test_msgr: use v2 addrs for simplemessenger
	msg/async: msgr2: don't force write event on every message received
	msg/async/ProtocolV2: be forgiving in server identity check
	msg/async/ProtocolV2: fault if we connect to the wrong peer
	msg/async: msgr2: clean cookie if connection failed in ACCEPT_SESSION
	msg/async/ProtocolV2: do not bump connect_seq for fault during ACCEPTING_SESSION
	msg/async: msgr2: don't send SESSION_RETRY_GLOBAL in handle_existing_connection
	msg/async: msgr2: organizing log messages
	msg/async: msgr2: fix connection fault when replacing
	msg/async: msgr2: fix replacing race handling
	msg/async: msgr2: fix connection race when existing connection is newer
	msg/async: msgr2: assign recv_stamp in handle_message
	msg/async: msgr2: fix peer_addrs discovery
	msg/async: msgr2: keep authorizer bufferlist across reconnects
	msg/async: msgr2: fix connection secret problems for WITH_SEASTAR builds
	msg/async: msgr2: send keepalive on connection race winner
	msg/async: msgr2: fix client address learning
	msg/async: msgr2: fix keepalive_ack message
	msg/async: msgr2: do not force updating rotating keys inline
	msg/async: msgr2: fix mark_down vs accept race
	msg/async: msgr2: unregister con from accept vs mark_down race
	auth/cephx/CephxSessionHandler: use connection_secret for encryption
	msg,cephx: establish a unique connection_secret for every connection
	msg/async: msgr2: use sha256_digest_t to print signature hex strings
	types.h,rgw: merge sha*_digest_t definitions
	msg/async: msgr2: close connection when no authorizer is given
	msg/async: msgr2: formatting fixes
	msg/async: msgr2: send client v2 address when only v1 address is defined
	msg/async: msgr2: add payload length to banner
	msg/async: msgr2: check protocol state after fast dispatch
	msg/async: msgr2: reduce log level for sending messages event
	msg/async: msgr2: call verify authorizer when CEPH_AUTH_NONE is used
	msg/async: msgr2: store peer entity name in the protocol
	msg/async: msgr2: apply sign/encrypt to messages data payload
	msg/async: msgr2: encryption/decryption of frames
	cephx: added encrypt/decrypt bufferlist method to session handler
	msg/async: msgr2: refactored the frame structures
	cephx: add sign bufferlist method
	options: msgr2 enable/disable signing and encrytion options
	msg/async: msgr2: cephx authentication
	msg/async: msgr2: implement reconnect
	msg/async: msgr2: fault handling
	msg/async: msgr2: messange exchange phase
	msg/async: msgr2: message flow handshake
	msg/async: msgr2: authentication phase
	msg/async: msgr2: exchange peer_type in banner phase
	test/msgr: cloned test_msgr test for testing msgr2 protocol
	msg/async: msgr2: banner exchange
	msg/async: asyncconnection: update the source address info
	msg/async: move base class Protocol its own source file

Reviewed-by: Sage Weil <sage@redhat.com>
2019-01-28 05:50:59 -06:00
Lenz Grimmer
ddd5a3b9e6
Merge pull request #25583 from rhcs-dashboard/timepicker
mgr/dashboard: Adds custom timepicker for grafana iframes

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
2019-01-28 12:27:48 +01:00
Kefu Chai
8b175ee4cc
Merge pull request #25745 from socketpair/precision
Formatters: improve precision of double numbers

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-01-28 17:29:50 +08:00
Kefu Chai
8fe1edbdb7
Merge pull request #25725 from cfanz/wip-mgr-handle-monmap
mgr: update daemon_state when necessary

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-01-28 16:44:53 +08:00
Kefu Chai
6c72f8f0a3
Merge pull request #26155 from tchaikov/wip-osd-cleanup
osd/PG: remove unused functions

Reviewed-by: Sage Weil <sage@redhat.com>
2019-01-28 16:43:05 +08:00
Kefu Chai
1c54d28bed
Merge pull request #26156 from tchaikov/wip-use-latest-setuptools
install-deps.sh: use the latest setuptools

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-01-28 16:42:30 +08:00
Kefu Chai
f52c22ebf5
Merge pull request #26138 from Songweibin/wip-rbd-py-cleanup
pybind/rbd: fix a typo in metadata_get comments

Reviewed-by: Mykola Golub <mgolub@suse.com>
2019-01-28 14:17:42 +08:00
Neha Ojha
9c1cd8c8f7
Merge pull request #26105 from neha-ojha/wip-37886
mgr/DaemonServer: log pgmap usage to cluster log

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2019-01-27 12:43:43 -08:00
Patrick Donnelly
ec0b649d3d
Merge PR #26086 into master
* refs/pull/26086/head:
	include/xlist: prevent item moves
	include/xlist: privatize item members

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2019-01-27 10:23:08 -08:00
Sage Weil
d1f4d30d53 msg/async/ProtocolV2: clear dispatch throttle on connection stop
This aligns us with the ProtocolV1 implementation.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-27 10:11:03 -06:00
Kefu Chai
d42daf378c
Merge pull request #25616 from s0nea/wip-dashboard-osd-recv-speed-default-vals
mgr/dashboard: consider config option default values

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
2019-01-27 22:17:49 +08:00
Kefu Chai
11c9331263
Merge pull request #24813 from wjwithagen/wjw-fix-alpine-compat
common: Fix Alpine compatability for TEMP_FAILURE_RETRY and ACCESSPERMS

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-01-27 21:42:51 +08:00
Kefu Chai
77109d7582
Merge pull request #25916 from tchaikov/wip-debian-librados-dev
debian: librados-dev should replace librados2-dev

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-01-27 21:41:39 +08:00
Kefu Chai
fa6a69288c
Merge pull request #26099 from jdurgin/wip-remove-ms-async-affinity
common/options: remove unused ms async affinity options

Reviewed-by: Sage Weil <sage@redhat.com>
2019-01-27 21:41:10 +08:00
Kefu Chai
0cec1bdaad
Merge pull request #26026 from tchaikov/wip-memstore-ceph-mutex
os/memstore: use ceph::mutex and friends

Reviewed-by: Sage Weil <sage@redhat.com>
2019-01-27 21:40:30 +08:00
Sage Weil
1881d9abba msg/async: fix should_use_msgr2 behavior (including monc)
Be consistent about whether we should use v2 to connect or not, and fix
teh monclient check to use the same logic.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-27 07:33:51 -06:00
Kefu Chai
3ba9eaa890
Merge pull request #26159 from dzafman/wip-gitignore
git: Ignore tags anywhere

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-01-27 21:18:35 +08:00
Ilya Dryomov
11ee8f02b0
Merge pull request #26150 from dillaman/wip-rados-lock-info
rados: return legacy address in 'lock info'

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2019-01-27 08:18:45 +01:00
Patrick Donnelly
032fda563c
Merge PR #25599 into master
* refs/pull/25599/head:
	mds: do not log internal journal error
	mds: clean up log level 2 messages
	mds: log session open stats

Reviewed-by: Zheng Yan <zyan@redhat.com>
2019-01-26 21:20:06 -08:00
David Zafman
6eec939e6f git: Ignore tags anywhere
Signed-off-by: David Zafman <dzafman@redhat.com>
2019-01-26 19:48:48 -08:00
Patrick Donnelly
a823134c4d
Merge PR #26096 into master
* refs/pull/26096/head:
	mds: remove cache drop asok command

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2019-01-26 16:51:20 -08:00
Patrick Donnelly
585624d86b
Merge PR #26087 into master
* refs/pull/26087/head:
	mds: fix MDSGather inheritance
	include: remove unnecessary inheritance

Reviewed-by: Sage Weil <sage@redhat.com>
2019-01-26 16:50:16 -08:00
Patrick Donnelly
03eb8d9e99
Merge PR #26083 into master
* refs/pull/26083/head:
	mds: trim cache after journal flush

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2019-01-26 16:48:58 -08:00
Patrick Donnelly
6858ba6625
Merge PR #26082 into master
* refs/pull/26082/head:
	client: flush cap release on session flush

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>
2019-01-26 16:48:07 -08:00
Sage Weil
6e933fe1b8 msg/async/AsyncMessenger: clear need_addr *after* we set our new addr
We check need_addr at the top without a lock held, so we need to be sure
we finished our work before we clear it, or else when there are two racing
threads the first will get the clock and clear the value and the second
will do nothing and see the unlearned value before the first finishes.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-26 17:01:14 -06:00
Sage Weil
d9f3022a51 msg/async/ProtocolV2: fix handling for v2 client connection with v1 addr
Switch it to be v2.  Reject the case where the client sends and addrvec, though;
that should only happen for clients that did_bind, and they should only connect to
v2 if they have a v2 bound addr.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-26 11:44:55 -06:00
Sage Weil
4c2e28f5b7 ceph_test_msgr: do not connect_to on the client side
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-26 11:44:55 -06:00
Sage Weil
985ec03cf9 msg/async: do not connect from server
We could have a fault on a server-side of a non-lossy connectoin where
there is a fault and we have outgoing data queued.  Since we are a server,
we cannot connect; we should just go into standby and wait for the other
end to reconnect, or for someone to mark us down.

This fixes a failure reproduced by Messenger/MessengerTest.SyntheticInjectTest/0
where it would assert(!policy.server) in the connect code.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-26 11:44:55 -06:00
Sage Weil
98a9a5e538 msg/async: do not use peer to addr detection; use getsockname()
If of relying on the peer to tell us what address we are connecting from,
look at how our local socket is bound, and use that address.

This removes the possibility for error because we will infer our address
locally and that will be the one place it is decide; the server will just
use our value.  As things were previously, we had to make the local and
remote inference match, which was fragile.

This does take away the client's ability to discover if it is traversing
NAT to reach the server and learning its public/external address.  I
don't think anybody has ever tested this, so it probably didn't even work,
and I've never heard it come up as a requirement.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-26 11:44:55 -06:00
Kefu Chai
8b55b47cc7 install-deps.sh: use latest setuptools
this change reverts "install-deps.sh: workaround setuptools' dependency on six"

This reverts commit 30ce5e5546.

tl;dr: this change addresses the failures of "make check" runs on arm64
builders when they try to build `mgr-dashboard-test-venv` target.

long story: without this change, we will fail to pull in
setuptools >= 36, and as a result pip will fail to import
`setuptools.build_meta` in `pip/_vendor/pep517/_in_process.py`. and will
a `BackendUnavailable` exception thrown by `_call_hook()` in
`pip/_vendor/pep517/wrappers.py`. since the issue addressed by 30ce5e55
has been addressed since setuptools >= 36.0.1, we should be safe to
upgrade to the latest setuptools now.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-27 01:33:10 +08:00
Kefu Chai
16e3320789 osd/PG: remove unused functions
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-27 01:18:25 +08:00
David Zafman
74239605a1
Merge pull request #23904 from xiexingguo/wip-27985
osd/PG: fix misused FORCE_RECOVERY[BACKFILL] flags

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
2019-01-25 16:44:22 -08:00
Sage Weil
2ca61bb460 msg/async/ProtocolV2: always send non-empty addrvec for self
If we don't know our address yet, send the peer a 0.0.0.0 or :: address with an empty
port and a populated nonce.  That way the peer can infer our final addr the same way
we do from learned_addr.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-25 14:17:34 -06:00
Sage Weil
5acf7ec1ff msg/async: never fill out port in myaddr if we didn't bind
If we are a client and didn't bind, then we should not fill in the port for our
address.  The one the peer sent us is just the random port our outgoing connection
happened to land on!

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-25 14:12:59 -06:00
Sage Weil
ea517edd2d ceph_test_msgr: use v2 addrs for simplemessenger
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-25 13:48:11 -06:00
Ricardo Dias
1b097e8164
msg/async: msgr2: don't force write event on every message received
Signed-off-by: Ricardo Dias <rdias@suse.com>
2019-01-25 19:42:37 +00:00
Patrick Donnelly
fef88d9581
Merge PR #25254 into master
* refs/pull/25254/head:
	mds: optimize resuming stale caps
	client: avoid unnecessary wakeup when handling RENEWCAPS
	client: set cap->wanted when adding new cap
	client: don't wakeup cap waiters twice when mds recovered
	mds: optimize revoking stale caps
	mds: put notable caps at the front of session's caps list
	mds: track if client has writeable range in Capability
	mds: add session pointer to Capability
	client: skip updating 'wanted' caps if caps are already issued
	client: sync 'retain caps' logical from kernel client

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-01-25 10:52:47 -08:00
Sebastian Wagner
2b9daafaf4
Merge pull request #25497 from jmolmo/wip_ansible_orchestrator
mgr/ansible: Create/Remove OSDs

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Sébastien Han <seb@redhat.com>
2019-01-25 19:37:22 +01:00
Kefu Chai
5931341c2e
Merge pull request #25848 from majianpeng/debug-buffer-compiler-error
common/buffer: fix compiler bug when enable DEBUG_BUFFER.

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-01-26 01:03:05 +08:00
Jason Dillaman
958b087f6b rados: return legacy address in 'lock info'
There is no need to differentiate v1 vs v2 addresses and we should
avoid breaking backwards compatibility in the output.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-01-25 09:12:15 -05:00
Kefu Chai
cc26a67890
Merge pull request #26022 from theanalyst/common-mclock-prio
common, osd: set mclock priority as 1 by default.

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
2019-01-25 22:07:17 +08:00
Kefu Chai
17d2fd5609
Merge pull request #26043 from tchaikov/wip-37975
osd: unmount store after service.shutdown()

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-01-25 22:06:27 +08:00
Kefu Chai
eedc1f1a50
Merge pull request #26053 from Aran85/pipeline-fix
osd: ec saves a write access to the memory under most circumstances

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-01-25 22:05:58 +08:00
Ricardo Marques
1b206291f8
Merge pull request #26119 from smithfarm/wip-fix-suse-buildrequires
rpm: fix xmlsec1 build dependency for dashboard make check

Reviewed-by: Ricardo Marques <rimarques@suse.com>
2019-01-25 12:55:38 +00:00
Yan, Zheng
e824b3d202 mds: optimize resuming stale caps
If client doesn't want any cap, there is no need to re-issue stale
caps.

A special case is that client wants some caps, but skipped updating
'wanted'. For this case, client needs to update 'wanted' when stale
session get renewed.

Fixes: http://tracker.ceph.com/issues/38043
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2019-01-25 17:16:04 +08:00
Sage Weil
5145120c2e
msg/async/ProtocolV2: be forgiving in server identity check
If we parse something out of mon_host or are given an addr by a user,
we should succeed as long as our target_addr is one of the destination's
addrs.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-25 09:15:19 +00:00
songweibin
65b803831b pybind/rbd: fix a typo in metadata_get comments
Signed-off-by: songweibin <song.weibin@zte.com.cn>
2019-01-25 16:56:05 +08:00
Ilya Dryomov
74e9992e97
Merge pull request #26120 from idryomov/wip-doc-osd-map-config
doc/rados/configuration: refresh osdmap section

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-01-25 08:51:36 +01:00
Patrick Donnelly
d7f6c1583b
mds: trim cache after journal flush
Dropping cache should trim the cache only after journal flush otherwise dirty
inodes will remain pinned in cache.

Fixes: http://tracker.ceph.com/issues/38010
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-01-24 21:49:48 -08:00