Commit Graph

96364 Commits

Author SHA1 Message Date
Yingxin Cheng
ef680adb59 test/crimson: fix perf_crimson_msgr to support v2
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:19 +08:00
Yingxin Cheng
eb925f4944 test/crimson: msgr should wait before shutdown
test_concurrent_dispatch() should wait for server->wait() before
shutting down its messengers.

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:19 +08:00
Yingxin Cheng
6a1dc38010 test/crimson: unit test for v2 protocol
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:19 +08:00
Yingxin Cheng
153e4877ae crimson/net: message frame exchanges (lossy part)
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:19 +08:00
Yingxin Cheng
02535f188a crimson/net: add keepalive timestamps to connection
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:19 +08:00
Yingxin Cheng
ea9e0056bd crimson/net: protocolv2 handshake frame exchanges
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:19 +08:00
Yingxin Cheng
44a9886b87 crimson/net: ProtocolV2 auth part
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:19 +08:00
Yingxin Cheng
7bc2abfd3b crimson/net: dummy auth for protocol v2
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:19 +08:00
Yingxin Cheng
a12875300a crimson/net: add AuthConnectionMeta to Protocol
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:19 +08:00
Yingxin Cheng
604ea739de crimson/net: add peer_global_id to Connection
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:19 +08:00
Yingxin Cheng
ce95af1cb6 crimson/net: banner exchange and HelloFrame
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:18 +08:00
Yingxin Cheng
7d71884bfb crimson/net: support frame read/write for ProtocolV2
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:18 +08:00
Yingxin Cheng
5f5b60a056 crimson/net: add target_addr to SocketConnection
represent the peer target_addr accepted/connected

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:18 +08:00
Yingxin Cheng
5d79386fbb crimson/net: add get_mytype() to Messenger
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:18 +08:00
Yingxin Cheng
5e7d387caf crimson/net: add global_seq in ProtocolV2
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:18 +08:00
Yingxin Cheng
e366e736ff crimson/net: skeleton code for ProtocolV2 logic
crimson ProtocolV2 class is following a state-machine design style:
* states are defined in ProtocolV2::state_t;
* call `execute_<state_name>()` methods to trigger different states;
* V2 logics are implemented in each execute_<state_name>() methods, and
  with explicit transitions to other states at the end of the execute_*;
* each state is associated with a write state defined in Protocol.h:
  - none: not allowed to send;
  - delay: messages can be queued, but will be delayed to send;
  - open: dispatch queued message/keepalive/ack;
  - drop: not send any messages, drop them all.

crimson ProtocolV2 is alike async ProtocolV2, with some considerations:
* explicit and encapsulated client/server handshake workflow.
* futurized-exception-based fault handling, which can interrupt protocol
  workflow at any time in each state.
* introduced SERVER_WAIT state, meaning to wait for peer-client's socket
  to reset or be replaced, and expect no further reads.
* introduced an explicit REPLACING state, async-msgr would be at the
  NONE state during replacing.

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:18 +08:00
Yingxin Cheng
674a740877 crimson/net: apple-to-apple server async-msgr
The simplest async-msgr server which will have the same behavior with
crimson-msgr server for apple-to-apple performance test.

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:18 +08:00
Yingxin Cheng
42b66ac99b test/crimson: improved perf tool for crimson-msgr
New features:
* --jobs: start multiple client messengers from core #1 ~ #jobs;
* --core: can assign server core to get away from busy client cores;
* --rounds: a client will send <rounds>/<jobs> messages;

Improved:
* Better configuration report;
* Report individual client results plus a summary;
* Validate if CPU number is sufficient before running;
* Sleep 1 second while connecting, so it won't hurt performance;
* Simplify client logic and bug fixes;

Removed unecessary features:
* finish_decode() for MOSDOp;
* keepalive ratio;

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:18 +08:00
Yingxin Cheng
f11e9b87a1 crimson/net: fix messenger creation
sharded data structures should only be allocated in core#0, or the
program will hang during exit.

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:18 +08:00
Yingxin Cheng
1e5852215a crimson/net: create SocketConnection based on address type
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:18 +08:00
Yingxin Cheng
e8ef8f2ce1 crimson/net: use entity_addrvec_t::front() inside messenger
Do not expect legacy-addr in order to bind v2 address.

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:18 +08:00
Yingxin Cheng
5779e4119a crimson/net: added msgr2 setting for SocketConnection
Provide entrypoint for the future ProtocolV2 class.

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:18 +08:00
Yingxin Cheng
32ad076ceb crimson/net: implement factory method of Socket
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:18 +08:00
Yingxin Cheng
9282d1a5d2 crimson/net: introduce SocketFRef
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 10:48:50 +08:00
Yingxin Cheng
2ab8b741d3 crimson/net: introduce protocol-level abstraction
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 10:48:50 +08:00
Yingxin Cheng
a1ed20810b crimson/net: batch messages instead of chaining futures
Instead of chaining writes with send_ready, connection will batch
messages in out_q, and will reap them by write_events() in the open
state.

The performance of pingpong is 3.7 times better from observation.

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-03-22 14:52:23 +08:00
Yingxin Cheng
55a76b4864 crimson/net: centralized write_event()
* introduce write_state_t to decouple write behaviors from states.
* replace `h.promise` with `state_changed`, with a more general way to
  change write behaviors according to state switches.
* centralize write_event() to dispatch writes in the open state.
* friendly interface for v1/v2 protocol abstraction.

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-03-22 14:52:23 +08:00
Sage Weil
2ea299b640 Merge PR #27044 into master
* refs/pull/27044/head:
	common: add bool log_to_file option

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-03-21 12:04:59 -05:00
Sage Weil
f89273f828 Merge PR #27048 into master
* refs/pull/27048/head:
	doc: add changelog for nautilus

Reviewed-by: Sage Weil <sage@redhat.com>
2019-03-21 11:48:29 -05:00
Kefu Chai
a13a5452ff
Merge pull request #27099 from tchaikov/wip-crimson-misc-logging
crimson/{net,mon,osd}: misc logging changes

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-03-22 00:14:01 +08:00
Kefu Chai
085fdf57fd
Merge pull request #27089 from tchaikov/wip-cmake-with-ninja
cmake: do not assume ${CMAKE_GENERATOR} == make

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-03-21 22:05:54 +08:00
Casey Bodley
9663480f3b
Merge pull request #27100 from ddiss/trivial_rgw_auth_keystone_gcc_warning
[trivial] rgw: fix minor compiler warning in keystone auth

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-03-21 10:05:18 -04:00
Kefu Chai
613a4670af
Merge pull request #27094 from tchaikov/wip-cmake-cmake-ubsan
cmake: workaround of false alarm from ubsan

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-03-21 22:03:14 +08:00
Kefu Chai
01cd5fb72c
Merge pull request #27088 from tchaikov/wip-update-seastar
cmake,crimson: pick up latest seastar

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-03-21 21:59:28 +08:00
Kefu Chai
4bf14a13f7
Merge pull request #26731 from wjwithagen/wjw-fix-cmake-os-release
build: FreeBSD does not have /etc/os-release

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-03-21 21:53:23 +08:00
Kefu Chai
bb67400d15
Merge pull request #26607 from tchaikov/wip-cython-language_level
pybind: set language_level for cythonize explicitly

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-03-21 21:51:14 +08:00
David Disseldorp
d807d1916f rgw: fix minor compiler warning in keystone auth
Signed-off-by: David Disseldorp <ddiss@suse.de>
2019-03-21 14:10:15 +01:00
Kefu Chai
5eb349f4ae
Merge pull request #27093 from tchaikov/wip-crimson-test-alien-echo
test/crimson: update to accomodate Dispatcher changes

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-03-21 21:00:10 +08:00
Kefu Chai
ea4f15f92e crimson/osd: more verbose logging
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-03-21 20:46:22 +08:00
Kefu Chai
2625c2cb8d crimson/{mon,osd}: don't log ms_dispatch messages
msgr already logs them

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-03-21 20:44:26 +08:00
Kefu Chai
5f196a2a10 crimson/net: log tx/rx messages
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-03-21 20:44:26 +08:00
Sage Weil
eb5e39ea08 Merge PR #27092 into master
* refs/pull/27092/head:
	mailmap: sort entries
	mailmap: sort organizationmap
	mailmap: updates for nautilus
	doc: mailmap updates for mimic
	mailmap fixes for 12.2.0

Reviewed-by: Sage Weil <sage@redhat.com>
2019-03-21 07:22:06 -05:00
Sebastian Wagner
cee38745ea
Merge pull request #26404 from sebastian-philipp/orchestrator-error-handling
mgr/orchestrator: Add error handling to interface

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Tim Serong <tserong@suse.com>
2019-03-21 12:30:13 +01:00
Kefu Chai
77553040fc cmake: workaround of false alarm from ubsan
to workaround a GCC bug, see
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88684

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-03-21 18:57:27 +08:00
Kefu Chai
f0eb1a9f27 test/crimson: update to accomodate Dispatcher changes
Dispatcher::ms_get_authorizer() was changed in f22d433a, so update the
test accordingly

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-03-21 18:54:11 +08:00
Abhishek Lekshmanan
38f07b3522 mailmap: sort entries
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-03-21 10:46:10 +01:00
Abhishek Lekshmanan
bd643d5156 mailmap: sort organizationmap
since this will make it easier for editing entries in the future

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-03-21 10:43:21 +01:00
Abhishek Lekshmanan
bd806d310b mailmap: updates for nautilus
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
2019-03-21 10:32:53 +01:00
Kefu Chai
80edd351bf cmake: disable -Werror when building seastar
when building seastar with a recent fmt, fmt complains:

src/fmt/include/fmt/time.h:13:2: error: #warning fmt/time.h is
deprecated, use fmt/chrono.h instead [-Werror=cpp]
 #warning fmt/time.h is deprecated, use fmt/chrono.h instead
  ^~~~~~~

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-03-21 17:04:38 +08:00
Kefu Chai
43d8d1962a seastar: pickup the latest seastar
for couple improvements

* dbf7b6b0: add fast path for small writes
* 2f357317: IPv6 support

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-03-21 17:04:38 +08:00