Commit Graph

97110 Commits

Author SHA1 Message Date
Yingxin Cheng
e289ec8521 test/crimson: wrap perf parameters as structured data
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-09 16:37:40 +08:00
Yingxin Cheng
478e9e8395 test/crimson: perf_crimson/async_server write test support
Allow server to have write/send workload with specified block size.

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-09 16:06:39 +08:00
Yuri Weinstein
dff2bf78be
Merge pull request #27407 from tchaikov/wip-rpm-python3-el7
rpm: use python3.4 on RHEL7 by default
2019-04-06 08:57:57 -07:00
Neha Ojha
91e4926b09
Merge pull request #27403 from cyberang3l/many_objects_per_pg_docs
doc: update documentation for the MANY_OBJECTS_PER_PG warning

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-04-05 16:03:01 -07:00
Casey Bodley
97f8a4aa1c
Merge pull request #27409 from cbodley/wip-vstart-rgw-debug
vstart: only add --debug-ms=1 in RGWDEBUG

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-04-05 16:41:25 -04:00
Casey Bodley
7e6d53233d vstart: only add --debug-ms=1 in RGWDEBUG
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-04-05 14:32:06 -04:00
Kefu Chai
7b15b682b1 cmake: should PYTHON3_VERSION_STRING of libpython3
and make sure the version matches exactly the requested one

in future, we should use FindPython.cmake

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-06 01:59:36 +08:00
Vangelis Tasoulas
24131fc59a
doc: Update documentation for the MANY_OBJECTS_PER_PG warning
The current documentation for the MANY_OBJECTS_PER_PG warning
states that The threshold can be raised to silence the health
warning by adjusting the mon_pg_warn_max_object_skew config
option on the monitors. It seems that this is not true (at least)
since the luminous times, and this option should be adjusted on
the managers.

I encountered this problem and I spend quite sometime injecting
the mon_pg_warn_max_object_skew to the monitors, added the option
ceph.conf and restarted the monitors several times but the warning
was not going away. I had to download the code to see what's
happening and I found out this:

$ git grep -A 3 mon_pg_warn_max_object_skew src/common/options.cc
src/common/options.cc:1480:    Option("mon_pg_warn_max_object_skew", Option::TYPE_FLOAT, Option::LEVEL_ADVANCED)
src/common/options.cc-1481-    .set_default(10.0)
src/common/options.cc-1482-    .set_description("max skew few average in objects per pg")
src/common/options.cc-1483-    .add_service("mgr"),

After I restarted the ceph-mgr service, the warning went away.

Signed-off-by: Vangelis Tasoulas <vangelis@tasoulas.net>
2019-04-05 19:53:35 +02:00
Patrick Donnelly
9473d99b61
Merge PR #27396 into master
* refs/pull/27396/head:
	doc: fixed typo in leadership names

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-04-05 10:49:25 -07:00
Patrick Donnelly
50206f33d9
Merge PR #27397 into master
* refs/pull/27397/head:
	doc: fixed caps

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-04-05 10:48:22 -07:00
Kefu Chai
cea9d18ced rpm: use python3.4 on RHEL7 by default
python3.4 is the native python3 before 7.6

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-06 00:28:39 +08:00
Casey Bodley
532458d9c1
Merge pull request #27400 from cbodley/wip-39118
rgw: limit entries in remove_olh_pending_entries()

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2019-04-05 12:02:02 -04:00
Sage Weil
8a5590b6aa Merge PR #26874 into master
* refs/pull/26874/head:
	OSD: OSDMapRef access by multiple threads is unsafe

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-04-05 08:57:51 -05:00
Sage Weil
cbcfe6e45f Merge PR #27132 into master
* refs/pull/27132/head:
	os/bluestore: new bluestore_debug_enforce_settings option.

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-04-05 08:57:31 -05:00
Sage Weil
c957b5a195 Merge PR #27317 into master
* refs/pull/27317/head:
	kv: make delete range optional on number of keys

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-04-05 08:57:14 -05:00
Sage Weil
819d484b37 Merge PR #27327 into master
* refs/pull/27327/head:
	mon/MonmapMonitor: clean up empty created stamp in monmap
	common/buffer: fix warnings

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-04-05 08:39:48 -05:00
Casey Bodley
3805ea635a rgw: limit entries in remove_olh_pending_entries()
If there are too many entries to send in a single osd op, the osd rejects
the request with EINVAL. This error happens in follow_olh(), which means
that requests against the object logical head (requests with no version
id) can't be resolved to the current object version. In multisite, this
also causes data sync to get stuck in retries

Fixes: http://tracker.ceph.com/issues/39118

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-04-05 09:26:07 -04:00
Servesha Dudhgaonkar
324426c966 doc: fixed caps
Signed-off-by: Servesha Dudhgaonkar <sdudhgao@redhat.com>
2019-04-05 17:29:05 +05:30
Servesha Dudhgaonkar
b373123c29 doc: fixed typo in leadership names
Signed-off-by: Servesha Dudhgaonkar <sdudhgao@redhat.com>
2019-04-05 17:16:50 +05:30
Ricardo Marques
ab8552f651
Merge pull request #27254 from ricardoasmarques/improve-iscsi-overview
mgr/dashboad: Improve iSCSI overview page

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2019-04-05 10:02:47 +01:00
Sebastian Wagner
effc2a3962
Merge pull request #27292 from sebastian-philipp/kubejacker-simplify
src/script/kubejacker: Fix and simplify

Reviewed-by: Jeff Layton <jlayton@redhat.com>
2019-04-05 10:09:24 +02:00
Kefu Chai
736d396ee0
Merge pull request #27121 from sebastian-philipp/orchestrator-doc-cli-fix-service-add
doc/orchestrator: Fix broken bullet points

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-04-05 13:52:44 +08:00
Kefu Chai
dbc32e518a
Merge pull request #27379 from xenolinux/wip-fix-bug1
doc : fixed capitalization

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-04-05 13:48:58 +08:00
Kefu Chai
9a60ae4286
Merge pull request #26710 from cyx1231st/wip-seastar-msgr-protocolv2
crimson/net: lossy connection for ProtocolV2

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-04-05 11:40:01 +08:00
Kefu Chai
19a7ce5d57
Merge pull request #27383 from tchaikov/wip-cmake-libpython3
cmake: check for MAJOR.MINOR version of python3

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-04-05 11:30:13 +08:00
Yingxin Cheng
0e564dbf7a crimson/net: more compact code with then()
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:19 +08:00
Yingxin Cheng
8ce92d0c3f crimson/net: clean auth related interfaces
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:19 +08:00
Yingxin Cheng
fcae1e6b44 crimson/net: scoped enum for proto_type
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:19 +08:00
Yingxin Cheng
feddee4e5c crimson/net: utilize lowres_system_clock with better performance
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:19 +08:00
Yingxin Cheng
82583c121c crimson/net: set policy from peer_type
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:19 +08:00
Yingxin Cheng
b6beb68885 crimson/net: fix Socket::read_exactly(0) case
Do not raise read_eof when try to read 0 bytes.

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:19 +08:00
Yingxin Cheng
ef680adb59 test/crimson: fix perf_crimson_msgr to support v2
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:19 +08:00
Yingxin Cheng
eb925f4944 test/crimson: msgr should wait before shutdown
test_concurrent_dispatch() should wait for server->wait() before
shutting down its messengers.

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:19 +08:00
Yingxin Cheng
6a1dc38010 test/crimson: unit test for v2 protocol
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:19 +08:00
Yingxin Cheng
153e4877ae crimson/net: message frame exchanges (lossy part)
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:19 +08:00
Yingxin Cheng
02535f188a crimson/net: add keepalive timestamps to connection
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:19 +08:00
Yingxin Cheng
ea9e0056bd crimson/net: protocolv2 handshake frame exchanges
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:19 +08:00
Yingxin Cheng
44a9886b87 crimson/net: ProtocolV2 auth part
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:19 +08:00
Yingxin Cheng
7bc2abfd3b crimson/net: dummy auth for protocol v2
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:19 +08:00
Yingxin Cheng
a12875300a crimson/net: add AuthConnectionMeta to Protocol
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:19 +08:00
Yingxin Cheng
604ea739de crimson/net: add peer_global_id to Connection
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:19 +08:00
Yingxin Cheng
ce95af1cb6 crimson/net: banner exchange and HelloFrame
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:18 +08:00
Yingxin Cheng
7d71884bfb crimson/net: support frame read/write for ProtocolV2
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:18 +08:00
Yingxin Cheng
5f5b60a056 crimson/net: add target_addr to SocketConnection
represent the peer target_addr accepted/connected

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:18 +08:00
Yingxin Cheng
5d79386fbb crimson/net: add get_mytype() to Messenger
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:18 +08:00
Yingxin Cheng
5e7d387caf crimson/net: add global_seq in ProtocolV2
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:18 +08:00
Yingxin Cheng
e366e736ff crimson/net: skeleton code for ProtocolV2 logic
crimson ProtocolV2 class is following a state-machine design style:
* states are defined in ProtocolV2::state_t;
* call `execute_<state_name>()` methods to trigger different states;
* V2 logics are implemented in each execute_<state_name>() methods, and
  with explicit transitions to other states at the end of the execute_*;
* each state is associated with a write state defined in Protocol.h:
  - none: not allowed to send;
  - delay: messages can be queued, but will be delayed to send;
  - open: dispatch queued message/keepalive/ack;
  - drop: not send any messages, drop them all.

crimson ProtocolV2 is alike async ProtocolV2, with some considerations:
* explicit and encapsulated client/server handshake workflow.
* futurized-exception-based fault handling, which can interrupt protocol
  workflow at any time in each state.
* introduced SERVER_WAIT state, meaning to wait for peer-client's socket
  to reset or be replaced, and expect no further reads.
* introduced an explicit REPLACING state, async-msgr would be at the
  NONE state during replacing.

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:18 +08:00
Yingxin Cheng
674a740877 crimson/net: apple-to-apple server async-msgr
The simplest async-msgr server which will have the same behavior with
crimson-msgr server for apple-to-apple performance test.

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:18 +08:00
Yingxin Cheng
42b66ac99b test/crimson: improved perf tool for crimson-msgr
New features:
* --jobs: start multiple client messengers from core #1 ~ #jobs;
* --core: can assign server core to get away from busy client cores;
* --rounds: a client will send <rounds>/<jobs> messages;

Improved:
* Better configuration report;
* Report individual client results plus a summary;
* Validate if CPU number is sufficient before running;
* Sleep 1 second while connecting, so it won't hurt performance;
* Simplify client logic and bug fixes;

Removed unecessary features:
* finish_decode() for MOSDOp;
* keepalive ratio;

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:18 +08:00
Yingxin Cheng
f11e9b87a1 crimson/net: fix messenger creation
sharded data structures should only be allocated in core#0, or the
program will hang during exit.

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:18 +08:00