Commit Graph

97260 Commits

Author SHA1 Message Date
Yingxin Cheng
7bc2abfd3b crimson/net: dummy auth for protocol v2
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:19 +08:00
Yingxin Cheng
a12875300a crimson/net: add AuthConnectionMeta to Protocol
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:19 +08:00
Yingxin Cheng
604ea739de crimson/net: add peer_global_id to Connection
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:19 +08:00
Yingxin Cheng
ce95af1cb6 crimson/net: banner exchange and HelloFrame
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:18 +08:00
Yingxin Cheng
7d71884bfb crimson/net: support frame read/write for ProtocolV2
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:18 +08:00
Yingxin Cheng
5f5b60a056 crimson/net: add target_addr to SocketConnection
represent the peer target_addr accepted/connected

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:18 +08:00
Yingxin Cheng
5d79386fbb crimson/net: add get_mytype() to Messenger
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:18 +08:00
Yingxin Cheng
5e7d387caf crimson/net: add global_seq in ProtocolV2
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:18 +08:00
Yingxin Cheng
e366e736ff crimson/net: skeleton code for ProtocolV2 logic
crimson ProtocolV2 class is following a state-machine design style:
* states are defined in ProtocolV2::state_t;
* call `execute_<state_name>()` methods to trigger different states;
* V2 logics are implemented in each execute_<state_name>() methods, and
  with explicit transitions to other states at the end of the execute_*;
* each state is associated with a write state defined in Protocol.h:
  - none: not allowed to send;
  - delay: messages can be queued, but will be delayed to send;
  - open: dispatch queued message/keepalive/ack;
  - drop: not send any messages, drop them all.

crimson ProtocolV2 is alike async ProtocolV2, with some considerations:
* explicit and encapsulated client/server handshake workflow.
* futurized-exception-based fault handling, which can interrupt protocol
  workflow at any time in each state.
* introduced SERVER_WAIT state, meaning to wait for peer-client's socket
  to reset or be replaced, and expect no further reads.
* introduced an explicit REPLACING state, async-msgr would be at the
  NONE state during replacing.

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:18 +08:00
Yingxin Cheng
674a740877 crimson/net: apple-to-apple server async-msgr
The simplest async-msgr server which will have the same behavior with
crimson-msgr server for apple-to-apple performance test.

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:18 +08:00
Yingxin Cheng
42b66ac99b test/crimson: improved perf tool for crimson-msgr
New features:
* --jobs: start multiple client messengers from core #1 ~ #jobs;
* --core: can assign server core to get away from busy client cores;
* --rounds: a client will send <rounds>/<jobs> messages;

Improved:
* Better configuration report;
* Report individual client results plus a summary;
* Validate if CPU number is sufficient before running;
* Sleep 1 second while connecting, so it won't hurt performance;
* Simplify client logic and bug fixes;

Removed unecessary features:
* finish_decode() for MOSDOp;
* keepalive ratio;

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:18 +08:00
Yingxin Cheng
f11e9b87a1 crimson/net: fix messenger creation
sharded data structures should only be allocated in core#0, or the
program will hang during exit.

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:18 +08:00
Yingxin Cheng
1e5852215a crimson/net: create SocketConnection based on address type
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:18 +08:00
Yingxin Cheng
e8ef8f2ce1 crimson/net: use entity_addrvec_t::front() inside messenger
Do not expect legacy-addr in order to bind v2 address.

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:18 +08:00
Yingxin Cheng
5779e4119a crimson/net: added msgr2 setting for SocketConnection
Provide entrypoint for the future ProtocolV2 class.

Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:18 +08:00
Yingxin Cheng
32ad076ceb crimson/net: implement factory method of Socket
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 11:21:18 +08:00
Yingxin Cheng
9282d1a5d2 crimson/net: introduce SocketFRef
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 10:48:50 +08:00
Yingxin Cheng
2ab8b741d3 crimson/net: introduce protocol-level abstraction
Signed-off-by: Yingxin Cheng <yingxincheng@gmail.com>
2019-04-05 10:48:50 +08:00
Kefu Chai
d1e83082b4 rpm: install python%{python3_pkgversion}-devel on el7
as RHEL 7.7 support a native python3.6, we cannot hardwire to python3.4
on RHEL7/CentOS7 anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-05 10:16:25 +08:00
Boris Ranto
8b3f0d5bcc cmake: check for MAJOR.MINOR version of python3
We can only check for MAJOR.MINOR version of python3 since
FindPython3Libs does not support checking for MAJOR.MINOR.PATCH version
of python3. We also need to make sure we use the PYTHON3 versions of
these variables.

This should fix a regression introduced by c961e00.

Signed-off-by: Boris Ranto <branto@redhat.com>
2019-04-05 10:09:58 +08:00
Patrick Donnelly
050004ce2c
Merge PR #27266 into master
* refs/pull/27266/head:
	cephfs-shell: Fix hidden files and directories list by ls command

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-04-04 15:36:14 -07:00
Ricardo Marques
1efcdf3000
Merge pull request #27380 from ricardoasmarques/fix-iscsi-target-submission
mgr/dashboard: Fix iSCSI target submission

Reviewed-by: Tiago Melo <tmelo@suse.com>
2019-04-04 23:35:14 +01:00
Patrick Donnelly
f7786d6cf5
Merge PR #27198 into master
* refs/pull/27198/head:
	vstart: generate environment script suitable for sourcing
	cephfs-shell: fix mode of script in tree

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-04-04 14:24:38 -07:00
Ricardo Marques
d63f206310 mgr/dashboard: Fix iSCSI target submission
cdIsInGroup should not be included in the request
and username/password should be an empty string
(instead of null)

Fixes: https://tracker.ceph.com/issues/39110

Signed-off-by: Ricardo Marques <rimarques@suse.com>
2019-04-04 21:31:59 +01:00
Patrick Donnelly
27df199891
Merge PR #27373 into master
* refs/pull/27373/head:
	doc: Add usage for shortcuts command in cephfs-shell

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-04-04 12:52:55 -07:00
Patrick Donnelly
f18eccb668
Merge PR #27303 into master
* refs/pull/27303/head:
	mds: add Octopus cephfs feature bit
	mds: verify current release has a cephfs feature bit
	mds: add cephfs feature bit for Nautilus
	mon: generalize and refactor lambda use
	qa: test featureful client with mimic base
	qa: remove obsolete snap upgrade tests

Reviewed-by: Zheng Yan <zyan@redhat.com>
2019-04-04 12:42:16 -07:00
Patrick Donnelly
901bb7cd04
Merge PR #27238 into master
* refs/pull/27238/head:
	mds: update projected_version when upgrading snaptable

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-04-04 12:40:27 -07:00
Patrick Donnelly
66a2276da3
Merge PR #26881 into master
* refs/pull/26881/head:
	mds: remove SimpleLock::replicate_relax()
	mds: avoid check session connection's features when issuing caps

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-04-04 12:39:08 -07:00
Ricardo Marques
ab0ac43982 mgr/dashboard: Log gateway_name on iscsi_client
Signed-off-by: Ricardo Marques <rimarques@suse.com>
2019-04-04 20:15:03 +01:00
Ricardo Marques
c988a44f06 mgr/dashboard: Display number of sessions on iSCSI overview page
Signed-off-by: Ricardo Marques <rimarques@suse.com>
2019-04-04 20:15:03 +01:00
Ricardo Marques
b7870c8051 mgr/dashboard: Adds 'tcmu-runner' to the 'user:rbd' backstore descr
Signed-off-by: Ricardo Marques <rimarques@suse.com>
2019-04-04 20:15:03 +01:00
Ricardo Marques
094d7935c3 mgr/dashboard: Disable backstore dropdown
iSCSI backstore is always visible but is disabled
if only one option is available.

Signed-off-by: Ricardo Marques <rimarques@suse.com>
2019-04-04 20:15:03 +01:00
Ricardo Marques
c588269e91 mgr/dashboard: Improve iSCSI overview page
iSCSI overview page will now use information
obtained from ceph-iscsi.

Fixes: https://tracker.ceph.com/issues/39024

Signed-off-by: Ricardo Marques <rimarques@suse.com>
2019-04-04 20:15:03 +01:00
Kefu Chai
8bd69f29af
Merge pull request #27358 from tchaikov/wip-multi-python3
rpm,cmake: use specified python3 version if any 

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
2019-04-04 23:50:32 +08:00
Kefu Chai
213bae2499
Merge pull request #27362 from ektich/ektich-patch-1
doc/rados/configuration: fix typo in mon-lookup-dns

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-04-04 23:41:00 +08:00
Kefu Chai
7cf8f6cbb4
Merge pull request #27338 from tchaikov/wip-crimson-wait-for-active
crimson/osd: lower debug level on i/o path

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-04-04 23:27:05 +08:00
Ricardo Marques
01679f44d8
Merge pull request #27378 from ricardoasmarques/fix-iscsi-disk-diff
mgr/dashboard: Fix iSCSI disk diff calculation

Reviewed-by: Ricardo Dias <rdias@suse.com>
2019-04-04 15:04:39 +01:00
Ricardo Marques
8b183d3d4c
Merge pull request #27262 from ricardoasmarques/check-gateway-in-use
mgr/dashboard: Check if gateway is in use before deletion

Reviewed-by: Ricardo Dias <rdias@suse.com>
2019-04-04 15:02:26 +01:00
Ricardo Marques
e97dff7d17
Merge pull request #27248 from ricardoasmarques/display-target-sessions
mgr/dashboard: Display the number of iSCSI active sessions

Reviewed-by: Ricardo Dias <rdias@suse.com>
2019-04-04 14:58:53 +01:00
Servesha Dudhgaonkar
2808373129 doc : fixed capitalization
Signed-off-by: Servesha Dudhgaonkar <sdudhgao@redhat.com>
2019-04-04 19:09:38 +05:30
Ricardo Marques
78595acff6 mgr/dashboard: Fix iSCSI disk diff calculation
The separator is now a '/' instead of '.'

Fixes: https://tracker.ceph.com/issues/39109

Signed-off-by: Ricardo Marques <rimarques@suse.com>
2019-04-04 13:35:53 +01:00
Vanush "Misha" Paturyan
3d935c3c53 doc/rados/configuration/mon-lookup-dns: fix typo
Signed-off-by: Vanush "Misha" Paturyan <ektich@gmail.com>
2019-04-04 12:37:54 +01:00
Varsha Rao
ceaf3bcf44 doc: Add usage for shortcuts command in cephfs-shell
Signed-off-by: Varsha Rao <varao@redhat.com>
2019-04-04 15:53:41 +05:30
Ricardo Marques
6c8fb920cb
Merge pull request #27350 from tspmelo/wip-fix-nfs-transport
mgr/dashboard: Fix deletion of NFS transports properties

Reviewed-by: Ricardo Dias <rdias@suse.com>
2019-04-04 09:03:48 +01:00
Mykola Golub
2ba794d736
Merge pull request #27326 from dillaman/wip-librados-unitest-logging
test/librados_test_stub: ensure the log flusher thread is started

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
2019-04-04 10:25:42 +03:00
Kefu Chai
8614b9b81d
Merge pull request #26016 from falcon78921/wip-doc-37957
doc: added section on creating RESTful API user

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-04-04 14:33:24 +08:00
Kefu Chai
7681a38ce5 rpm: pass %{python3_version} to cmake
to force cmake to use the python3 and python3 modules for building
python3 bindings

on the debian side, it's okay to continue using "-DWITH_PYTHON3=ON", as

- cmake does normalize "ON" to 3
- debian's cmake extension lives on /usr/lib/python3/dist-packages/
  not in a specific /usr/lib/python3.x/dist-packages directory

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-04 13:30:48 +08:00
Kefu Chai
c961e002d2 cmake: check for libpython of the same version of interpreter
actually cython or python3-cython's dependencies should take care of it.
but we should at least get it right on our side if we check it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-04 13:30:48 +08:00
Kefu Chai
d8c13e8604 cmake: use specified python3 version if any
use might have multiple python3 installed, some of them has/have all
dependencies installed and is good enough for building Ceph. we should
not always use the latest python installed in the system and complain that
there is missing dependencies, even if user has installed all the
python3 dependencies for the older python3.

put in other words, if user only installs cython module for python3.4, but
she has both python3.6 and python3.4 in her system. we should not force
her to uninstall python3.6 for installing Ceph.

this change also aligns with MGR_PYTHON_VERSION. i am not applying the
same change to WITH_PYTHON2, because python2 is already stablized. and distros
are not likely to release new python2 releases.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-04 13:30:48 +08:00
xie xingguo
6a8aedc107 qa: add new test case for pulling error
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-04-04 11:04:43 +08:00