Commit Graph

557 Commits

Author SHA1 Message Date
Kefu Chai
db09ea8531
Merge pull request #28794 from tchaikov/wip-crimson-option-help-seastar
crimson/osd: add "--help-seastar" command line option

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-06-30 15:52:32 +08:00
Kefu Chai
9e3e5d435e doc/dev/crimson.rst: update with stuff related to seastar options
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-29 23:16:42 +08:00
Kefu Chai
a66c27ad26 doc/dev/crimson.rst: update logging related doc
as crimson's logging levels are now connected those of ceph, there is no
need to note down the difference between them anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-29 21:41:00 +08:00
Varsha Rao
709f76c528 doc: Updates link to Sepia lab
Signed-off-by: Varsha Rao <varao@redhat.com>
2019-06-28 19:47:50 +05:30
Kefu Chai
07e02c47ff
Merge pull request #28674 from tchaikov/wip-doc/crimson
doc/dev: add crimson.rst

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-06-21 19:18:11 +08:00
Kefu Chai
ad639c7977
Merge pull request #28462 from myoungwon/wip-doc-dedup
doc: add a deduplication document

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-06-21 19:13:44 +08:00
myoungwon oh
cd41b64460 doc: add a deduplication document
Signed-off-by: Myoungwon Oh <omwmw@sk.com>
2019-06-21 14:23:41 +09:00
Kefu Chai
3ec6dfa289 doc/dev: add crimson.rst
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-06-21 12:29:07 +08:00
Sage Weil
18eacb6ed8 Merge PR #22446 into master
* refs/pull/22446/head:
	relicense LGPL-2.1 code as LGPL-2.1 or LGPL-3.0
	COPYING: add reference to a few Apache-2.0 licensed source files
	debian/copyright: sync with COPYING
	COPYING: LGPL2.1 -> LGPL-2.1
	COPYING: extra bits from debian/copyright

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
2019-06-12 09:11:49 -05:00
Neha Ojha
de5c5d8468
Merge pull request #21722 from mslovy/wip-parial-recovery
osd: partial recovery strategy based on PGLog

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-05-10 11:48:23 -07:00
Neha Ojha
984073e44d doc/dev/osd_internals/async_recovery: update cost calculation
Signed-off-by: Neha Ojha <nojha@redhat.com>
2019-05-08 20:19:32 -07:00
letterwuyu
7148dda076 osd: supplement doc
Signed-off-by: lishuhao <lishuhao@unitedstack.com>
2019-05-06 20:17:43 +08:00
lishuhao
20e26e8395 osd: partial recovery add doc
Signed-off-by: Ning Yao <yaoning@unitedstack.com>
Signed-off-by: lishuhao <lishuhao@unitedstack.com>
2019-05-06 15:30:35 +08:00
David Zafman
bc300055e7 doc: Describe recovery and backfill prioritizations
Fixes: http://tracker.ceph.com/issues/39011

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-05-02 21:58:44 -07:00
Samuel Just
5e36acdb6f osd/: mechanically rename RecoveryMachine/State/Ctx to Peering*
I'm going to extract this logic and reuse it in crimson.  Recovery* has
always been a confusing name as it implements neither log-based recovery
nor backfill.  Rather, it's mainly the buisiness logic for agreeing on
an authoritative log and some ancillary things such as scrub/backfill
reservation.

$ for i in $(git grep -l 'RecoveryMachine'); do sed -i 's/RecoveryMachine/PeeringMachine/g' $i; done
$ for i in $(git grep -l 'RecoveryState'); do sed -i 's/RecoveryState/PeeringState/g' $i; done
$ for i in $(git grep -l 'RecoveryCtx'); do sed -i 's/RecoveryCtx/PeeringCtx/g' $i; done

Signed-off-by: Samuel Just <sjust@redhat.com>
2019-05-01 11:22:11 -07:00
Kefu Chai
49117a0414
Merge pull request #27741 from tchaikov/doc/dev/wip-corpus.rst
doc/dev/corpus.rst: correct instructions

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-04-25 13:30:10 +08:00
Kefu Chai
692f4b1a05 doc/dev/corpus.rst: correct instructions
* should keep all OSD up and running, otherwise some tests will wait for
  healthy cluster for ever.
* should start 2 MGR for an active-standby setting -- better coverage
this way

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-24 19:49:58 +08:00
Sage Weil
cf9dfa79c1 doc/dev/erasure-coded-pool: update
Signed-off-by: Sage Weil <sage@redhat.com>
2019-04-22 12:36:29 -05:00
Sage Weil
2f361a6eee relicense LGPL-2.1 code as LGPL-2.1 or LGPL-3.0
The primary motivation to relicense is a desire to integrate with projects
that are licensed under the Apache License version 2.0.  Although opinions
vary, there are some who argue the the LGPL-2.1 and Apache-2.0 licenses
are not fully compatible.  We would like to avoid the ambiguity and
potential for controversy.

Projects we would like to consume that are Apache-2.0 licensed include
Seastar, OpenSSL (which is in the process of relicensing to Apache-2.0),
and Swagger (swagger.io).  Note that some of these are dynamically linked
or consumed via a high-level language and may or may not require a change
to LGPL-3.0, but providing the option for LGPL-3.0 certainly avoids any
uncertainty.

A few other source files are already incorporated into Ceph that claim an
Apache-2.0 license:

    src/common/deleter.h
    src/common/sstring.h
    src/include/cpp-btree

The Ceph developers would further like to provide a license option that is
more modern than the current LGPL-2.1.  LGPL-3.0 includes updated,
clarified language around several issues and is widely considered
more modern, superior license.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-04-22 11:22:55 -05:00
Kefu Chai
7f7fdc7211 doc/corpus: update to adapt the change from autotools to cmake
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-04-16 00:25:53 +08:00
Changcheng Liu
fa8dffe883 doc: correct ceph example in corpus section
Signed-off-by: Changcheng Liu <changcheng.liu@intel.com>
2019-04-08 09:26:24 +08:00
Changcheng Liu
4953a6e9e3 doc: correct example to use vstart to run up cluster
Signed-off-by: Changcheng Liu <changcheng.liu@intel.com>
2019-04-08 09:15:52 +08:00
Nathan Cutler
0e63fed768
Merge pull request #27159 from smithfarm/wip-38934
doc: move Developer Guide to its own subdirectory

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2019-03-27 12:17:37 +01:00
David Zafman
769cdc8294 doc: Document new pg state and changes to auto repair behavior
Fixes: http://tracker.ceph.com/issues/38616

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-03-25 16:03:36 -07:00
Nathan Cutler
64fc87a483 doc: move Developer Guide to its own subdirectory
Fixes: ef3ac5692b
Fixes: https://tracker.ceph.com/issues/38934
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2019-03-25 16:34:36 +01:00
David Zafman
9fd4b062f1 doc: Fix the pg states and auto repair config options
Fixes: http://tracker.ceph.com/issues/38896

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-03-22 19:58:00 -07:00
Sage Weil
e57af1d41d msg: remove SimpleMessenger
Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-17 07:57:45 -05:00
Sage Weil
937f28e6a6 doc/releases/nautilus: add msgr2 refs
Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-11 10:31:49 -05:00
Sage Weil
bd52f35fd0 Merge PR #26444 into master
* refs/pull/26444/head:
	ceph_features: deprecate MON_SINGLE_PAXOS
	osdmap: map versions to releases
	mon: retire some `osd set/unset` flags

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-03-04 17:26:12 -06:00
Changcheng Liu
839c07822a doc: correct vstart.sh file location
Signed-off-by: Changcheng Liu <changcheng.liu@intel.com>
2019-03-04 16:55:31 +08:00
Joao Eduardo Luis
a20ce5f427 osdmap: map versions to releases
Signed-off-by: Joao Eduardo Luis <joao@suse.de>
2019-03-02 11:36:23 +00:00
Sage Weil
5eb77a2b73 Merge PR #26614 into master
* refs/pull/26614/head:
	osd: Implement lazy omap usage statistics

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-02-28 18:08:29 -06:00
Brad Hubbard
9e21ef0e40 osd: Implement lazy omap usage statistics
Opportunistic gathering of omap statistics during deep scrub.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2019-02-26 14:19:53 +10:00
Xie Xingguo
59a99dfbd7
Merge pull request #26611 from falcon78921/wip-doc-log-based-pg
doc: updated reference link for log based PG

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-02-26 08:42:37 +08:00
James McClune
0dc5000124 doc: updated reference link for log based PG
Added :ref: label to Log Based PG page. Referenced <log-based-pg>
label on Asynchronous Recovery page.

Fixes: https://tracker.ceph.com/issues/38465
Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>
2019-02-24 16:08:44 -05:00
James McClune
93b904b32a doc: updated governance link for Ceph project leads
Added :ref: label to Governance page. Referenced <governance>
label on Development Workflow page.

Fixes: https://tracker.ceph.com/issues/38464
Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>
2019-02-24 15:58:02 -05:00
Ricardo Dias
d5fab790ef
doc/msgr2: session establishment failure scenarios
Signed-off-by: Ricardo Dias <rdias@suse.com>
2019-02-19 12:14:51 +00:00
Sage Weil
bc7f567467 msg/async/ProtocolV2: refuse incoming reconnect attempt intended for someone else
The same scenario that was possible on client_ident is also possible
for reconnect: on the accepting side we recognized the client_addrs
but the client is trying to reach someone else.  Since the cookie is a
random 64-bit value it is unlikely, but still possible, that we will
match an existing session that is not for the client.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-11 11:02:28 -06:00
Sage Weil
e399fc7514 msg/async/ProtocolV2: refuse incoming connection intended for someone else
If we get a client_ident frame, and they are trying to talk to someone
else, drop the connection.

This is an inelegant workaround to http://tracker.ceph.com/issues/38247.
A nicer fix would be to restructure the protocol so that the client knows
who they connected to before they try to open a session.  That is a bigger
change that can follow...

Fixes: http://tracker.ceph.com/issues/38247
Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-11 09:29:30 -06:00
Sage Weil
2e9c61de49 doc/dev/msgr2: bring up to date with new HELLO, different RECONNECT
Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-07 17:05:10 -06:00
Sage Weil
86c99bf023 msg/async/ProtocolV2: separate IDENT into {CLIENT,SERVER}_IDENT
This is less ambiguous and easier to document since the frame payloads
are different.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-07 17:05:10 -06:00
Sage Weil
5e4df2a509 doc/dev/cephx: re-wordwrap
Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-07 12:10:34 -06:00
Sage Weil
c1102f043e doc/dev/cephx: document nautilus change to cephx
Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-07 12:10:33 -06:00
Sage Weil
c7ee66c3e5 auth,msg/async/ProtocolV2: negotiate connection modes
The modes are:

- crc: crc32c checksums to protect against bit errors.  No secrecy or
  authenticity guarantees, so a MITM could alter traffic in flight.
- secure: cryptographic secrecy and authenticity proection (i.e, encrypted
  and signed).

We do not include a 'signed' mode that provides authenticity without
secrecy because the cryptographic protocols appear to be faster than
SHA-2.

New settings:

- ms_cluster_mode  : mode(s list) for intra-cluster connections
- ms_service_mode  : mode(s list) for daemons to allow
- ms_client_mode   : mode(s list) for clients to allow

Also,

- ms_mon_cluster_mode  : mon <-> mon connections
- ms_mon_service_mode  : mon <-> daemon or client connections

The msgr2 protocol is expanded slightly to negotiate a mode.  Client
shares it's allowed/preferred modes, and server picks one as auth finishes.
Negotiation is independent of the authentication, except that the
authentiction mode may precluse certain choices. Specifically, AUTH_NONE
does not support 'secure', only 'crc'.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-07 12:10:33 -06:00
Sage Weil
c1a2d1be52 doc/dev/msgr2: update docs to match implementation for auth frames
Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-07 06:53:03 -06:00
Sage Weil
1c40968bd5 msg/async: separate TAG_AUTH_REQUEST_MORE and TAG_AUTH_REPLY_MORE
Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-07 06:53:03 -06:00
Sage Weil
141c305c12 doc/dev/cephx: document current implementation of cephx
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-29 12:28:36 -06:00
Sage Weil
86d74e9ad7 Merge PR #25818 into master
* refs/pull/25818/head:
	mon: adjust mon sync, extra_probe_peers to use addrvec

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-01-08 14:15:28 -06:00
Sage Weil
3d34c4ebd2 mon: adjust mon sync, extra_probe_peers to use addrvec
The peer addr stuff via asok is a bit fragile because the user must
provide an exact addrvec matching the mon to avoid some weirdness, but
it's rarely used, and the fix would be some robustness/tolerance in the
messenger that is a bigger project than this.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-08 13:08:31 -06:00
Sage Weil
9badd953f4 doc/dev/msgr2: checksum frame header (tag + length)
Ensure we catch transport errors early (before we, say, try to allocate
a missized payload buffer).

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-07 15:01:21 -06:00