Commit Graph

93216 Commits

Author SHA1 Message Date
Sage Weil
5709fb9122 osd: go active even if mon only accepted our v1 addr
We may bind to v1 and v2 addrs, but we need to behave if the mon only
recognized our v1 addr and still go active.  That's because whether they
see our v2 addr depends on whether we connected to the mon via the v1 or
v2 port, and the mon may not be binding to v2 (yet, or ever).

Add a legacy_equals to entity_addrvec_t, and use that instead of
probably_equals for the OSD boot checks.  The probably_equals returns true
if the IP address portion of the address is empty.. which should never
happen in the OSD boot case since we have learned our real IP long before
we try to send osd_boot.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-12-21 15:30:18 -06:00
Ricardo Dias
dd05584021 test/msgr: add test for msgr2 protocol
Signed-off-by: Ricardo Dias <rdias@suse.com>
2018-12-21 15:30:18 -06:00
Sage Weil
a1401bd3f7 msg/async/ProtocolV2: share socket_addr and all addrs during handshake
Also, drop the sloppy addr checks.. I'm not sure we need them?

Signed-off-by: Sage Weil <sage@redhat.com>
2018-12-21 15:30:18 -06:00
Sage Weil
3031a65cbb msg/async: print socket_addr for the connection
When we bind to multiple addrs, print the one the current connection is
using for the given connection.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-12-21 15:30:18 -06:00
Ricardo Dias
08d9f8aa08 msg/async: msgr2 protocol placeholder
This commit adds messenger V2 protocol. This V2 protocol is a clone of
the V1 protocol with the additional support for address vectors.
After the banner exchange the peers exchange their address vectors.

Signed-off-by: Ricardo Dias <rdias@suse.com>
2018-12-21 15:30:18 -06:00
Ricardo Dias
8ab05504f0 msg/async: move ProtocolV1 class to its own source file
Signed-off-by: Ricardo Dias <rdias@suse.com>
2018-12-21 15:30:18 -06:00
Sage Weil
3a52d3cf03 msg/async: keep listen addr in ServerSocket, pass to new connections
When we accept a connection, we want to know what listening addr we
accepted on.  Because the addr can change after we create teh listening socket
(when we learn the addr and fill in the IP portion), instead store the position
in our myaddrs addrvec.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-12-21 15:30:18 -06:00
Sage Weil
7ae4affef5 msg/async/AsyncMessenger: fix set_addr_unknowns
Fill in IP address from a v1 or v2 address into any v1 or v2 addresses.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-12-21 15:30:18 -06:00
Sage Weil
6e6159c899 Merge PR #25620 into master
* refs/pull/25620/head:
	os/bluestore: more robust handling for lack of per-pool stats cases.

Reviewed-by: Sage Weil <sage@redhat.com>
2018-12-21 10:12:42 -06:00
Sage Weil
42de7e43a4 Merge PR #25632 into master
* refs/pull/25632/head:
	test: add a json tester
	ceph_json: parser sets data on all types when parsing

Reviewed-by: Sage Weil <sage@redhat.com>
2018-12-21 09:49:37 -06:00
Kefu Chai
e57649dd9c
Merge pull request #22800 from wjwithagen/wjw-ctest-timeout
cmake: allow setting of the CTest timeout during building.

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-12-21 23:40:27 +08:00
Kefu Chai
28b91c5305
Merge pull request #25661 from joscollin/wip-doc-changing-monitors
doc: fix typo in add-or-rm-mons

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-12-21 14:46:48 +08:00
Kefu Chai
3dcb9fdfe6
Merge pull request #25639 from oliveiradan/wip-krb_pr25566_cleanup_oliveiradan
auth/krb: Fix Kerberos build warnings

Reviewed-by: Sage Weil <sage@redhat.com>
2018-12-21 14:36:51 +08:00
Kefu Chai
bfd5aabcf7
Merge pull request #25537 from jecluis/dev/wip-mgr-module-error
mgr: make module error message more descriptive

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2018-12-21 14:09:40 +08:00
Kefu Chai
b5cb2115ba
Merge pull request #25595 from rzarzynski/wip-ec-drop_bl_crosstalk_dependency
ec: lrc doesn't depend on crosstalks between bufferlists anymore.

Reviewed-by: Sage Weil <sage@redhat.com>
2018-12-21 14:06:08 +08:00
Kefu Chai
5d9440b203
Merge pull request #25503 from ceph/rbd-mirror-ceph-base
spec: requires ceph base instead of common

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-12-21 14:04:34 +08:00
Jos Collin
6dba3865c9
doc: fix typo in add-or-rm-mons
Signed-off-by: Jos Collin <jcollin@redhat.com>
2018-12-21 10:47:04 +05:30
Jason Dillaman
c84cc4a542
Merge pull request #25655 from gregsfortytwo/wip-rbd-snap-doc
doc: Clean up the snapshot consistency note

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-12-20 18:07:36 -05:00
Sage Weil
fc4adfd27a Merge PR #25577 into master
* refs/pull/25577/head:
	test: allow ceph_kvstorebench to be build on FreeBSD

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-12-20 17:02:15 -06:00
Sage Weil
4d61afd33a Merge PR #25508 into master
* refs/pull/25508/head:
	kv/rocksdb_cache: Remove C style struct hack undefined behavior

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2018-12-20 17:01:54 -06:00
Sage Weil
bb804eb895 Merge PR #25236 into master
* refs/pull/25236/head:
	ceph-volume: Add venv to vstart cmake target
	mgr/orchestrator: Add test orchestrator

Reviewed-by: Jeff Layton <jlayton@redhat.com>
2018-12-20 17:01:04 -06:00
Greg Farnum
505025c641 doc: Clean up the snapshot consistency note
The old note made it sound like we weren't crash-consistent, and had a
confusing section about needing to clone before mounting.
You *do* need to clone the snapshot before mounting it, but that has
nothing to do fs freezing or consistency.

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2018-12-20 10:33:38 -08:00
Daniel Oliveira
1f32b2b37c auth/krb: Fix Kerberos build warnings
Signed-off-by: Daniel Oliveira <doliveira@suse.com>
2018-12-20 11:16:00 -07:00
Kefu Chai
539a16f376
Merge pull request #25207 from cyx1231st/wip-crimson-msgr-dispatch-events
crimson/net: encapsulate protocol implementations with states (remaining part)

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-12-20 23:51:20 +08:00
Yingxin
1e6a0a9a42 crimson/net: let conn directly verify the authorizer with dispatcher
Signed-off-by: Yingxin <yingxin.cheng@intel.com>
2018-12-21 03:42:42 +08:00
Yingxin
03d435f79d crimson/net: remove the messenger gate
There is no need to use messenger gate if SocketConnection themselves
are properly gated.

Signed-off-by: Yingxin <yingxin.cheng@intel.com>
2018-12-21 03:10:17 +08:00
Yingxin
f56fcd095a crimson/net: encapsulate protocol implementations with accept/connect
Signed-off-by: Yingxin <yingxin.cheng@intel.com>
2018-12-21 03:10:09 +08:00
Yingxin
2dbe7499ea crimson/net: no open connections in accepting_conns
It should an atomic operation to move the connection from
accepting_conns to connections when switch to the open state.

Signed-off-by: Yingxin <yingxin.cheng@intel.com>
2018-12-21 03:09:05 +08:00
Yingxin
1643f95791 crimson/net: don't execute_open() if exception is thrown
Signed-off-by: Yingxin <yingxin.cheng@intel.com>
2018-12-21 03:09:05 +08:00
Yingxin
b22cf59c83 crimson/net: encapsulate protocol implementations with open state
Signed-off-by: Yingxin <yingxin.cheng@intel.com>
2018-12-21 03:09:05 +08:00
Yingxin
60d5dd0c24 crimson/net: remove unecessary future dependencies for accept/dispatch
Signed-off-by: Yingxin <yingxin.cheng@intel.com>
2018-12-21 03:09:05 +08:00
Yingxin
0142e5440d crimson/net: dispatch events in SocketConnection
* move dispatch(), and exception handling logics in accept() and
  connect() from SocketMessenger into SocketConnection, so we can manage
  the state transition in the same class and at the same abstraction
  level.
* gate the dangling futures in SocketConnection, because the
  connection's smart_ptr won't be hold by messenger any more during
  exception handling.
* don't return close() inside SocketConnection to prevent recursive
  gating -- dead lock.

Signed-off-by: Yingxin <yingxin.cheng@intel.com>
2018-12-21 03:08:55 +08:00
Kefu Chai
35185b9e71
Merge pull request #25647 from tchaikov/wip-ftbfs-aarch64-gcc-8.1
test/crimson: fix FTBFS of unittest_seastar_perfcounters on arm64

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-12-20 23:42:38 +08:00
Kefu Chai
7b99e65fad
Merge pull request #25641 from tchaikov/cmake-rbd-tracing
cmake: let rbd_api depend on librbd-tp

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-12-20 22:16:07 +08:00
Sebastian Wagner
4eb9fdad65
Merge pull request #25340 from sebastian-philipp/orchestrator-json-output
mgr/orchestrator: Add JSON output to CLI commands

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Tim Serong <tserong@suse.com>
2018-12-20 15:14:01 +01:00
Sage Weil
49907ea497 Merge PR #25630 into master
* refs/pull/25630/head:
	osd/OSDMap: cancel mapping if target osd is out

Reviewed-by: Sage Weil <sage@redhat.com>
2018-12-20 06:48:41 -06:00
Sage Weil
8bd90b1d0c Merge PR #25598 into master
* refs/pull/25598/head:
	common: avoid bl::get_current_ptr()-driven ref counting in denc.h.

Reviewed-by: Sage Weil <sage@redhat.com>
2018-12-20 06:48:24 -06:00
Kefu Chai
1189e2e3ea
Merge pull request #25585 from SUSE/wip-remove-cython-subinterpreter-check
cmake,deb,rpm: remove cython 0.29's subinterpreter check, re-enable build with cython 0.29+

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-12-20 19:30:08 +08:00
Kefu Chai
fdd8d2e95e test/crimson: fix FTBFS of unittest_seastar_perfcounters on arm64
this should address the GCC bug which causes following failure:

/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/test/crimson/test_perfcounters.cc:35:9:
internal compiler error: in tsubst_decomp_names, at cp/pt.c:16537
         for (const auto &[path, perf_counter_ref] : by_path) {
         ^~~

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-12-20 18:50:17 +08:00
Sebastian Wagner
3164551e3f ceph-volume: Add venv to vstart cmake target
Adds new add_custom_target `ceph-volume-venv-setup`
that creates a virtual environment, and
calls `python setup.py develop`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2018-12-20 10:57:02 +01:00
Sebastian Wagner
933b2cfc28 mgr/orchestrator: Add test orchestrator
1. To be able to run the cli without an external orchestrator.
2. Run the CLI in Teuthology.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2018-12-20 10:56:49 +01:00
Kefu Chai
9309c24504 cmake: let rbd_api depend on librbd-tp
whoever includes tracing/librbd.h should depends on librbd-tp.

this fix the FTBFS of:

ceph/src/librbd/librbd.cc:50:10: fatal error: tracing/librbd.h: No such
file or directory
 #include "tracing/librbd.h"
          ^~~~~~~~~~~~~~~~~~
compilation terminated.
src/librbd/CMakeFiles/rbd_api.dir/build.make:62: recipe for target
'src/librbd/CMakeFiles/rbd_api.dir/librbd.cc.o' failed

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-12-20 13:38:30 +08:00
Sage Weil
3a2b090623 Merge PR #25622 into master
* refs/pull/25622/head:
	Objecter.cc: make op resend logic more readable

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2018-12-19 21:23:26 -06:00
Sage Weil
9f3cf00b79 Merge PR #25360 into master
* refs/pull/25360/head:
	qa/workunits/mon/pg_autoscaler: clean up pools afterwards
	qa/suites/rados/singletone/all/pg-autoscaler: whitelist health warnings
	qa/tasks/ceph: wait for splits/merges before final scrub
	mon/OSDMonitor: be tidy with target_size_ratio and pre-nautilus code
	mgr/pg_autoscaler: simplify conditions
	qa/suites/rados: add simple pg-autoscaler test
	qa/workunits/cephtool/test.sh: pg_autoscale_mode=off while testing pg_num etc
	doc/rados/operations: document autoscaler and its health warnings
	mgr/pg_autoscaler: add pg autoscaler module
	pybind/mgr/mgr_util: move format_ helpers out of status module
	mon/OSDMonitor: accept optional target_size_{bytes,ratio} to 'osd pool create'
	mon/OSDMonitor: remove max_split_count configurable
	osd/osd_types: pool_opts_t: int -> int64_t
	osd/osd_types: pool_opts: fix whitespace
	osd/osd_types: pool_opts_t: make encoding feature-dependent
	mgr/devicehealth: pg_num_min 1 for device_health_metrics pool
	mon/OSDMonitor: accept optional pg_num_min to 'osd pool create'
	mon/OSDMonitor: apply osd_pool_default_pg_autoscale_mode to new pools
	pybind/mgr/mgr_module: some accessors
	mon/MgrMonitor: enable progress module by default
	osd/osd_types: add pool pg_autoscale_mode, pg_num_min, target_size_{bytes,ratio} properties
	osdc/Objecter: revise get_latest_version locking
	os/memstore: ignore OP_COLL_SET_BITS
	qa: generalise REQUIRE_MEMSTORE
	mgr: drop GIL in get_config
	mon: add 'size' arg to `osd pool create`
	mon: use pg_num_target for checks during creation
	mgr: revise locking in getter paths
	common/options: add `mon_target_pg_per_osd`
	mgr: expose OSDMap.pool_raw_used_rate

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2018-12-19 21:22:35 -06:00
Sage Weil
b255e49259 qa/workunits/mon/pg_autoscaler: clean up pools afterwards
This avoids a huge pg merge from 100s to 4, which takes a long time and
makes the teuthology scrub cleanup time out.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-12-19 14:37:01 -06:00
Sage Weil
b8d45b262c qa/suites/rados/singletone/all/pg-autoscaler: whitelist health warnings
Signed-off-by: Sage Weil <sage@redhat.com>
2018-12-19 14:37:01 -06:00
Sage Weil
6f06e394e4 qa/tasks/ceph: wait for splits/merges before final scrub
Signed-off-by: Sage Weil <sage@redhat.com>
2018-12-19 14:37:01 -06:00
Yehuda Sadeh
b6b2e7f71a
Merge pull request #25056 from fghaas/doc-rgw-openstack
doc: Improve OpenStack integration and multitenancy docs for radosgw

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Kai Wagner <kwagner@suse.com>
Reviewed-by: Maxime Guyot <maxime@root314.com>
2018-12-19 12:28:55 -08:00
jtlayton
8dd93330e4
Merge pull request #25610 from jtlayton/wip-orchestrator
mgr/orchestrator: fix _list_services display
2018-12-19 14:11:46 -05:00
Sage Weil
b6822c8a36 Merge PR #25614 into master
* refs/pull/25614/head:
	cls/rgw: update rgw_cls_usage_log_trim_op encode version

Reviewed-by: Sage Weil <sage@redhat.com>
2018-12-19 11:45:29 -06:00