Commit Graph

94004 Commits

Author SHA1 Message Date
Patrick Donnelly
cae0756639
Merge PR #25973 into master
* refs/pull/25973/head:
	qa: use simpler fs fail to bring fs down
	MDSMonitor: add fs fail command

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Douglas Fuller <dfuller@redhat.com>
2019-01-19 17:18:08 -08:00
Patrick Donnelly
f046f14f4a
Merge PR #25971 into master
* refs/pull/25971/head:
	qa: test hex dump_inode
	pybind/ceph_argparse: accept int with non-decimal base

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-01-19 17:17:04 -08:00
Patrick Donnelly
f8b2a5be06
Merge PR #25650 into master
* refs/pull/25650/head:
	client: add ceph_stat/ceph_lstat/ceph_fstat api

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
2019-01-19 17:16:17 -08:00
Patrick Donnelly
6e1d7c4b39
qa: use simpler fs fail to bring fs down
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-01-18 23:37:17 -08:00
David Galloway
51bfb6791f
Merge pull request #25919 from yuriw/wip-yuriw-distros-master
qa/tests: added rhel 7.6
2019-01-18 13:03:17 -05:00
Sage Weil
0a638eafd8 Merge PR #26019 into master
* refs/pull/26019/head:
	mon: implement hiding commands in ceph tool

Reviewed-by: Sage Weil <sage@redhat.com>
2019-01-18 11:27:07 -06:00
Casey Bodley
92ea0a6873
Merge pull request #25996 from yehudasa/wip-37885
qa/rgw/tempest: either force os_type or select random distro

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2019-01-18 11:45:29 -05:00
Noah Watkins
9afdedc25b
Merge pull request #26015 from noahdesu/osd-find-fsid
cli: dump osd-fsid as part of osd find <id>

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sébastien Han <seb@redhat.com>
2019-01-18 08:26:47 -08:00
David Zafman
99ddd3666b
Merge pull request #22797 from dzafman/wip-19753
osd: Deny reservation if expected backfill size would put us over bac…

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-01-18 07:42:00 -08:00
Sage Weil
c390230477 Merge PR #25132 into master
* refs/pull/25132/head:
	os/bluestore: implement kv update on bluefs extents info change.
	os/bluestore: use original algorithm to calculate BlueFS gift size for explicit allocation.
	os/bluestore: get rid of bluefs space gifting performed in background
	os/bluestore: do KV/FM/allocator opening in two stages.
	os/bluestore: detach FreelistManager from DB instance.
	test/osd/store_test: add test case for explicit bluefs expanding.
	os/bluestore: implement explicit bluefs space allocation at slow device
	os/bluestore: unify allocator dump on bluefs space failures
	os/bluestore: stop keeping bluefs_extents at KV

Reviewed-by: Sage Weil <sage@redhat.com>
2019-01-18 09:38:59 -06:00
Kefu Chai
011dd30934
Merge pull request #25972 from noahdesu/doc-release-ver-sort
doc: sort releases by date and version

Reviewed-by: Nathan Cutler <ncutler@suse.com>
2019-01-18 22:52:50 +08:00
Kefu Chai
3028c234c5
Merge pull request #25891 from ifed01/wip-ifed-fix-compress-errors
os/bluestore: do not assert on non-zero err codes from  compress() call

Reviewed-by: Sage Weil <sage@redhat.com>
2019-01-18 22:41:08 +08:00
Kefu Chai
f057e5e1fb
Merge pull request #15168 from qwren/qat-crypto
crypto: add openssl support for RGW encryption

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-01-18 22:39:13 +08:00
Kefu Chai
c8cf73015e
Merge pull request #25961 from p-na/wip-pna-fix-argparse
ceph_argparse: fix --verbose

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-01-18 22:38:21 +08:00
Alfredo Deza
73aafe5b20
Merge pull request #25324 from votdev/fix_raw_input
ceph-volume: Adapt code to support Python3

Reviewed-by: Alfredo Deza <adeza@redhat.com>
2019-01-18 09:19:27 -05:00
Sebastian Wagner
ad20b27034
Merge pull request #25912 from sebastian-philipp/orchestrator-drive-group
mgr/orchestrator: Extend DriveGroupSpec
2019-01-18 14:59:48 +01:00
Sebastian Wagner
a95327d90c
Merge pull request #26006 from jtlayton/wip-orchestrator
Clean up field names in ServiceDescription and add a service field

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2019-01-18 14:57:54 +01:00
Sage Weil
b6c0cec27a Merge PR #25981 into master
* refs/pull/25981/head:
	crush: switch to *get_or_create_class_id* helper whenever possible

Reviewed-by: Sage Weil <sage@redhat.com>
2019-01-18 07:16:36 -06:00
Sebastian Wagner
33e6f39b2f mgr/orchestrator: Extend DriveGroupSpec
Also: Added tox.

Co-authored-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2019-01-18 13:30:14 +01:00
Lenz Grimmer
4caeb717dc
Merge pull request #25602 from familyuu/pool
mgr/dashboard: Add pool cache tiering details tab

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2019-01-18 13:07:49 +01:00
Tim Serong
a969c9d406
Merge pull request #25651 from votdev/improve_basemgrmodule
mgr: Allow modules to get/set other module options

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2019-01-18 19:15:57 +10:00
Kefu Chai
1f8a97c448
Merge pull request #25304 from tchaikov/wip-crimson-osd
crimson/osd: crimson osd driver

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-01-18 14:07:42 +08:00
Kefu Chai
6d38cce8e5 crimson/osd: add mkfs()
we will need it for preparing crimson-osd without help of ceph-osd.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-18 12:42:08 +08:00
Kefu Chai
33928e0a38 crimson/osd: send beacon to mon periodically
so monitor won't mark crimson-osd down and out

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-18 12:42:08 +08:00
Kefu Chai
cd668c29e4 crimson/osd: enable crimson-osd to boot
* add state.h to encapsulate the state represeting different stages
  related to booting an OSD. the boot process of an OSD can be blocked
  by
  - waiting for PG consuming updated osdmaps
  - waiting for osdmaps marking osd.{whoami} up
  - waiting for new osdmaps to bring this osd up to speed.
  - waiting for current OSD to be healthy
  we could chain these "waits" in a more seastarized way, and let
OSD::start() wait on the future returned by this chain. but that'd
requires adding some seastar::shard_future<> as member variables of
`OSD` class, which is a little bit more convoluted than the state
machine approach used in this change. we could switch over to the
`future<>` chain approach, if we found that these futures could have
more consumers than merely `OSD::start()`.
* all osdmaps are now stored in an `std::map` in `OSD`, we can
  improve it by
  - caching it using an LRU cache
  - trimming the stale ones
  - persisting the evicted maps into the meta collection in ObjectStore
* superblock is not persited to store, neither is it read from the
  store.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-18 12:42:08 +08:00
Kefu Chai
2a732749d6 test/crimson: s/get_connection/connect_to/
in aa3d6f66, we removed Messenger::get_connection(), so update the test
accordingly.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-18 12:39:39 +08:00
Kefu Chai
4013dd73c3 messages/MOSDBeacon: include PaxosServiceMessage.h
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-18 12:39:39 +08:00
Kefu Chai
f847f6ac5d crimson: integrate mon::Client into OSD
and register them as dispatchers using ChainedDispatchers

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-18 12:39:37 +08:00
Kefu Chai
60f5b50f47 crimson: add ChainedDispatchers
it will be used to glue multiple dispatchers for subscribing events
emitted by messenger

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-18 12:36:59 +08:00
Kefu Chai
81d78fe5fe crimson: add prototype of crimson/osd
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-18 12:36:59 +08:00
Kefu Chai
9fae689b6e crimson: add a dummy object store
CyanStore only provides just-enough facility for booting an OSD.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-18 12:32:21 +08:00
Kefu Chai
fc21dcf4cf crimson: set src for message
monitor will panic at seeing a MOSDBoot message which is not from an
OSD. see OSDMonitor::preprocess_boot()
```
ceph_assert(m->get_orig_source_inst().name.is_osd());
```

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-18 12:32:21 +08:00
Kefu Chai
7875183c76 crimson: ignore v2 only addvec
crimson need to be v2 compatible. but let's leave this for another PR.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-18 12:32:21 +08:00
Kefu Chai
3aa8d90b36 cmake: silence -Wsign-compare in libfmt
* silence -Wsign-compare in libfmt
* silence -Wattributes in seastar

to silence warnings like:

/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/seastar/fmt/include/fmt/format.h:
In instantiation of 'constexpr bool fmt::v5::internal::find(Ptr, Ptr, T,
Ptr&) [with bool IS_CONSTEXPR = false; T = char; Ptr = const wchar_t*]':
/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/seastar/fmt/include/fmt/format.h:2154:32:
required from 'constexpr void
fmt::v5::internal::parse_format_string(fmt::v5::basic_string_view<Char>,
Handler&&)::writer::operator()(const Char*, const Char*) [with bool
IS_CONSTEXPR = false; Char = wchar_t; Handler =
fmt::v5::format_handler<fmt::v5::arg_formatter<fmt::v5::back_insert_range<fmt::v5::internal::basic_buffer<wchar_t>
> >, wchar_t,
fmt::v5::basic_format_context<std::back_insert_iterator<fmt::v5::internal::basic_buffer<wchar_t>
>, wchar_t> >&]'
/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/seastar/fmt/include/fmt/format.h:2164:3:
required from 'constexpr void
fmt::v5::internal::parse_format_string(fmt::v5::basic_string_view<Char>,
Handler&&) [with bool IS_CONSTEXPR = false; Char = wchar_t; Handler =
fmt::v5::format_handler<fmt::v5::arg_formatter<fmt::v5::back_insert_range<fmt::v5::internal::basic_buffer<wchar_t>
> >, wchar_t,
fmt::v5::basic_format_context<std::back_insert_iterator<fmt::v5::internal::basic_buffer<wchar_t>
>, wchar_t> >&]'
/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/seastar/fmt/include/fmt/format.h:3346:39:
required from 'typename Context::iterator fmt::v5::vformat_to(typename
ArgFormatter::range, fmt::v5::basic_string_view<Char>,
fmt::v5::basic_format_args<Context>) [with ArgFormatter =
fmt::v5::arg_formatter<fmt::v5::back_insert_range<fmt::v5::internal::basic_buffer<wchar_t>
> >; Char = wchar_t; Context =
fmt::v5::basic_format_context<std::back_insert_iterator<fmt::v5::internal::basic_buffer<wchar_t>
>, wchar_t>; typename Context::iterator =
std::back_insert_iterator<fmt::v5::internal::basic_buffer<wchar_t> >;
typename ArgFormatter::range =
fmt::v5::back_insert_range<fmt::v5::internal::basic_buffer<wchar_t> >]'
/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/seastar/fmt/include/fmt/format.h:3457:64:
required from here

/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/seastar/fmt/include/fmt/format.h:2120:14:
error: comparison of integer expressions of different signedness: 'const
wchar_t' and 'char' [-Werror=sign-compare]
     if (*out == value)
         ~~~~~^~~~~~~~
cc1plus: all warnings being treated as errors

where libfmt compares a wchar_t with a literal '}', which is char.
because the former is unsigned, and the latter is of a signed type,
GCC is annoyed. but since both of them are ASCII, and when performing
comparison, the signed one is converted to unsigned, then result of
comparison is correct per-se. hence, it's safe to silence this very
warning.

and warning like:

/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/seastar/src/core/future-util.cc:61:5:
required from 'seastar::future<> seastar::sleep_abortable(typename
Clock::duration, seastar::abort_source&) [with Clock =
std::chrono::_V2::steady_clock; typename Clock::duration =
std::chrono::duration<long int, std::ratio<1, 1000000000> >]'
/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/seastar/src/core/future-util.cc:68:105:
required from here
/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/seastar/src/core/future-util.cc:48:28:
error: 'seastar::sleep_abortable(typename Clock::duration,
seastar::abort_source&)::sleeper::sleeper(typename Clock::duration,
seastar::abort_source&) [with Clock = std::chrono::_V2::steady_clock;
typename Clock::duration = std::chrono::duration<long int, std::ratio<1,
1000000000> >]::<lambda()>' declared with greater visibility than the
type of its field 'seastar::sleep_abortable(typename Clock::duration,
seastar::abort_source&)::sleeper::sleeper(typename Clock::duration,
seastar::abort_source&) [with Clock = std::chrono::_V2::steady_clock;
typename Clock::duration = std::chrono::duration<long int, std::ratio<1,
1000000000> >]::<lambda()>::<this capture>' [-Werror=attributes]
                 : tmr([this] { done.set_value(); }) {
                            ^
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-18 12:32:21 +08:00
Kefu Chai
8e180acf95 seastar: pick up the cmake fix
to pick up the fix for seastar/issues/576

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-18 12:32:21 +08:00
Kefu Chai
71239019b2 crimson: set default keyring path
otherwise crimson-osd won't be able to load the keyring

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-18 12:32:21 +08:00
Kefu Chai
1d6b3e7344 crimson: add entity_addrvec_t interfaces to Messenger
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-18 12:32:19 +08:00
Kefu Chai
07365e71b3 crimson/monc: add subcription helpers
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-18 12:30:15 +08:00
Kefu Chai
90361af7e3 crimson/monc: add start() method
to boot strap mon::Client once seastar and msgr is ready. this allows us
to allocate mon::Client on stack.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-18 12:30:15 +08:00
Kefu Chai
9622fac190 test/crimson/monc: start/stop perf counter
in CephContext::CephContext(), we assume that
ceph::common::local_perf_coll() is ready when a CephContext is to be
contructed. so we need to prepare start it before creating CephContext.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-18 12:30:15 +08:00
Patrick Donnelly
0684ab14a4
mon: implement hiding commands in ceph tool
Otherwise ceph.in doing get_command_descriptions sees the command does not
exist and will print an error.

Fixes: https://tracker.ceph.com/issues/37956

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-01-17 20:12:10 -08:00
Sage Weil
71d629b5a5 Merge PR #25900 into master
* refs/pull/25900/head:
	qa/tasks/ceph.py: bracket addrvecs in mon_host etc
	vstart.sh: bracket addrvec on mon_host for msgr2-only mode
	unittest_addrs: entity_addr_t: strengthen tests slightly
	common/ceph_argparse: make parse_ip_port_vec handle list of addrs or addrvecs
	common/ceph_argparse: parse_ip_port_vec returns addrvecs, not addrds
	msg/msg_types: entity_addrvec_t: require brackets for size >1
	msg/msg_types: entity_addrvec_t: allow brackets when parsing addrvec to match output
	msg/msg_types: entity_addrvec_t: allow only ',' as an addrvec separator
	msg/msg_types: entity_addr_t: we should not parse an addrvec
	msg/msg_types: entity_addr_t: fix empty string parse cases
	msg/msg_types: entity_addr_t: is_ipv6() and is_ipv4()

Reviewed-by: Ricardo Dias <rdias@suse.com>
2019-01-17 21:02:59 -06:00
Sage Weil
39f9e8db4d Merge PR #25998 into master
* refs/pull/25998/head:
	msg/Dispatcher: remove force_new arg from ms_get_authorizer()
	crimson/net: drop authenticator retry
	msg/simple: remove forced authorizer refresh

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-01-17 18:51:33 -06:00
Sage Weil
4f48314a7a Merge PR #25937 into master
* refs/pull/25937/head:
	qa: specify filestore for misc tests
	osd/objectstore: change osd_objectstore default to bluestore

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-01-17 18:51:16 -06:00
Xie Xingguo
00b7ac8670
Merge pull request #25693 from shun-s/silence-mon-load-metadata
MDSMonitor: silence unable to load metadata

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
2019-01-18 08:49:23 +08:00
Noah Watkins
4d01b2c96e cli: dump osd-fsid as part of osd find <id>
Dumps the osd-fsid uuid as part of the `osd find <id>` command.
Currently this uuid is only available as part of `osd dump` but
ceph-ansible has a use case to interrogate a single osd without needing
the entire osdmap dump.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2019-01-17 11:16:44 -08:00
Sage Weil
b5e5ee6f40 Merge PR #25849 into master
* refs/pull/25849/head:
	qa/suites/rados/upgrade: one mon per node, and enable-msgr2 at end
	qa/rados/thrash-old-clients: avoid msgr2
	mon: make bootstrap rank check more robust
	mon: clean up probe debug output a bit
	msg/async: use v1 for v1 <-> [v2,v1] peers
	msg/async/AsyncMessenger: drop single-use _send_to
	mon/HealthMonitor: raise MON_MSGR2_NOT_ENABLED if mons not bound to msgr2
	doc/rados/operations/health-checks: document MON_* health warnings
	mon/MonMapMonitor: add 'mon enable-msgr2' command
	mon: respawn if rank addr changes
	mon/MonMap: calc_addr_mons() after setting rank addrvec

Reviewed-by: Ricardo Dias <rdias@suse.com>
2019-01-17 11:04:30 -06:00
Sage Weil
7a89787cde qa/suites/rados/upgrade: one mon per node, and enable-msgr2 at end
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-17 10:03:14 -06:00
Sage Weil
f7b6a4b440 qa/rados/thrash-old-clients: avoid msgr2
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-17 10:03:14 -06:00
Patrick Donnelly
b62daf3b25
qa: test hex dump_inode
Fixes: http://tracker.ceph.com/issues/24721

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-01-17 07:38:12 -08:00