Commit Graph

93952 Commits

Author SHA1 Message Date
Sage Weil
b6c0cec27a Merge PR #25981 into master
* refs/pull/25981/head:
	crush: switch to *get_or_create_class_id* helper whenever possible

Reviewed-by: Sage Weil <sage@redhat.com>
2019-01-18 07:16:36 -06:00
Lenz Grimmer
4caeb717dc
Merge pull request #25602 from familyuu/pool
mgr/dashboard: Add pool cache tiering details tab

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2019-01-18 13:07:49 +01:00
Tim Serong
a969c9d406
Merge pull request #25651 from votdev/improve_basemgrmodule
mgr: Allow modules to get/set other module options

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2019-01-18 19:15:57 +10:00
Kefu Chai
1f8a97c448
Merge pull request #25304 from tchaikov/wip-crimson-osd
crimson/osd: crimson osd driver

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-01-18 14:07:42 +08:00
Kefu Chai
6d38cce8e5 crimson/osd: add mkfs()
we will need it for preparing crimson-osd without help of ceph-osd.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-18 12:42:08 +08:00
Kefu Chai
33928e0a38 crimson/osd: send beacon to mon periodically
so monitor won't mark crimson-osd down and out

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-18 12:42:08 +08:00
Kefu Chai
cd668c29e4 crimson/osd: enable crimson-osd to boot
* add state.h to encapsulate the state represeting different stages
  related to booting an OSD. the boot process of an OSD can be blocked
  by
  - waiting for PG consuming updated osdmaps
  - waiting for osdmaps marking osd.{whoami} up
  - waiting for new osdmaps to bring this osd up to speed.
  - waiting for current OSD to be healthy
  we could chain these "waits" in a more seastarized way, and let
OSD::start() wait on the future returned by this chain. but that'd
requires adding some seastar::shard_future<> as member variables of
`OSD` class, which is a little bit more convoluted than the state
machine approach used in this change. we could switch over to the
`future<>` chain approach, if we found that these futures could have
more consumers than merely `OSD::start()`.
* all osdmaps are now stored in an `std::map` in `OSD`, we can
  improve it by
  - caching it using an LRU cache
  - trimming the stale ones
  - persisting the evicted maps into the meta collection in ObjectStore
* superblock is not persited to store, neither is it read from the
  store.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-18 12:42:08 +08:00
Kefu Chai
2a732749d6 test/crimson: s/get_connection/connect_to/
in aa3d6f66, we removed Messenger::get_connection(), so update the test
accordingly.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-18 12:39:39 +08:00
Kefu Chai
4013dd73c3 messages/MOSDBeacon: include PaxosServiceMessage.h
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-18 12:39:39 +08:00
Kefu Chai
f847f6ac5d crimson: integrate mon::Client into OSD
and register them as dispatchers using ChainedDispatchers

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-18 12:39:37 +08:00
Kefu Chai
60f5b50f47 crimson: add ChainedDispatchers
it will be used to glue multiple dispatchers for subscribing events
emitted by messenger

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-18 12:36:59 +08:00
Kefu Chai
81d78fe5fe crimson: add prototype of crimson/osd
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-18 12:36:59 +08:00
Kefu Chai
9fae689b6e crimson: add a dummy object store
CyanStore only provides just-enough facility for booting an OSD.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-18 12:32:21 +08:00
Kefu Chai
fc21dcf4cf crimson: set src for message
monitor will panic at seeing a MOSDBoot message which is not from an
OSD. see OSDMonitor::preprocess_boot()
```
ceph_assert(m->get_orig_source_inst().name.is_osd());
```

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-18 12:32:21 +08:00
Kefu Chai
7875183c76 crimson: ignore v2 only addvec
crimson need to be v2 compatible. but let's leave this for another PR.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-18 12:32:21 +08:00
Kefu Chai
3aa8d90b36 cmake: silence -Wsign-compare in libfmt
* silence -Wsign-compare in libfmt
* silence -Wattributes in seastar

to silence warnings like:

/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/seastar/fmt/include/fmt/format.h:
In instantiation of 'constexpr bool fmt::v5::internal::find(Ptr, Ptr, T,
Ptr&) [with bool IS_CONSTEXPR = false; T = char; Ptr = const wchar_t*]':
/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/seastar/fmt/include/fmt/format.h:2154:32:
required from 'constexpr void
fmt::v5::internal::parse_format_string(fmt::v5::basic_string_view<Char>,
Handler&&)::writer::operator()(const Char*, const Char*) [with bool
IS_CONSTEXPR = false; Char = wchar_t; Handler =
fmt::v5::format_handler<fmt::v5::arg_formatter<fmt::v5::back_insert_range<fmt::v5::internal::basic_buffer<wchar_t>
> >, wchar_t,
fmt::v5::basic_format_context<std::back_insert_iterator<fmt::v5::internal::basic_buffer<wchar_t>
>, wchar_t> >&]'
/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/seastar/fmt/include/fmt/format.h:2164:3:
required from 'constexpr void
fmt::v5::internal::parse_format_string(fmt::v5::basic_string_view<Char>,
Handler&&) [with bool IS_CONSTEXPR = false; Char = wchar_t; Handler =
fmt::v5::format_handler<fmt::v5::arg_formatter<fmt::v5::back_insert_range<fmt::v5::internal::basic_buffer<wchar_t>
> >, wchar_t,
fmt::v5::basic_format_context<std::back_insert_iterator<fmt::v5::internal::basic_buffer<wchar_t>
>, wchar_t> >&]'
/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/seastar/fmt/include/fmt/format.h:3346:39:
required from 'typename Context::iterator fmt::v5::vformat_to(typename
ArgFormatter::range, fmt::v5::basic_string_view<Char>,
fmt::v5::basic_format_args<Context>) [with ArgFormatter =
fmt::v5::arg_formatter<fmt::v5::back_insert_range<fmt::v5::internal::basic_buffer<wchar_t>
> >; Char = wchar_t; Context =
fmt::v5::basic_format_context<std::back_insert_iterator<fmt::v5::internal::basic_buffer<wchar_t>
>, wchar_t>; typename Context::iterator =
std::back_insert_iterator<fmt::v5::internal::basic_buffer<wchar_t> >;
typename ArgFormatter::range =
fmt::v5::back_insert_range<fmt::v5::internal::basic_buffer<wchar_t> >]'
/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/seastar/fmt/include/fmt/format.h:3457:64:
required from here

/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/seastar/fmt/include/fmt/format.h:2120:14:
error: comparison of integer expressions of different signedness: 'const
wchar_t' and 'char' [-Werror=sign-compare]
     if (*out == value)
         ~~~~~^~~~~~~~
cc1plus: all warnings being treated as errors

where libfmt compares a wchar_t with a literal '}', which is char.
because the former is unsigned, and the latter is of a signed type,
GCC is annoyed. but since both of them are ASCII, and when performing
comparison, the signed one is converted to unsigned, then result of
comparison is correct per-se. hence, it's safe to silence this very
warning.

and warning like:

/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/seastar/src/core/future-util.cc:61:5:
required from 'seastar::future<> seastar::sleep_abortable(typename
Clock::duration, seastar::abort_source&) [with Clock =
std::chrono::_V2::steady_clock; typename Clock::duration =
std::chrono::duration<long int, std::ratio<1, 1000000000> >]'
/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/seastar/src/core/future-util.cc:68:105:
required from here
/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/seastar/src/core/future-util.cc:48:28:
error: 'seastar::sleep_abortable(typename Clock::duration,
seastar::abort_source&)::sleeper::sleeper(typename Clock::duration,
seastar::abort_source&) [with Clock = std::chrono::_V2::steady_clock;
typename Clock::duration = std::chrono::duration<long int, std::ratio<1,
1000000000> >]::<lambda()>' declared with greater visibility than the
type of its field 'seastar::sleep_abortable(typename Clock::duration,
seastar::abort_source&)::sleeper::sleeper(typename Clock::duration,
seastar::abort_source&) [with Clock = std::chrono::_V2::steady_clock;
typename Clock::duration = std::chrono::duration<long int, std::ratio<1,
1000000000> >]::<lambda()>::<this capture>' [-Werror=attributes]
                 : tmr([this] { done.set_value(); }) {
                            ^
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-18 12:32:21 +08:00
Kefu Chai
8e180acf95 seastar: pick up the cmake fix
to pick up the fix for seastar/issues/576

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-18 12:32:21 +08:00
Kefu Chai
71239019b2 crimson: set default keyring path
otherwise crimson-osd won't be able to load the keyring

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-18 12:32:21 +08:00
Kefu Chai
1d6b3e7344 crimson: add entity_addrvec_t interfaces to Messenger
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-18 12:32:19 +08:00
Kefu Chai
07365e71b3 crimson/monc: add subcription helpers
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-18 12:30:15 +08:00
Kefu Chai
90361af7e3 crimson/monc: add start() method
to boot strap mon::Client once seastar and msgr is ready. this allows us
to allocate mon::Client on stack.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-18 12:30:15 +08:00
Kefu Chai
9622fac190 test/crimson/monc: start/stop perf counter
in CephContext::CephContext(), we assume that
ceph::common::local_perf_coll() is ready when a CephContext is to be
contructed. so we need to prepare start it before creating CephContext.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-18 12:30:15 +08:00
Sage Weil
71d629b5a5 Merge PR #25900 into master
* refs/pull/25900/head:
	qa/tasks/ceph.py: bracket addrvecs in mon_host etc
	vstart.sh: bracket addrvec on mon_host for msgr2-only mode
	unittest_addrs: entity_addr_t: strengthen tests slightly
	common/ceph_argparse: make parse_ip_port_vec handle list of addrs or addrvecs
	common/ceph_argparse: parse_ip_port_vec returns addrvecs, not addrds
	msg/msg_types: entity_addrvec_t: require brackets for size >1
	msg/msg_types: entity_addrvec_t: allow brackets when parsing addrvec to match output
	msg/msg_types: entity_addrvec_t: allow only ',' as an addrvec separator
	msg/msg_types: entity_addr_t: we should not parse an addrvec
	msg/msg_types: entity_addr_t: fix empty string parse cases
	msg/msg_types: entity_addr_t: is_ipv6() and is_ipv4()

Reviewed-by: Ricardo Dias <rdias@suse.com>
2019-01-17 21:02:59 -06:00
Sage Weil
39f9e8db4d Merge PR #25998 into master
* refs/pull/25998/head:
	msg/Dispatcher: remove force_new arg from ms_get_authorizer()
	crimson/net: drop authenticator retry
	msg/simple: remove forced authorizer refresh

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-01-17 18:51:33 -06:00
Sage Weil
4f48314a7a Merge PR #25937 into master
* refs/pull/25937/head:
	qa: specify filestore for misc tests
	osd/objectstore: change osd_objectstore default to bluestore

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-01-17 18:51:16 -06:00
Xie Xingguo
00b7ac8670
Merge pull request #25693 from shun-s/silence-mon-load-metadata
MDSMonitor: silence unable to load metadata

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
2019-01-18 08:49:23 +08:00
Sage Weil
b5e5ee6f40 Merge PR #25849 into master
* refs/pull/25849/head:
	qa/suites/rados/upgrade: one mon per node, and enable-msgr2 at end
	qa/rados/thrash-old-clients: avoid msgr2
	mon: make bootstrap rank check more robust
	mon: clean up probe debug output a bit
	msg/async: use v1 for v1 <-> [v2,v1] peers
	msg/async/AsyncMessenger: drop single-use _send_to
	mon/HealthMonitor: raise MON_MSGR2_NOT_ENABLED if mons not bound to msgr2
	doc/rados/operations/health-checks: document MON_* health warnings
	mon/MonMapMonitor: add 'mon enable-msgr2' command
	mon: respawn if rank addr changes
	mon/MonMap: calc_addr_mons() after setting rank addrvec

Reviewed-by: Ricardo Dias <rdias@suse.com>
2019-01-17 11:04:30 -06:00
Sage Weil
7a89787cde qa/suites/rados/upgrade: one mon per node, and enable-msgr2 at end
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-17 10:03:14 -06:00
Sage Weil
f7b6a4b440 qa/rados/thrash-old-clients: avoid msgr2
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-17 10:03:14 -06:00
Lenz Grimmer
a1d6f9c72d
Merge pull request #25654 from tspmelo/wip-missing-i18n
mgr/dashboard: Add missing frontend I18N

Reviewed-by: Sebastian Krah <skrah@suse.com>
2019-01-17 16:33:43 +01:00
Lenz Grimmer
1efa21c41d
Merge pull request #25997 from zmc/wip-db-graf-validate
mgr/dashboard: Fix /api/grafana/validation

Reviewed-by: Volker Theile <vtheile@suse.com>
2019-01-17 16:25:56 +01:00
Sage Weil
e62f45d1ba msg/Dispatcher: remove force_new arg from ms_get_authorizer()
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-17 06:33:32 -06:00
Kefu Chai
a37d19dc9b crimson/monc: set name using a setter
* set entity_name using a setter not pass it to constructor, because
  the entity_name is retrieved in seastar's app.run() by ConfigProxy,
  while it'd be simpler if we can instantiate mon::Client in main()
  as a local variable, instead of managing it on heap using a smart
  pointer. so we cannot pass the entity_name as a parameter of ctor.
* also cleanup the #include's, as they are included already in the
  header.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-17 20:23:13 +08:00
Kefu Chai
6621f79997 crimson: pass entity_name and cluster to ctor of ConfigProxy
as we always need to set entity_name and cluster before start using
ConfigProxy, and we do not read these settings from config file,
this makes these two setting special. so it'd be simpler to just
pass them as parameters of constructor.

and more importantly, we need to parse the command line arguments
using ceph_argparse_early_args() first, as it will consume the parsed
arguments, and leave the unparsed ones in the input parameter of `args`,
and then, we can pass the unparsed args to app.run().

it's not a perfect solution. as there are some options that both parsers
are interested, for instance, `-c` -- ceph take it as the conf file's
path, while seastar takes it as the number of cores to use. but let's
feed ceph's parser first. unless it's fine to drop the backward
compatibility of command line syntax of ceph-osd.

Signed-off-by: Kefu Chai <kchai@redhat.com>
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2019-01-17 20:23:12 +08:00
Kefu Chai
61d41cf099 crimson/auth: add domain specific error types
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-17 20:23:12 +08:00
Lenz Grimmer
bb4e49bacd
Merge pull request #25870 from votdev/validate_ip_addr
mgr/dashboard: Improve RGW address parser

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
2019-01-17 13:08:48 +01:00
Xie Xingguo
66712c360c
Merge pull request #25959 from xiexingguo/wip-memory-leak
crush: fix memory leak

Reviewed-by: Sage Weil <sage@redhat.com>
2019-01-17 18:23:23 +08:00
Xie Xingguo
18b3f73ecb
Merge pull request #25941 from xiexingguo/wip-warn-enable-upmap
mgr/balancer: blame if upmap won't actually work

Reviewed-by: Sage Weil <sage@redhat.com>
2019-01-17 18:22:28 +08:00
Lenz Grimmer
d6826b892e
Merge pull request #25836 from votdev/improve_rgw_status_endpoint
mgr/dashboard: Improve exception handling in /api/rgw/status

Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2019-01-17 09:05:34 +01:00
Gregory Farnum
619304ba83
Merge pull request #25950 from gregsfortytwo/wip-githubmap
Wip githubmap
2019-01-16 21:09:29 -08:00
Kefu Chai
035e9f534a
Merge pull request #25751 from batrick/stackstringstream-reset
common: reinit StackStringStream on clear

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-01-17 10:43:35 +08:00
xie xingguo
93203fd5de crush: switch to *get_or_create_class_id* helper whenever possible
Because:
- it is simpler
- it is safer

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-01-17 10:41:43 +08:00
Song Shun
c4c73e686c MDSMon: silence unable to load metadata
silence unable to load metadata

Signed-off-by: Song Shun <song.shun3@zte.com.cn>
2019-01-17 09:02:00 +08:00
Sage Weil
7bb583f2bd crimson/net: drop authenticator retry
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-16 15:44:06 -06:00
Sage Weil
708f020115 msg/simple: remove forced authorizer refresh
This synchronous check has always been kludgey; remove it and just fault
instead, just like we did with 794a8f9cf5
in AsyncMessenger.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-16 15:39:53 -06:00
Patrick Donnelly
b57f896ed5
Merge PR #25994 into master
* refs/pull/25994/head:
	qa: silence read-only WRN for damage testing

Reviewed-by: Jeff Layton <jlayton@redhat.com>
2019-01-16 12:43:35 -08:00
Zack Cerza
132eb8b798 mgr/dashboard: Fix /api/grafana/validation
If the Grafana URL contains a trailing slash, this endpoint ends up
returning a 404, disabling the integration entirely. Fix that.

Signed-off-by: Zack Cerza <zack@redhat.com>
2019-01-16 13:36:23 -07:00
Patrick Donnelly
ac302de7b7
qa: silence read-only WRN for damage testing
Fixes: http://tracker.ceph.com/issues/37944

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-01-16 11:55:54 -08:00
Sage Weil
63ba6d249a Merge PR #25934 into master
* refs/pull/25934/head:
	msg/msg_type: entity_addr_t: fix legacy decode
	msg/msg_types: make set_sockaddr() work with AF_UNSPEC (i.e., zeroed)
	msg/msg_types: make set_sockaddr() a bit more robust
	msg/async: fix IP inference

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-01-16 13:12:35 -06:00
Vikhyat Umrao
8a694fc2f9 qa: specify filestore for misc tests
Signed-off-by: Vikhyat Umrao <vumrao@redhat.com>
Signed-off-by: Sage Weil <sage@redhat.com>
2019-01-16 13:09:19 -06:00