Commit Graph

83810 Commits

Author SHA1 Message Date
Sage Weil
7940f66b64 mon/MonClient: subscribe to config, handle updates
Signed-off-by: Sage Weil <sage@redhat.com>
2018-03-06 14:44:48 -06:00
Sage Weil
38db0a289f mon/ConfigMonitor: add monitor service for distributing configs
Add 'config' subscribe option to get MConfigs on auth and when config is
updated.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-03-06 14:44:48 -06:00
Sage Weil
43a9d164d6 mon/ConfigMap: parsed cluster-wide config
Breaks down options by global, type, daemon, and can filter options by
crush location.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-03-06 14:44:48 -06:00
Sage Weil
081e52c7bf messages/MConfig: send config map
Signed-off-by: Sage Weil <sage@redhat.com>
2018-03-06 14:44:47 -06:00
Sage Weil
9647e50886 common/config: set_mon_vals()
Signed-off-by: Sage Weil <sage@redhat.com>
2018-03-06 14:44:47 -06:00
Sage Weil
37fe36fbe5 common/config: some cleanup; rm_val
Signed-off-by: Sage Weil <sage@redhat.com>
2018-03-06 14:44:47 -06:00
Sage Weil
d4cc161566 cleanup: src/.libs -> build/lib
Clean up remaining traces of automake's .libs.  This makes unittest_*
tests easier to run by hand (mostly).

Signed-off-by: Sage Weil <sage@redhat.com>
2018-03-06 14:44:47 -06:00
Sage Weil
f8d4e5b5e9 common/config: rewrite (╯°□°)╯︵ ┻━┻
This is too complete a rewrite to reasonably break down into small steps,
and even if I could, it would be harder to review that way than to simply
review the new implementation.  The semantics of the old one were so weird
that it's harder to reason about the change in behavior than to simply
review the new behavior.

That's my story, at least, and I'm sticking to it!

So, here are the highlights:

 - $foo meta expansions are evaluated at get_val() time.  This means the
weird bool arguments to set_val specifying whether things were expanded
are removed (they didn't make any sense unless you were thinking about the
old implementation).
 - for every option, we track values from any inputs (config, mon,
   override).  At get_val() time, we pick the highest priority one.
 - diff() is rewritten to be simple and to show you all of the above.
 - internal interfaces are simplified, and in terms of Option::value_t
   whenever possible.
 - unit tests simplified somewhat based on the above.

Known issues:

 - legacy values get pushed out in select cases.  Notably if foo=$bar
   and bar is updated, we do not update $foo (there is no dependency
   tracking to do this efficiently).

Signed-off-by: Sage Weil <sage@redhat.com>
2018-03-06 14:44:47 -06:00
Sage Weil
55f902f267 common/config: whitespace
Signed-off-by: Sage Weil <sage@redhat.com>
2018-03-06 14:44:10 -06:00
Sage Weil
dfd07f3820 common/config: streamline get_val_generic
Lookup value first; avoid a schema lookup entirely.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-03-06 14:44:10 -06:00
Sage Weil
db655f800a common: refactor default config value handling
- pass an optional map<string,string> instead of an arg list
- remember default values in md_config_t
- some nicer helpers

Signed-off-by: Sage Weil <sage@redhat.com>
2018-03-06 14:44:10 -06:00
Sage Weil
955537776b os/bluestore: pass rocksdb options via map, not global config
Signed-off-by: Sage Weil <sage@redhat.com>
2018-03-06 14:44:10 -06:00
Sage Weil
33a333ebaa kv/KeyValueDB: pass kv_options map to ctor
This will let us avoid passing options via global config options.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-03-06 14:44:09 -06:00
Sage Weil
e6612ade1d common/config: make debug_* regular options
Signed-off-by: Sage Weil <sage@redhat.com>
2018-03-06 14:44:09 -06:00
Sage Weil
08eaca61be crush/CrushWrapper: get_full_location by name
Signed-off-by: Sage Weil <sage@redhat.com>
2018-03-06 14:44:09 -06:00
Sage Weil
4161e804d7 common/config: move parse_value() into Option
It is more reuable there.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-03-06 14:44:09 -06:00
Sage Weil
02e57753e5 common/config: new get_val() variant
Signed-off-by: Sage Weil <sage@redhat.com>
2018-03-06 14:44:09 -06:00
Sage Weil
da11800d30 common/config: record is_daemon
Signed-off-by: Sage Weil <sage@redhat.com>
2018-03-06 14:44:09 -06:00
Sage Weil
64a5332766 common/config: find_option
Signed-off-by: Sage Weil <sage@redhat.com>
2018-03-06 14:44:09 -06:00
Sage Weil
004357a03b common: don't pass alt_def_args if we don't use them
Signed-off-by: Sage Weil <sage@redhat.com>
2018-03-06 14:44:09 -06:00
Sage Weil
35caf2e106 common/options: convert bool safe to a flags field
Signed-off-by: Sage Weil <sage@redhat.com>
2018-03-06 14:44:09 -06:00
Sage Weil
c391f476af mon/ConfigKeyService: make STORE_PREFIX public
Signed-off-by: Sage Weil <sage@redhat.com>
2018-03-06 14:44:09 -06:00
Sage Weil
dd216198b5
Merge pull request #20681 from liewegas/wip-fork-vs-msgr
common,rbd-nbd: fix up prefork behavior vs AsyncMessenger singletons

Reviewed-by: Haomai Wang <haomai@xsky.com>
2018-03-06 14:43:05 -06:00
Nathan Cutler
233fe8338e
Merge pull request #17530 from smithfarm/wip-21264
tools: cleanup: rip out ceph-rest-api

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>
2018-03-06 15:22:20 +01:00
Sage Weil
f4bb81553c Merge PR #20688 into wip-sage-testing-20180306.131906
* refs/pull/20688/head:
	doc: Drop the output of pg query
	doc: Update Monitoring OSDs and PGs

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2018-03-06 07:19:46 -06:00
Sage Weil
2a978666fc githubmap: update contributors
Signed-off-by: Sage Weil <sage@redhat.com>
2018-03-06 07:19:45 -06:00
Alfredo Deza
83348aef08
Merge pull request #20718 from ceph/wip-rm23163
ceph-volume tests alleviate libvirt timeouts when reloading

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
2018-03-06 08:04:46 -05:00
Kefu Chai
59195f12da
Merge pull request #20726 from b-ranto/wip-doc-upgrade
build-doc: Upgrade ceph python libraries

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-03-06 19:47:41 +08:00
Kefu Chai
23eb14de52
Merge pull request #20103 from openattic/wip-mgr-dashboard_v2
mgr/dashboard_v2: Initial submission of a web-based management UI (replacement for the existing dashboard)

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-03-06 19:08:57 +08:00
Kefu Chai
2e4f06ea54
Merge pull request #20659 from tchaikov/wip-msg/async/dpdk/launch-on-coreid
msg/async: execute on core specified by core_id not its index

Reviewed-by: shangfufei <shangfufei@inspur.com>
Reviewed-by: Haomai Wang <haomai@xsky.com>
2018-03-06 17:10:52 +08:00
Nathan Cutler
bebcdbeb76 tests: workunits: drop rest/test.py workunit
This workunit is only used by the two rest-api test cases that are also being removed.

Fixes: http://tracker.ceph.com/issues/21264
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-03-06 09:34:43 +01:00
Nathan Cutler
b69530e647 tests: rados suite: drop rest-api test cases
Fixes: http://tracker.ceph.com/issues/21264
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-03-06 06:58:59 +01:00
Gregory Farnum
017c846449
Merge pull request #20519 from gregsfortytwo/wip-22882-linger-locking
osdc/Objecter: take budgets across a LingerOp instead of on child Ops

Reviewed-by:  Jason Dillaman <dillaman@redhat.com>
2018-03-05 21:49:27 -08:00
Greg Farnum
aba55b5fb6 osdc: create and use take_linger_budget for prefetching watch/notify budgets
We don't actually take any budget, as we don't want to hit the
total op limits (especially on eg rbd-mirror), but this prevents
our per-message budgeting from causing a deadlock.

Fixes: http://tracker.ceph.com/issues/22882

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2018-03-05 21:45:42 -08:00
Gregory Farnum
7b9a715189
Merge pull request #20517 from gregsfortytwo/wip-22114-beacon-blocking-2
mon: mark OSD beacons and pg_create messages as no_reply

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Joao Eduardo Luis <joao@suse.de>
2018-03-05 21:29:10 -08:00
Jos Collin
3d6a61efff
Merge pull request #20720 from ctrlaltdel/typo
doc: Fix typo s/applicatoin/application/

Reviewed-by: Jos Collin <jcollin@redhat.com>
2018-03-06 09:22:19 +05:30
Kefu Chai
c454b1d1d6
Merge pull request #20535 from ifed01/wip-ifed-bring-fio
test/fio: enable objectstore FIO plugin building without the need to install and build FIO source code

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-03-06 08:36:39 +08:00
Kefu Chai
ade7193be8
Merge pull request #20708 from majianpeng/bluefs-bug-fix
os/bluestore: fix wrong usage for BlueFS::_allocate.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2018-03-06 08:31:07 +08:00
Jianpeng Ma
3e56ecb7b1 os/bluestore: fix wrong usage for BlueFS::_allocate.
In fact, function _allocate is atomic function which don't
allocate less than want space.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2018-03-06 16:11:17 +08:00
Jianpeng Ma
d1af5b060c os/bluestore/BlueFS: cleanup code for more readable.
And remove the useless judgement.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2018-03-06 16:11:17 +08:00
Kefu Chai
41ae53385b
Merge pull request #20699 from yaozongyou/fix-potential-memory-leak
common: fix potential memory leak in HTMLFormatter

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-03-06 08:29:06 +08:00
Kefu Chai
a1c4846fbc
Merge pull request #20701 from tchaikov/wip-23212
os/bluestore: recalc_allocated() when decoding bluefs_fnode_t

Reviewed-by: Jianpeng Ma <jianpeng.ma@intel.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2018-03-06 08:28:12 +08:00
Kefu Chai
f9a934a211
Merge pull request #20544 from liewegas/wip-min-kv
os/bluestore: kv_max -> kv_min

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2018-03-06 08:27:20 +08:00
Kefu Chai
a1fdc5e280
Merge pull request #20713 from smithfarm/wip-spdk-python3
spdk: advance to upstream dc82989d

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-03-06 08:22:05 +08:00
John Spray
86d7f3f72c
Merge pull request #20131 from jcsp/wip-doc-encoding
doc/dev: add a brief guide to serialization

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-03-05 22:33:34 +00:00
Boris Ranto
93cd098103 build-doc: Upgrade ceph python libraries
If you have python ceph bindings installed on your system, the doc-build
script will fail since pip detects them and fails to install the updated
bindings in the virtualenv. This fixes the issue by adding --upgrade
flag to pip so that it overrides the system-installed libraries.

Signed-off-by: Boris Ranto <branto@redhat.com>
2018-03-05 22:46:39 +01:00
Boris Ranto
ae36980ce4
Merge pull request #20717 from b-ranto/wip-restful-doc
doc: Improve mgr/restful module documentation

Reviewed-by: John Spray <john.spray@redhat.com>
2018-03-05 22:37:49 +01:00
Nathan Cutler
b7e638f7e4 build/ops: deb: move python-jinja2 dependency to mgr
87399bea83 introduced an explicit dependency on
python-jinja2, but mistakenly as an overall build dependency instead of as a
runtime dependency of ceph-mgr as intended.

Fixes: http://tracker.ceph.com/issues/22457
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-03-05 22:04:34 +01:00
Nathan Cutler
0dd34c81f2 doc: PendingReleaseNotes: deprecate/drop ceph-rest-api
References: http://tracker.ceph.com/issues/21264
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-03-05 22:01:51 +01:00
Nathan Cutler
495742f7dd tools: cleanup: rip out ceph-rest-api
Obsoleted by the mgr REST API.

Fixes: http://tracker.ceph.com/issues/21264
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-03-05 22:01:51 +01:00