Commit Graph

83803 Commits

Author SHA1 Message Date
Sage Weil
f8d4e5b5e9 common/config: rewrite (╯°□°)╯︵ ┻━┻
This is too complete a rewrite to reasonably break down into small steps,
and even if I could, it would be harder to review that way than to simply
review the new implementation.  The semantics of the old one were so weird
that it's harder to reason about the change in behavior than to simply
review the new behavior.

That's my story, at least, and I'm sticking to it!

So, here are the highlights:

 - $foo meta expansions are evaluated at get_val() time.  This means the
weird bool arguments to set_val specifying whether things were expanded
are removed (they didn't make any sense unless you were thinking about the
old implementation).
 - for every option, we track values from any inputs (config, mon,
   override).  At get_val() time, we pick the highest priority one.
 - diff() is rewritten to be simple and to show you all of the above.
 - internal interfaces are simplified, and in terms of Option::value_t
   whenever possible.
 - unit tests simplified somewhat based on the above.

Known issues:

 - legacy values get pushed out in select cases.  Notably if foo=$bar
   and bar is updated, we do not update $foo (there is no dependency
   tracking to do this efficiently).

Signed-off-by: Sage Weil <sage@redhat.com>
2018-03-06 14:44:47 -06:00
Sage Weil
55f902f267 common/config: whitespace
Signed-off-by: Sage Weil <sage@redhat.com>
2018-03-06 14:44:10 -06:00
Sage Weil
dfd07f3820 common/config: streamline get_val_generic
Lookup value first; avoid a schema lookup entirely.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-03-06 14:44:10 -06:00
Sage Weil
db655f800a common: refactor default config value handling
- pass an optional map<string,string> instead of an arg list
- remember default values in md_config_t
- some nicer helpers

Signed-off-by: Sage Weil <sage@redhat.com>
2018-03-06 14:44:10 -06:00
Sage Weil
955537776b os/bluestore: pass rocksdb options via map, not global config
Signed-off-by: Sage Weil <sage@redhat.com>
2018-03-06 14:44:10 -06:00
Sage Weil
33a333ebaa kv/KeyValueDB: pass kv_options map to ctor
This will let us avoid passing options via global config options.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-03-06 14:44:09 -06:00
Sage Weil
e6612ade1d common/config: make debug_* regular options
Signed-off-by: Sage Weil <sage@redhat.com>
2018-03-06 14:44:09 -06:00
Sage Weil
08eaca61be crush/CrushWrapper: get_full_location by name
Signed-off-by: Sage Weil <sage@redhat.com>
2018-03-06 14:44:09 -06:00
Sage Weil
4161e804d7 common/config: move parse_value() into Option
It is more reuable there.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-03-06 14:44:09 -06:00
Sage Weil
02e57753e5 common/config: new get_val() variant
Signed-off-by: Sage Weil <sage@redhat.com>
2018-03-06 14:44:09 -06:00
Sage Weil
da11800d30 common/config: record is_daemon
Signed-off-by: Sage Weil <sage@redhat.com>
2018-03-06 14:44:09 -06:00
Sage Weil
64a5332766 common/config: find_option
Signed-off-by: Sage Weil <sage@redhat.com>
2018-03-06 14:44:09 -06:00
Sage Weil
004357a03b common: don't pass alt_def_args if we don't use them
Signed-off-by: Sage Weil <sage@redhat.com>
2018-03-06 14:44:09 -06:00
Sage Weil
35caf2e106 common/options: convert bool safe to a flags field
Signed-off-by: Sage Weil <sage@redhat.com>
2018-03-06 14:44:09 -06:00
Sage Weil
c391f476af mon/ConfigKeyService: make STORE_PREFIX public
Signed-off-by: Sage Weil <sage@redhat.com>
2018-03-06 14:44:09 -06:00
Sage Weil
dd216198b5
Merge pull request #20681 from liewegas/wip-fork-vs-msgr
common,rbd-nbd: fix up prefork behavior vs AsyncMessenger singletons

Reviewed-by: Haomai Wang <haomai@xsky.com>
2018-03-06 14:43:05 -06:00
Nathan Cutler
233fe8338e
Merge pull request #17530 from smithfarm/wip-21264
tools: cleanup: rip out ceph-rest-api

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>
2018-03-06 15:22:20 +01:00
Sage Weil
f4bb81553c Merge PR #20688 into wip-sage-testing-20180306.131906
* refs/pull/20688/head:
	doc: Drop the output of pg query
	doc: Update Monitoring OSDs and PGs

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2018-03-06 07:19:46 -06:00
Sage Weil
2a978666fc githubmap: update contributors
Signed-off-by: Sage Weil <sage@redhat.com>
2018-03-06 07:19:45 -06:00
Alfredo Deza
83348aef08
Merge pull request #20718 from ceph/wip-rm23163
ceph-volume tests alleviate libvirt timeouts when reloading

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
2018-03-06 08:04:46 -05:00
Kefu Chai
59195f12da
Merge pull request #20726 from b-ranto/wip-doc-upgrade
build-doc: Upgrade ceph python libraries

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-03-06 19:47:41 +08:00
Kefu Chai
23eb14de52
Merge pull request #20103 from openattic/wip-mgr-dashboard_v2
mgr/dashboard_v2: Initial submission of a web-based management UI (replacement for the existing dashboard)

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-03-06 19:08:57 +08:00
Kefu Chai
2e4f06ea54
Merge pull request #20659 from tchaikov/wip-msg/async/dpdk/launch-on-coreid
msg/async: execute on core specified by core_id not its index

Reviewed-by: shangfufei <shangfufei@inspur.com>
Reviewed-by: Haomai Wang <haomai@xsky.com>
2018-03-06 17:10:52 +08:00
Nathan Cutler
bebcdbeb76 tests: workunits: drop rest/test.py workunit
This workunit is only used by the two rest-api test cases that are also being removed.

Fixes: http://tracker.ceph.com/issues/21264
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-03-06 09:34:43 +01:00
Nathan Cutler
b69530e647 tests: rados suite: drop rest-api test cases
Fixes: http://tracker.ceph.com/issues/21264
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-03-06 06:58:59 +01:00
Gregory Farnum
017c846449
Merge pull request #20519 from gregsfortytwo/wip-22882-linger-locking
osdc/Objecter: take budgets across a LingerOp instead of on child Ops

Reviewed-by:  Jason Dillaman <dillaman@redhat.com>
2018-03-05 21:49:27 -08:00
Greg Farnum
aba55b5fb6 osdc: create and use take_linger_budget for prefetching watch/notify budgets
We don't actually take any budget, as we don't want to hit the
total op limits (especially on eg rbd-mirror), but this prevents
our per-message budgeting from causing a deadlock.

Fixes: http://tracker.ceph.com/issues/22882

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2018-03-05 21:45:42 -08:00
Gregory Farnum
7b9a715189
Merge pull request #20517 from gregsfortytwo/wip-22114-beacon-blocking-2
mon: mark OSD beacons and pg_create messages as no_reply

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Joao Eduardo Luis <joao@suse.de>
2018-03-05 21:29:10 -08:00
Jos Collin
3d6a61efff
Merge pull request #20720 from ctrlaltdel/typo
doc: Fix typo s/applicatoin/application/

Reviewed-by: Jos Collin <jcollin@redhat.com>
2018-03-06 09:22:19 +05:30
Kefu Chai
c454b1d1d6
Merge pull request #20535 from ifed01/wip-ifed-bring-fio
test/fio: enable objectstore FIO plugin building without the need to install and build FIO source code

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-03-06 08:36:39 +08:00
Kefu Chai
ade7193be8
Merge pull request #20708 from majianpeng/bluefs-bug-fix
os/bluestore: fix wrong usage for BlueFS::_allocate.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2018-03-06 08:31:07 +08:00
Jianpeng Ma
3e56ecb7b1 os/bluestore: fix wrong usage for BlueFS::_allocate.
In fact, function _allocate is atomic function which don't
allocate less than want space.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2018-03-06 16:11:17 +08:00
Jianpeng Ma
d1af5b060c os/bluestore/BlueFS: cleanup code for more readable.
And remove the useless judgement.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2018-03-06 16:11:17 +08:00
Kefu Chai
41ae53385b
Merge pull request #20699 from yaozongyou/fix-potential-memory-leak
common: fix potential memory leak in HTMLFormatter

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-03-06 08:29:06 +08:00
Kefu Chai
a1c4846fbc
Merge pull request #20701 from tchaikov/wip-23212
os/bluestore: recalc_allocated() when decoding bluefs_fnode_t

Reviewed-by: Jianpeng Ma <jianpeng.ma@intel.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2018-03-06 08:28:12 +08:00
Kefu Chai
f9a934a211
Merge pull request #20544 from liewegas/wip-min-kv
os/bluestore: kv_max -> kv_min

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2018-03-06 08:27:20 +08:00
Kefu Chai
a1fdc5e280
Merge pull request #20713 from smithfarm/wip-spdk-python3
spdk: advance to upstream dc82989d

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-03-06 08:22:05 +08:00
John Spray
86d7f3f72c
Merge pull request #20131 from jcsp/wip-doc-encoding
doc/dev: add a brief guide to serialization

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-03-05 22:33:34 +00:00
Boris Ranto
93cd098103 build-doc: Upgrade ceph python libraries
If you have python ceph bindings installed on your system, the doc-build
script will fail since pip detects them and fails to install the updated
bindings in the virtualenv. This fixes the issue by adding --upgrade
flag to pip so that it overrides the system-installed libraries.

Signed-off-by: Boris Ranto <branto@redhat.com>
2018-03-05 22:46:39 +01:00
Boris Ranto
ae36980ce4
Merge pull request #20717 from b-ranto/wip-restful-doc
doc: Improve mgr/restful module documentation

Reviewed-by: John Spray <john.spray@redhat.com>
2018-03-05 22:37:49 +01:00
Nathan Cutler
b7e638f7e4 build/ops: deb: move python-jinja2 dependency to mgr
87399bea83 introduced an explicit dependency on
python-jinja2, but mistakenly as an overall build dependency instead of as a
runtime dependency of ceph-mgr as intended.

Fixes: http://tracker.ceph.com/issues/22457
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-03-05 22:04:34 +01:00
Nathan Cutler
0dd34c81f2 doc: PendingReleaseNotes: deprecate/drop ceph-rest-api
References: http://tracker.ceph.com/issues/21264
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-03-05 22:01:51 +01:00
Nathan Cutler
495742f7dd tools: cleanup: rip out ceph-rest-api
Obsoleted by the mgr REST API.

Fixes: http://tracker.ceph.com/issues/21264
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2018-03-05 22:01:51 +01:00
Yuri Weinstein
8eb3c1f88e
Merge pull request #20542 from liewegas/wip-bluefs-buffered-io
common/options: bluefs_buffered_io=true by default

Reviewed-by: Mark Nelson <mnelson@redhat.com>
2018-03-05 09:43:36 -08:00
Ilya Dryomov
c13d196256
Merge pull request #20721 from idryomov/wip-krbd-parent-overlap-test
qa: krbd parent-overlap test

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-03-05 18:39:35 +01:00
Igor Fedotov
939805165e test/fio: enable objectstore FIO plugin building without the need to install and build FIO source code
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
2018-03-05 20:35:08 +03:00
Jason Dillaman
3ff15beb97
Merge pull request #20695 from mikechristie/wip-doc-mnc-update-iscsi-kernel
doc: update iSCSI upstream kernel to 4.16

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-03-05 12:27:09 -05:00
John Spray
d9a289849a
Merge pull request #20663 from tchaikov/wip-ceph.in-py3
ceph.in: use a different variable for holding thrown exception

Reviewed-by: John Spray <john.spray@redhat.com>
2018-03-05 15:36:27 +00:00
Francois Deppierraz
25edd6115a Doc: Fix typo s/applicatoin/application/
Signed-off-by: Francois Deppierraz <francois@ctrlaltdel.ch>
2018-03-05 15:44:19 +01:00
Ilya Dryomov
bd05dbd694
Merge pull request #20714 from idryomov/wip-krbd-msgr-segments-test
qa: krbd msgr-segments test

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-03-05 15:25:56 +01:00