Commit Graph

93441 Commits

Author SHA1 Message Date
Kefu Chai
953abf78e7
Merge pull request #25726 from tchaikov/wip-37751
osd: unlock osd_lock when tweaking osd settings

Reviewed-by: Sage Weil <sage@redhat.com>
2019-01-01 23:49:27 +08:00
Kefu Chai
edfc199473
Merge pull request #25733 from tchaikov/wip-37719-workaround
librbd: workaround an ICE of GCC

Reviewed-by: Sage Weil <sage@redhat.com>
2019-01-01 23:44:23 +08:00
Kefu Chai
98a7674369 crimson: workaround an ICE of GCC
this change works around the FTBFS on arm64:

/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/crimson/common/config_proxy.h:74:13:
internal compiler error: in tsubst_decomp_names, at cp/pt.c:16537
             for (auto& [obs, keys] : rev_obs) {
             ^~~
Please submit a full bug report,
with preprocessed source if appropriate.

it seems that this issue is a dup of
https://bugzilla.redhat.com/show_bug.cgi?id=1639019 .

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-01 16:38:16 +08:00
Kefu Chai
17849cac84 librbd: workaround an ICE of GCC
GCC is somehow annoyed at seeing the combination of decltype and
initializer_list in this place. i tried to remove the `if` clause, and
only left the `else` block, GCC was happy with that change. i also tried
to pass an empty `{}` to `decltype(reply.lockers)`, and GCC was also
happy with that. so i guess there are multiple factors taking effect in
this problem. probably any of them could be the last straw that breaks
GCC.

but we cannot have a minimal reproducer for this issue here without more
efforts. and `reply.lockers` is empty after `reply` is constructed, so
it would be simpler if we just add the locker info to it instead of
assigning a newly constructed `map` to it.

Fixes: http://tracker.ceph.com/issues/37719
Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-01-01 16:26:08 +08:00
Kefu Chai
dab8ff4480
Merge pull request #25640 from joscollin/wip-mon-parse
mon,osd: avoid str copy in parse

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-01-01 14:39:59 +08:00
Kefu Chai
bf3b53d887
Merge pull request #25534 from trociny/wip-36244
mgr: fix crash due to multiple sessions from daemons with same name

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-01-01 14:12:23 +08:00
Sage Weil
db5a69abd2 Merge PR #25597 into master
* refs/pull/25597/head:
	mgr/hello: define some module options
	pybind/mgr/mgr_module: normalize defaults to str when type is missing
	mgr/telegraf: specify option types
	mgr/telemetry: specify option types
	mgr/zabbix: specify option types
	mgr/localpool: document options and specify in native types
	mgr/devicehealth: document options and specify in native types
	mgr/balancer: document and specify options
	common/options: make runtime vs not runtime explicit, not type-dependent
	mon/MgrMonitor: mark module options with FLAG_MGR
	mon/MgrMonitor: make find_module_option handle localized option names
	pybind/mgr/mgr_module: set values as string
	mgr: return options as appropriate python type
	mgr/PythonCompat: python 3 cludges
	pybind/mgr/mgr_module: make use of defined default value
	mgr/PyModule: populate ModuleOptions and expose to mon
	mon/ConfigMonitor: unify module options with built-in options
	mon/MgrMonitor: populate options
	mon/MgrMap: define ModuleOptions and include in map
	common/options: expand type helpers and make them static
	common/options: pin enums to values

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2018-12-31 22:31:18 -06:00
Sage Weil
49eab70fbc Merge PR #25652 into master
* refs/pull/25652/head:
	osd/OSDMap: disallow new upmaps on pgs that are pending merge
	osd/PG: align past_intervals and last_epoch_clean for fabricated merge target

Reviewed-by: Neha Ojha <nojha@redhat.com>
2018-12-31 22:30:40 -06:00
Sage Weil
58a6bcb748 Merge PR #25730 into master
* refs/pull/25730/head:
	Revert "os/objectstore: add new op OP_CREATE for create a new object."

Reviewed-by: Sage Weil <sage@redhat.com>
2018-12-31 22:30:03 -06:00
Sage Weil
f3ce547eb3 Merge PR #25670 into master
* refs/pull/25670/head:
	os/bluestore: KernelDevice::read() does the EIO mapping now.

Reviewed-by: Sage Weil <sage@redhat.com>
2018-12-31 22:28:47 -06:00
Sage Weil
3c9a562755 Merge PR #25408 into master
* refs/pull/25408/head:
	mgr/hello: Make use of `HandleCommandResult`.

Reviewed-by: Sage Weil <sage@redhat.com>
2018-12-31 15:24:26 -06:00
Sage Weil
d8d16ba6aa qa/workunits/rados/test_health_warnings: prevent out osds
Fixes: http://tracker.ceph.com/issues/37776
Signed-off-by: Sage Weil <sage@redhat.com>
2018-12-31 11:18:05 -06:00
Sage Weil
e516855487 osd: reliably send pg_created messages to the mon
The OSD has to reliably deliver a pg_created message to the mon in order
for the mon to clear the pool's CREATING flag.  Previously, a mon
connection reset would drop the message.

Restructure this to:

- queue a message any time a PG peers and the pool as the CREATING flag
- track pending messages in OSDService
- resend on mon connect
- prune messages for pools that no longer have the CREATING flag

This new strategy can result in resends of these messages to the mon in
cases where the mon already knows the PG was created.  However, pool
creation is rare, and these extra messages are cheap.  And we can avoid
this overhead if we like by limiting the number of PGs that the mon can
create explicitly if we choose (by lowering mon_osd_max_initial_pgs).

Fixes: http://tracker.ceph.com/issues/37775
Signed-off-by: Sage Weil <sage@redhat.com>
2018-12-31 11:05:03 -06:00
Kefu Chai
46073b4de0
Merge pull request #25437 from Aran85/fix-objecter-reset
objecter: avoid race when reset down osd's session

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-01-01 00:56:52 +08:00
Kefu Chai
1771e2b0ad
Merge pull request #21138 from zouaiguo/fix_mgr_restful_osd_error
mgr/restful: fix py got exception when get osd info

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-01-01 00:52:38 +08:00
Kefu Chai
491c2545f3 Revert "os/objectstore: add new op OP_CREATE for create a new object."
This reverts commit 22f73ba62e.

Fixes: http://tracker.ceph.com/issues/37774
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-12-31 23:04:25 +08:00
Kefu Chai
906c365c77
Merge pull request #22385 from majianpeng/touch-omit-kvdb
os/objectstore: add a new op OP_CREATE.

Reviewed-by: Sage Weil <sage@redhat.com>
2018-12-31 22:35:09 +08:00
Kefu Chai
a0352c83fe
Merge pull request #25653 from rhcs-dashboard/37717-update-health-test
mgr/dashboard: updated API test

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-12-31 22:30:13 +08:00
Kefu Chai
8e94b08150 osd: unlock osd_lock when tweaking osd settings
unlock osd_lock when serving "debug kick_recovery_wq" command

we need to unlock osd_lock temporarily when updating the osd settings,
otherwise we will run into assert failure. because
OSD::handle_conf_change() acquires the osd_lock which is not a recursive
lock.

Fixes: http://tracker.ceph.com/issues/37751
Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-12-30 23:19:02 +08:00
Kefu Chai
5c628a1cc9 osd: use unlock_guard for unlock osd temporarily
when OSD::do_command() gets called, osd_lock is acquired. but when
serving some of these commands, we need to call methods which also
acquire the osd_lock by themselves. for instance,
OSD::handle_conf_change() gets called by cct->_conf.apply_changes().
to allow them to do so, we unlock osd_lock before calling those methods,
and re-lock it after done with them.

unlock_guard is introduced to unlock and re-lock the lock in a RAII style.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-12-30 23:18:09 +08:00
Kefu Chai
55959be90a
Merge pull request #25723 from falcon78921/wip-doc-37770
doc: fix the formatting of HTTP Frontends documentation

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-12-30 17:40:48 +08:00
James McClune
6f3100b816 doc: cleanup HTTP Frontends documentation
Added literal block to /etc/ceph/ceph.conf example.

Fixes: https://tracker.ceph.com/issues/37770
Signed-off-by: James McClune <jmcclune@mcclunetechnologies.net>
2018-12-29 12:40:50 -05:00
Jos Collin
2805f34f39
mon: avoid str copy in parse
Avoids the input string copy.
It doesn't make any difference when the input string is copied.

Signed-off-by: Jos Collin <jcollin@redhat.com>
2018-12-29 20:03:39 +05:30
Jos Collin
a0b3562b24
doc: Fixed double spaces
Signed-off-by: Jos Collin <jcollin@redhat.com>
2018-12-26 11:24:48 +05:30
Jos Collin
c1ddc49b2d
doc: Fixed Create a Cluster url
Fixes: http://tracker.ceph.com/issues/37764
Signed-off-by: Jos Collin <jcollin@redhat.com>
2018-12-26 11:24:11 +05:30
Xie Xingguo
0ccdc79938
Merge pull request #25702 from wjwithagen/wjw-fix-bluestore-warning
bluestore: silence Clang warning on possible uninitialize usuage

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2018-12-26 08:52:57 +08:00
Patrick Donnelly
cebcf238b0
Merge PR #25591 into master
* refs/pull/25591/head:
	mds: clean up log messages for standby-replay

Reviewed-by: Zheng Yan <zyan@redhat.com>
2018-12-25 10:21:04 -08:00
Willem Jan Withagen
b23220ca81 bluestore: silence Clang warning on possible uninitialize usuage
```
/home/wjw/ceph.head/ceph/src/os/bluestore/BlueStore.cc:5935:15: note: initialize the variable 'reserved' to silence this warning
  int reserved;
              ^
               = 0
1 warning generated.
```

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2018-12-25 16:07:16 +01:00
alfonsomthd
4ac2e3f864 mgr/dashboard: updated health controller & API test
Updated health controller & test to reflect changes introduced
in 'df' payload.

Return 'total_used_raw_bytes' instead of 'total_used_bytes'
to match CLI 'bin/rados df' used/avail summary in
Landing Page (frontend component).

Do not return 'stats_by_class' to save bandwidth as they are
not needed (right now) in the dashboard.

Fixes: https://tracker.ceph.com/issues/37717

Signed-off-by: Alfonso Martínez <almartin@redhat.com>
2018-12-24 14:45:34 +01:00
Kefu Chai
e0d538d3f5
Merge pull request #25688 from wjwithagen/wjw-fix-kerberos
install-deps.sh: Add Kerberos requirement for FreeBSD

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-12-24 19:38:18 +08:00
Willem Jan Withagen
4d04ce251a build: Add Kerberos requirement for FreeBSD
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2018-12-24 10:52:58 +01:00
alfonsomthd
1d8bb72683 mgr/dashboard: pool stats not returned by default
* All pool controller methods with same default value
for stats flag.
* Stats requested explicitly by frontend service.
* Updated API tests accordingly.

Fixes: https://tracker.ceph.com/issues/36740

Signed-off-by: Alfonso Martínez <almartin@redhat.com>
2018-12-24 08:48:38 +01:00
linbing
dbbde30648 os/bluestore: fixup access a destroy cond cause deadlock or undefine
behavior.

Fixes: https://tracker.ceph.com/issues/37733
Signed-off-by: linbing <linbing@t2cloud.net>
2018-12-24 08:11:06 +08:00
Kefu Chai
94cb394d62
Merge pull request #25675 from dzafman/wip-36525
test: Disable duplicate request command test during scrub testing

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-12-24 00:16:00 +08:00
Kefu Chai
8928f1ffd7
Merge pull request #25582 from trociny/wip-osd-perf-query-wrs
osd: Weighted Random Sampling for dynamic perf stats

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-12-24 00:12:47 +08:00
Kefu Chai
18a518615e osd/PrimaryLogPG: do not count failed read in delta_stats
ctx->delta_stats is accumulated to PG.info.stats.stats in
ReplicatedBackend::submit_transaction() before queueing the transaction.

and `stats` is used for stats reporting. so i think we should not take
the failed ops into account.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2018-12-23 22:49:53 +08:00
Kefu Chai
98078f1748
Merge pull request #25584 from thinkercui/bugfix
osd/PrimaryLogPG: fix the extent length error of the sync read

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2018-12-23 22:30:58 +08:00
Kefu Chai
d351e02880
Merge pull request #25626 from xiexingguo/wip-no-upmap-for-merge
mgr/balancer: skip auto-balancing for pools with pending pg-merge

Reviewed-by: Sage Weil <sage@redhat.com>
2018-12-23 22:28:49 +08:00
Kefu Chai
5cbec0f22e
Merge pull request #25645 from sebastian-philipp/orchestrator-ansible-minor-cleanup
mgr/ansible: Python 3 fix

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-12-23 22:28:12 +08:00
Patrick Donnelly
8722fbab07
Merge PR #25619 into master
* refs/pull/25619/head:
	mds: use frag_vec_t for efficient frag storage
	mds: do not copy session map accidentally
	mds: use vector for subtree access
	include/frag: use more efficient small vectors
	StackStringStream: add strv method short-cut
	include: add small_vector encode/decode/<<
	mds: scope using directives

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2018-12-22 20:16:13 -08:00
Sage Weil
f3e4e56fa5 Merge PR #25618 into master
* refs/pull/25618/head:
	mgr/dashboard: Updated API pool tests

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2018-12-22 16:04:55 -06:00
Patrick Donnelly
017e7c587c
Merge PR #25648 into master
* refs/pull/25648/head:
	client: add latency statistics for a read/fsync operation

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-12-22 12:34:57 -08:00
Patrick Donnelly
7b08cd79fe
Merge PR #24797 into master
* refs/pull/24797/head:
	mds: handle state change race

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-12-22 12:33:19 -08:00
Patrick Donnelly
7e8c6838af
Merge PR #23126 into master
* refs/pull/23126/head:
	wherever: guard handle_conf_change() from concurrent execution
	config: drop config_proxy::lock when invoking config observer

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-12-22 12:27:57 -08:00
Jason Dillaman
ce13562a74
Merge pull request #25636 from trociny/wip-prometheus-rbd-ns
pybind/mgr/prometheus: rbd stats namespace support

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-12-22 10:25:32 -05:00
Jason Dillaman
4d4418b3af
Merge pull request #25666 from trociny/wip-instance-watcher-assert
rbd_mirror: assert no requests on destroying InstanceWatcher

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-12-22 10:25:17 -05:00
Jason Dillaman
b87bfb663c
Merge pull request #25667 from trociny/wip-37672
test/librados_test_stub: handle object doesn't exist gracefully 

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-12-22 10:24:22 -05:00
David Zafman
554ea73cb5 test: Disable duplicate request command test during scrub testing
Scrub testing requires an orderly control of scrubbing.  Most but not
all the time, the duplicate scrub request is ignored because the first
request hasn't finished.  Teuthology enables this environment variable
in the workunit handling.

Fixes: https://tracker.ceph.com/issues/36525

Signed-off-by: David Zafman <dzafman@redhat.com>
2018-12-21 18:28:23 -08:00
Kefu Chai
34fa1b6922
Merge pull request #25671 from tchaikov/wip-seastar-pthread
seastar: pickup the change to link pthread

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2018-12-22 09:51:52 +08:00
Kefu Chai
fe89e0e96c
Merge pull request #25608 from tchaikov/bluestore_fbsd
os/bluestore: support for FreeBSD

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Sage Weil <sage@redhat.com>
2018-12-22 09:39:28 +08:00