Commit Graph

109018 Commits

Author SHA1 Message Date
Kefu Chai
b9e282f0e3 admin/build-doc: use "ceph" for prefix of functions defined by libcephfs
if some python module tries to `import cephfs`, it should not fail.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-03-11 22:43:08 +08:00
Kefu Chai
1183f26719 admin/build-doc: pass -rpath to linker
to make sure the dummy cythonized python binding is able to find the dummy C binding.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-03-11 22:43:08 +08:00
Kefu Chai
c39f46fc6c
Merge pull request #33844 from tchaikov/wip-crimson-mgr-reconnect
crimson/mgr: reconnect in ms_handle_reset()

Reviewed-by: Xuehan Xu <xxhdx1985126@163.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2020-03-11 14:30:03 +08:00
Kefu Chai
100c375fd4
Merge pull request #33849 from tchaikov/wip-crimson-fmt-time-point
crimson: extract formatter for time_point types

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
2020-03-11 10:06:30 +08:00
Gregory Farnum
d0db97d475
Merge pull request #32963 from Dmole/patch-1
doc: operating.rst clean up

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2020-03-10 11:04:59 -07:00
Radoslaw Zarzynski
e7f10d47c1
Merge pull request #33582 from rzarzynski/wip-doc-fix-confignames-log_based_pg
doc/dev/log_based_pg.rst: fix names of configurables.

Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-03-10 18:48:17 +01:00
Kefu Chai
e978412524 crimson: extract formatter for time_point types
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-03-11 00:16:55 +08:00
Nathan Cutler
a1c459882d
Merge pull request #33778 from smithfarm/wip-drop-is_opensuse
rpm: drop "is_opensuse" conditional in SUSE-specific bcond block

Reviewed-by: Tim Serong <tserong@suse.com>
2020-03-10 15:25:57 +01:00
Kefu Chai
9318c18218 crimson/mgr: reconnect in ms_handle_reset()
since `reconnect()` does not wait for `conn->close()` anymore, we
can actually call `reconnect()` right in `ms_handle_reset()`.

also, to avoid connecting mgr too frequently, connect the mgr with
specified interval (1 second by default).

send the MMgrOpen message in `ms_handle_connect()` to make it explicit
that we sends this message when the connection is established.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-03-10 21:13:47 +08:00
Kefu Chai
74001778ef crimson/mgr: s/tick/report/
prepare for the change to reconnect right when handling connection reset

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-03-10 21:13:47 +08:00
Nathan Cutler
ac9383adcf
Merge pull request #33819 from smithfarm/wip-bci-octopus
script/backport-create-issue: start backporting to octopus

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
2020-03-10 11:50:56 +01:00
Kefu Chai
eb488f8f67
Merge pull request #33458 from tchaikov/wip-test-objectstore
objectstore: cleanups to avoid using `g_ceph_context`

Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-03-10 10:51:30 +08:00
Neha Ojha
6117a0d4db
Merge pull request #33281 from ideepika/wip-set-osd-pool-size-extra-param-check
mon/OSDMonitor: add flag `--yes-i-really-mean-it` for setting pool size 1

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-03-09 19:14:50 -07:00
Deepika Upadhyay
21508bd9dd mon/OSDMonitor: add flag --yes-i-really-mean-it for setting pool size 1
Adds option `mon_allow_pool_size_one` which will be disabled by default
to ensure pools are not configured without replicas.
If the user still wants to use pool size 1, they will have to change the
value of `mon_allow_pool_size_one` to true and then have to pass flag
`--yes-i-really-mean-it` to cli command:

Example:
`ceph osd pool test set size 1 --yes-i-really-mean-it`

Fixes: https://tracker.ceph.com/issues/44025
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
2020-03-09 23:27:36 +05:30
Jason Dillaman
e06946b6b5
Merge pull request #33676 from pponnuvel/strncpy_avoid_one_byte_copy
test/librbd: avoid copying/overwriting a byte unnecessarily
2020-03-09 13:48:33 -04:00
Jason Dillaman
797ed05e14
Merge pull request #33681 from changchengx/blockguard_right_exclusive
librbd: explicitly specify BlockGuard to be right exclusive

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-03-09 13:47:39 -04:00
Jason Dillaman
499a4e50f0
Merge pull request #33712 from yison/rwl-fix-block-extents-issue
librbd: fix block extents issue of rwl writeback cache

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-03-09 13:46:13 -04:00
Kefu Chai
50cf889e78
Merge pull request #33196 from xxhdx1985126/wip-crimson-pglog-to-pr
crimson: add pglog support to crimson's data write path

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-03-09 23:09:05 +08:00
Nathan Cutler
dc1d025b7e script/backport-create-issue: start backporting to octopus
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2020-03-09 15:37:37 +01:00
Sage Weil
5206bfbd48 Merge PR #33793 into master
* refs/pull/33793/head:
	qa/suites/rados/cephadm/upgrade: new start point
	qa/tasks/cephadm: put bootstrap config etc directly in /etc/ceph
	cephadm: shell: default to config and keyring in /etc/ceph, if present

Reviewed-by: Ricardo Marques <rimarques@suse.com>
2020-03-09 08:28:57 -05:00
Sage Weil
f3ffbb88f6 Merge PR #33808 into master
* refs/pull/33808/head:
	mgr/cephadm: apply: fill in default placement if none is provided
	mgr/cephadm: make placement truly optional (default to count=1)
	mgr/cephadm: allow count == 0
	mgr/cephadm: remove magic labels

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2020-03-09 08:28:37 -05:00
Lenz Grimmer
da52dec552
Merge pull request #33756 from tspmelo/wip-remove-ngx-store
mgr/dashboard: Remove ngx-store

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2020-03-09 12:30:50 +01:00
Kefu Chai
ec9c315ea0
Merge pull request #33691 from rhcs-dashboard/vstart-enable-nfs-ganesa-mgmt-dashboard
vstart.sh: enable nfs-ganesha mgmt. in dashboard.

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Varsha Rao <varao@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-03-09 19:25:05 +08:00
Kefu Chai
f41fd1faef
Merge pull request #33797 from tchaikov/wip-crimson-cleanups
crimson: cleanups

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2020-03-09 18:22:29 +08:00
Kefu Chai
b57403de20
Merge pull request #32440 from rosinL/wip-spdk
os/bluestore/spdk: Fix the overflow error of parsing spdk coremask

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-03-09 11:54:58 +08:00
Kefu Chai
6582a688ae crimson/mgr: close() in background
as per Yingxin,

application code is not required to wait for the `close()` future, it
would be safe to ignore it, because:
- `close()` will shutdown its socket synchronously;
- `close()` will create an internal `ConnectionRef` when it's closing;
- `Messenger` will wait for all connections closed during `shutdown()`;

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-03-09 11:49:57 +08:00
Kefu Chai
1e37793c3d common/buffer.cc: silence ASan warning
silences following warning
```
../src/common/buffer.cc:472:9: runtime error: member access within null pointer of type 'struct raw'
```

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-03-09 11:49:57 +08:00
Kefu Chai
c64c47a687 crimson/osd: reorder includes
to follow
https://google.github.io/styleguide/cppguide.html#Names_and_Order_of_Includes

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-03-09 11:49:57 +08:00
Kefu Chai
3dfdf0e86b crimson/mgr: use periodical timer for report
* always rearm the timer when handling MMgrConfigure
* remove `mgr::Client::tick_period`

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-03-09 11:49:57 +08:00
Sage Weil
78b24f5783 Merge PR #33776 into master
* refs/pull/33776/head:
	test: Add flush_pg_stats to avoid race with getting num_shards_repaired

Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-03-08 22:05:52 -05:00
Sage Weil
c0524e31c2 mgr/cephadm: apply: fill in default placement if none is provided
Most stateless daemons get 2x (so there is a standby).  Monitoring items
get just 1x.

By default we do 5 monitors, which will gracefully degrade to one per host
if the cluster has <5 hosts.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-08 21:19:52 -05:00
Chunsong Feng
d79efbabb8 os/bluestore/spdk: Fix the overflow error of parsing spdk coremask
coremask supports up to 256 bits in DPDK19.05, but the use of stoll in
NVMEManager::try_get limits the maximum use to 64 bits. Parse coremask by
hex character from low to high.

Fixes: https://tracker.ceph.com/issues/43044
Signed-off-by: Hu Ye <yehu5@huawei.com>
Signed-off-by: Chunsong Feng <fengchunsong@huawei.com>
Signed-off-by: luo rixin <luorixin@huawei.com>
2020-03-09 09:43:41 +08:00
Sage Weil
ab24887736 Merge PR #33804 into master
* refs/pull/33804/head:
	cephadm: ls: warn if daemon type (version) is not supported
	cephadm: report grafana version
	cephadm: report prometheus, node-exporter, alertmanager versions
	cephadm: use None (not '<no value>') for monitoring daemon version

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-03-08 19:57:06 -05:00
Sage Weil
f4ff4f96ec Merge PR #33792 into master
* refs/pull/33792/head:
	doc/cephadm: fix formatting for osd section
	doc/cephadm: update 'adding mons' section to suggest/prefer 'apply'
	doc/cephadm: fix formatting, typos
	mgr/cephadm: implement apply_mon
	mgr/cephadm: allow mon creation without explicit ip or addr
	mgr/cephadm: allow _apply_service to delete mon daemon's data
	mgr/cephadm: remove mon from monmap before removing mon
	mgr/cephadm: do not remove mon if it breaks quorum

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-03-08 17:29:00 -05:00
Sage Weil
4ba3e5d3b0 cephadm: ls: warn if daemon type (version) is not supported
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-08 17:24:27 -05:00
Sage Weil
02f2ddf3dc cephadm: report grafana version
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-08 17:23:24 -05:00
Sage Weil
6b0463cae5 Merge PR #33802 into master
* refs/pull/33802/head:
	mgr/cephadm: sanity check upgrade version
	mgr/cephadm: only need to invalidate once here
	mgr/cephadm: upgrade requires root mode for now

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-03-08 16:49:38 -05:00
Sage Weil
9fcad77bb5 Merge PR #33800 into master
* refs/pull/33800/head:
	mgr/cephadm: fix prom config generation when hosts have no labels or addrs

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-03-08 16:38:28 -05:00
Sage Weil
6d5c758117 Merge PR #33795 into master
* refs/pull/33795/head:
	mgr/orch: collapse 'SPEC' and 'PLACEMENT' columns in 'orch ls'

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-03-08 16:38:15 -05:00
Ramana Raja
746a2a9180
Merge pull request #33788 from ajarr/wip-44438
test_volumes: fix _verify_clone_attrs call

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2020-03-08 23:06:50 +05:30
Sage Weil
0bb29996ae mgr/cephadm: make placement truly optional (default to count=1)
If no placement information is provided at all, assume 1 daemon over any
host.

This could perhaps be improved with a default placement that varies by
daemon type...

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-08 12:05:47 -05:00
Sage Weil
f61f8e4d79 mgr/cephadm: allow count == 0
Scale a service down to 0 without removing the spec.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-08 12:04:22 -05:00
Sage Weil
387813ab5f mgr/cephadm: remove magic labels
Remove the magic label behavior.  It makes the code confusing, it
makes the overall behavior hard to explain, and it makes the PlacementSpec
meaning different than what Rook is doing.

Instead, if you want mons on hosts with label 'mon', then say 'label:mon'.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-08 12:00:45 -05:00
Yuval Lifshitz
e66b911299
Merge pull request #33686 from yuvalif/fix_data_corruption_in_cls_queue_head
cls/queue: fix data corruption in urgent data
2020-03-08 13:17:02 +02:00
Sage Weil
5bd44d4338 mgr/cephadm: fix prom config generation when hosts have no labels or addrs
The inventory for a host might be {}, which evaluates as false.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-07 21:45:29 -06:00
Sage Weil
0360bea127 Merge PR #33742 into master
* refs/pull/33742/head:
	msg/async: s/nowait/always_async/ in EventCenter::submit_to().
	msg/async: perform recv reset immediately if called inside EC.

Reviewed-by: Sage Weil <sage@redhat.com>
2020-03-07 20:18:23 -06:00
Sage Weil
47d7c75731 cephadm: report prometheus, node-exporter, alertmanager versions
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-07 16:49:38 -06:00
Sage Weil
6ef96b4881 cephadm: use None (not '<no value>') for monitoring daemon version
Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-07 16:34:08 -06:00
Sage Weil
b6084c333d qa/suites/rados/cephadm/upgrade: new start point
The starting cephadm needs to look for default ceph.conf etc in /etc/ceph
for cephadm.py to be happy.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-07 15:19:07 -06:00
Sage Weil
96220c0c05 qa/tasks/cephadm: put bootstrap config etc directly in /etc/ceph
This puts the conf and keyring in /etc/ceph earlier rather than later,
making them useful for debugging a live system *during* bootstrap.  It's
also less code.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-03-07 15:18:45 -06:00