Commit Graph

123512 Commits

Author SHA1 Message Date
Dan Mick
9d37c6efbd monitoring/grafana/build/Makefile: pull dashboards from local dir
Use the dashboard definition files in this workspace directly

Signed-off-by: Dan Mick <dmick@redhat.com>
2021-05-26 13:37:25 -07:00
Dan Mick
444d6f6623 monitoring/grafana/build/Makefile: Add ARCH variable
Allow building for other archs, in particular arm64

Signed-off-by: Dan Mick <dmick@redhat.com>
2021-05-26 13:37:25 -07:00
Dan Mick
508b1d387f monitoring/grafana/build/Makefile: fully qualify source image
Some build machines may not have a default docker repo configured.
Specify docker.io.

Signed-off-by: Dan Mick <dmick@redhat.com>
2021-05-26 13:37:24 -07:00
Adam C. Emerson
935d3e079d
Merge pull request #41465 from adamemerson/wip-50169
rgw: Simplify log shard probing and err on the side of omap

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-05-26 13:52:57 -04:00
Kefu Chai
1b7904fc45
Merge pull request #41554 from rzarzynski/wip-crimson-simplify-ox-lt-mgmt
crimson/osd: simplify the management of OpsExecuter's life-time.

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-05-26 23:03:46 +08:00
Radoslaw Zarzynski
96f74185be crimson/osd: simplify the management of OpsExecuter's life-time.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-05-26 14:30:27 +00:00
Sage Weil
1d6b331cb7 Merge PR #41510 into master
* refs/pull/41510/head:
	doc/cephfs/nfs: remove documented limitation

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
2021-05-26 10:12:06 -04:00
Kefu Chai
e32589d1ad
Merge pull request #41547 from t-msn/wip-update-cephspec
ceph.spec.in: install gcc-toolset-9-gcc-c++ for rhel only

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-05-26 18:44:31 +08:00
Misono Tomohiro
f38def9273 ceph.spec.in: install gcc-toolset-9-gcc-c++ for rhel only
Otherwise fedora 33 complains there is no gcc-toolset-9-gcc-c++
when running "WITH_SEASTAR=true ./install_deps.sh"

Related to: 36759b5363
Signed-off-by: Misono Tomohiro <misono.tomohiro@jp.fujitsu.com>
2021-05-26 16:21:34 +09:00
Kefu Chai
31c84c0b3c
Merge pull request #41545 from tchaikov/wip-vstart-fix
vstart.sh: pass the addition option to parse_block_devs()

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-05-26 15:18:22 +08:00
Kefu Chai
bf3ea637c1 vstart.sh: use || instead of "-o"
to silence the warning like:

 SC2166: Prefer [ p ] || [ q ] as [ p -o q ] is not well defined.

see also
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/test.html

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-05-26 14:02:53 +08:00
Kefu Chai
f9a3921eff vstart.sh: pass the addition option to parse_block_devs()
to address the regression introduced by
3ea5242e38

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-05-26 14:00:58 +08:00
Kefu Chai
557d2b8314
Merge pull request #41543 from runsisi/wip-fix-clay-doc
doc: add missing crush-device-class={device-class} pair for clay code profile

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-05-26 13:55:03 +08:00
Kefu Chai
cc76e883f3
Merge pull request #41542 from tchaikov/wip-vstart-cleanup
vstart: cleanups

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2021-05-26 12:58:37 +08:00
Kefu Chai
0edaff5d3c
Merge pull request #41536 from rzarzynski/wip-crimson-fix-ox-lifetime
crimson/osd: extend lifetime of OpsExecuter to match all_completed

Reviewed-by: Xuehan Xu <xuxuehan@360.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-05-26 11:42:58 +08:00
Kefu Chai
3ea5242e38 vstart: define helper for parsing block_devs options
for better readability, and to simplify the code

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-05-26 11:18:42 +08:00
Kefu Chai
6de7b86e09 vstart: use block_devs for {blue,sea}store_dev
so the logic handling bluestore_dev and seastore_dev can be merged.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-05-26 11:18:42 +08:00
Kefu Chai
b851c8a351 vstart: use here document to define usage
for better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-05-26 11:18:42 +08:00
luo.runbing
d9315c88d5
doc: add missing crush-device-class={device-class} pair for clay code profile
`crush-device-class` is optional for `ceph osd erasure-code-profile set`,
add it for the sake of completeness

Signed-off-by: luo.runbing <luo.runbing@zte.com.cn>
2021-05-26 11:14:38 +08:00
Kefu Chai
ac83f83ad6
Merge pull request #41484 from liewegas/cephadm-progress
mgr/cephadm: progress item for service apply

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
2021-05-26 10:36:31 +08:00
Kefu Chai
a35da2957b
Merge pull request #41452 from ifed01/wip-ifed-fix-no-track
os/bluestore: track bluestore_warn_on_no_per_pg_omap cfg parameter ch…

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
2021-05-26 10:33:58 +08:00
Kefu Chai
cbce718252
Merge pull request #41453 from ifed01/wip-ifed-stray-omap
os/bluestore: improve stray omap logging during fsck.

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
2021-05-26 10:33:22 +08:00
Kefu Chai
2ba42c5e5e
Merge pull request #41398 from aclamk/wip-aclamk-avl-block-picker-improve
os/bluestore: Improve _block_picker function

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-05-26 10:32:14 +08:00
Kefu Chai
866850ff38
Merge pull request #41488 from liewegas/cephadm-conf
config,mgr: expose ceph.conf path to mgr modules

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-05-26 10:31:33 +08:00
Kefu Chai
cd1b9bbbd1
Merge pull request #41459 from liewegas/devid-underscores
common/blkdev: remove double _'s from device_id

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-05-26 10:30:08 +08:00
Kefu Chai
89a8d3c939
Merge pull request #41491 from tchaikov/wip-ceph-daemon-ann
pybind/ceph_daemon: add type annotations

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2021-05-26 10:28:49 +08:00
Kefu Chai
dbf7ac3c32
Merge pull request #41493 from tchaikov/wip-pybind-impport-from-future
pybind: do not import from "__future__" anymore

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
2021-05-26 10:26:57 +08:00
Kefu Chai
6560d60229
Merge pull request #41498 from tchaikov/wip-cmake-libasan.6
cmake/modules/FindSanitizers: prefer libasan.6

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-05-26 10:23:03 +08:00
Kefu Chai
c25d8e977e
Merge pull request #41401 from rzarzynski/wip-crimson-injectdataerr
crimson/osd, common: implement the inject{m,}dataerr admin commands

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-05-26 09:23:35 +08:00
Kefu Chai
9d23089f43
Merge pull request #41541 from liu-chunmei/seastore-add-devs
crimson/seastore: add --seastore-devs in vstart.sh

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-05-26 08:29:13 +08:00
Kefu Chai
ce8a314dee
Merge pull request #41533 from tchaikov/wip-doc-rgw-conf
doc/radosgw: use confval directive to define options

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-05-26 08:18:27 +08:00
Radoslaw Zarzynski
87effa76fd crimson/osd: implement the injectmdataerr admin command.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-05-26 08:14:59 +08:00
Radoslaw Zarzynski
e4ec9c5342 crimson/osd: implement the injectdataerr admin command.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-05-26 08:14:59 +08:00
Radoslaw Zarzynski
3387c1c5ae crimson/os: implement inject_{m,}data_error in AlienStore.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-05-26 08:05:51 +08:00
Kefu Chai
d65f36ebe9
Merge pull request #41434 from tchaikov/wip-cmd-getval
common/cmdparse: use string_view for the key and return val by retval

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-05-26 08:03:46 +08:00
chunmei-liu
0dc5b76e2f crimson/seastore: add --seastore-devs in vstart.sh
to support /dev/xxx as seastore device

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
2021-05-25 15:41:24 -07:00
Adam C. Emerson
71ae2dd27a rgw: Simplify log shard probing and err on the side of omap
In the multigeneration version we no longer care whether entries
exist, since we never delete and recreate empty logs. Remove logic
that marked entirely empty shards as DNE under the assumption that
they would be deleted if so.

Fixes: https://tracker.ceph.com/issues/50169

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2021-05-25 16:56:37 -04:00
Sage Weil
9ab9cc26e2 Merge PR #41007 into master
* refs/pull/41007/head:
	qa/tasks/cephfs/test_nfs: fix info test
	doc/cephfs/fs-nfs-exports: document --ingress --virtual-ip
	mgr/nfs: move ingress vs virtual_ip check to cluster interface
	PendingReleaseNotes: clarify deprecated
	PendingReleaseNotes: note breaking CLI changes
	doc/cephadm/nfs: document nfs+ingress
	qa/suites/rados/cephadm/smoke-roleless: test nfs, nfs + ingress
	mgr/nfs: take --ingress argument to 'nfs cluster create'
	mgr/cephadm: adjust debug output for device refresh
	mgr/cephadm: ingress: fix log msg
	mgr/cephadm: fix logging of config/placement errors
	common/options: enable nfs module for new clusters
	cephadm: --stop-signal=SIGTERM
	mgr/orchestrator: default nfs pool, namespaces
	mgr/cephadm: nfs: create pool if it doesn't yet exist
	doc/cephadm/nfs: update
	mgr/nfs: change 'nfs cluster info'
	mgr/nfs: take optional virtual_ip for deploying ingress
	mgr/nfs: remove 'nfs cluster update'
	mgr/nfs: factor out ganesha pool creation
	mgr/nfs: delete -> rm for CLI
	mgr/nfs: add some type annotations
	python-common: fix IngressSpec yaml dump
	mgr/cephadm: ingress: remove eth0 default
	qa/tasks/cephadm: allow mounting volumes in shell
	cephadm: add -v arg to shell
	qa/tasks/vip: add 'vip.exec' task
	mgr/orchestrator: add --port arg to 'orch apply nfs'
	mgr/cephadm: nfs: add purge
	mgr/cephadm: ingress: support nfs
	mgr/cephadm: do not reconfigure daemons on deleted services
	mgr/cephadm: nfs: shell out to rados tool for conf creation
	mgr/cephadm: nfs: add rank to grace file from mgr module
	mgr/cephadm: nfs: bind ganesha to appropriate ip:port
	mgr/cephadm: enable ranked daemons for nfs
	mgr/cephadm: support creation of daemons with ranks
	mgr/cephadm: make _plan show removed daemon names
	mgr/cephadm/schedule: assign/map ranks
	mgr/cephadm: add rank[_generation] properties
	mgr/cephadm/inventory: store optional rank_map along with specs
	mgr/cephadm: include service_name is generated DaemonDescription
	mgr/orchestrator: include service_name in DaemonDescription dump
	mgr/cephadm/inventory: fix deleted check
	mgr/cephadm: simplify
	mgr/cephadm/schedule: make placement shuffle deterministic
	mgr/cephadm: document CephadmService flags

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Varsha Rao <varao@redhat.com>
2021-05-25 16:17:44 -04:00
Sage Weil
bfbc7b3078 Merge PR #41539 into master
* refs/pull/41539/head:
	doc/cephadm: fix prompts in service-management.rst

Reviewed-by: Sage Weil <sage@redhat.com>
2021-05-25 16:17:21 -04:00
Zac Dover
db3af8e3d2 doc/cephadm: fix prompts in service-management.rst
This PR formats the prompts in service-managment.rst
properly.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2021-05-26 05:22:56 +10:00
Kefu Chai
8b16defd4f doc/radosgw: use confval directive to define options
less repeating this way

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-05-25 23:56:31 +08:00
Radoslaw Zarzynski
acd5823a7f crimson/osd: extend lifetime of OpsExecuter to match all_completed.
f7181ab2f6 has optimized the client
parallelism. To achieve that `PG::do_osd_ops()` were converted to
return basically future pair of futures. Unfortunately, the life-
time management of `OpsExecuter` was kept intact. In the result,
the object was valid only till fullfying the outer future while,
due to the `rollbacker` instances, it should be available till
`all_completed` becomes available.

This issue can explain the following problem has been observed
in a Teuthology job [1].

```
DEBUG 2021-05-20 08:03:22,617 [shard 0] osd - do_op_call: method returned ret=-17, outdata.length()=0 while num_read=1, num_write=0
DEBUG 2021-05-20 08:03:22,617 [shard 0] osd - rollback_obc_if_modified: object 19:e17d4708:test-rados-api-smithi095-38404-2::foo:head got erro
r generic:17, need_rollback=false
=================================================================
==33626==ERROR: AddressSanitizer: heap-use-after-free on address 0x60d0000b9320 at pc 0x560f486b8222 bp 0x7fffc467a1e0 sp 0x7fffc467a1d0
READ of size 4 at 0x60d0000b9320 thread T0
    #0 0x560f486b8221  (/usr/bin/ceph-osd+0x2c610221)
    #1 0x560f4880c6b1 in seastar::continuation<seastar::internal::promise_base_with_type<boost::intrusive_ptr<MOSDOpReply> >, seastar::noncopy
able_function<crimson::interruptible::interruptible_future_detail<crimson::osd::IOInterruptCondition, crimson::errorator<crimson::unthrowable_
wrapper<std::error_code const&, crimson::ec<(std::errc)11> > >::_future<crimson::errorated_future_marker<boost::intrusive_ptr<MOSDOpReply> > >
 > ()>, seastar::future<void>::then_impl_nrvo<seastar::noncopyable_function<crimson::interruptible::interruptible_future_detail<crimson::osd::
IOInterruptCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)11> > >::_future<crimson:
:errorated_future_marker<boost::intrusive_ptr<MOSDOpReply> > > > ()>, crimson::interruptible::interruptible_future_detail<crimson::osd::IOInte
rruptCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)11> > >::_future<crimson::error
ated_future_marker<boost::intrusive_ptr<MOSDOpReply> > > > >(seastar::noncopyable_function<crimson::interruptible::interruptible_future_detail
<crimson::osd::IOInterruptCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)11> > >::_
future<crimson::errorated_future_marker<boost::intrusive_ptr<MOSDOpReply> > > > ()>&&)::{lambda(seastar::internal::promise_base_with_type<boos
t::intrusive_ptr<MOSDOpReply> >&&, seastar::noncopyable_function<crimson::interruptible::interruptible_future_detail<crimson::osd::IOInterruptCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)11> > >::_future<crimson::errorated_future_marker<boost::intrusive_ptr<MOSDOpReply> > > > ()>&, seastar::future_state<seastar::internal::monostate>&&)#1}, void>::run_and_dispose() (/usr/bin/ceph-osd+0x2c7646b1)
    #2 0x560f5352c3ae  (/usr/bin/ceph-osd+0x374843ae)
    #3 0x560f535318ef  (/usr/bin/ceph-osd+0x374898ef)
    #4 0x560f536e395a  (/usr/bin/ceph-osd+0x3763b95a)
    #5 0x560f532413d9  (/usr/bin/ceph-osd+0x371993d9)
    #6 0x560f476af95a in main (/usr/bin/ceph-osd+0x2b60795a)
    #7 0x7f7aa0af97b2 in __libc_start_main (/lib64/libc.so.6+0x237b2)
    #8 0x560f477d2e8d in _start (/usr/bin/ceph-osd+0x2b72ae8d)

```

[1]: http://pulpito.front.sepia.ceph.com/rzarzynski-2021-05-20_07:28:16-rados-master-distro-basic-smithi/6124735/

The commit deals with the problem by repacking the outer future.
An alternative could be in switching from `std::unique_ptr` to
`seastar::shared_ptr` for managing `OpsExecuter`.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-05-25 14:47:33 +00:00
Sage Weil
218eec938d qa/tasks/cephfs/test_nfs: fix info test
Signed-off-by: Sage Weil <sage@newdream.net>
2021-05-25 10:15:45 -04:00
Sage Weil
ada85604ab doc/cephfs/fs-nfs-exports: document --ingress --virtual-ip
Signed-off-by: Sage Weil <sage@newdream.net>
2021-05-25 10:15:45 -04:00
Sage Weil
f2e78bf2e3 mgr/nfs: move ingress vs virtual_ip check to cluster interface
Signed-off-by: Sage Weil <sage@newdream.net>
2021-05-25 10:15:45 -04:00
Sage Weil
fe03c85b43 PendingReleaseNotes: clarify deprecated
Signed-off-by: Sage Weil <sage@newdream.net>
2021-05-25 10:15:45 -04:00
Sage Weil
0ca1400e4b PendingReleaseNotes: note breaking CLI changes
Signed-off-by: Sage Weil <sage@newdream.net>
2021-05-25 10:15:45 -04:00
Sage Weil
e51c3dcf01 doc/cephadm/nfs: document nfs+ingress
Signed-off-by: Sage Weil <sage@newdream.net>
2021-05-25 10:15:45 -04:00
Sage Weil
e2db373698 qa/suites/rados/cephadm/smoke-roleless: test nfs, nfs + ingress
Still missing a full client mount test, though!

Signed-off-by: Sage Weil <sage@newdream.net>
2021-05-25 10:15:45 -04:00
Sage Weil
81f0b6984f mgr/nfs: take --ingress argument to 'nfs cluster create'
It is likely that the rook/k8s variation of ingress will not take a
virtual_ip argument.  We want to make sure that ingress yes/no can be
specified independent of the virtual_ip.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-05-25 10:15:45 -04:00