Commit Graph

126108 Commits

Author SHA1 Message Date
Yaarit Hatuka
7ca39fa92b rpm, debian: move smartmontools and nvme-cli to ceph-base
We wish to be able to scrape SMART and NVMe metrics from OSD and MON
nodes. For this we require / recommend smartmontools and nvme-cli
dependencies for both the ceph-osd and ceph-mon packages.  However, the
sudoers file (which is required for invoking `smartctl` by user 'ceph')
was installed only in the ceph-osd package.  Since different packages
cannot own the same file, and because we want to be able to scrape from
every daemon, we move the dependencies and the sudoers installation to
ceph-base. For generalization, we rename:
	sudoers.d/ceph-osd-smartctl -> sudoers.d/ceph-smartctl

Fixes: https://tracker.ceph.com/issues/50657
Signed-off-by: Yaarit Hatuka <yaarit@redhat.com>
2021-08-25 13:20:23 +00:00
Ilya Dryomov
8a3448f6a9
Merge pull request #42806 from zdover23/wip-doc-2021-08-17-rbd-nomad-3-of-x
doc: rewrite configure ceph-csi to "conf. nomad"

Reviewed-by: Etienne Menguy <etienne.menguy@croit.io>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2021-08-24 17:41:57 +02:00
Kefu Chai
ea890d7ead
Merge pull request #42845 from tchaikov/wip-51483
common/options: validate see-also

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-08-24 23:32:39 +08:00
Zac Dover
795877cd49 doc: rewrite configure ceph-csi to "conf. nomad"
This PR rewrites the sections
     - Configure Ceph-CSI
     - Configure Nomad

in the rbd-nomad.rst Chapter of the RBD
Guide.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2021-08-24 16:55:30 +02:00
Sebastian Wagner
8c54a705e2
Merge pull request #42857 from dsavineau/cephadm_tcmalloc_ceph_daemons_only
cephadm: Set tcmalloc env var for Ceph daemons

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-08-24 16:55:02 +02:00
Sebastian Wagner
626a7bf31a
Merge pull request #42833 from oleeander/cephadm_openntpd
cephadm: check for openntpd.service as time sync service

Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-08-24 16:54:50 +02:00
Sebastian Wagner
d2b5417420
Merge pull request #42824 from pcuzner/cephadm-add-ports
cephadm:Add listening ports to gather-facts output

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>
2021-08-24 16:54:34 +02:00
Sebastian Wagner
c5d0e393c1
Merge pull request #42699 from sebastian-philipp/cephadm-duplicate-ips
cephadm: list-networks: Avoid duplicated IPs

Reviewed-by: Francesco Pantano <fpantano@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sage Weil <sage@newdream.net>
2021-08-24 16:54:14 +02:00
Sebastian Wagner
d611ee9034
Merge pull request #42405 from Daniel-Pivonka/cephadm_rm_offline_host
mgr/cephadm: add ability to remove offline host

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-08-24 16:52:28 +02:00
Ilya Dryomov
b8921a0c9a
Merge pull request #42805 from zdover23/wip-doc-2021-08-17-rbd-nomad-2-of-x
doc: improve "Create a Pool" in rbd-nomad.rst

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2021-08-24 16:32:42 +02:00
Kefu Chai
a66daecbea cmake: s/Python_EXECUTABLE/Python3_EXECUTABLE/
as FindPython3.cmake only sets Python3_EXECUTABLE for us, we should
stick with Python3_EXECUTABLE instead of Python_EXECUTABLE.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-08-24 22:22:37 +08:00
Kefu Chai
6cfdd40cad common/options: validate see-also
y2c.py is like a compiler which translates .yaml to .cc and .h files,
it does not have access to all .yaml files. to validate the dangling
see-also issue, we need to do this with a "linker".

in this change, validate-options.py is introduced to check if any of
option name included by the see-also property is valid.

Fixes: https://tracker.ceph.com/issues/51483
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-08-24 22:22:37 +08:00
Zac Dover
1715989e06 doc: improve "Create a Pool" in rbd-nomad.rst
This PR improves the English in the "Create
a Pool" section of the "RBD & Nomad Integration"
chapter of the RBD Guide.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2021-08-24 16:16:24 +02:00
Kefu Chai
56bf728807
Merge pull request #42826 from tchaikov/wip-test-fair-mutex
test/common/test_fair_mutex: increase NR_ROUNDS from 256 to 512

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-08-24 22:04:28 +08:00
Kefu Chai
60811bff82
Merge pull request #42881 from wjwithagen/wjw-fix-vstart-mgr
src/vstart.sh: put modules option for MGR in the [mgr] block.

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2021-08-24 21:51:34 +08:00
Kefu Chai
75ea7222cd
Merge pull request #42882 from cyx1231st/wip-seastar-cleanup
crimson/common: minor cleanups to futures

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-08-24 21:49:42 +08:00
Ilya Dryomov
ed13301c20
Merge pull request #42802 from Rethan/fix-rbd-bench-unit
rbd: avoid overflow of ios and clarify io-size limit for bench

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2021-08-24 13:19:03 +02:00
Samuel Just
8aa1400f49
Merge pull request #42841 from xxhdx1985126/wip-52305
crimson/common: keep ref count of crimson::interruptible::interrupt_cond

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-08-23 20:12:03 -07:00
haoyixing
4a287e8fcd rbd: avoid overflow of ios and clarify io-size limit for bench
When doing rbd bench, we record done ios to print progress, current it's unsigned.
Suppose we do a bench of io-size 512B and io-total 4T, that means a total number of
8G ios which causes an overflow.
And we don't support io-size greater than 4G, so change help message.

Signed-off-by: haoyixing <haoyixing@kuaishou.com>
2021-08-24 10:28:50 +08:00
Yingxin Cheng
0f058fdf18 crimson/interruptible_future: remove unused code
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-08-24 09:39:57 +08:00
Yingxin Cheng
4129f3e43e crimson/common: remove unused futurize::apply()
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-08-24 09:39:49 +08:00
Patrick Donnelly
ea04087786 Merge PR #42371 into master
* refs/pull/42371/head:
	mgr/volumes: Fix a race during clone cancel
	mgr/volumes: Fail subvolume removal if it's in progress

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2021-08-23 20:02:31 -04:00
Neha Ojha
c434eb660b
Merge pull request #42884 from ideepika/wip-remove-unused-submodules
.gitmodules: remove thrift submodule

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-08-23 16:48:17 -07:00
Gregory Farnum
02bda6d799
Merge pull request #41335 from noonedeadpunk/doc/stretch_mon
doc/rados/operations/stretch-mode: Adjust description of mon.e placement

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2021-08-23 16:13:54 -07:00
Ali Maredia
54142666e5
Merge pull request #42053 from TRYTOBE8TME/wip-rgw-dpp-4
rgw: DPP addition 5

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2021-08-23 15:34:41 -04:00
Igor Fedotov
a49f10e760
Merge pull request #42439 from ifed01/wip-ifed-more-errors-shared-blob-repair
os/bluestore: fix additional errors during missed shared blob repair.

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
2021-08-23 18:03:41 +03:00
Yuri Weinstein
75d1961570
Merge pull request #41569 from MrFreezeex/rbd-mirror-snap-perf-dump
rbd-mirror: add perf counters to snapshot replayer
2021-08-23 07:29:40 -07:00
Kefu Chai
b7cace06f1
Merge pull request #42871 from tchaikov/wip-cmake-exclude-grafonnet-lib
cmake: exclude "grafonnet-lib" target from "all" 

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Patrick Seidensal <pseidensal@suse.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
2021-08-23 22:24:53 +08:00
Yuval Lifshitz
2f456a5ded
Merge pull request #42434 from zenomri/wip-omri-rgw-tracing
[WIP] -  RGW tracing implementation
2021-08-23 17:01:54 +03:00
Stephan Müller
0c8cb21d5e
Merge pull request #41380 from Devp00l/wip-rgw-multisite-doc
doc: Improve RGW multisite documentation

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Alex Settle <asettle@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-08-23 15:52:57 +02:00
Alfonso Martínez
92e6244a9f
Merge pull request #42864 from aaryanporwal/vrt-doc
doc/dev: mgr/dashboard: Visual regression tests for ceph dashboard

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2021-08-23 14:59:00 +02:00
Willem Jan Withagen
82d1e68525 src/vstart.sh: put modules option for MGR in the [mgr] block.
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2021-08-23 14:53:47 +02:00
Yuval Lifshitz
e31783c680
Merge pull request #42739 from Matan-B/lua-install-packages-fix
rgw: fix install packages function
2021-08-23 15:46:54 +03:00
Alfonso Martínez
7291426fd0
Merge pull request #42865 from rhcs-dashboard/fix-rgw-bucket-list-api
mgr/dashboard: stats=false not working when listing buckets

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2021-08-23 14:43:56 +02:00
Sebastian Wagner
fb7cb4b50d
Merge pull request #42051 from melissa-kun-li/asyncssh
mgr/cephadm: replace execnet and remoto with asyncssh

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-08-23 14:29:01 +02:00
Ilya Dryomov
9b50733526
Merge pull request #42804 from zdover23/wip-doc-2021-08-17-rbd-nomad-1-of-x
doc: improve RBD "Block Devices and Nomad"

Reviewed-by: Etienne Menguy <etienne.menguy@croit.io>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2021-08-23 13:46:31 +02:00
Kefu Chai
8dd7ac98cc
Merge pull request #42879 from tchaikov/wip-seastar
seastar: pick up change silencing gcc warning

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-08-23 19:10:37 +08:00
Avan Thakkar
95543bb150 mgr/dashboard: stats=false not working when listing buckets
Fixes: https://tracker.ceph.com/issues/51154
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
2021-08-23 15:57:54 +05:30
Xuehan Xu
05e4a7677e crimson/common: all interruptible future invocations should have already had interrupt condition in place
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-08-23 18:23:41 +08:00
Xuehan Xu
454a3731d9 crimson/common: correct comment error about interruptible future
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-08-23 18:23:41 +08:00
Xuehan Xu
d01f0eb69d test/crimson: add interruptible async unit test
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-08-23 18:23:41 +08:00
Xuehan Xu
85c89155dc crimson/common: keep ref count of crimson::interruptible::interrupt_cond
Currently, interrupt conditionss are transfered between inner and outer continuation
chains via a tls interrupt_cond variable. This simple strategy leads to problem when it
comes to mixing normal future/continuation procedures and seastar::thread. When seastar::async()
is called, the reactor can directly invoke the passed functor and lead to two different
scenarios:

1.if a seastar::get/yield() inside the passed lambda, the interrupt_cond should be erased at the
end of the continuation execution when it is yielded back;
2.otherwise, the interrupt_cond should be not erased.

There can be so many possible sequences of yielding of several different fibers that we can hardly
judge at the end of the continuation execution whether there was a yielding during the current
execution, which means we can't be able to know whether the tls interrupt_cond should be erased.
There could be other scenarios where the current strategy fails. To end this kind of issues
once and for all, we involve the ref counting mechinary.

Fixes: https://tracker.ceph.com/issues/52305
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-08-23 18:19:20 +08:00
Sebastian Wagner
8b768153b8
cephadm: Add test_command_list_networks
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
2021-08-23 12:12:24 +02:00
Sebastian Wagner
419dfd6afa
cephadm: test_networks: Move tests from test_cephadm
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
2021-08-23 12:12:24 +02:00
Sebastian Wagner
b65822fc8c
cephadm: list-networks: Avoid duplicated IPs
Fixes: https://tracker.ceph.com/issues/52083
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
2021-08-23 12:12:24 +02:00
Sebastian Wagner
1978cd3323
cephadm: Fix TestCephVolume.test_noop
Fixes `PermissionError: [Errno 13] Permission denied: '/var/lib/ceph/760b...5a5fbb'`

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
2021-08-23 12:08:22 +02:00
Aaryan Porwal
d4f0d949af doc/dev: mgr/dashboard: Visual regression tests for ceph dashboard
Fixes: https://tracker.ceph.com/issues/52370

Signed-off-by: Aaryan Porwal <aaryanporwal2233@gmail.com>
2021-08-23 14:18:15 +05:30
Mykola Golub
320e059c95
Merge pull request #41405 from ideepika/wip-rbd-update-feature-test
test/librbd: add unit tests for rbd update features

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
2021-08-23 11:33:41 +03:00
Xuehan Xu
52e32806c8 crimson/common: trace TLS crimson::interruptible::interrupt_cond<InterruptCond> modifications
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-08-23 11:03:15 +08:00
Yingxin Cheng
bd3cb2f67c crimson/errorator: remove unecessary template param
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-08-23 09:46:31 +08:00