Commit Graph

118505 Commits

Author SHA1 Message Date
Sebastian Wagner
9f7c9bf411
Merge pull request #38560 from pcuzner/cephadm-maintenance-mode
cephadm: add host-maintenance subcommand

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
2021-01-07 11:02:15 +01:00
Sebastian Wagner
c6e4b9ab9c mgr/cephadm: make ceph orch {restart|...} asynchronous
Othwerwiese we'd block the CLI handler thread.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2021-01-07 10:52:02 +01:00
Sebastian Wagner
80cbe7ecaa
Merge pull request #38681 from pcuzner/enable-exporter
cephadm: enable exporter deployment at bootstrap

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2021-01-07 10:30:53 +01:00
Sebastian Wagner
3f93251603
Merge pull request #38704 from varshar16/wip-fix-rook-rm-service
mgr/rook: Remove service_type param from remove_service()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Travis Nielsen <tnielsen@redhat.com>
2021-01-07 10:29:49 +01:00
Mahati Chamarthy
ccd1047b25 librbd/cache: Implement aio_read operation
... and retire entries i.e. flush to OSD.
Support writesame, compare_and_write, discard
and invalidate IO operations with tests.

Signed-off-by: Lisa Li <xiaoyan.li@intel.com>
Signed-off-by: Mahati Chamarthy <mahati.chamarthy@intel.com>
Signed-off-by: Changcheng Liu <changcheng.liu@intel.com>
2021-01-07 09:54:25 +05:30
Jason Dillaman
9870f7a3d4
Merge pull request #38025 from MahatiC/wip-ssd-write
librbd/cache: Implement aio_write operation

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2021-01-06 22:45:09 -05:00
Xiubo Li
c87392ccb0 mds: add scrub perf counters support
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2021-01-07 07:53:39 +08:00
Yuri Weinstein
b84ff2b744
Merge pull request #38795 from yuriw/wip-yuriw-octopus-x-master
qa/tests: changed the dirs structure

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-01-06 15:23:56 -08:00
Patrick Donnelly
9450398ede
Merge PR #38792 into master
* refs/pull/38792/head:
	.github: add MDSMonitor to cephfs label

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-01-06 14:30:21 -08:00
Patrick Donnelly
7b39daab79
Merge PR #38587 into master
* refs/pull/38587/head:
	doc: add explanatory note about cache trim rate setting

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-01-06 13:34:38 -08:00
Yuri Weinstein
962df165b2 qa/tests: changed the dirs structure
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2021-01-06 13:12:44 -08:00
Yuri Weinstein
fd78328ca0
Merge pull request #38787 from yuriw/wip-yuriw-octopus-x-master
qa/tests: octopus-x suite added more workloads

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-01-06 12:47:06 -08:00
Neha Ojha
ca4bc20f9f
Merge pull request #38789 from neha-ojha/wip-fix-cephadm-task
qa/tasks/cephadm.py: do not create rbd pool by default

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2021-01-06 12:31:50 -08:00
Patrick Donnelly
2bce009d99
qa: add debug vstart_runner option
The default was changed to INFO recently but there was no way to restore
visibility of DEBUG messages.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-01-06 11:36:49 -08:00
Patrick Donnelly
cdabf0d507
.github: add MDSMonitor to cephfs label
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-01-06 10:59:41 -08:00
Patrick Donnelly
725a4b8b42
mon/MDSMonitor: add missing frozen checks
This has no known impact on testing.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-01-06 10:56:21 -08:00
Patrick Donnelly
2fa494f581
common: remove dead option
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-01-06 10:53:52 -08:00
Yuri Weinstein
03dc6cb3b5 qa/tests: octopus-x suite added more workloads
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2021-01-06 10:23:40 -08:00
Neha Ojha
f3af610721 qa/tasks/cephadm.py: do not create rbd pool by default
rados/cephadm/smoke* does not use the install task and the adjust-ulimits
dependency is met as a part of it. create_rbd_pool needs adjust-ulimits,
so for now we will disable create_rbd_pool by default and only set it
to true for the upgrade suite.

Signed-off-by: Neha Ojha <nojha@redhat.com>
2021-01-06 17:35:15 +00:00
Patrick Donnelly
d7cacef125
qa: get pool holding file data from file layout
Otherwise it always looks at the default data pool. For ec pools, this
may not be where the file data is.

Fixes: https://tracker.ceph.com/issues/48756
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-01-06 07:11:20 -08:00
Kefu Chai
1897b8cf57
Merge pull request #38714 from tchaikov/wip-better-vstart
vstart: format the customized options in ceph.conf

Reviewed-by: luo rixin <luorixin@huawei.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-01-06 22:53:34 +08:00
Patrick Donnelly
06fff9aa82
Merge PR #38769 into master
* refs/pull/38769/head:
	doc/cephfs: add data pool-MDS instructions link

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
2021-01-06 06:50:24 -08:00
Patrick Donnelly
eb47e990c3
mds: avoid spurious sleeps
Like trim_interval, don't sleep for small amounts of time. This avoids
spurious sleeps like:

    2020-12-25T00:14:22.242+0000 7f6a95884700 20 mds.0.cache upkeep thread waiting interval 0.000000108s
    2020-12-25T00:14:22.242+0000 7f6a95884700 20 mds.0.cache upkeep thread waiting interval 0.000000108s
    2020-12-25T00:14:22.242+0000 7f6a95884700 20 mds.0.cache upkeep thread waiting interval 0.000000108s
    2020-12-25T00:14:22.242+0000 7f6a95884700 20 mds.0.cache upkeep thread waiting interval 0.000000108s
    2020-12-25T00:14:22.242+0000 7f6a95884700 20 mds.0.cache upkeep thread waiting interval 0.000000108s
    2020-12-25T00:14:22.242+0000 7f6a95884700 20 mds.0.cache upkeep thread waiting interval 0.000000108s
    2020-12-25T00:14:22.242+0000 7f6a95884700 20 mds.0.cache upkeep thread waiting interval 0.000000108s

Also, fix the same issue in the Client.

Fixes: https://tracker.ceph.com/issues/48753
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-01-06 06:42:56 -08:00
Mahati Chamarthy
55099fb05b librbd/cache: Implement aio_write operation
...along with load_existing_entries and shutdown cache
operation

Signed-off-by: Lisa Li <xiaoyan.li@intel.com>
Signed-off-by: Mahati Chamarthy <mahati.chamarthy@intel.com>
Signed-off-by: Changcheng Liu <changcheng.liu@intel.com>
2021-01-06 19:06:32 +05:30
Sebastian Wagner
6b3c46f508 doc/dev: resurrect hidden dev/cephadm.rst
some time ago, a cephadm folder was added that did hide the
old dev documentation page.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2021-01-06 14:13:11 +01:00
Zac Dover
725d294218 doc/cephfs: add data pool-MDS instructions link
- This commit adds a link to the "Create a Ceph
File System" page. The link that it adds is to the
"Adding a data pool to the MDS" subsection of the
file layouts page.
- s/mds/file system/

Fixes: https://tracker.ceph.com/issues/48531
Signed-off-by: Zac Dover <zac.dover@gmail.com>
2021-01-06 15:52:24 +10:00
Kefu Chai
485ddccc35
Merge pull request #38713 from tchaikov/wip-ceph-annotations
ceph.in: return bytes as outbuf in error path, add type annotations

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-01-06 11:34:09 +08:00
Yan, Zheng
087e5cd8d6 mds: don't use dentry_key_t in C_IO_Dir_Commit_Ops
dentry_key_t uses std::string_view to access corresponding dentry's name.
C_IO_Dir_Commit_Ops() is executed by worker thread. It's possible dentry
gets freed before C_IO_Dir_Commit_Ops() gets executed.

Signed-off-by: "Yan, Zheng" <ukernel@gmail.com>
2021-01-06 11:24:13 +08:00
Yin Congmin
2093c88229 tools: update immutable object cache water mark
Signed-off-by: Yin Congmin <congmin.yin@intel.com>
2021-01-06 09:18:48 +08:00
Yin Congmin
b4b7161a12 doc/rbd: add the configuration description of immutable object cache throttle
Signed-off-by: Yin Congmin <congmin.yin@intel.com>
2021-01-06 08:45:45 +08:00
Neha Ojha
6398ca5ab3
Merge pull request #38777 from neha-ojha/wip-cephadm-rbd
qa: allow the cephadm task to create an RBD pool and add workload to use it

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>
2021-01-05 16:31:37 -08:00
Neha Ojha
fcff116216 qa/suites/upgrade/octopus-x: add rados_loadgenbig workload
Signed-off-by: Neha Ojha <nojha@redhat.com>
2021-01-05 21:36:04 +00:00
Neha Ojha
276a4fb8b0 qa/tasks/cephadm.py: create RBD pool
The ceph task already does this and a bunch of tests rely on this pool
being already present. Can be disabled by setting create_rbd_pool to False.

Signed-off-by: Neha Ojha <nojha@redhat.com>
2021-01-05 21:32:41 +00:00
Or Ozeri
0440604c35 librbd: check image size on crypto format
This commit adds a check to verify that an image is big enough before crypto formatting it.

Signed-off-by: Or Ozeri <oro@il.ibm.com>
2021-01-05 20:28:04 +02:00
Or Ozeri
1521f822bc librbd: nit fixes
This commit includes very small miscellaneous changes to librbd

Signed-off-by: Or Ozeri <oro@il.ibm.com>
2021-01-05 20:27:30 +02:00
Or Ozeri
069a8639fb librbd: fix crypto discard handling
This commit fixes a bug in the discard implementation of the crypto layer

Signed-off-by: Or Ozeri <oro@il.ibm.com>
2021-01-05 20:27:17 +02:00
Or Ozeri
2049102d37 librbd: start crypto layer sub-requests at crypto layer
This commit changes sub-requests created by the crypto layer to not go through the upper layers.
This is mainly to avoid deadlocks caused by an upper cache layer.

Signed-off-by: Or Ozeri <oro@il.ibm.com>
2021-01-05 20:25:11 +02:00
Patrick Donnelly
edd6d3ddcd
Merge PR #38742 into master
* refs/pull/38742/head:
	doc/cephfs/nfs: Add more details about path and pseudo path
	doc/cephfs/nfs: Add note about how to fetch port details with rook
	doc/cephfs/nfs: Add warning note about management of dashboard created exports

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
2021-01-05 08:38:41 -08:00
Jason Dillaman
20141ed67b
Merge pull request #38735 from changchengx/pwl_load_existing_cache
librbd/cache/pwl: correct update previous_sync_point_entry

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2021-01-05 09:33:49 -05:00
Jason Dillaman
6b40dddb27
Merge pull request #38520 from lixiaoy1/wip-doc-pwl
doc: add RBD persistent write-back cache

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2021-01-05 09:12:20 -05:00
lixiaoy1
948e1e82e2 doc: add RBD persistent write-back cache
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
2021-01-05 09:37:30 -05:00
Varsha Rao
948ea79fb0 doc/cephfs/nfs: Add more details about path and pseudo path
Fixes: https://tracker.ceph.com/issues/48731
Signed-off-by: Varsha Rao <varao@redhat.com>
2021-01-05 18:04:34 +05:30
Varsha Rao
e626e54b72 doc/cephfs/nfs: Add note about how to fetch port details with rook
Fixes: https://tracker.ceph.com/issues/48731
Signed-off-by: Varsha Rao <varao@redhat.com>
2021-01-05 18:04:34 +05:30
Varsha Rao
8ccb57069a doc/cephfs/nfs: Add warning note about management of dashboard created exports
Fixes: https://tracker.ceph.com/issues/48731
Signed-off-by: Varsha Rao <varao@redhat.com>
2021-01-05 18:04:34 +05:30
Jan Fajerski
2fa311a462 Merge PR #32776 into master
* refs/pull/32776/head:
	win32*.sh: fetch WNBD dependency
	win32*.sh: bump win32 boost dependency
	cmake: skip osd, Lua and neorados on Windows
	win32*.sh: improve lib handling
	win32*.sh: update openssl location
	win32*.sh: cleanup dependency build dir when targetting Windows
	win32*.sh: Strip binaries individually
	win32*.sh: Cleanup zip archive creation
	cmake,win32*.sh: Add OS=(ubuntu|suse) flag, for distro settings
	win32*.sh: Use nproc for CPU count when targeting win32
	test: Include compat.h to find uid_t/gid_t/gidlist when cross compiling
	win32*.sh: improve win32 dependency check
	win32*.sh: isolate network operations when building win32 dependencies
	cmake: drop Python dependency when targeting Windows
	win32*.sh: cleanup Windows build script
	win32*.sh: fix patch command
	win32*.sh: update Windows build scripts and readme

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2021-01-05 12:50:37 +01:00
Volker Theile
3fe715201c cephadm: Various properties like 'last_refresh' do not contain timezone
Fixes: https://tracker.ceph.com/issues/48068

Signed-off-by: Volker Theile <vtheile@suse.com>
2021-01-05 12:22:32 +01:00
Kefu Chai
02178b04d6
Merge pull request #38699 from rosinL/boost-paralllel
do_cmake: add a default value to BOOST_J

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-01-05 19:06:57 +08:00
zdover23
94e7c6f95a
Merge pull request #38745 from zdover23/wip-doc-jan-2021-dev-guide-basic-workflow-reorg
doc/dev: restructure basic-workflow.rst

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-01-05 18:34:19 +10:00
Kefu Chai
62bac2989d
Merge pull request #38749 from tchaikov/wip-cmake-check-nasm
cmake: only display the result of checking nasm once

Reviewed-by: Changcheng Liu <changcheng.liu@aliyun.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-01-05 10:32:48 +08:00
Patrick Donnelly
dc3bd41d42
Merge PR #38703 into master
* refs/pull/38703/head:
	test: move snap delete operation to non-root directory
	client: optionally check client permission on rmsnap()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-01-04 09:49:00 -08:00