Commit Graph

116831 Commits

Author SHA1 Message Date
Yan, Zheng
ff10bdbe16 mds: abort/pause/resume scrubs in multiple mds
Limit scrub abort/pause/resume commands to mds.0. mds.0 sends messages
to other mds, asks them to abort/pause/resume scrubs.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2020-11-16 09:02:18 +08:00
Yan, Zheng
ad5471f4cc mds: track scrub status in multiple mds
Scrubs are always initialized from mds.0. So mds.0 can ensure that scrub
tags are unique globally. mds.0 periodically gathers scrubs running in
itself and in other mds. A scrub is finished only if it's not running in
any mds.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2020-11-16 09:02:18 +08:00
Yan, Zheng
03908aa043 mds: remove on_finish from {CInode,CDir}::scrub_info_t
A CInode/CDir is scrubbed no longer means corresponding subtree is fully
scrubbed. The on_finish in {CInode,CDir}::scrub_info_t become useless.
This patch also removes code that flushs journal if scrub has repaired
anything.  Later patch will add the code back at different place.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2020-11-16 09:02:18 +08:00
Yan, Zheng
86fb5b48a8 Continuation: don't delete self while there are in-processing stages
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2020-11-16 09:02:17 +08:00
Yan, Zheng
f01ebadeda mds: auth pin CInode when validating its disk state
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2020-11-16 09:02:17 +08:00
Simon Gao
c8c3ba439a mds: rdlock file/nest lock when accumulating stats of subtree dirfrags
Signed-off-by: Simon Gao <simon29rock@gmail.com>
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2020-11-16 09:02:17 +08:00
Simon Gao
e5c4e8c0b0 mds: multiple mds scrub support
If a non-auth object is encountered during scrubbing, forward scrub
to the object's auth mds.

Fixes: https://tracker.ceph.com/issues/12274
Signed-off-by: Simon Gao <simon29rock@gmail.com>
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2020-11-16 09:02:17 +08:00
Yan, Zheng
458e7d6559 include/frag: add encode/decode functions for fragset_t
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2020-11-16 09:02:17 +08:00
Yan, Zheng
38c6fc8c7d mds: remove object can't be scrubbed immediately from scrub stack
This avoid checking objects that can't be scrubbed repeatly.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2020-11-16 09:02:17 +08:00
Yan, Zheng
a4c9bfc5b4 mds: prevent dirfrag scrub/fragment from running at the same time
When a CDir is in scrub stack, mds should not split/merge it.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2020-11-16 09:02:17 +08:00
Simon Gao
b43af152ba mds: change scrub traverse from post-order to breadth-first search
After using breadth-first search, scrubing a dir inode does not need
to wait until all of its descendant difrags/inodes are scrubbed. This
simplfies scrub code a lot. The downside is that a scrubbed dir inode
no longer implies corresponding subtree has been fully scrubbed. It
makes later scrub (without force option) less efficient.

Signed-off-by: Simon Gao <simon29rock@gmail.com>
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2020-11-16 09:02:17 +08:00
Yan, Zheng
547c1ed9f1 mds: make both CInode and CDir as entities of scrub
Making CDir as entity of scrub is preparetion for scrubbing across
multiple mds. When subtree bound is encountered, scrub should be
forwarded to subtree's auth mds. The auth mds adds CDir to scrub stack.

Signed-off-by: Simon Gao <simon29rock@gmail.com>
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2020-11-16 09:02:17 +08:00
Yan, Zheng
afb2e12078 mds: remove ScrubStack::scrubstack
comment says it's hack for dout. I don't see any reason it's needed.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2020-11-16 09:02:16 +08:00
Patrick Donnelly
07cba31a03
Merge PR #34842 into master
* refs/pull/34842/head:
	qa/tasks/vstart_runner.py: disable the ns unsharing if using localhost

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-11-15 12:39:11 -08:00
Patrick Donnelly
11b5847e7e
Merge PR #37746 into master
* refs/pull/37746/head:
	client: add a dedicated thread for the Client tick
	client: try to renew caps and flush old cap releases

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-11-15 12:38:05 -08:00
Patrick Donnelly
3541c8de12
Merge PR #37982 into master
* refs/pull/37982/head:
	qa/cephfs: add code for when config is None in __init__

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
2020-11-15 12:37:02 -08:00
Patrick Donnelly
359c8b0ddc
Merge PR #38016 into master
* refs/pull/38016/head:
	qa: switch to 'osdop_read' instead of 'op_r'

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-11-15 12:35:58 -08:00
Patrick Donnelly
680ed1a889
Merge PR #38056 into master
* refs/pull/38056/head:
	pybind/mgr/volumes: use = for equality comparision

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-11-15 12:34:00 -08:00
Brad Hubbard
a6c6ffa39f
Merge pull request #38077 from tchaikov/wip-doc-scrub-max
doc/rados/operation: add link to scrub_max_interval

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-11-16 06:19:09 +10:00
Kefu Chai
f47f13f446
Merge pull request #38038 from bk201/wip-48209
mgr/dashboard: fix cephadm e2e test failure on deleting OSDs

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-11-16 01:29:46 +08:00
Kefu Chai
0463a774c9
Merge pull request #37908 from dzafman/wip-47930
test: Fix race in TEST_recovery_scrub test

Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-11-16 01:00:56 +08:00
Kefu Chai
3c58e34f10
Merge pull request #38043 from jhonxue/wip-fix-bluestore-bitmap-allocator
osd: fix bluestore bitmap allocator calculate wrong last_pos with hint

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2020-11-16 00:38:35 +08:00
Kefu Chai
e0782a0e12
Merge pull request #37963 from aclamk/wip-bluestore-no-debug-misc
os/bluestore: Removed bluestore_debug_misc option

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2020-11-16 00:37:51 +08:00
Kefu Chai
d121f4b31e
Merge pull request #37999 from trociny/wip-48153
os/bluestore: fix "end reached" check in collection_list_legacy

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-11-16 00:34:12 +08:00
Kefu Chai
6f87bbcf71
Merge pull request #37630 from zjcmszh/aio_rmxattr
pybind/rados: add Ioctx::aio_rmxattr() and test aio_rmxattr()

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-11-16 00:32:22 +08:00
Kefu Chai
e3e3b10f5d
Merge pull request #37976 from batrick/msg-destructor
messages: use final specifier for destructor

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-11-16 00:31:21 +08:00
Kefu Chai
f7962b1767
Merge pull request #37992 from rishabh-d-dave/dashboard-qa-skipTest
qa/mgr: mgr_test_case: raise SkipTest instead of calling skipTest()

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
2020-11-16 00:30:42 +08:00
Kefu Chai
085facd179
Merge pull request #37998 from tchaikov/wip-log-mon-cleanup
log,mon: use libfmt to format string and use range-based loop

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
2020-11-16 00:18:17 +08:00
Kefu Chai
6120bb257e
Merge pull request #38026 from wjwithagen/wjw-fix-test-mon-build_initial
test/mon: FreeBSD returns ENOENT on resolv error

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-11-16 00:09:15 +08:00
Willem Jan Withagen
5ae4827970 test/mon: Accept ENOENT as returned value on FreeBSD
FreeBSD returns ENOENT on resolv error

Tracker: https://tracker.ceph.com/issues/48183
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2020-11-15 10:09:23 +01:00
Kefu Chai
2a32514706 doc/rados/operation: add link to scrub_max_interval
add the reference to the per-pool setting when explaing
PG_NOT_DEEP_SCRUBBED.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-11-15 11:50:12 +08:00
J. Eric Ivancich
67c8ee09fe
Merge pull request #38082 from adamemerson/wip-anti-nihilism
Asyncening 0: The Yielding Anti-Nihilist

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
2020-11-14 22:35:23 -05:00
Patrick Donnelly
81bcf9b40f
Merge PR #38071 into master
* refs/pull/38071/head:
	.github: add labeler for cephfs

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-11-14 10:04:12 -08:00
Kefu Chai
364e385a04
Merge pull request #38067 from dillaman/wip-github-rbd-label
github: add 'rbd' auto-labelling

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-11-15 00:53:38 +08:00
Kefu Chai
b1ddf085bb
Merge pull request #38013 from drakonstein/master
rpm,deb: change sudoers file mode to 440

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
2020-11-14 19:56:40 +08:00
Brad Hubbard
1464a7d588
Merge pull request #38073 from badone/wip-use-right-tunable-for-pg-scrub-doc
doc: Use correct tunable for PG_NOT_SCRUBBED

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-11-14 14:49:27 +10:00
Michael Fritch
1ab7a5753c
Merge pull request #37883 from diwilli/patch-1
cephadm: Set listen-addresses on alertmanager container

Reviewed-by: Patrick Seidensal <pseidensal@suse.com>
2020-11-13 19:27:59 -07:00
Michael Fritch
d2df27ef63
Merge pull request #38011 from mgfritch/cephadm-sudo-log-dir
cephadm: verify root privileges before log dir create

Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>
Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
2020-11-13 19:23:34 -07:00
Patrick Donnelly
20d9754dba
.github: add labeler for cephfs
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2020-11-13 15:17:36 -08:00
Jason Dillaman
273b9c6333 github: add 'rbd' auto-labelling
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-11-13 14:43:21 -05:00
Adam C. Emerson
6674a0d4a9 rgw: No null_yield in rgw_rest
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2020-11-13 14:35:15 -05:00
Adam C. Emerson
1fcc217e89 rgw: No null_yield in rgw_role
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2020-11-13 14:35:15 -05:00
Adam C. Emerson
89cb013e7a rgw: no null_yield in rgw_tools
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2020-11-13 14:35:15 -05:00
Adam C. Emerson
dec41841de rgw: No null_yield in rgw_torrent
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2020-11-13 14:35:15 -05:00
Adam C. Emerson
7e7c3f5614 rgw: No null_yield in rgw_user
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2020-11-13 14:35:15 -05:00
Adam C. Emerson
5760fb630e rgw: No null_yield in rgw_zone
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2020-11-13 14:35:15 -05:00
Adam C. Emerson
01ce52e52e rgw: No null_yield in RGWOp
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2020-11-13 14:35:15 -05:00
Adam C. Emerson
4cd42ce332 rgw/services: No null_yield
Null yield is fine at top levels where we know what kind of thread
we're running into, but do not belong in general use functions.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2020-11-13 14:35:15 -05:00
Neha Ojha
a0e1a8f173
Merge pull request #38057 from zdover23/wip-doc-dev-guide-essentials-source-clean-update
doc/dev: update essentials.rst

Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-11-13 09:42:20 -08:00
Neha Ojha
a0e7802863
Merge pull request #38036 from zdover23/wip-doc-dev-guide-basic-workflow-make-check-nov-2020
doc/dev: improve make check text

Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-11-13 09:38:28 -08:00