Commit Graph

122172 Commits

Author SHA1 Message Date
Sage Weil
5adef5f766 mgr/orchestrator: report osds as osd.unmanaged as appropriate
If there is no osdspec_affinity or service_name (from unit.meta), then
report as 'osd.unmanaged'.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-09 16:23:20 -04:00
Samuel Just
055ebe381d
Merge pull request #39216 from myoungwon/wip-manifest-dedup-test
osd, test: reworks for manifest dedup test cases

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-04-09 12:42:18 -07:00
Sage Weil
2b63ae25c9 mgr/orchestrator: remove IMAGE ID from 'orch ls'
This is not very useful at this level:
 - we see it from 'orch ps'
 - it can be a mix of ids during upgrade
 - some services may have multiple images at steady state (e.g., ingress)

Signed-off-by: Sage Weil <sage@newdream.net>
2021-04-09 15:35:17 -04:00
J. Eric Ivancich
098eddaeee rgw: fix radoslist stuck loop
When an incomplete multipart upload has in excess of 1000 parts,
looping over those parts was not handled property causing an infinite
loop. The paging/marker is now handled correctly.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2021-04-09 15:28:01 -04:00
Kefu Chai
b026037d44 qa/suites: test upgrade/octopus-x on focal instead bionic
so we can solely build on focal in future once all other bionic facets
in qa/ is removed or replaced.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-10 00:51:40 +08:00
Yuri Weinstein
5f2f793af2
Merge pull request #40623 from ronen-fr/wip-ronenf-revert-40077
osd: Revert "osd: Try other PGs when reservation failures occur"

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-04-09 07:45:55 -07:00
Yuri Weinstein
63a3452248
Merge pull request #40606 from myoungwon/wip-49427-2
osd: recover unreadable snapshot before reading refcount info

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-04-09 07:44:42 -07:00
Jos Collin
90ec86d8ff
qa: Update the mdsmap schema in mgr/dashboard/test_health.py
Fixed the error:
tasks.mgr.dashboard.helper._ValError: In `input['fs_map']['filesystems'][0]['mdsmap']`: unknown keys: {'flags_state'}
Traceback (most recent call last):
File "/home/jenkins-build/build/workspace/ceph-api/qa/tasks/mgr/dashboard/test_health.py", line 273, in test_full_health
self.assertSchema(data, schema)

Fixes: https://tracker.ceph.com/issues/48683
Signed-off-by: Jos Collin <jcollin@redhat.com>
2021-04-09 17:34:19 +05:30
Alfonso Martínez
8bad7360ef mgr/dashboard: fix errors when creating NFS export.
- Fix daemon raw config parsing.
- Handle error when no rgw daemons found.

Fixes: https://tracker.ceph.com/issues/49925
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
2021-04-09 10:51:21 +02:00
myoungwon oh
9d7844b305 osd: avoid for the two copy to cancel each other
add the op to blocked_list, then return early
if the destination of copy-from already exists and
the head object is same

fixes: https://tracker.ceph.com/issues/49726

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-04-09 10:15:04 +09:00
Josh Durgin
e734c2d123
Merge pull request #40510 from aclamk/wip-bluestore-sharding-rst
doc: Add BlueStore sharding documentation

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-04-08 15:36:27 -07:00
Patrick Donnelly
682a55e8a2
mds: reject lookup ino requests for mds dirs
Fixes: https://tracker.ceph.com/issues/49922
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-04-08 12:21:42 -07:00
Patrick Donnelly
4702be4f47
qa: test standby-replay with fs:workloads
Fixes: https://tracker.ceph.com/issues/50045
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-04-08 11:37:56 -07:00
Patrick Donnelly
9d7a42e709
Merge PR #40486 into master
* refs/pull/40486/head:
	mds: trim cache regularly for standby-replay
	mds: remove extra heap release

Reviewed-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
2021-04-08 11:35:05 -07:00
Patrick Donnelly
1cfc753e9a
Merge PR #40520 into master
* refs/pull/40520/head:
	mds/scrub: background scrub error fixes

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-04-08 11:34:21 -07:00
Patrick Donnelly
2f1d3aa629
Merge PR #40633 into master
* refs/pull/40633/head:
	mds: ensure export_pin rank < max_mds

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-04-08 11:33:41 -07:00
Patrick Donnelly
b02fa8b9de
Merge PR #40638 into master
* refs/pull/40638/head:
	mds: do not show the default auth if it's unambiguous
	mds: switch to rank number instead

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-04-08 11:33:08 -07:00
Patrick Donnelly
e1748e7a83
mon: check mdsmap is resizeable before promoting standby-replay
If any MDS is up:creating, some rank data structures may not exist yet.

Fixes: https://tracker.ceph.com/issues/50215
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-04-08 11:24:49 -07:00
Kefu Chai
8e7e4420d0
Merge pull request #40617 from tchaikov/wip-system-pmem
install-deps.sh: install libpmem libraries if WITH_PMEM is set

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-04-08 22:57:23 +08:00
Patrick Donnelly
1b8ee16c13
Merge PR #40467 into master
* refs/pull/40467/head:
	doc: detail `fs snapshot mirror daemon status` mgr command
	doc: s/<fs>/<fs_name>/g

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sébastien Han <seb@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
2021-04-08 07:28:10 -07:00
Kefu Chai
8202aa3ddf
Merge pull request #40654 from rzarzynski/wip-crimson-notify-lifetime
crimson/osd: fix the lifetime of Notify during timeouts

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-04-08 20:37:41 +08:00
Ilya Dryomov
99f3a452ef
Merge pull request #40641 from idryomov/wip-require-ceph-common-for-ioc
packaging: require ceph-common for immutable object cache daemon

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-04-08 13:07:44 +02:00
Radoslaw Zarzynski
03d35060d6 crimson/osd: improve debugs around Watch / Notify.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-04-08 09:49:51 +00:00
Radoslaw Zarzynski
aa4c90254e crimson/osd: fix the lifetime of Notify during timeouts.
This fixes a segfault during `LibRadosWatchNotify.WatchNotify2Timeout`.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-04-08 09:49:27 +00:00
Venky Shankar
bfaf5e9929 doc: detail fs snapshot mirror daemon status mgr command
@leseb noticed that this interface did not have any documentation. Thanks!

Fixes: http://tracker.ceph.com/issues/50229
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2021-04-08 15:13:58 +05:30
Venky Shankar
9d60ec8ceb doc: s/<fs>/<fs_name>/g
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2021-04-08 15:13:58 +05:30
Kefu Chai
69a7ed4eab run-make-check: enable WITH_RBD_RWL when WITH_PMEM is true
so we can at least build-test the rwl cache plugin in "make check" run.

this is an opt-in option which is only enabled if WITH_PMEM is "true",
we will set it in the corresponding ceph-build job.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-08 17:16:46 +08:00
Kefu Chai
17d2bc3707 install-deps.sh: install libpmem libraries if WITH_PMEM is set
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-08 17:15:15 +08:00
Kefu Chai
5c486aa77d do_freebsd.sh: disable WITH_SYSTEMD
as FreeBSD does not has systemd at the time of writing, so it does not
make sense to build or install systemd support.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-08 16:59:39 +08:00
Kefu Chai
8c44876be4 cmake: s/HAVE_MSGHDR/WITH_SYSTEMD/
as FreeBSD also has msghdr but it does not have systemd, or
flags like MFD_ALLOW_SEALING, O_TMPFILE or F_SEAL_GROW. so
use WITH_SYSTEMD for enabling journald backend of logging system.

also move the option of "WITH_SYSTEMD" up so that the src/CMakeLists.txt
is able to see the variable of WITH_SYSTEMD defined by it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-04-08 16:58:50 +08:00
myoungwon oh
efd89f9beb osd: do not assert() in the case of no obc
Upon rollback, we should handle ENOENT case,
so what we should do here is to return NOOP.

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-04-08 17:38:14 +09:00
myoungwon oh
779e60e5bc osd: remove unnecessary ref handling in _delete_oid
Let's consider the following case when handling a delete op.
1. Delete --> whiteouted
2. Make clone

In this case, current code clears chunk_map and calls dec_all_manifest_refcount()
in _delete_oid() even if the clone still has the references.

To fix this, This commit remove unnecessary ref handling in _delete_oid, and
makes finish_ctx() to handle ref handling, which can aware of whether the
clone is created or not.

Also, remove oi.size == 0 condition in finish_ctx() to handle ref. counting
upon a delete op with whitedouted clone.

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-04-08 17:35:59 +09:00
Adam Kupczyk
a2067a02d7 doc: Add BlueStore sharding documentation
Provide sharding explanation, configuration parameters explanation.
Provide resharding tool example into Pacific format.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
Co-authored-by: Josh Durgin <jdurgin@redhat.com>
2021-04-08 09:27:58 +02:00
Kefu Chai
e42c93c785
Merge pull request #40596 from tchaikov/wip-cmake-clean
cmake: remove created directory when "clean" target is run

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2021-04-08 15:26:07 +08:00
Rishabh Dave
511434ac5a qa/cephfs: remove create_keyring_file from cephfs_test_case.py
Since teuthology.orchestra.remote.mktemp() can write a temporary file
and not just create it, create_keyring_file() is now redundant.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2021-04-08 12:11:00 +05:30
Rishabh Dave
e213849581 qa/cephfs: don't use sudo to write files in /tmp
Files in /tmp cannot be written by any user( including the root user)
other than the file owner even if the permission mode on the file is
777.

Fixes: https://tracker.ceph.com/issues/49466
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2021-04-08 11:56:00 +05:30
Venky Shankar
714d51301b
Merge pull request #40342 from batrick/snap_schedule-qa-fixes
qa: snap_schedule test debugging

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2021-04-08 11:51:05 +05:30
Jos Collin
26b7e5855b
doc: add lsflags command to Administrative Commands document
Fixes: https://tracker.ceph.com/issues/48682
Signed-off-by: Jos Collin <jcollin@redhat.com>
2021-04-08 11:09:18 +05:30
Jos Collin
58b62f10d9
qa: test fs lsflags command
Fixes: https://tracker.ceph.com/issues/48682
Signed-off-by: Jos Collin <jcollin@redhat.com>
2021-04-08 11:09:18 +05:30
Jos Collin
e5e0f859d3
mon: add command to print fs flags
Fixes: https://tracker.ceph.com/issues/48682
Signed-off-by: Jos Collin <jcollin@redhat.com>
2021-04-08 11:09:17 +05:30
Jos Collin
0838e9fc03
mds: print each flag value
Fixes: https://tracker.ceph.com/issues/48683
Signed-off-by: Jos Collin <jcollin@redhat.com>
2021-04-08 11:09:17 +05:30
Yuval Lifshitz
c43878d0bd
Merge pull request #40625 from TRYTOBE8TME/wip-rgw-notifications
rgw/test: Changing force-branch to master
2021-04-08 08:32:45 +03:00
YuanXin
db13b9abd5 osd: avoid two copy with same src cancel each other
For cache tier, if some head object has two snaps, the two snaps share the same clone object,
and the clone object was flush/evicted from cache pool, when a rollback requests and a read
snap request to these two snaps at the same time will generate two promote requests to the
same clone object, these two promote requests will generate two copy ops with same src, than
the second copy op will cancel the first copy op by calling cancel_copy and kick_object_context_blocked,
but after calling kick_object_context_blocked, a new promote request corresponding to first
copy op will be restarted and generate a new copy op, the new copy op will cancel the second
copy op again, so two promote requests will cancel their copy op each other and run into dead
loop.

Fixes: https://tracker.ceph.com/issues/49409

Signed-off-by: YuanXin <yuanxin@didiglobal.com>
2021-04-08 13:58:08 +09:00
Kefu Chai
dfca3f8cb1
Merge pull request #40622 from batrick/dev-make
doc: update some docs still referring to make

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-04-08 12:55:37 +08:00
Kefu Chai
50287ee333
Merge pull request #40647 from rzarzynski/wip-crimson-notify-send_comp-broken-debug
crimson/osd: fix broken debug in Notify::send_completion().

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
2021-04-08 11:58:56 +08:00
Patrick Donnelly
dd2e31a336
Merge PR #40204 into master
* refs/pull/40204/head:
	client: cleanup _preadv_pwritev_locked()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
2021-04-07 19:37:40 -07:00
Patrick Donnelly
c20ee8eedf
Merge PR #40427 into master
* refs/pull/40427/head:
	mds: skip the buffer in UnknownPayload::decode()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-04-07 19:37:00 -07:00
Patrick Donnelly
d06d66d88d
Merge PR #40611 into master
* refs/pull/40611/head:
	qa: retry cloning the ceph.git

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-04-07 19:35:58 -07:00
Kefu Chai
a9fe167c1a
Merge pull request #40636 from tchaikov/wip-50196
qa/tasks/mgr: skip test_diskprediction_local on python>=3.8

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2021-04-08 10:15:33 +08:00
Yanhu Cao
3017b78225 mds: ensure export_pin rank < max_mds
Signed-off-by: Yanhu Cao <gmayyyha@gmail.com>
2021-04-08 09:51:29 +08:00