Commit Graph

144072 Commits

Author SHA1 Message Date
Patrick Donnelly
db438cf990
qa/cephfs: add probabilistic ignorelist for pg_health
PG_AVAILABILITY/PG_DEGRADED warnings are dominating fs runs. We want the
underlying issue fixed but it cannot continue to fail all of our tests 100% of
the time. Use a probabilistic addition of these warnings to the ignorelist.

Fixes: https://tracker.ceph.com/issues/64984
Related-to: https://tracker.ceph.com/issues/52624
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2024-03-22 09:13:22 -04:00
Ronen Friedman
154411328e
Merge pull request #56337 from ronen-fr/wip-rf-qslow
osd/scrub: no "slow response" warning for queued reservations

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2024-03-22 12:50:21 +02:00
Ivo Almeida
17f3760628
Merge pull request #56372 from ivoalmeida/fix-edit-subvolume-group-size
mgr/dashboard: fix subvolume group edit size

Reviewed-by: Nizamudeen A <nia@redhat.com>
2024-03-22 09:40:02 +00:00
Nizamudeen A
625e312e86
Merge pull request #55953 from afreen23/wip-pg-targets
mgr/dashboard: Mark placement targets as non-required

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: ivoalmeida <NOT@FOUND>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2024-03-22 14:04:03 +05:30
Venky Shankar
cc26800f66 Merge PR #54581 into main
* refs/pull/54581/head:
	doc/dev: update quiesce developer document
	qa: wrap quiesce verification to dump debugging on error
	qa: update quiesce tests for control via locallock
	qa: set archive path in vstart_runner
	qa: refactor CephFSMount.kill_background to optionally kill all background jobs
	qa: use kwarg for rank parameter
	qa: simplify calls to (rank|mds)_(tell|asok)
	Revert "pybind/mgr/volumes: block quiesce for critical .meta file"
	mds: remove is_root indication on quiesce_inode op
	mds: prevent new lock cache cons when invalidating an existing one
	mds: use XLOCK_WAIT For local lock xlockers
	mds: prevent new wrlocks on LocalLock if there exists any xlock waiter
	mds: block import discover when parent directory inode is quiesced
	mds: avoid issuing exclusive caps to clients lacking w caps
	mds: print lock cache during invalidation
	mds: use inodeno_t to track quiesce requests
	mds: dispatch quiesce_inode ops after dir traversal
	mds: remove quiescelock handling for SimpleLock type
	mds: quiescelock as local lock + cap masking
	qa: run quiesce unit tests in fs:functional
	qa: add quiesce protocol unit tests
	qa: detect partial migrations during large config of dist epin
	qa: use stdin-killer to timeout run_shell_payload
	qa: simplify run_shell argument processing
	doc: add dev docs for quiesce protocol
	pybind/mgr/volumes: block quiesce for critical .meta file
	mds: add vxattr to block quiesce on an inode
	mds: convert encoded ephemeral dist pin to flags
	mds: add counter to throttle quiesce
	mds: add quiesce set feature flag
	mds: skip non-head inodes for quiesce
	mds: add quiesce op
	mds: print all SimpleLock flags in debug output
	mds: pretty print mutation when dumping lock
	mds: add new inode quiescelock
	mds: use 128 bits for waiters on MDSCacheObject
	mds: provide mechanism to authpin while freezing
	mds: add command to get specific op
	mds: finish request before completing internal req
	mds: complete internal op if killed
	mds: avoid killing dead requests
	mds: add command to kill request
	mds: add path argument to `ops` and `dump tree` to stream result to local file
	mds: print internal_request filepaths if present
	mds: add more information to debug message
	mds: remove redundant parenthesis
	mds: implement Mutation::dump method
	mds: make LockType fields const
	mds: annotate mdr with try_rdlock_snap_layout failure
	mds: refactor if into switch
	mds: call Locker method using this
	mds: simplify assert
	mds: dump locks passed to Locker::acquire_locks
	mds: add LockOp::print method for debugging
	mds: use new insert template via print
	mds: add request result to mutation for analysis by tests
	mds: add comment on locking order rules
	mds: allow specifying rdlock position
	mds: remove dead method
	common: provide a template for object dumps
	common: support long running ops without slow warnings
	common: simplify loop
	common: add JSONFormatterFile class
	common: use more efficient vector for stack
	include: use larger int for large gathers

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
2024-03-22 12:54:29 +05:30
Venky Shankar
b4d5d7270c Merge PR #56219 into main
* refs/pull/56219/head:
	qa: change log-whitelist to log-ignorelist

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
2024-03-22 12:51:33 +05:30
Venky Shankar
f2a1a2f4f4 Merge PR #56245 into main
* refs/pull/56245/head:
	mds/quiesce-db: always clear the db if a membership is lost

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2024-03-22 12:50:11 +05:30
Venky Shankar
8449bab924
Merge pull request #56298 from batrick/i64986
qa/cephfs: ignorelist clog of MDS_UP_LESS_THAN_MAX

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2024-03-22 12:46:58 +05:30
Venky Shankar
ba0deb568a Merge PR #56299 into main
* refs/pull/56299/head:
	qa/suites/fs/nfs: use standard health ignorelist

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2024-03-22 12:45:10 +05:30
Nizamudeen A
ae777cc01b
Merge pull request #56054 from rhcs-dashboard/open-swagger-new-tab
mgr/dashboard: open api swagger in new tab

Reviewed-by: afreen23 <NOT@FOUND>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2024-03-22 11:42:06 +05:30
Rongqi Sun
29008b1d1a
Merge pull request #56379 from Svelar/doc_oE
doc/install: fix typos in openEuler-installation doc
2024-03-22 13:56:29 +08:00
Yingxin
e143d4a45d
Merge pull request #56246 from xxhdx1985126/wip-64957
crimson/os/seastore/btree: check for reserved ptrs when determining children stability

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
2024-03-22 13:56:03 +08:00
Nizamudeen A
82821585f5
Merge pull request #56237 from rhcs-dashboard/store-token-in-cookie
mgr/dashboard: securely store remote cluster token

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: ivoalmeida <NOT@FOUND>
2024-03-22 11:24:28 +05:30
Xuehan Xu
7dcd74ac09 crimson/os/seastore/btree: always check the stability of extents within
the current transaction's view

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2024-03-22 11:58:09 +08:00
Rongqi Sun
593ab1ad41 doc/install: fix typos in openEuler-installation doc
Signed-off-by: Rongqi Sun <sunrongqi@huawei.com>
2024-03-22 03:12:09 +00:00
Samuel Just
dbfb661eb9
Merge pull request #55886 from tchaikov/wip-ceph.spec-gcc-13
ceph.spec.in: bump gcc-toolset to 13 and use it on rhel>=8

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2024-03-21 14:24:39 -07:00
Adam King
6f35d3430f
Merge pull request #56369 from adk3798/nvmeof-new-fields
mgr/cephadm: add some NVMEof gateway fields to be configurable by cephadm

Reviewed-by: Gil Bregman <gbregman@il.ibm.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
2024-03-21 15:33:24 -04:00
Dan Mick
0d302a85fe
Merge pull request #56283 from Svelar/remove_WITH_ZBD
run-make: remove WITH_ZBD
2024-03-21 11:20:11 -07:00
Ivo Almeida
a19f2b6e3c mgr/dashboard: fix subvolume group edit size
Removed the 'infinite' string which was set when no value was provided

Fixes: https://tracker.ceph.com/issues/65049

Signed-off-by: Ivo Almeida <ialmeida@redhat.com>
2024-03-21 16:15:56 +00:00
Ivo Almeida
2c6a6de14b
Merge pull request #55691 from ivoalmeida/snapshot-schedule-retention-fix
mgr/dashboard: fix retention add for subvolume

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2024-03-21 15:36:26 +00:00
Gil Bregman
a16ac81c43 mgr/cephadm: add nvmeof configuration fields
Fixes: https://tracker.ceph.com/issues/64821

Signed-off-by: Gil Bregman <gbregman@il.ibm.com>
2024-03-21 11:18:25 -04:00
Gil Bregman
86a592730b python-common/ceph/deployment: add nvmeof configuration fields
Fixes: https://tracker.ceph.com/issues/64821

Signed-off-by: Gil Bregman <gbregman@il.ibm.com>
2024-03-21 11:18:05 -04:00
Casey Bodley
0c72fcc26a
Merge pull request #56008 from kchheda3/wip-notification-subsys
rgw/notification: add rgw notification specific debug log subsystem

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
2024-03-21 15:08:35 +00:00
Ronen Friedman
40d5064baa osd/scrub: removing the slow reservation response configuration option
Removing 'osd_scrub_slow_reservation_response', as it is not in use
anymore.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2024-03-21 17:06:13 +02:00
Yuri Weinstein
e6d2e6c44c
Merge pull request #55348 from chenyuanrun/fix-io-ctx-impl-flags
librados: use CEPH_OSD_FLAG_FULL_FORCE for IoCtxImpl::remove

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
2024-03-21 07:48:01 -07:00
Yuri Weinstein
d61ec169c2
Merge pull request #50622 from cofractal/close-range
common: use close_range on Linux

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2024-03-21 07:47:34 -07:00
Rongqi Sun
ab1dd05300 run-make: remove WITH_ZBD
This commit remove 'WITH_ZBD' in run-make.sh because of: https://github.com/ceph/ceph/pull/55054

Fixes: https://tracker.ceph.com/issues/64962
Signed-off-by: Rongqi Sun <sunrongqi@huawei.com>
2024-03-21 12:42:49 +00:00
Adam King
f0dda24397
Merge pull request #56278 from sachinpunadikar/vstart_ganesha_fix
vstart: Ganesha should not be started in DEBUG mode

Reviewed-by: Adam King <adking@redhat.com>
2024-03-21 07:01:47 -04:00
Afreen
a2cf128544 mgr/dashboard:Mark placement targets as non-required in bucket form
Fixes https://tracker.ceph.com/issues/64708

- adds info text explaining about placement targets
- adds fieldset to security, policy and tags for a11y
- removes placement target as required field
- when no placement target provided, request omits placement_target
  field
- hides placement atrgets ection for edit form since its not possible to
  edit after creation
- updates rgw-bucket.service.spec.ts unit test
- minor cleanups related to unncesseary css present, and changes in help
  text for object locking
- updated e2e tests for buckets to incorporate changes

Signed-off-by: Afreen <afreen23.git@gmail.com>
2024-03-21 16:05:53 +05:30
Ivo Almeida
971bfd8b4d
Merge pull request #56284 from ivoalmeida/snapshot-schedule-monthly-retention-frequency
mgr/dashboard: fix M retention frequency display

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
2024-03-21 09:10:57 +00:00
Laura Flores
88f8db5c4b
Merge pull request #56146 from ljflores/wip-tracker-64725
qa/suites/rados/singleton: add POOL_APP_NOT_ENABLED to ignorelist
2024-03-20 16:50:47 -05:00
Nizamudeen A
cab8141bf1
Merge pull request #56183 from rhcs-dashboard/fix-tags-null
mgr/dashboard: fix empty tags 

Reviewed-by: afreen23 <NOT@FOUND>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2024-03-21 00:47:04 +05:30
Nizamudeen A
09e55fb7c5
Merge pull request #52617 from TomNewChao/main
mgr/dashboard/frontend:Ceph dashboard supports multiple languages

Reviewed-by: Svelar <NOT@FOUND>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2024-03-21 00:28:38 +05:30
Yuri Weinstein
c1e6f35145
Merge pull request #55841 from Matan-B/wip-matanb-mon-rmsnap-fix
mon/OSDMonitor: fix rmsnap command

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2024-03-20 08:43:22 -07:00
Yuri Weinstein
bf30425f5e
Merge pull request #55409 from cybozu/add-ceph-exporter-to-deb-package
debian: add ceph-exporter package

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
2024-03-20 08:40:57 -07:00
Yuri Weinstein
98a7421080
Merge pull request #53308 from NitzanMordhai/wip-nitzan-qa-tasks-with-crush-rules
suites: qa tasks with crush rules

Reviewed-by: Samuel Just <sjust@redhat.com>
2024-03-20 08:37:45 -07:00
Patrick Donnelly
719d30d277
doc/dev: update quiesce developer document
To include changes relating to it now being a local lock that prevents mutable
caps.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2024-03-20 10:56:58 -04:00
Patrick Donnelly
48766b336d
qa: wrap quiesce verification to dump debugging on error
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2024-03-20 10:56:58 -04:00
Patrick Donnelly
c0f0997e17
qa: update quiesce tests for control via locallock
- Verify multirank quiesce incl. caps issued.
- Unset splitauth experimental

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2024-03-20 10:56:58 -04:00
Patrick Donnelly
561965131e
qa: set archive path in vstart_runner
For storing misc. test artifacts.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2024-03-20 10:56:58 -04:00
Patrick Donnelly
bb71023ced
qa: refactor CephFSMount.kill_background to optionally kill all background jobs
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2024-03-20 10:56:58 -04:00
Patrick Donnelly
71fd9a34a9
qa: use kwarg for rank parameter
Otherwise it gets included in the *args list. This is necessary after commit
`qa: simplify calls to (rank|mds)_(tell|asok)`.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2024-03-20 10:56:58 -04:00
Patrick Donnelly
464ef51525
qa: simplify calls to (rank|mds)_(tell|asok)
Instead of requiring the caller to put the arguments in a list, allow passing
as regular arguments.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2024-03-20 10:56:58 -04:00
Patrick Donnelly
92f1ed3ed1
Revert "pybind/mgr/volumes: block quiesce for critical .meta file"
This reverts commit 16e50abbd42a4b4f740816ba5f60bc259a6a96bf.

This flag is no longer necessary as the volumes plugin issues quiesce calls
against the data (i.e. root) directory of the subvolume rather than the
subvolume directory (with its associated .meta file).

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2024-03-20 10:56:58 -04:00
Patrick Donnelly
667c9a9c5a
mds: remove is_root indication on quiesce_inode op
This is no longer necessary with the change to a LocalLock quiescelock.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2024-03-20 10:56:57 -04:00
Patrick Donnelly
9b4a93f2b7
mds: prevent new lock cache cons when invalidating an existing one
The previous scheme invalidated a lock cache and then immediately removed it
from its Capability list. The lock cache would eventually be deleted but a new
one could be constructed shortly after. The main reason for this is that simply
invalidating the lock cache does not drive a state change in the local locks
preventing new writers. This is mostly important for acquiring the quiescelock.

This commit also corrects a bug where a MDLockCache would be created for a
given opcode type (like create) when the capability does not have the issued
cap (CEPH_CAP_DIR_CREATE). The bug would not cause any negative side-effects
but would hold locks unnecessarily when only MDS ops (and not the client
executing ops asynchronously) are acquiring the locks.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2024-03-20 10:56:57 -04:00
Patrick Donnelly
f813b8c511
mds: use XLOCK_WAIT For local lock xlockers
This avoids waking up all waiters when only WAIT_XLOCK waiters should wake.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2024-03-20 10:56:57 -04:00
Patrick Donnelly
bddd3c7201
mds: prevent new wrlocks on LocalLock if there exists any xlock waiter
Otherwise, an xlock waiter can become starved as a LocalLock supports multiple
writers.

Strictly speaking, a new lock state would be appropriate for this but we cheat
frequently with the LocalLock -- there is only one state. All transition checks
are already manually performed by the Locker.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2024-03-20 10:56:57 -04:00
Patrick Donnelly
1ec6817cc0
mds: block import discover when parent directory inode is quiesced
This is to prevent two racing ranks quiescing some root from exporting a tree
under a completely quiesced directory (inode). The state of that imported tree
may take time to quiesce and cause the root to be QUIESCED before all inodes
under it are actually quiesced.

If a dirfrag to be imported is discovered before the parent is quiesced, then
the quiesce traversal will issue a quiesce_inode op normally for parent which
will attempt to authpin the parent. That will block if the export is still
in-progress (causing quiesce to wait for the export to finish or abort).

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2024-03-20 10:56:57 -04:00
Patrick Donnelly
772f04afdf
mds: avoid issuing exclusive caps to clients lacking w caps
Signed-off-by: Leonid Usov <leonid.usov@ibm.com>
2024-03-20 10:56:57 -04:00