Commit Graph

145533 Commits

Author SHA1 Message Date
Zac Dover
a84ec3a1c0 doc/cephfs: separate commands into sections
Separate commands so that each command has its own subsection in the
section "FS Subvolumes" in the file doc/cephfs/fs-volumes.rst.
Previously, the list of commands for manipulating subvolumes was one
long, unbroken list and the beginning of one section could easily be
mistaken for the end of the previous section.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2024-05-20 16:29:44 +10:00
Kefu Chai
5e689efb37
Merge pull request #57011 from tchaikov/wip-cmake-legacy-option-header
cmake: link against legacy-option-headers instead of depending on it

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2024-05-19 21:56:02 +08:00
Zac Dover
0434d879d2
Merge pull request #57542 from zdover23/wip-doc-2024-05-18-ceph-fs-volumes-subvolume-quiesce-squid-note
doc/cephfs: Squid and later - subvolume quiesce

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2024-05-19 21:09:06 +10:00
Matan Breizman
e03f7be26d
Merge pull request #57383 from athanatos/sjust/wip-65867-user-version
crimson: fix user_version handling

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>
2024-05-19 13:30:28 +03:00
Matan Breizman
14a1ef9258
Merge pull request #56114 from xxhdx1985126/wip-seastore-teuthology-max-test-attr-len
src/test/TestRados: add max-attr-len to control the max length of attributes sent to OSDs

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2024-05-19 13:28:24 +03:00
NitzanMordhai
fd7b38b264
Merge pull request #55008 from NitzanMordhai/wip-nitzan-clear-data-digest-for-crc-check
osd: full-object read CRC mismatch due to 'truncate' modifying oi.size w/o clearing 'data_digest'
2024-05-19 10:29:44 +03:00
NitzanMordhai
06e4c6f205 ceph_test_rados_io_pp: Add cxx test for append zero test
1. adding allow_ec_overwrite option for cxx test
2. adding new test for crc failuer check with append zero length

Fixes: https://tracker.ceph.com/issues/53240
Signed-off-by: Nitzan Mordechai <nmordech@redhat.com>
2024-05-19 05:03:15 +00:00
NitzanMordhai
6662e6bc21 test/pybind: check crc fail after append zero
Add test for zero crc check failed.

Fixes: https://tracker.ceph.com/issues/53240
Signed-off-by: Nitzan Mordechai <nmordech@redhat.com>
2024-05-19 05:03:15 +00:00
jiawd
afd42666ea osd: Adjust code style
Fixes: https://tracker.ceph.com/issues/53240
Signed-off-by: jiawd <jiawendong@xtaotech.com>
2024-05-19 05:03:15 +00:00
jiawd
83d3523bb0 osd: Remove the duplicate oi.clear_data_digest
Fixes: https://tracker.ceph.com/issues/53240
Signed-off-by: jiawd <jiawendong@xtaotech.com>
2024-05-19 05:03:15 +00:00
jiawd
aeafd2e9a4 osd: full-object read crc is mismatch, because truncate modify oi.size and forget to clear data_digest
when write before truncate, need trim length, if truncate is to 0,
write is [0~128k], write change to [0~0], do nothing, oi.size is 0, x1 = set_data_digest(crc32(-1)).
write is [128k~128k], write change to [128k~0], truncate oi.size to offset 128k, x2 = set_data_digest(crc32(x1)).
write is [256k~128k], write change to [256k~0], truncate oi.size to offset 256k, x3 = set_data_digest(crc32(x2)).
...
write is [4063232~128k], write change to [4063232~0], truncate oi.size to offset 4063232, xn = set_data_digest(crs32(xn-1))
Now, we can see oi.size is 4063232, and data_digest is 0xffffffff, because thelength of in_data of crc is 0 every time.
when read verify crc will reply EIO. (EC pool).

so, when truncate in write, need clear data_digest and DIGEST flag,
when write before truncate, need to trim length, when offset over than oi.size, don't truncate oi.size to offset.

Fixes: https://tracker.ceph.com/issues/53240
Signed-off-by: jiawd <jiawendong@xtaotech.com>
2024-05-19 05:03:14 +00:00
Zac Dover
be63ca6d03 doc/cephfs: Squid and later - subvolume quiesce
Add a note to the "Subvolume quiesce" section that says that the
information in the section applies only to the Squid and later releases
of Ceph. This is included here so that I don't overwrite the Reef and
Quincy documentation with irrelevant information, and so that I don't
overwrite the Squid information with blank space where the "Subvolume
quiesce" section should be.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2024-05-19 10:00:29 +10:00
Ilya Dryomov
b6f5f455c1
Merge pull request #57433 from idryomov/wip-65813
librbd: don't crash on a zero-length read if buffer is NULL

Reviewed-by: Ramana Raja <rraja@redhat.com>
2024-05-18 22:30:29 +02:00
Ilya Dryomov
f319358771
Merge pull request #56310 from ajarr/wip-64800
librbd: create rbd_trash object during pool initialization and namespace creation

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2024-05-18 22:28:56 +02:00
Zac Dover
a6fb74948f
Merge pull request #57534 from zdover23/wip-doc-2024-05-17-ceph-fs-volumes-2-of-x
doc/cephfs: edit fs-volumes.rst (2 of x)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2024-05-18 21:27:45 +10:00
Rishabh Dave
b494674fd0
Merge pull request #57528 from rishabh-d-dave/fix-test_rootsquash_nofeature
qa/cephfs: add MDS_CLIENTS_BROKEN_ROOTSQUASH to ignorelist

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2024-05-17 17:56:22 +05:30
Rishabh Dave
19ee59ecab
Merge pull request #57496 from rishabh-d-dave/block-test_idem_unaffected_root_squash
qa/cephfs: block buggy tests in test_admin.py

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2024-05-17 17:56:01 +05:30
Rishabh Dave
6859fe6b2e
Merge pull request #57333 from rishabh-d-dave/fs-fail-cache-kclient
qa/cephfs: set joinable on FS before exiting tests in TestFSFail

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2024-05-17 17:55:44 +05:30
Zac Dover
0a3981a011 doc/cephfs: edit fs-volumes.rst (2 of x)
Edit doc/cephfs/fs-volumes to the section "Cloning Snapshots" (but not
including the section "Cloning Snapshots".

Follows https://github.com/ceph/ceph/pull/57415

Signed-off-by: Zac Dover <zac.dover@proton.me>
2024-05-17 20:46:28 +10:00
Aashish Sharma
f72021bb9e
Merge pull request #57497 from rhcs-dashboard/fix-66058-main
mgr/dashboard: Fix host count per cluster and total hosts count on multi-cluster overview page


Reviewed-by: Nizamudeen A <nia@redhat.com>
2024-05-17 14:37:10 +05:30
Ilya Dryomov
c38bb1b353
Merge pull request #57498 from idryomov/wip-link-to-mon-osd-blocklist-default-expire
common/options: link to mon_osd_blocklist_default_expire from RBD

Reviewed-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
Reviewed-by: N Balachandran <nibalach@redhat.com>
2024-05-17 09:53:09 +02:00
Ilya Dryomov
1579f3649e
Merge pull request #52560 from petrutlucian94/rbd_service_restart_test
qa: add ceph-rbd windows service restart test 

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2024-05-17 09:09:38 +02:00
Nizamudeen A
407d44ee9b
Merge pull request #57253 from rhcs-dashboard/core_events_migration_for_c9
mgr/k8sevents: update V1Events to CoreV1Events

Reviewed-by: Dan Mick <dmick@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2024-05-17 12:01:02 +05:30
Aashish Sharma
b72a68301a mgr/dashboard: Fix host count per cluster and total hosts count on multi-cluster overview page
Fixes: https://tracker.ceph.com/issues/66058

Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2024-05-17 10:28:07 +05:30
Rishabh Dave
b7d07700d6 qa/cephfs: block buggy tests in test_admin.py
Block test_idem_unaffected_root_squash temporarily and
test_multifs_single_path_rootsquash.

This test fails due to a known bug. Block it temporarily so that
test_admin.py can run fully and PRs under QA can be tested fully.
Otherwise, this test fails and that halts test_admin.py, which leaves
the PR partially untested.

This failure is then seen as an unrelated failure which lets the buggy
code get merged. This has happened recently.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2024-05-17 10:18:13 +05:30
Yingxin
fb2023ea3c
Merge pull request #57474 from xxhdx1985126/wip-seastore-rewrite-bug-fix
crimson/os/seastore/transaction_manager: correct the offset of the data copied from the original extents

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2024-05-17 10:36:22 +08:00
Patrick Donnelly
bfe574c6ce
Merge PR #57302 into main
* refs/pull/57302/head:
	qa/tasks/quiescer: dump ops in parallel

Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
2024-05-16 21:12:51 -04:00
Kefu Chai
fcb2eb7761
Merge pull request #57260 from tchaikov/test_back_trace-with-asan
test/common/test_back_trace: skip one more frame when testing w/ ASan

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2024-05-17 08:46:33 +08:00
Kefu Chai
84484c0730
Merge pull request #57429 from tchaikov/dencoder-unregister
tools/ceph-dencoder: unregister dencoders in dtor

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2024-05-17 08:41:59 +08:00
Rishabh Dave
281183dd14 qa/cephfs: add MDS_CLIENTS_BROKEN_ROOTSQUASH to ignorelist
MDS_CLIENTS_BROKEN_ROOTSQUASH is generated and expected by
test_rootsquash_nofeature but it hasn't be added to ignorelist as a
result of which QA code marks the job as failed even though all tests
finished running successfully.

Introduced-by: bccc8ceb47
Fixes: https://tracker.ceph.com/issues/66075
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2024-05-16 22:13:28 +05:30
Rishabh Dave
faa30e03f3 qa/cephfs: set joinable on FS before exiting tests in TestFSFail
After running TestFSFail, CephFSTestCase.tearDown() fails attempting
to unmount CephFS. Set joinable on FS and wait for the MDS to be up
before exiting the test. This will ensure that unmounting is
successful in teardown.

Fixes: https://tracker.ceph.com/issues/65841
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2024-05-16 22:11:01 +05:30
Patrick Donnelly
15f734ec62
qa/tasks/quiescer: dump ops in parallel
Since this --flags=locks takes the mds_lock and dumps thousands of ops, this
may take a long time to complete for each individual MDS. The entire quiesce
set may timeout (and all q ops killed) before we finish dumping ops.

Fixes: https://tracker.ceph.com/issues/65823
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2024-05-16 12:11:49 -04:00
Yuri Weinstein
c2fb93636e
Merge pull request #56640 from NitzanMordhai/wip-nitzan-public-addr-check-subnet-unreachable
common/pick_address: check if address in subnet all public address

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Prashant D <pdhange@redhat.com>
2024-05-16 08:12:51 -07:00
Casey Bodley
b25c182d5b
Merge pull request #57479 from ktdreyer/wip-66016-qatlib-arches
cmake: disable WITH_QATLIB/ZIP on non-x86

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2024-05-16 15:15:23 +01:00
NitzanMordhai
48db64c217
Merge pull request #56743 from NitzanMordhai/wip-nitzan-backword-forword-dencoder-tests
suites: adding dencoder test multi versions
2024-05-16 15:40:11 +03:00
Ilya Dryomov
572909629c common/options: link to mon_osd_blocklist_default_expire from RBD
"number of seconds to blocklist - set to 0 for OSD default" in the
description of rbd_blocklist_expire_seconds refers to the value that is
controlled by mon_osd_blocklist_default_expire.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2024-05-16 12:50:00 +02:00
Venky Shankar
999ca78a1a Merge PR #56944 into main
* refs/pull/56944/head:
	qa: add a YAML to ignore MGR_DOWN warning

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2024-05-16 14:52:13 +05:30
Yingxin
d50b94f93f
Merge pull request #55735 from xxhdx1985126/wip-crimson-errorator-void-handler
crimson/common/errorator: disallow void-returning error handlers

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
2024-05-16 15:57:48 +08:00
Yingxin
a8099a6c30
Merge pull request #57432 from cyx1231st/wip-seastore-cleanup-objdata-read
crimson/os/seastore/object_data_handler: cleanup read()

Reviewed-by: Myoungwon Oh <myoungwon.oh@samsung.com>
Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>
2024-05-16 14:18:46 +08:00
Venky Shankar
e0efed739d Merge PR #56941 into main
* refs/pull/56941/head:
	mds: find a new head for the batch ops when the head is dead

Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2024-05-16 11:29:17 +05:30
Venky Shankar
deb2cddb7e Merge PR #57275 into main
* refs/pull/57275/head:
	qa/fsx: use a specified sha1 to build the xfstest-dev

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
2024-05-16 11:26:45 +05:30
nmordech@redhat.com
3f26a965f6 suites: adding dencoder test multi versions
We are currently conducting regular ceph-dencoder tests for backward compatibility.
However, we are omitting tests for forward compatibility.
This suite will introduce tests against the ceph-objects-corpus to address forward
compatibility issues that may arise.
the script will install N-2 version and run against the latest version corpus objects
that we have, then install N-1 to N version and check them as well.

Signed-off-by: Nitzan Mordechai <nmordech@redhat.com>
2024-05-16 05:16:17 +00:00
Patrick Donnelly
cbb1da1189
Merge PR #57454 into main
* refs/pull/57454/head:
	mds/quiesce-db: optimize peer updates
	mds/quiesce-db: track db epoch separately from the membership epoch
	mds/quiesce-db: test that a peer on a newer membership epoch can ack a root

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2024-05-15 23:01:16 -04:00
Patrick Donnelly
70ed3825f8
Merge PR #57274 into main
* refs/pull/57274/head:
	mds: don't stall the asok thread for flush commands
	qa/quiescer: relax some timing requirements in the quiescer

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2024-05-15 22:56:38 -04:00
Patrick Donnelly
3e92f50796
Merge PR #57329 into main
* refs/pull/57329/head:
	qa: unmount clients before damaging the fs

Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2024-05-15 22:56:11 -04:00
Patrick Donnelly
76e57bc03e
Merge PR #57334 into main
* refs/pull/57334/head:
	mds: remove erroneous debug message

Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
2024-05-15 22:55:19 -04:00
Yingxin
d0cccde9ea
Merge pull request #57476 from cyx1231st/wip-seastore-drop-unused
crimson/os/seastore/transaction_manager: drop unused code

Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>
2024-05-16 09:34:52 +08:00
Laura Flores
492cb97a16
Merge pull request #55985 from Svelar/make-dist
make-dist: remove old cruft recursively
2024-05-15 17:08:28 -05:00
Ramana Raja
4117b8e374 common/options: make 'rbd_validate_pool' a dev option
'rbd_validate_pool' option is for testing purposes only. So change
its level appropriately to 'dev'.

Signed-off-by: Ramana Raja <rraja@redhat.com>
2024-05-15 15:32:24 -04:00
Ramana Raja
0bcb5cfce9 librbd/image: create rbd_trash object during RBD pool initialization
... and RBD namespace creation.

It was not possible to remove a RBD image when OSDs were full and the
'rbd_trash' object was not already created in the image's pool or pool
namespace. The 'rbd_trash' object was created in a pool or namespace
during the first instance of image removal from the pool or namespace.
If no images were ever removed from a RBD pool or namespace and the
OSDs became full, removal of images using the CLI failed. The failure
occured when trying to move the images to trash since the 'rbd_trash'
object was missing in the pool or namespace.

Fix this issue by creating the rbd_trash object in a pool when
initalizing the pool as a RBD pool and when creating a RBD namespace.

Fixes: https://tracker.ceph.com/issues/64800
Signed-off-by: Ramana Raja <rraja@redhat.com>
2024-05-15 15:32:15 -04:00