Commit Graph

145538 Commits

Author SHA1 Message Date
Xuehan Xu
211ca3792e crimson/os/seastore/cached_extent: EXIST_MUTATION_PENDING also
considered as mutatione_pending

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2024-05-20 11:19:54 +08:00
Xuehan Xu
d4c78f1524 crimson/os/seastore/cached_extent: adjust
CachedExtent::on_replace_prior()

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2024-05-20 11:19:54 +08:00
Xuehan Xu
72d721ad72 crimson/os/seastore/btree: misc consolidations
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2024-05-20 11:19:54 +08:00
Xuehan Xu
fecf501c41 crimson/os/seastore/cache: drop unnecessary friend declarations
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2024-05-20 11:19:54 +08:00
Xuehan Xu
1b8a781f66 crimson/os/seastore/btree: add CachedExtent::on_rewrite() and
CachedExtent::is_rewrite()

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2024-05-20 11:19:51 +08:00
Xuehan Xu
4b155f3ee5 crimson/os/seastore/lba_manager: extents pass to LBALeafNode::update()
must either has a null parent tracker or point to the current leaf node

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2024-05-20 11:12:44 +08:00
Kefu Chai
5e689efb37
Merge pull request #57011 from tchaikov/wip-cmake-legacy-option-header
cmake: link against legacy-option-headers instead of depending on it

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2024-05-19 21:56:02 +08:00
Zac Dover
0434d879d2
Merge pull request #57542 from zdover23/wip-doc-2024-05-18-ceph-fs-volumes-subvolume-quiesce-squid-note
doc/cephfs: Squid and later - subvolume quiesce

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2024-05-19 21:09:06 +10:00
Matan Breizman
e03f7be26d
Merge pull request #57383 from athanatos/sjust/wip-65867-user-version
crimson: fix user_version handling

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>
2024-05-19 13:30:28 +03:00
Matan Breizman
14a1ef9258
Merge pull request #56114 from xxhdx1985126/wip-seastore-teuthology-max-test-attr-len
src/test/TestRados: add max-attr-len to control the max length of attributes sent to OSDs

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2024-05-19 13:28:24 +03:00
NitzanMordhai
fd7b38b264
Merge pull request #55008 from NitzanMordhai/wip-nitzan-clear-data-digest-for-crc-check
osd: full-object read CRC mismatch due to 'truncate' modifying oi.size w/o clearing 'data_digest'
2024-05-19 10:29:44 +03:00
NitzanMordhai
06e4c6f205 ceph_test_rados_io_pp: Add cxx test for append zero test
1. adding allow_ec_overwrite option for cxx test
2. adding new test for crc failuer check with append zero length

Fixes: https://tracker.ceph.com/issues/53240
Signed-off-by: Nitzan Mordechai <nmordech@redhat.com>
2024-05-19 05:03:15 +00:00
NitzanMordhai
6662e6bc21 test/pybind: check crc fail after append zero
Add test for zero crc check failed.

Fixes: https://tracker.ceph.com/issues/53240
Signed-off-by: Nitzan Mordechai <nmordech@redhat.com>
2024-05-19 05:03:15 +00:00
jiawd
afd42666ea osd: Adjust code style
Fixes: https://tracker.ceph.com/issues/53240
Signed-off-by: jiawd <jiawendong@xtaotech.com>
2024-05-19 05:03:15 +00:00
jiawd
83d3523bb0 osd: Remove the duplicate oi.clear_data_digest
Fixes: https://tracker.ceph.com/issues/53240
Signed-off-by: jiawd <jiawendong@xtaotech.com>
2024-05-19 05:03:15 +00:00
jiawd
aeafd2e9a4 osd: full-object read crc is mismatch, because truncate modify oi.size and forget to clear data_digest
when write before truncate, need trim length, if truncate is to 0,
write is [0~128k], write change to [0~0], do nothing, oi.size is 0, x1 = set_data_digest(crc32(-1)).
write is [128k~128k], write change to [128k~0], truncate oi.size to offset 128k, x2 = set_data_digest(crc32(x1)).
write is [256k~128k], write change to [256k~0], truncate oi.size to offset 256k, x3 = set_data_digest(crc32(x2)).
...
write is [4063232~128k], write change to [4063232~0], truncate oi.size to offset 4063232, xn = set_data_digest(crs32(xn-1))
Now, we can see oi.size is 4063232, and data_digest is 0xffffffff, because thelength of in_data of crc is 0 every time.
when read verify crc will reply EIO. (EC pool).

so, when truncate in write, need clear data_digest and DIGEST flag,
when write before truncate, need to trim length, when offset over than oi.size, don't truncate oi.size to offset.

Fixes: https://tracker.ceph.com/issues/53240
Signed-off-by: jiawd <jiawendong@xtaotech.com>
2024-05-19 05:03:14 +00:00
Zac Dover
be63ca6d03 doc/cephfs: Squid and later - subvolume quiesce
Add a note to the "Subvolume quiesce" section that says that the
information in the section applies only to the Squid and later releases
of Ceph. This is included here so that I don't overwrite the Reef and
Quincy documentation with irrelevant information, and so that I don't
overwrite the Squid information with blank space where the "Subvolume
quiesce" section should be.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2024-05-19 10:00:29 +10:00
Ilya Dryomov
b6f5f455c1
Merge pull request #57433 from idryomov/wip-65813
librbd: don't crash on a zero-length read if buffer is NULL

Reviewed-by: Ramana Raja <rraja@redhat.com>
2024-05-18 22:30:29 +02:00
Ilya Dryomov
f319358771
Merge pull request #56310 from ajarr/wip-64800
librbd: create rbd_trash object during pool initialization and namespace creation

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2024-05-18 22:28:56 +02:00
Zac Dover
a6fb74948f
Merge pull request #57534 from zdover23/wip-doc-2024-05-17-ceph-fs-volumes-2-of-x
doc/cephfs: edit fs-volumes.rst (2 of x)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2024-05-18 21:27:45 +10:00
Rishabh Dave
b494674fd0
Merge pull request #57528 from rishabh-d-dave/fix-test_rootsquash_nofeature
qa/cephfs: add MDS_CLIENTS_BROKEN_ROOTSQUASH to ignorelist

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2024-05-17 17:56:22 +05:30
Rishabh Dave
19ee59ecab
Merge pull request #57496 from rishabh-d-dave/block-test_idem_unaffected_root_squash
qa/cephfs: block buggy tests in test_admin.py

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2024-05-17 17:56:01 +05:30
Rishabh Dave
6859fe6b2e
Merge pull request #57333 from rishabh-d-dave/fs-fail-cache-kclient
qa/cephfs: set joinable on FS before exiting tests in TestFSFail

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2024-05-17 17:55:44 +05:30
Zac Dover
0a3981a011 doc/cephfs: edit fs-volumes.rst (2 of x)
Edit doc/cephfs/fs-volumes to the section "Cloning Snapshots" (but not
including the section "Cloning Snapshots".

Follows https://github.com/ceph/ceph/pull/57415

Signed-off-by: Zac Dover <zac.dover@proton.me>
2024-05-17 20:46:28 +10:00
Aashish Sharma
f72021bb9e
Merge pull request #57497 from rhcs-dashboard/fix-66058-main
mgr/dashboard: Fix host count per cluster and total hosts count on multi-cluster overview page


Reviewed-by: Nizamudeen A <nia@redhat.com>
2024-05-17 14:37:10 +05:30
Ilya Dryomov
c38bb1b353
Merge pull request #57498 from idryomov/wip-link-to-mon-osd-blocklist-default-expire
common/options: link to mon_osd_blocklist_default_expire from RBD

Reviewed-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
Reviewed-by: N Balachandran <nibalach@redhat.com>
2024-05-17 09:53:09 +02:00
Ilya Dryomov
1579f3649e
Merge pull request #52560 from petrutlucian94/rbd_service_restart_test
qa: add ceph-rbd windows service restart test 

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2024-05-17 09:09:38 +02:00
Nizamudeen A
407d44ee9b
Merge pull request #57253 from rhcs-dashboard/core_events_migration_for_c9
mgr/k8sevents: update V1Events to CoreV1Events

Reviewed-by: Dan Mick <dmick@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2024-05-17 12:01:02 +05:30
Aashish Sharma
b72a68301a mgr/dashboard: Fix host count per cluster and total hosts count on multi-cluster overview page
Fixes: https://tracker.ceph.com/issues/66058

Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2024-05-17 10:28:07 +05:30
Rishabh Dave
b7d07700d6 qa/cephfs: block buggy tests in test_admin.py
Block test_idem_unaffected_root_squash temporarily and
test_multifs_single_path_rootsquash.

This test fails due to a known bug. Block it temporarily so that
test_admin.py can run fully and PRs under QA can be tested fully.
Otherwise, this test fails and that halts test_admin.py, which leaves
the PR partially untested.

This failure is then seen as an unrelated failure which lets the buggy
code get merged. This has happened recently.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2024-05-17 10:18:13 +05:30
Yingxin
fb2023ea3c
Merge pull request #57474 from xxhdx1985126/wip-seastore-rewrite-bug-fix
crimson/os/seastore/transaction_manager: correct the offset of the data copied from the original extents

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2024-05-17 10:36:22 +08:00
Patrick Donnelly
bfe574c6ce
Merge PR #57302 into main
* refs/pull/57302/head:
	qa/tasks/quiescer: dump ops in parallel

Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
2024-05-16 21:12:51 -04:00
Kefu Chai
fcb2eb7761
Merge pull request #57260 from tchaikov/test_back_trace-with-asan
test/common/test_back_trace: skip one more frame when testing w/ ASan

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2024-05-17 08:46:33 +08:00
Kefu Chai
84484c0730
Merge pull request #57429 from tchaikov/dencoder-unregister
tools/ceph-dencoder: unregister dencoders in dtor

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2024-05-17 08:41:59 +08:00
Rishabh Dave
281183dd14 qa/cephfs: add MDS_CLIENTS_BROKEN_ROOTSQUASH to ignorelist
MDS_CLIENTS_BROKEN_ROOTSQUASH is generated and expected by
test_rootsquash_nofeature but it hasn't be added to ignorelist as a
result of which QA code marks the job as failed even though all tests
finished running successfully.

Introduced-by: bccc8ceb47
Fixes: https://tracker.ceph.com/issues/66075
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2024-05-16 22:13:28 +05:30
Rishabh Dave
faa30e03f3 qa/cephfs: set joinable on FS before exiting tests in TestFSFail
After running TestFSFail, CephFSTestCase.tearDown() fails attempting
to unmount CephFS. Set joinable on FS and wait for the MDS to be up
before exiting the test. This will ensure that unmounting is
successful in teardown.

Fixes: https://tracker.ceph.com/issues/65841
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2024-05-16 22:11:01 +05:30
Patrick Donnelly
15f734ec62
qa/tasks/quiescer: dump ops in parallel
Since this --flags=locks takes the mds_lock and dumps thousands of ops, this
may take a long time to complete for each individual MDS. The entire quiesce
set may timeout (and all q ops killed) before we finish dumping ops.

Fixes: https://tracker.ceph.com/issues/65823
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2024-05-16 12:11:49 -04:00
Yuri Weinstein
c2fb93636e
Merge pull request #56640 from NitzanMordhai/wip-nitzan-public-addr-check-subnet-unreachable
common/pick_address: check if address in subnet all public address

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Prashant D <pdhange@redhat.com>
2024-05-16 08:12:51 -07:00
Casey Bodley
b25c182d5b
Merge pull request #57479 from ktdreyer/wip-66016-qatlib-arches
cmake: disable WITH_QATLIB/ZIP on non-x86

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2024-05-16 15:15:23 +01:00
NitzanMordhai
48db64c217
Merge pull request #56743 from NitzanMordhai/wip-nitzan-backword-forword-dencoder-tests
suites: adding dencoder test multi versions
2024-05-16 15:40:11 +03:00
Ilya Dryomov
572909629c common/options: link to mon_osd_blocklist_default_expire from RBD
"number of seconds to blocklist - set to 0 for OSD default" in the
description of rbd_blocklist_expire_seconds refers to the value that is
controlled by mon_osd_blocklist_default_expire.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2024-05-16 12:50:00 +02:00
Venky Shankar
999ca78a1a Merge PR #56944 into main
* refs/pull/56944/head:
	qa: add a YAML to ignore MGR_DOWN warning

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2024-05-16 14:52:13 +05:30
Yingxin
d50b94f93f
Merge pull request #55735 from xxhdx1985126/wip-crimson-errorator-void-handler
crimson/common/errorator: disallow void-returning error handlers

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
2024-05-16 15:57:48 +08:00
Yingxin
a8099a6c30
Merge pull request #57432 from cyx1231st/wip-seastore-cleanup-objdata-read
crimson/os/seastore/object_data_handler: cleanup read()

Reviewed-by: Myoungwon Oh <myoungwon.oh@samsung.com>
Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>
2024-05-16 14:18:46 +08:00
Venky Shankar
e0efed739d Merge PR #56941 into main
* refs/pull/56941/head:
	mds: find a new head for the batch ops when the head is dead

Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2024-05-16 11:29:17 +05:30
Venky Shankar
deb2cddb7e Merge PR #57275 into main
* refs/pull/57275/head:
	qa/fsx: use a specified sha1 to build the xfstest-dev

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
2024-05-16 11:26:45 +05:30
nmordech@redhat.com
3f26a965f6 suites: adding dencoder test multi versions
We are currently conducting regular ceph-dencoder tests for backward compatibility.
However, we are omitting tests for forward compatibility.
This suite will introduce tests against the ceph-objects-corpus to address forward
compatibility issues that may arise.
the script will install N-2 version and run against the latest version corpus objects
that we have, then install N-1 to N version and check them as well.

Signed-off-by: Nitzan Mordechai <nmordech@redhat.com>
2024-05-16 05:16:17 +00:00
Patrick Donnelly
cbb1da1189
Merge PR #57454 into main
* refs/pull/57454/head:
	mds/quiesce-db: optimize peer updates
	mds/quiesce-db: track db epoch separately from the membership epoch
	mds/quiesce-db: test that a peer on a newer membership epoch can ack a root

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2024-05-15 23:01:16 -04:00
Patrick Donnelly
70ed3825f8
Merge PR #57274 into main
* refs/pull/57274/head:
	mds: don't stall the asok thread for flush commands
	qa/quiescer: relax some timing requirements in the quiescer

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2024-05-15 22:56:38 -04:00
Patrick Donnelly
3e92f50796
Merge PR #57329 into main
* refs/pull/57329/head:
	qa: unmount clients before damaging the fs

Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2024-05-15 22:56:11 -04:00