Commit Graph

139120 Commits

Author SHA1 Message Date
Matan Breizman
45c6c5474d
Merge pull request #52599 from Matan-B/wip-matanb-crimson-cbt-fix
qa: Fix cbt usage with Crimson

Reviewed-by: Samuel Just <sjust@redhat.com>
2023-07-25 09:58:58 +03:00
Matan Breizman
f3b4c1deb8
Merge pull request #52598 from Matan-B/wip-matanb-crimson-centos-8
qa/suites/crimson-rados: Use centos8 for testing

Reviewed-by: Samuel Just <sjust@redhat.com>
2023-07-25 09:57:01 +03:00
Venky Shankar
e99c17f271 Merge PR #48038 into main
* refs/pull/48038/head:
	client test: Add fsync to ll_preadv_pwritev test
	libcephfs: Option to write + fsync via ceph_ll_nonblocking_readv_writev
	Client: Hook nonblocking fsync into the write path of ll_preadv_pwritev
	Client: Add non-blocking fsync
	Client/Inode: wait_for_caps fixups
	Client: change several waitfor_* to use Context list
	test: Add nonblocking I/O client test
	libcephfs: Add nonblocking readv/writev I/O interface
	Client: Add ll_preadv_pwritev to expose non-blocking I/O to libcephfs
	Client: Add non-blocking helper classes
	Client: Break some code into new methods in prep for non-blocking I/O
	Buffers: Add function to buffer.h to copy bufferlist to an iovec
	ObjectCacher: Prepare file_write path for non-blocking I/O

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2023-07-25 09:34:36 +05:30
Ilya Dryomov
f3a659f33d
Merge pull request #52607 from zap51/patch-1
doc/rbd/rbd-cloudstack.rst: include support for multiple monitors

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2023-07-25 04:49:40 +02:00
Anthony D'Atri
591e698803
Merge pull request #49889 from bluikko/bluikko-doc-srv-lookup-link
doc/rados/configuration: add links to MON DNS
2023-07-24 18:39:53 -04:00
Jayanth Reddy
5bce61ecf3 doc/rbd/rbd-cloudstack.rst: include support for multiple monitors
Signed-off-by: Jayanth Reddy <45934027+zap51@users.noreply.github.com>
2023-07-24 23:02:53 +02:00
Frank S. Filz
13162f6ecc client test: Add fsync to ll_preadv_pwritev test
Signed-off-by: Frank S. Filz <ffilzlnx@mindspring.com>
2023-07-24 11:49:05 -07:00
Frank S. Filz
201a355f36 libcephfs: Option to write + fsync via ceph_ll_nonblocking_readv_writev
Signed-off-by: Frank S. Filz <ffilzlnx@mindspring.com>
2023-07-24 11:49:05 -07:00
Frank S. Filz
6809ff8c12 Client: Hook nonblocking fsync into the write path of ll_preadv_pwritev
Signed-off-by: Frank S. Filz <ffilzlnx@mindspring.com>
2023-07-24 11:49:04 -07:00
Frank S. Filz
a181c1a0db Client: Add non-blocking fsync
We will need the ability to do an non-blocking write that finishes with
fsync so we need non-blocking fsync.

Signed-off-by: Frank S. Filz <ffilzlnx@mindspring.com>
2023-07-24 11:49:04 -07:00
Frank S. Filz
a8d0158d0d Client/Inode: wait_for_caps fixups
The non-blocking flush requires us to be able to re-add to
wait_for_caps but if we simply add to the list, we get stuck in an
infinite loop. Add a wait_for_caps_pending list to add to, and then
when done signalling, we move the wait_for_caps_pending items onto the
wait_for_caps list.

Also in handle_cap_flush_ack(), we need to complete the caps flushing
before signalling since with non-blocking flush, we will be actually
examining the caps from the completion rather than signalling a
condition variable in the completion.

Signed-off-by: Frank S. Filz <ffilzlnx@mindspring.com>
2023-07-24 11:49:04 -07:00
Frank S. Filz
bff94987bb Client: change several waitfor_* to use Context list
Change waitfor_caps, waitfor_safe and waitfor_commit to Context list.

To make a non-blocking version of fsync (to be used for non-blocking write
and commit), we need to be able to signal an arbitrary Context on completion
of either of these lists.

add_nonblocking_onfinish_to_context_list Adds such a Context to the list.

Signed-off-by: Frank S. Filz <ffilzlnx@mindspring.com>
2023-07-24 11:49:04 -07:00
Frank S. Filz
6b50da636d test: Add nonblocking I/O client test
Signed-off-by: Frank S. Filz <ffilzlnx@mindspring.com>
2023-07-24 11:49:04 -07:00
Frank S. Filz
b4e39f3ecc libcephfs: Add nonblocking readv/writev I/O interface
Signed-off-by: Frank S. Filz <ffilzlnx@mindspring.com>
2023-07-24 11:49:03 -07:00
Frank S. Filz
94b55cbb99 Client: Add ll_preadv_pwritev to expose non-blocking I/O to libcephfs
Signed-off-by: Frank S. Filz <ffilzlnx@mindspring.com>
2023-07-24 11:49:03 -07:00
Frank S. Filz
1210ddf7a1 Client: Add non-blocking helper classes
Signed-off-by: Frank S. Filz <ffilzlnx@mindspring.com>
2023-07-24 11:49:03 -07:00
Frank S. Filz
162935c036 Client: Break some code into new methods in prep for non-blocking I/O
These bits of code need to be invoked from a separate spot when we
introduce non-blocking I/O, so break them out now.

Signed-off-by: Frank S. Filz <ffilzlnx@mindspring.com>
2023-07-24 11:49:03 -07:00
Frank S. Filz
d3c3bf5b77 Buffers: Add function to buffer.h to copy bufferlist to an iovec
Signed-off-by: Frank S. Filz <ffilzlnx@mindspring.com>
2023-07-24 11:49:03 -07:00
Frank S. Filz
bf98aec6d3 ObjectCacher: Prepare file_write path for non-blocking I/O
For non-blocking I/O, we will want to be able to override
block_writes_upfront so rename the member cfg_block_writes_upfront and add
an option to pass block_writes_upfront as a parameter along with a member
access method so caller can pass cfg_block_writes_upfront.

Signed-off-by: Frank S. Filz <ffilzlnx@mindspring.com>
2023-07-24 11:49:02 -07:00
Casey Bodley
3f09d56bed
Merge pull request #52521 from cbodley/wip-62059
valgrind: update suppression for SyscallParam under call_init

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
2023-07-24 14:06:01 -04:00
Neha Ojha
7cb0e36fc4
Merge pull request #52601 from idryomov/wip-doc-redundant-tiering-note
doc/rados: remove redundant cache tiering deprecation note

Reviewed-by: Neha Ojha <nojha@redhat.com>
2023-07-24 10:06:23 -07:00
Rishabh Dave
211621588b
Merge pull request #51934 from kotreshhr/fix-assert-rename
mds: Fix the linkmerge assert check

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
2023-07-24 21:44:27 +05:30
Yuval Lifshitz
cc450e23e6
Merge pull request #52473 from vedanshbhartia/null_deref
rgw: add coverity annotations and asserts for null dereferences
2023-07-24 19:11:44 +03:00
Yuval Lifshitz
589535e0e7
Merge pull request #52437 from vedanshbhartia/coverity_missing_break
rgw: Add coverity annotation for missing break statement in switch
2023-07-24 19:11:01 +03:00
Yuval Lifshitz
2ad5a66cd8
Merge pull request #52436 from vedanshbhartia/coverity_logical_dead
rgw: Remove logically dead code
2023-07-24 19:10:08 +03:00
Yuval Lifshitz
4aa76cc98a
Merge pull request #52434 from vedanshbhartia/coverity_1511179
rgw: Fix potential null pointer dereference in rgw_user.cc
2023-07-24 19:08:06 +03:00
Yuval Lifshitz
45b81cdba6
Merge pull request #52433 from vedanshbhartia/coverity_1510266
rgw: When finding bucket owner info, handle account name being empty
2023-07-24 19:07:31 +03:00
Yuval Lifshitz
7876f44d2a
Merge pull request #52432 from vedanshbhartia/coverity_1511177
rgw: Fix potential null dereference error in bucket name parsing
2023-07-24 19:06:42 +03:00
Yuval Lifshitz
c065a24d98
Merge pull request #52253 from vedanshbhartia/coverity_1515338
rgw: Init res id with cls_2pc_reservation::NO_ID
2023-07-24 19:05:04 +03:00
Yuval Lifshitz
c4754156da
Merge pull request #51648 from vedanshbhartia/coverity_1510684
rgw: refactor D3nDataCache::d3n_io_write to avoid resource leaks
2023-07-24 19:04:08 +03:00
Casey Bodley
84e169d5b7
Merge pull request #51772 from caisan/multisite-hang-period-update
rgw:multisite hang when update and commit period

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
2023-07-24 10:30:39 -04:00
Casey Bodley
8612ec0641
Merge pull request #52056 from cbodley/wip-56993
rgw: s3 object lock avoids overflow in retention date

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2023-07-24 10:28:14 -04:00
Casey Bodley
6abe46cff1
Merge pull request #52036 from adamemerson/wip-rgw-mdlog-yield
rgw: Thread optional_yield through RGWMetadataLog operations

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2023-07-24 10:27:57 -04:00
Casey Bodley
063721da35 valgrind: update suppression for SyscallParam under call_init
centos9 has a slightly different call stack here

Fixes: https://tracker.ceph.com/issues/62059

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-07-24 10:16:18 -04:00
Matan Breizman
3f87e4da85
Merge pull request #52529 from xxhdx1985126/wip-61943
crimson/osd/osd_operations: add pipeline to LogMissingRequest to sync it

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2023-07-24 16:54:17 +03:00
Ilya Dryomov
12ccd29466 doc/rados: remove redundant cache tiering deprecation note
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2023-07-24 15:01:26 +02:00
Matan Breizman
82d686b27f qa/suites/crimson-rados/perf: add monitoring_profiles for cbt
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2023-07-24 11:05:40 +00:00
Matan Breizman
fe83ca08f9 qa/tasks/cbt.py: Fix perf package naming
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2023-07-24 09:02:43 +00:00
Matan Breizman
d26137f6e0 qa/suites/crimson-rados: Use centos8 for testing
centos_latest was moved to centos9, we only schedule centos8
Crimson builds.
Use centos8 until Crimson will support 9.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2023-07-24 07:11:13 +00:00
Ali Masarwa
d22df1378d
Merge pull request #52439 from AliMasarweh/wip-alimasa-persistant-q-observability
RGW: fix issue in observability over the persistent topics queue
2023-07-24 10:06:36 +03:00
Xuehan Xu
0b7c47db78 crimson/osd/osd_operations: add pipeline to LogMissingRequest to sync it
with ReplicateRequest

This commit is a follow-up for e141428949
and f4d336a2db

Fixes: https://tracker.ceph.com/issues/61943
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2023-07-24 13:39:55 +08:00
Ville Ojamo
06cb565215 doc/rados/configuration: add links to MON DNS
Add two links to MON DNS lookup and an anchor in the destination document.

Signed-off-by: Ville Ojamo <14869000+bluikko@users.noreply.github.com>
2023-07-24 10:15:01 +07:00
Yingxin
8c4d93c2d7
Merge pull request #52221 from xxhdx1985126/wip-collection-bug-fix
crimson/os/seastore/collection_manager: fill CollectionNode::decoded on clean reads

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2023-07-24 09:56:37 +08:00
Yingxin
cf49dbed53
Merge pull request #52446 from xxhdx1985126/wip-seastore-lazy-read-patch
crimson/os/seastore/transaction_manager: data loss issues

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Xinyu Huang <xinyu.huang@intel.com>
2023-07-24 09:55:54 +08:00
Soumya Koduri
01bc98b489
Merge pull request #52445 from soumyakoduri/wip-skoduri-syncpolicy-fixes
rgw/sync-policy: Correct "sync status" & "sync group" commands

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2023-07-23 23:11:14 +05:30
Soumya Koduri
094460c26b rgw: Read sync status from only the zones allowed to sync from
Using multisite sync policy, zones within a zonegroup can be configured
not to sync from certain zones. Filter such zones while reading sync
status.

Fixes: https://tracker.ceph.com/issues/62014
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2023-07-23 19:51:07 +05:30
Soumya Koduri
fa5759922f rgw/sync-policy: Do not allow empty group names
Fixes: https://tracker.ceph.com/issues/62015
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2023-07-23 19:51:07 +05:30
zdover23
4868f0b6fb
Merge pull request #52587 from zdover23/wip-doc-2023-07-22-rados-configuration-ceph-conf-4-of-x
doc/rados: edit ceph-conf.rst (4 of x)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-07-23 22:11:24 +10:00
Zac Dover
ac47899863 doc/rados: edit ceph-conf.rst (4 of x)
Edit doc/rados/configuration/ceph-conf.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-07-23 21:56:05 +10:00
Igor Fedotov
d71eea965d
Merge pull request #51820 from ifed01/wip-ifed-fragmentation-info
os/bluestore: introduce allocator state histogram

Reviewed-by: Adam Kupczyk <akupczyk@ibm.com>
2023-07-22 17:38:13 +03:00