Commit Graph

139815 Commits

Author SHA1 Message Date
Venky Shankar
b81699f339 Merge PR #52111 into main
* refs/pull/52111/head:
	ceph: allow xlock state to be LOCK_SYNC when putting it
	ceph: allow xlock state to be LOCK_PREXLOCK when putting it

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-08-22 13:27:17 +05:30
Venky Shankar
0af06bd5d0 Merge PR #52335 into main
* refs/pull/52335/head:
	client: move the Inode to new auth mds session when changing auth cap
	client: check the xlists in ~MetaSession()

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
2023-08-22 13:26:02 +05:30
Venky Shankar
172ad82232 Merge PR #52520 into main
* refs/pull/52520/head:
	mds: acquire inode snaplock in open

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>
2023-08-22 13:25:07 +05:30
Venky Shankar
f2e17e40e7 Merge PR #52741 into main
* refs/pull/52741/head:
	qa/cephfs: switch to python3 for centos stream 9

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
2023-08-22 13:23:40 +05:30
Venky Shankar
97b76a480b qa: add POOL_APP_NOT_ENABLED to ignorelist for cephfs tests
Fixes: http://tracker.ceph.com/issues/62508
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2023-08-22 11:44:55 +05:30
Zac Dover
ea1dc3c027 doc/start: refactor ABC test chart
Refactor the ABC test chart so that the information about which tests
have been run is presented in the center of the chart instead of, as it
was before, in a superscript.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-08-22 15:30:21 +10:00
Venky Shankar
ac3ab1a203 PendingReleaseNotes: add a note for mds_session_metadata_threshold mds config
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2023-08-22 00:47:27 -04:00
Venky Shankar
84df4b3d0c test: add test to verify that a buggy client is blocklisted
... when its session metadata is bloated due to buildup of
`completed_requests`.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
2023-08-22 00:47:27 -04:00
Venky Shankar
59dd587ddb mds: add perf counter to track number of sessions evicted due to metadata threshold being exceeded
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2023-08-22 00:47:27 -04:00
Venky Shankar
bc6814d72a mds: blocklist clients with "bloated" session metadata
Buggy clients (or maybe a MDS bug) causes a huge buildup of
`completed_requests` metadata in its session information.
This could cause the MDS to go read-only when its flushing
session metadata to the journal since the bloated metadata
causes the ODSOp payload to exceed the maximum write size.

Blocklist such clients so as to allow the MDS to continue
servicing requests.

Fixes: http://tracker.ceph.com/issues/61947
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2023-08-22 00:47:26 -04:00
zdover23
de99a529f4
Merge pull request #53069 from zdover23/wip-doc-2023-08-22-start-os-recommendations-platforms
doc/start: update "platforms" table

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-08-22 14:36:42 +10:00
zdover23
9863c9fffd
Merge pull request #53068 from zdover23/wip-doc-2023-08-21-README-md-cmake-options
doc/README.md - improve "CMake Options"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-08-22 14:35:19 +10:00
Yingxin
90f710f748
Merge pull request #53046 from cyx1231st/wip-crimson-msgr-perf-random-client-nonce
crimson/tools/perf_crimson_msgr: randomize client nonce

Reviewed-by: Samuel Just <sjust@redhat.com>
2023-08-22 09:29:49 +08:00
Casey Bodley
100bb99706
Merge pull request #52673 from tobias-urdin/options-anon-auth-presigned
rgw: handle http options CORS with v4 auth

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2023-08-21 15:52:25 -04:00
Casey Bodley
23e071a93c Merge PR #52868 into main
* refs/pull/52868/head:
	mailmap: update affiliations for reef release
	script: fix credits.sh showing 0 lines changed for mapped emails
	script: credits.sh uses grep -E

Reviewed-by: Joshua Baergen <jbaergen@digitalocean.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Zack Cerza <zack@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Parth Arora <paarora@redhat.com>
Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>
Reviewed-by: zmc <NOT@FOUND>
2023-08-21 15:16:45 -04:00
Zac Dover
78bf082498 doc/start: update "platforms" table
Update the table that reports which versions of which Linux releases
have been used in tests of Ceph.

Fixes: https://tracker.ceph.com/issues/62354
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-08-22 03:10:19 +10:00
Ronen Friedman
f74ca2bb33 osd: minor changes to the descriptions of some configuration items
... mostly scrub-related.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-08-21 10:04:32 -05:00
Casey Bodley
0fbd834348 mailmap: update affiliations for reef release
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-08-21 10:22:51 -04:00
Zac Dover
2a9b13bd7b doc/README.md - improve "CMake Options"
Add subheadings to the "CMake Options" section of ceph/README.md. Remove
modal verbs when appropriate and make sentences tidier.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-08-22 00:13:34 +10:00
Venky Shankar
38b4e13d8f Revert "qa/cephfs: fix ior build failure"
PR #52416 is incomplete and got merged on the basis of a linked
test run which passed, however, subsequent update to the PR
caused the test to start failing again.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
2023-08-21 19:33:42 +05:30
cloudbehl
9fcc176162 mgr/dashboard: Overview graph improvemntments
Fixes: https://tracker.ceph.com/issues/62367

Signed-off-by: cloudbehl <cloudbehl@gmail.com>
2023-08-21 19:11:57 +05:30
Aashish Sharma
a4b76efd31
Merge pull request #52915 from rhcs-dashboard/rgw-multisite-sync-card
mgr/dashboard: multisite sync status card for rgw overview dashboard

Reviewed-by: Nizamudeen A <nia@redhat.com>
2023-08-21 18:11:12 +05:30
Aashish Sharma
da65bbc20a mgr/dashboard: multisite sync status card for rgw overview dashboard
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit 1d6f19e53b68c180a2d0301889974949fe899a2c)
2023-08-21 12:48:44 +05:30
Lucian Petrut
a09b70dfd8 cmake: honor WITH_FMT_HEADER_ONLY with vendored fmt lib
We're using the vendored fmt lib when there is no system library
available. However, there is an inconsistency: the
WITH_FMT_HEADER_ONLY setting is ignored by the vendored library.

In order to address this, we'll use the fmt-header-only alias
if WITH_FMT_HEADER_ONLY is set.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2023-08-21 06:39:26 +00:00
Venky Shankar
83ab48ad02 Merge PR #52867 into main
* refs/pull/52867/head:
	doc: modify and add descriptions for  cephfs-shell commands.

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
2023-08-21 11:03:48 +05:30
Yingxin
44f4d78b0d
Merge pull request #52128 from myoungwon/wip-fix-invalid-replay
crimson/os/seastore/journal/cbj: generalize scan_valid_records()

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2023-08-21 09:49:44 +08:00
zdover23
ee1539f11a
Merge pull request #53055 from zdover23/wip-doc-2023-08-19-README-md-building-ceph
doc/README.md - edit "Building Ceph"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-08-20 20:44:23 +10:00
Zac Dover
5bb3942365 doc/README.md - edit "Building Ceph"
Improve the "Building Ceph" section of ceph/README.md.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-08-20 09:16:54 +10:00
Leonid Usov
749c770676 mds/Server: mark a cap acquisition throttle event in the request
Fixes: https://tracker.ceph.com/issues/59067
Signed-off-by: Leonid Usov <leonid.usov@ibm.com>
2023-08-18 18:11:29 +03:00
Xuehan Xu
cf105932a7 crimson/os/seastore/omap_manager: fix the entry leak issue in
BtreeOMapManager::omap_list()

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2023-08-18 20:23:35 +08:00
zdover23
dd5f8e078f
Merge pull request #53035 from zdover23/wip-doc-2023-08-17-README-md-contributing-code
doc: edit README.md - contributing code

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-08-18 20:26:59 +10:00
myoungwon oh
39ea5add3a crimson/os/seastore/journal/cbj: remove return_record and read_record
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2023-08-18 03:23:44 +00:00
myoungwon oh
ef1a6a90e0 crimson/os/seastore/journal/cbj: check if batched records are written at once before rolling
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2023-08-18 03:23:27 +00:00
myoungwon oh
15ae64c331 crimson/os/seastore: genralize read_validate_record_metadata and read_validate_data
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2023-08-18 03:22:45 +00:00
Yingxin Cheng
91c65e2238 crimson/tools/perf_crimson_msgr: randomize client nonce
In case we want to start multiple clients simultaneously.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2023-08-18 09:57:14 +08:00
Yingxin
b3bbd9f53a
Merge pull request #53023 from xxhdx1985126/wip-seastore-daggling-ref
crimson/os/seastore: fix daggling reference of oid in SeaStore::Shard::stat()

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2023-08-18 09:19:47 +08:00
Zac Dover
9433a3e62f doc: edit README.md - contributing code
Edit the "Contributing Code" section of the "ceph/README.md" file.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-08-18 04:53:30 +10:00
Ali Masarwa
89a4f7308e RGW | Bucket Notification: Fix the issue of miscounting the entries to remove
Signed-off-by: Ali Masarwa <ali.saed.masarwa@gmail.com>
2023-08-17 18:38:23 +03:00
Avan
82e2614584
Merge pull request #53022 from rhcs-dashboard/upgrade-start-enhancement
mgr/dashboard: support cluster upgrade even if the check for upgrade fails

Reviewed-by: cloudbehl <NOT@FOUND>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2023-08-17 20:46:30 +05:30
Casey Bodley
902bcd7bc4 script: fix credits.sh showing 0 lines changed for mapped emails
script was showing 0 for several contributors because the commit Author:
did not match their mapped address:
```
Number of lines added and removed, by authors
...
   306	     0 Radoslaw Zarzynski <rzarzynski@redhat.com>
   307	     0 Mike Perez <miperez@redhat.com>
   308	     0 Michael J. Kidd <linuxkidd@redhat.com>
   309	     0 Lukas Mayer <lmayer@wind.gmbh>
   310	     0 Luis Henriques <lhenriques@suse.com>
   311	     0 Kyle McGough <kmcgough@digitalocean.com>
   312	     0 João Eduardo Luís <joao@suse.de>
   313	     0 JinyongHa <jy200.ha@samsung.com>
   314	     0 Ilya Dryomov <idryomov@redhat.com>
```

in the case of Ilya and Radoslaw:
```
$ git log --no-merges --pretty='%ae' v17.2.6..v18.2.0 | sed -e "s/'/ /g" | sort -u | grep -e idryomov -e rzarzyns
idryomov@gmail.com
rzarzyns@redhat.com
```

this output gets fed into `git log --numstat --author="$mail"` to fetch
the stats, but that command maps idryomov@gmail.com -> idryomov@redhat.com
and rzarzyns@redhat.com -> rzarzynski@redhat.com so no commits matched.
disabling the mapping for this command with --no-mailmap allows it to
fetch their stats correctly:
```
    14   11809 Radoslaw Zarzynski <rzarzynski@redhat.com>
    16   10051 Ilya Dryomov <idryomov@redhat.com>
```

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-08-17 10:54:46 -04:00
Casey Bodley
f13c545cf8 script: credits.sh uses grep -E
resolves warning spam:
> egrep: warning: egrep is obsolescent; using grep -E

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-08-17 10:54:46 -04:00
Lucian Petrut
d40edd3ac0 rbd-wnbd: fix llvm issues
We're fixing a few rbd-wnbd issues that are currently ignored
by mingw-gcc but not by llvm:

* checking if an uint is smaller than 0
* qualified method names must be used when passing the address
* duplicate symbol "shutdown_lock"
* add missing const cast when passing WNBD interface

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2023-08-17 13:31:51 +00:00
Lucian Petrut
ba3c6d76a7 test: avoid converting addresses to "long"
On Windows x64 hosts, "long" (4B) is not large enough to hold
an address.

For this reason, we're updating "test_json_formattable.cc"
to use "long long" instead.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2023-08-17 13:31:51 +00:00
Lucian Petrut
6f80c86d46 common,test: avoid deprecated result_of_t
std::result_of_t was deprecated in c++17 and removed in c++20.
gcc kept it around for backwards compatibility, however it was
removed in clang.

For this reason, we'll need to use std::invoke_result_ot instead,
which has a slightly different syntax.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2023-08-17 13:31:12 +00:00
Tobias Urdin
dbc0a4ed2d rgw/auth: move http options v4 logic to get_v4_canonical_method()
Signed-off-by: Tobias Urdin <tobias.urdin@binero.com>
2023-08-17 11:47:34 +00:00
Tobias Urdin
814cae52ba rgw/auth: handle HTTP OPTIONS with v4 auth
This adds code to properly verify the signature
for HTTP OPTIONS calls that is preflight CORS
requests passing the expected method in the
access-control-request-method header.

Fixes: https://tracker.ceph.com/issues/62033
Signed-off-by: Tobias Urdin <tobias.urdin@binero.com>
2023-08-17 11:47:34 +00:00
Tobias Urdin
135b6fc20e rgw/auth: check auth for http options requests
If we get a HTTP OPTIONS request we previously always
allowed the anonymous engine to handle the request even
if other auth was given in the request.

Fixes: https://tracker.ceph.com/issues/62033
Signed-off-by: Tobias Urdin <tobias.urdin@binero.com>
2023-08-17 11:47:34 +00:00
tengjie5
6869ba38f2
doc: modify and add descriptions for cephfs-shell commands.
Fixes: https://tracker.ceph.com/issues/62074
Signed-off-by: Teng Jie <tengjie5@asiainfo.com>
2023-08-17 17:57:57 +08:00
Ilya Dryomov
9640080ae3
Merge pull request #53030 from petrutlucian94/rbd_tests
test/librbd/fsx: fix wnbd check

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2023-08-17 11:39:53 +02:00
Xuehan Xu
0b0ed4cdef osd/PGLog: return a ready future when found missing objects
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2023-08-17 14:54:40 +08:00