Commit Graph

140749 Commits

Author SHA1 Message Date
Casey Bodley
12b1a560c6
Merge pull request #52144 from linuxbox2/wip-nochown-bucket
rgw/file: make setattr(...) a no-op on buckets

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2023-10-02 15:38:35 +01:00
Casey Bodley
8da4ba8a35
Merge pull request #52960 from cbodley/wip-62411
rgw/lc: remove_bucket_config() doesn't update xattrs on bucket delete

Reviewed-by Matt Benjamin <mbenjamin@redhat.com>
2023-10-02 15:38:16 +01:00
Casey Bodley
3b4ba8b7e9
Merge pull request #53266 from VVoidV/lbr-chunk-upload-buffer-fix
rgw: improve buffer list utilization in the chunkupload scenario

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2023-10-02 15:37:46 +01:00
Casey Bodley
12bc70b170
Merge pull request #53525 from RickyMaRui/fix-SignatureDoesNotMatch
rgw: fix SignatureDoesNotMatch when extra headers start with 'x-amz'

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2023-10-02 15:37:25 +01:00
Casey Bodley
7c4c73ad17
Merge pull request #53602 from cbodley/wip-62938
rgw: s3website doesn't prefetch for web_dir() check

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2023-10-02 15:37:08 +01:00
Casey Bodley
9fbb530312
Merge pull request #53614 from xrmeng8756/master
rgw: fix rgw rate limiting RGWRateLimitInfo class decode_json max_rea…

Reviewed-by:  Jiffin Tony Thottan <jthottan@redhat.com>
2023-10-02 15:36:53 +01:00
Casey Bodley
84dcf80155
Merge pull request #53676 from smanjara/data-sync-fairness-crash
rgw/multisite: call drain before flushing markers in incremental sync

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2023-10-02 15:36:07 +01:00
Casey Bodley
98cdc09608
Merge pull request #53680 from cbodley/wip-62989
rgw/keystone: EC2Engine uses reject() for ERR_SIGNATURE_NO_MATCH

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2023-10-02 15:35:47 +01:00
Casey Bodley
123c60797e
Merge pull request #53684 from cbodley/wip-rgw-keystone-yield
rgw/async: use optional_yield for keystone and kms requests

Reviewed-by: Adam Emerson <aemerson@redhat.com>
Reviewed-by: Ali Maredia <amaredia@redhat.com>
2023-10-02 15:35:20 +01:00
Casey Bodley
c78168d188
Merge pull request #53691 from max-licc/radosgw_start_watcher_crash
rgw: fix unwatch crash at radosgw startup

Reviewed-by: Adam Emerson <aemerson@redhat.com>
2023-10-02 15:34:51 +01:00
Casey Bodley
100d81aa06
Merge pull request #53714 from cbodley/wip-63004
[CVE-2023-43040] rgw: Fix bucket validation against POST policies

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2023-10-02 15:34:25 +01:00
Adam King
0bca13c8bf
Merge pull request #53724 from adk3798/rm-get_unit_name_by_instance
cephadm: remove get_unit_name_by_instance func

Reviewed-by: John Mulligan <jmulligan@redhat.com>
2023-10-02 09:46:04 -04:00
Adam King
1bfcd77a17
Merge pull request #53715 from adk3798/other-haproxy-container-version
cephadm: fix haproxy version with certain containers

Reviewed-by: John Mulligan <jmulligan@redhat.com>
2023-10-02 09:41:38 -04:00
zdover23
e1ea0a18df
Merge pull request #53744 from zdover23/wip-doc-2023-10-02-architecture-16-of-x
doc/architecture: edit "Calculating PG IDs"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-10-02 23:01:54 +11:00
Ilya Dryomov
237aa221eb qa/suites/krbd: stress test for recovering from watch errors
Fixes: https://tracker.ceph.com/issues/63010
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2023-10-02 12:21:12 +02:00
Ronen Friedman
39e370099d
Merge pull request #53606 from Patty8122/control-reaches-end-fix
test: corrected control reaches end by adding a return

Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
2023-10-02 08:37:18 +03:00
Zac Dover
69a585fbf4 doc/architecture: edit "Calculating PG IDs"
Edit the section "Calcluating PG IDs" in doc/architecture.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-10-02 10:22:16 +10:00
Patty8122
346981a7c2 test: corrected control reaches end by adding a return
Signed-off-by: Patty8122 <divyapattisapu@uchicago.edu>
2023-10-01 18:09:40 -05:00
Ali Masarwa
51e9525bae
Merge pull request #53580 from AliMasarweh/wip-alimasa-persistent-q-migration
RGW | Bucket Notification: migrating old entries to support persistency control

Reviewed-by: yuvalif <ylifshit@redhat.com>
2023-10-01 16:38:40 +03:00
Yuri Weinstein
5b70a6b920
Merge pull request #46692 from pdvian/wip-unreachable-osd
osd: Report health error if OSD public address is not within subnet

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-10-01 05:59:26 -07:00
Yuri Weinstein
c9f29cc362
Merge pull request #52509 from ericqzhao/main
mgr: add throttle policy for DaemonServer

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2023-10-01 05:57:01 -07:00
zdover23
e427b9e3b1
Merge pull request #53740 from zdover23/wip-doc-2023-09-30-architecture-15-of-x
doc/architecture: edit several sections

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-10-01 22:50:24 +11:00
Adam King
b949914ca1
Merge pull request #53618 from adk3798/cephadm-refactor-more-net-utils
cephadm: move more funcs into net_utils.py

Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
2023-09-30 18:41:29 -04:00
Zac Dover
9657c45fc1 doc/architecture: edit several sections
Edit the following sections in doc/architecture.rst:

 1. Dynamic Cluster Management
 2. About Pools
 3. Mapping PGs to OSDs

The tone of "Dynamic Cluster Management" remains a bit too close to the
tone of marketing material, in my opinion, but I will return to firm it
up when I have finished a once-over of architecture.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-10-01 08:25:34 +10:00
Adam King
1b1e91ea7b
Merge pull request #53702 from adk3798/cephadm-refactor-registry-login
cephadm: move registry_login to container_engines.py

Reviewed-by: John Mulligan <jmulligan@redhat.com>
2023-09-30 17:07:45 -04:00
Adam King
2367fabaea
Merge pull request #53617 from adk3798/cephadm-refactor-more-data-utils
cephadm: move more funcs into data_utils.py

Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
2023-09-30 17:05:47 -04:00
Adam King
e2b54e39bd
Merge pull request #53612 from adk3798/fix-refreshed-column
mgr/cephadm: fix REFRESHED column of orch ps being unpopulated

Reviewed-by: Michael Fritch <mfritch@suse.com>
2023-09-30 16:59:42 -04:00
Adam King
91cbbaf799 cephadm: re-format black data_utils.py
Signed-off-by: Adam King <adking@redhat.com>
2023-09-30 15:28:44 -04:00
Adam King
915ac7d74f cephadm: move more funcs into data_utils.py
Signed-off-by: Adam King <adking@redhat.com>
2023-09-30 15:28:40 -04:00
Adam King
6306e5c316
Merge pull request #53682 from phlogistonjohn/jjm-cephadm-move-pathify
cephadm: move pathify & get_file_timestamp to file_utils

Reviewed-by: Adam King <adking@redhat.com>
2023-09-30 15:25:59 -04:00
Adam King
46e4c40b91
Merge pull request #53683 from phlogistonjohn/jjm-cephadm-move-getters
cephadm: move context getters

Reviewed-by: Adam King <adking@redhat.com>
2023-09-30 15:22:12 -04:00
zdover23
8f07179437
Merge pull request #53726 from zdover23/wip-doc-2023-09-29-architecture-14-of-x
doc/architecture: edit "Replication"

Reviewed-by: Cole Mitchell <cole.mitchell.ceph.@gmail.com>
2023-09-30 10:12:15 +10:00
Laura Flores
3deccad920
Merge pull request #53605 from sayantani11/osd-1
script/backport-resolve-issue: Update script with latest versions
2023-09-29 18:34:45 -04:00
mengxiangrui
f021b96302 rgw: fix rgw rate limiting RGWRateLimitInfo class decode_json max_read_bytes and max_write_bytes field mismatch
Fixes: https://tracker.ceph.com/issues/62955
Signed-off-by: xiangrui meng <mengxr@chinatelecom.cn>
2023-09-29 20:02:49 +08:00
Pere Diaz Bou
63f0a0df14 os/bluestore: fix _extend_log seq advance
when extending the log, the sequence was left on a bad state because it would first create a transaction to update with the current seq number but leave the "real" transaction with the same sequence number which should be `extend_log_transaction.seq + 1`.

Signed-off-by: Pere Diaz Bou <pdiabou@redhat.com>
2023-09-29 13:17:03 +02:00
Pere Diaz Bou
8c68a503f1 os/bluestore: test _extend_log sequence advance
Signed-off-by: Pere Diaz Bou <pdiabou@redhat.com>
2023-09-29 13:16:42 +02:00
zdover23
1d10b71792
Merge pull request #53723 from zdover23/wip-doc-2023-09-29-architecture-13-of-x
doc/architecture: edit "Data Scrubbing"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-09-29 16:02:14 +10:00
Zac Dover
4cf22e195a doc/architecture: edit "Replication"
Edit "Replication" in the "Smart Daemons Enable Hyperscale" section of
doc/architecture.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-09-29 15:30:14 +10:00
Zac Dover
d7b991646f doc/architecture: edit "Data Scrubbing"
Edit the "Data Scrubbing" listitem in the list of benefits conferred by
the use by OSDs of the aggregate power of the cluster, in the section
"Smart Daemons Enable Hyperscale" in doc/architecture.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-09-29 15:26:13 +10:00
Venky Shankar
6d8679ebcd Merge PR #53594 into main
* refs/pull/53594/head:
	qa: fix "no orch backend set" in nfs suite

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-09-29 10:01:17 +05:30
zdover23
8e5dda2493
Merge pull request #53707 from zdover23/wip-doc-2023-09-28-architecture-12-of-x
doc/architecture: edit "OSD Membership and Status"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-09-29 07:35:45 +10:00
Adam King
34fa171ff3 cephadm: remove get_unit_name_by_instance func
As it is one line, quite simple, and only
had a single caller, it was decided we'd remove
this function as part of the cephadm refactor.

Signed-off-by: Adam King <adking@redhat.com>
2023-09-28 14:39:53 -04:00
Zac Dover
dbd85336b1 doc/architecture: edit "OSD Membership and Status"
Edit "OSD Membership and Status" in the "Smart Daemons Enable
Hyperscale" section of doc/architecture.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-09-29 04:28:56 +10:00
Adam King
5540cd1882 cephadm: fix haproxy version with certain containers
Some builds of haproxy containers' output
from "haproxy -v" start with

HAProxy version

rather than

HA-Proxy version

no reason on our end not to accept both

Signed-off-by: Adam King <adking@redhat.com>
2023-09-28 11:07:21 -04:00
Joshua Baergen
98bfb71cb3 rgw: Fix bucket validation against POST policies
It's possible that user could provide a form part as a part of a POST
object upload that uses 'bucket' as a key; in this case, it was
overriding what was being set in the validation env (which is the real
bucket being modified). The result of this is that a user could actually
upload to any bucket accessible by the specified access key by matching
the bucket in the POST policy in said POST form part.

Fix this simply by setting the bucket to the correct value after the
POST form parts are processed, ignoring the form part above if
specified.

Fixes: https://tracker.ceph.com/issues/63004

Signed-off-by: Joshua Baergen <jbaergen@digitalocean.com>
2023-09-28 11:04:35 -04:00
Casey Bodley
2f5657011f
Merge pull request #53608 from cbodley/wip-62536
qa/rgw/sts: keycloak task installs java manually

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2023-09-28 15:37:20 +01:00
Rishabh Dave
e404413e89 mon/AuthMonitor: make "ceph auth caps" print error messsages...
generated by MDSAuthCaps::parse(). Specifically, in this case, this
command is supposed to print an error message when permission flag passed
for MDS capability is incorrect.

This needs the method AuthMonitor::_update_or_create_entity() to be
fixed. The issue is that stderr stream is not passed from the block under
"else if" clause of "ceph auth caps" command to MDSAuthCaps::_parse()
method. Let's fix this by adding a parameter for stderr stream to every
method in stack that is between this block and MDSAuthCaps::parse()
method.

qa.tasks.cephfs.test_admin.TestPermErrMsg contains tests that checks
whether or not all the Ceph commands that accept MDS capabilites print
the error message when permission flag in this capaibility is incorrect.
Add a test to this class to check if "ceph auth caps" also print an error
message when perm flag in MDS capability sting is incorrect.

Fixes: https://tracker.ceph.com/issues/63020
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2023-09-28 19:52:03 +05:30
Matt Benjamin
c53f3bc319 rgw/file: make setattr(...) a no-op on buckets
Shallow fix for apparent unstable behavior after nfs "chown" on
an RGW bucket via RGW NFS.  While we allow buckets to be created
(and subject to ordinary rules, deleted), chown against a bucket
hasn't been tested and potentially is not valid.  Prevent it
altogether for now--if permissions would allow it, chown will
succeed but won't have any effect.

Fixes: https://tracker.ceph.com/issues/61689

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2023-09-28 09:55:25 -04:00
Ali Masarwa
60eb713084 RGW | Bucket Notification: migrating old entries to support persistency control
Signed-off-by: Ali Masarwa <ali.saed.masarwa@gmail.com>
2023-09-28 16:30:23 +03:00
Ilya Dryomov
2094a0450d qa/suites/krbd: rename singleton to singleton-msgr-failures
A "singleton without msgr-failures" is wanted in the next commit.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2023-09-28 12:06:11 +02:00