Commit Graph

138721 Commits

Author SHA1 Message Date
Vedansh Bhartia
ca3a1dbbca rgw: add coverity annotations and asserts for null dereferences
Signed-off-by: Vedansh Bhartia <vedanshbhartia@gmail.com>
2023-07-16 21:35:46 +05:30
Pedro Gonzalez Gomez
4de76bf550
Merge pull request #50558 from rhcs-dashboard/fix-dataUnits-graph-plotting
mgr/dashboard: fix a bug where data would plot wrongly

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2023-07-05 12:29:20 +02:00
Matan
e509829189
Merge pull request #51852 from Matan-B/wip-matanb-crimson-watch-notify-logs
crimson/osd/watch: Add logs around Watch/Notify

Reviewed-by: Samuel Just <sjust@redhat.com>
2023-07-05 11:33:31 +03:00
Ilya Dryomov
fa09e736f3
Merge pull request #52086 from pkalever/fix-mirror-status-desc
rbd-mirror: fix image replayer shut down description on force promote

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2023-07-05 00:22:53 +02:00
zdover23
c5d4dbf405
Merge pull request #52207 from zdover23/wip-doc-2023-06-27-rados-operations-crush-map-edits-2-of-x-2nd-attempt
doc/rados: edit crush-map-edits (2 of x)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-07-05 05:22:33 +10:00
Zac Dover
ae04debe9e doc/rados: edit crush-map-edits (2 of x)
Edit doc/rados/operations/crush-map-edits.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Co-authored-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-07-05 05:11:25 +10:00
zdover23
baeb4d541e
Merge pull request #52308 from zdover23/wip-doc-2023-07-05-seqdiag-test
doc/dev: remove seqdiag assets

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2023-07-05 03:55:15 +10:00
Zac Dover
ce96ddd4c7 doc/dev: remove seqdiag assets
Remove seqdiag assets to determine whether the docs can be built if they
are absent. (Currently they cannot be built when they are present.) If
this works, then these diagrams will be replaced, probably with .png
files.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-07-05 03:37:10 +10:00
zdover23
0f303c0149
Merge pull request #51422 from yuvalif/wip-yuval-cephadm-lua-doc
doc/rgw/lua: add info uploading a script in cephadm deployment

Reviewed-by: Zac Dover <zac.dover@proton.me>
2023-07-04 03:47:40 +10:00
Casey Bodley
813fb284c7
Merge pull request #52247 from cbodley/wip-57905
rgw: rgwx-skip-decrypt also skips decompression of encrypted objects

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
2023-07-03 11:19:36 -04:00
Mark Kogan
19f97fc5a9
Merge pull request #52249 from mkogan1/wip-t61661-d3n-valg
rgw: d3n: fix valgrind reported leak related to libaio ops
2023-07-03 17:06:21 +03:00
Soumya Koduri
0c5a77f4c9
Merge pull request #52160 from soumyakoduri/wip-skoduri-lc-acl
rgw/set_attrs: Do not modify mtime

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2023-07-03 10:20:48 +05:30
J. Eric Ivancich
7d93aa83bb
Merge pull request #51983 from clwluvw/aws4-date-header
rgw: pick http_date in case of http_x_amz_date absence

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2023-07-02 16:30:26 -04:00
Matan Breizman
3548489f62 crimson/osd/watch: don't print bufferlists
The bufferlists clutter the logs and don't infrom of anything useful:
```
DEBUG 2023-05-26 23:36:29,780 [shard 2] osd - send_completion sending notify replies: {notify_reply_t{watcher_gid=4220, watcher_cookie=139786421930752, bl=buffer::list(len=0,

)}, notify_reply_t{watcher_gid=4220, watcher_cookie=139786422032096, bl=buffer::list(len=0,

)}}
```

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2023-07-02 13:57:49 +00:00
Matan Breizman
b1c05f9ace crimson/osd/watch: Add logs around Watch/Notify
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2023-07-02 13:57:49 +00:00
Matan
c42efbf587
Merge pull request #49044 from rzarzynski/wip-crimson-ertr-plain-rets
crimson/ertr: let ErrVisitorT return plain value if ValueFuncT returns seastar::future

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: chunmei-liu <chunmei.liu@intel.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2023-07-02 14:02:01 +03:00
Matan
55fcd87221
Merge pull request #52032 from Matan-B/wip-matanb-crimson-no-ignore
crimson/osd: don't ignore start_pg_operation returned future

Reviewed-by: Samuel Just <sjust@redhat.com>
2023-07-02 13:53:25 +03:00
Matan
6bd7cd4c38
Merge pull request #51147 from liu-chunmei/multicore-osd
Crimson/osd: support multicore osd

Reviewed-by: Samuel Just <sjust@redhat.com>
2023-07-02 13:15:02 +03:00
Matan
1f1800b26e
Merge pull request #51916 from cyx1231st/wip-seastar-msgr-multi-core-2
crimson/net: support connections in multiple shards

Reviewed-by: Samuel Just <sjust@redhat.com>
2023-07-02 13:05:04 +03:00
Anthony D'Atri
00edceddba
Merge pull request #52280 from zdover23/wip-doc-2023-07-01-radosgw-multisite
doc/radosgw: refine "Zones" in multisite.rst
2023-07-01 09:46:19 -04:00
Zac Dover
8dab6c066c doc/radosgw: refine "Zones" in multisite.rst
Describe in greater detail the function of Zones in serving S3 objects,
as requested by Alexander Patrakov here:
https://github.com/ceph/ceph/pull/49981#issuecomment-1615866374.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-07-01 21:51:15 +10:00
zdover23
f79906237a
Merge pull request #52277 from zdover23/wip-doc-2023-07-01-radosgw-s3select-overview
doc/radosgw: s/s3select/S3 Select/

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-07-01 21:29:24 +10:00
Zac Dover
71c327ef4f doc/radosgw: s/s3select/S3 Select/
Correct several misspelling of "S3 Select". Hat tip to Anthony D'Atri,
who caught this in an earlier PR.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-07-01 21:14:30 +10:00
J. Eric Ivancich
892fd71829
Merge pull request #47595 from 5cs/fix-lc-head-marker
rgwlc: fix lc head marker point to non-exist lc entry

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2023-06-30 11:33:15 -04:00
J. Eric Ivancich
fb9d481188
Merge pull request #50667 from ivancich/wip-add-reupload-testing
qa/rgw: test that multipart re-upload does not leave any orphans

Reviewed-by: Cory Snyder <csnyder@iland.com>
2023-06-30 11:30:27 -04:00
Soumya Koduri
88a05a4600
Merge pull request #52252 from soumyakoduri/wip-skoduri-dbstore
rgw/dbstore: Update README

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2023-06-30 17:44:02 +05:30
Yingxin
ee16718511
Merge pull request #52217 from zhscn/fix-file-local-variable
crimson/os/seastore/omap_manager: correct editor settings

Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2023-06-30 10:32:24 +08:00
zdover23
cb5c7ecf4a
Merge pull request #52250 from zdover23/wip-doc-2023-06-29-radosgw-s3select-basic-workflow
doc/radosgw: edit "Basic Workflow" in s3select.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-06-30 02:00:13 +10:00
Zac Dover
4d2c09b683 doc/radosgw: edit "Basic Workflow" in s3select.rst
Edit the "Basic Workflow" section in doc/radosgw/s3select.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-06-30 01:27:52 +10:00
Soumya Koduri
467c76e3dd rgw/dbstore: Update README
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2023-06-29 16:00:43 +05:30
Mark Kogan
f834e10cfe rgw: d3n: fix valgrind reported leak related to libaio ops
Fixes: https://tracker.ceph.com/issues/61661

The valgrind leak indication is a false positive in this case,
it is because the libaio internal thread have not timed out yet
when radosgw is terminated.

```
man aio_init
...
aio_idle_time
This field specifies the amount of time in seconds that a worker thread
should wait for further requests before terminating, after having
completed a previous request.   The
default value is 1.
...
```
for the sake of teuthology reducing the timeout

waiting for 2 minutes for example like below would also prevent the leak report
```
❯ env
LD_LIBRARY_PATH=/mnt/nvme5n1p1/src-git/ceph--up--master-clang/build/lib/:$LD_LIBRARY_PATH
PYTHONPATH=$PYTHONPATH:/mnt/nvme5n1p1/src-git/ceph--up--master-clang/build/lib/cython_modules/lib.3
RAGWEED_CONF=$(realpath ./ragweed.conf) RAGWEED_STAGES=prepare,check tox
-- -v |& ccze -Aonolookups ;   sleep 2m | pv -t ; pkill radosgw
```

Signed-off-by: Mark Kogan <mkogan@redhat.com>
2023-06-29 08:13:49 +00:00
Radoslaw Zarzynski
5cd4a0ec53
Merge pull request #52008 from rishabh-d-dave/improvements-authmon
mon: improvements and cleanup for AuthMonitor

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-06-29 09:32:03 +02:00
Yingxin Cheng
a870946423 crimson/osd/heartbeat: relax the order of replacement reset and accept
With the new implementation in messenger, the order of replacement reset
and accept events cannot be determined because they are from different
connections.

Modify the heatbeat logic to tolerate the both cases.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2023-06-29 14:27:58 +08:00
Casey Bodley
b91ff3fd86 rgw: fetch_remote_obj() preserves RGW_ATTR_COMPRESSION of encrypted objects
if the source object was both compressed and encrypted, preserve its
original compression attribute so it can be decompressed the same way it
was originally compressed

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-06-28 15:19:19 -04:00
Casey Bodley
ea45da747d rgw: fetch_remote_obj() will never verify etags of encrypted objects
fetch_remote_obj() transfers objects in their encrypted form, so does
not have access to the decrypted data for checksum verification

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-06-28 15:19:19 -04:00
Casey Bodley
8d35759eb8 rgw: rgwx-skip-decrypt also skips decompression of encrypted objects
compression is applied before encryption. so if we skip decryption, we
can't decompress either

Fixes: https://tracker.ceph.com/issues/57905

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-06-28 15:19:19 -04:00
Ilya Dryomov
a0082f63e8
Merge pull request #52157 from pkalever/log-improve
rbd-mirror: add some assorted logs

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2023-06-28 11:45:50 +02:00
Soumya Koduri
31666d71aa rgw: Do not update mtime in set_attrs
As per https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingMetadata.html,
the only way for users to modify object metadata is to make a copy of the object and
set the metadata. Hence retain old mtime for any other attr changes

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2023-06-28 13:11:26 +05:30
zdover23
2767f5a72c
Merge pull request #52194 from zdover23/wip-doc-2023-06-26-radosgw-s3select-overview
doc/radosgw: edit "Overview" in s3select.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-06-28 16:43:08 +10:00
Zac Dover
e051dd1a75 doc/radosgw: edit "Overview" in s3select.rst
Edit the "Overview" section in doc/radosgw/s3select.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-06-28 16:16:00 +10:00
Venky Shankar
809d475814 Merge PR #49971 into main
* refs/pull/49971/head:
	doc/cephfs: document MDS_CLIENTS_LAGGY health warning
	qa: ignore warnings
	qa: add test cases to check client eviction if an OSD is laggy
	mds,messages: enable beacon to report clients lagginess
	mds: do not evict client on laggy osds
	common: add new config option to defer client eviction
	osd: add method to check for laggy osds

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-06-28 10:23:54 +05:30
Yingxin
2c617c35d5
Merge pull request #52090 from xxhdx1985126/wip-fixedkv-btree-merge-fix
crimson/os/seastore/btree: should add left's size when merging levels…

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2023-06-28 12:53:06 +08:00
Prasanna Kumar Kalever
2f85530c85 rbd-mirror: add some logs
Added some logs as their values where not very clear while parsing though the
log files.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2023-06-28 10:16:01 +05:30
Zhang Song
3663b22916 crimson/os/seastore/omap_manager: correct editor settings
Signed-off-by: Zhang Song <zhangsong325@gmail.com>
2023-06-28 10:14:45 +08:00
Patrick Donnelly
f72990b451
Merge PR #51858 into main
* refs/pull/51858/head:
	pybind/mgr/devicehealth: do not crash if db not ready

Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Yaarit Hatuka <yaarithatuka@gmail.com>
2023-06-27 13:43:38 -04:00
Venky Shankar
94e8649304 Merge PR #52170 into main
* refs/pull/52170/head:
	qa: assign file system affinity for replaced MDS

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
2023-06-27 17:51:31 +05:30
Aashish Sharma
a58490c1bd
Merge pull request #51283 from rhcs-dashboard/disable-delete
mgr/dashboard: disable delete on multisite

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
2023-06-27 17:28:12 +05:30
Prasanna Kumar Kalever
947a53677d rbd-mirror: fix image replayer shut down description on force promote
On force promote if the opposite site is down then we currently show
image status description as "local image linked to unknown peer"

Previously:
----------
$ rbd --cluster=site-b mirror image status pool1/img1
img1:
  global_id:   a73341a6-8302-4c97-ac6e-278083fd347e
  state:       up+stopping_replay
  description: local image linked to unknown peer
  service:     admin on localhost.localdomain
  last_update: 2023-06-15 19:47:45
  peer_sites:
    name: site-a
    state: up+stopped
    description: local image is primary
    last_update: 2023-06-15 19:47:32
  snapshots:
    9 .mirror.primary.a73341a6-8302-4c97-ac6e-278083fd347e.1f101367-277f-42f0-8308-e51201d0529a (peer_uuids:[c46c6d97-f59b-4591-9d35-d7ff9d0d72f7])

Currently:
---------
$ rbd --cluster=site-b mirror image status pool1/img1
img1:
  global_id:   2a6d61e1-8e76-42c4-af76-8f61ce65c7e2
  state:       up+stopped
  description: orphan (force promoting)
  service:     admin on localhost.localdomain
  last_update: 2023-06-15 19:29:22
  peer_sites:
    name: site-a
    state: down+stopped
    description: local image is primary
    last_update: 2023-06-15 19:29:05
  snapshots:
    9 .mirror.primary.2a6d61e1-8e76-42c4-af76-8f61ce65c7e2.99f82a30-0241-4e51-8428-7a2376d137f6 (peer_uuids:[3150c6ef-aeee-45dc-8d0e-5dc5a53d88eb])

Fixes: https://tracker.ceph.com/issues/52913
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2023-06-27 16:37:18 +05:30
Venky Shankar
f370b581f6 qa: assign file system affinity for replaced MDS
Otherwise, the MDS that just got replaced can transition to a rank
for another file system and the test cannot deterministically infer
which MDS needs to checked.

Fixes: http://tracker.ceph.com/issues/61764
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2023-06-27 09:23:14 +05:30
Yingxin Cheng
2fed9397d5 crimson/net: cleanup, rename is_fixed_cpu
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2023-06-27 10:31:11 +08:00