Commit Graph

136954 Commits

Author SHA1 Message Date
Casey Bodley
1bc8d66495 rgw/aio: librados_op() requires librados::IoCtx
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-03-12 18:56:54 -04:00
Casey Bodley
48d53b401e
Merge pull request #50385 from cbodley/wip-cmake-asan
cmake: some fixes and suppressions for WITH_ASAN under vstart.sh

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2023-03-12 14:29:25 -04:00
Guillaume Abrioux
370e0ed0fe
Merge pull request #50473 from guits/cv-fix-cv-tests
ceph-volume: update the OS before deploying Ceph
2023-03-12 08:11:19 +01:00
Anthony D'Atri
1680aff2c0
Merge pull request #50483 from cbodley/wip-rgw-beast-server-name
rgw/beast: add extra_response_headers for Server name
2023-03-11 19:07:33 -05:00
Casey Bodley
f6a49aeff4
Merge pull request #50168 from cbodley/wip-23838
rgw: store torrent file for GetObjectTorrent in object attrs instead of omap

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2023-03-11 15:09:02 -05:00
Casey Bodley
f0a3f733cf
Merge pull request #50346 from vedanshbhartia/rm-57938
rgw: use unique_ptr for flat_map emplace in BucketTrimWatche

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2023-03-11 14:39:46 -05:00
Casey Bodley
3a1d3752aa
Merge pull request #50429 from cbodley/wip-58932
rgw/admin: 'bucket stats' displays non-empty mtime

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2023-03-11 14:34:06 -05:00
Casey Bodley
028e730822
Merge pull request #50211 from bbc/add_quota_containers_rgw_rest_header
rgw: add containers quota to swift rest headers

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2023-03-11 14:33:19 -05:00
Casey Bodley
ba9949744a
Merge pull request #49466 from linuxbox2/wip-lc-size
rgw/notifications: fetch object state to get size, in rgw_lc.cc

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
2023-03-11 14:32:47 -05:00
Casey Bodley
e64a0de467
Merge pull request #47040 from hualongfeng/qat_batch_mode
Crypto: Add QAT batch mode

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2023-03-11 14:31:47 -05:00
Casey Bodley
7b56006e0e doc/rgw: remove rgw_thread_pool_size from frontend options
this is a config variable, not a frontend option. and the description is
not accurate

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-03-11 14:15:31 -05:00
Casey Bodley
ffeacc0959 rgw/beast: add extra_response_headers for Server name
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-03-11 13:45:29 -05:00
zdover23
27325272c4
Merge pull request #50470 from zdover23/wip-doc-2023-03-10-landing-page-uniquely
doc/index: remove "uniquely" from landing page

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-03-11 07:20:11 +10:00
Yuri Weinstein
2ef67f139c
Merge pull request #49963 from ifed01/wip-ifed-fix-prefixed-kv-iterator
kv/RocksDBStore: don't use real wholespace iterator for prefixed access

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: Cory Snyder <csnyder@iland.com>
2023-03-10 12:13:07 -08:00
Yuri Weinstein
3049ada40c
Merge pull request #49958 from ifed01/wip-ifed-report-min-alloc-size
os/bluestore: report min_alloc_size through "ceph osd metadata"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-03-10 12:11:29 -08:00
Yuri Weinstein
98bd899d7a
Merge pull request #49768 from rzarzynski/wip-crimson-fmt9-compile-guard
crush: fix compilation guard around specialization for libfmt => 9

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Matan Breizman <Matan.Brz@gmail.com>
2023-03-10 12:11:03 -08:00
Yuri Weinstein
d601267d9d
Merge pull request #49524 from rzarzynski/wip-osd-snaptrimmer-awaitasyncwork-better-ret
osd: slightly improve readability in handling of snap trim errors

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
2023-03-10 12:09:16 -08:00
Laura Flores
d0049e4aa7
Merge pull request #48948 from ljflores/wip-telemetry-show-all
mgr/telemetry: make sure histograms are formatted in `all` commands
2023-03-10 13:32:01 -06:00
J. Eric Ivancich
fe9eb488d0
Merge pull request #50348 from ivancich/wip-restore-bucket-index
rgw: experimental support for restoring a lost bucket index

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2023-03-10 12:32:58 -05:00
Yuri Weinstein
a492b0843c
Merge pull request #50465 from ceph/wip-yuri-branch_fix-main
qa/tests: added reef branch

Reviewed-by: Laura Flores <lflores@redhat.com>
2023-03-10 08:06:55 -08:00
Guillaume Abrioux
491e88ede5 ceph-volume: add bluefs_buffered_io: false in functional tests
This is a workaround to make dmcrypt scenarios pass the CI.

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
2023-03-10 16:44:16 +01:00
Yuval Lifshitz
d70c5a2294
Merge pull request #49807 from DHB-liuhong/lh-bucket-notification-cache
rgw: add bucket notification cache

reviewed-by: cbodley, yuvalif
2023-03-10 17:33:13 +02:00
Guillaume Abrioux
663c914c59 ceph-volume: update the OS before deploying Ceph
ceph-volume tests are failing, OSDs never get up and running.
For some reason, updating the OS early in the testing workflow
addresses that issue in the CI.

-- to be continued ... --

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
2023-03-10 16:01:19 +01:00
Casey Bodley
8b637c4c09 rgw/admin: hide bencoded torrent data from 'object stat'
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-03-10 09:58:21 -05:00
Venky Shankar
bd5895229c Merge PR #50013 into main
* refs/pull/50013/head:
	mgr/volumes: avoid returning -ESHUTDOWN back to cli

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
2023-03-10 17:48:24 +05:30
Venky Shankar
bbdc17cb86 Merge PR #48280 into main
* refs/pull/48280/head:
	qa: test new client with old cluster

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-03-10 16:50:45 +05:30
Venky Shankar
f2709dd181 Merge PR #49102 into main
* refs/pull/49102/head:
	qa: add test for schedule auto deactivation
	doc/cephfs: add note about path usage in snap-schedule
	mgr/snap_schedule: add debug log for missing or wrong path

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
2023-03-10 16:45:35 +05:30
Zac Dover
6e5be7ad66 doc/index: remove "uniquely" from landing page
Change "Ceph uniquely delivers[...]" to "Ceph delivers[...]"

Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-03-10 20:35:18 +10:00
Feng Hualong
c097b2469e test/rgw: Add optional_yield in rgw_crypto
Signed-off-by: Feng Hualong <hualong.feng@intel.com>
2023-03-10 17:36:03 +08:00
Feng Hualong
0ce6d03bbc Crypto: Add QAT batch mode
Now, the current code does not allow qat to exert
its acceleration advantage, but leads to poor performance
of qat. The QAT batch mode is implemented here, so that
the QAT performance can be displayed.

When we scale up the number of concurrent requests and worry about
QAT instance will be bottleneck, we can fall back to CPU.

And there add a parameter to set the times of the number of QAT
instance to wait for free instance, which can avoid QAT to be
not busy and make sure QAT full of utilization as much as possible.

max_queue_size is up to max_requests

Add optional_yield in RGWPutObj_BlockEncrypt and
RGWGetObj_BlockDecrypt. Make it with coroutine and non-coroutine mode.

Signed-off-by: Feng Hualong <hualong.feng@intel.com>
2023-03-10 17:02:42 +08:00
dparmar18
179e4bcae9 qa: test new client with old cluster
Fixes: https://tracker.ceph.com/issues/53573
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2023-03-10 13:15:17 +05:30
liuhong
6de6fd851a rgw: add bucket notification cache
Bucket notification supports reading the rule configuration from the cache first, rather than directly from the bottom layer

Signed-off-by: liuhong <liuhong_yewu@cmss.chinamobile.com>
2023-03-10 08:34:05 +01:00
Venky Shankar
bfe00d4bbc Merge PR #45224 into main
* refs/pull/45224/head:
	mds: add support for snapshots alternate_name

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: David Disseldorp <ddiss@suse.de>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-03-10 10:15:59 +05:30
Venky Shankar
81e27bdbc7 Merge PR #50107 into main
* refs/pull/50107/head:
	client: fix CEPH_CAP_FILE_WR caps reference leakage in _write()

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-03-10 10:09:09 +05:30
Venky Shankar
9fd4cb3a03 Merge PR #50127 into main
* refs/pull/50127/head:
	client: correct quota check in Client::_rename()

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
2023-03-10 09:57:15 +05:30
Yuri Weinstein
be459d5e82 qa/tests: added reef branch
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2023-03-09 15:05:00 -08:00
Yuri Weinstein
7ebcdeac82
Merge pull request #49870 from ifed01/wip-ifed-rm-range-threshold-on-the-fly
kv/rocksdbstore: apply rocksdb_delete_range_threshold on the fly

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
2023-03-09 14:52:42 -08:00
Yuri Weinstein
3848f449d7
Merge pull request #49628 from Rethan/feat-osd-op-thread-timeout
osd: allow osd_op_thread_timeout and suicide_timeout to be adjusted o…

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2023-03-09 14:52:12 -08:00
Laura Flores
af986901a9
Merge pull request #50199 from athanatos/sjust/wip-scrub-event-helpers-50088
scrub: use a generic interface for scheduling timer based events
2023-03-09 15:30:23 -06:00
Laura Flores
7d3685de04
Merge pull request #50283 from ronen-fr/wip-rf-partial-unpub
osd/scrub: only call publish_stats_to_osd() where allowed and needed
2023-03-09 14:54:47 -06:00
Laura Flores
cda85937c4
Merge pull request #50065 from ajarr/fix-58691
mgr: store names of modules that register RADOS clients in the MgrMap
2023-03-09 13:09:04 -06:00
Laura Flores
8a9ee6f6a1
Merge pull request #49546 from Matan-B/wip-matanb-pull-push-naming
osd/ReplicatedBackend: Rename Push/Pull Info
2023-03-09 12:55:18 -06:00
J. Eric Ivancich
6ea11b1305 rgw: add rgw-restore-bucket-index
Adds an experimental script that allows a bucket index of a
non-versioned bucket to be restored by applying `radosgw-admin object
reindex ...` to all objects in the specified bucket. The objects in
the bucket are determined by scanning the data pool for head objects
containing the bucket's marker.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2023-03-09 11:00:57 -05:00
J. Eric Ivancich
be0583c308 rgw: add an "object reindex" subcommand to radosgw-admin
This subcommand will take a bucket and an object (or a list of objects
in a file) and add those objects to the bucket's index. It does so by
preparing the bucket index entry update and then allowing the
so-called "dir suggest" mechansim, which is triggered by listing the
bucket, to completing the bucket index entry. This mechanism is
triggered by listing the bucket.  Using this mechanism has the
advantage of doing this lazily that both allows the reindex operation
to run more quickly and distributes the workload over time.

This subcommand can help restore lost bucket index entries.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2023-03-09 11:00:57 -05:00
Venky Shankar
950a61ac3a Merge PR #50195 into main
* refs/pull/50195/head:
	cephfs-top: Handle `METRIC_TYPE_NONE` fields for sorting

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-03-09 19:50:38 +05:30
Nizamudeen A
67c2b69a95
Merge pull request #50456 from rhcs-dashboard/custom-image-kcli-bootstrap
mgr/dashboard: custom image for kcli bootstrap script

Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2023-03-09 19:40:01 +05:30
Nizamudeen A
be8f1e7f4a mgr/dashboard: custom image for kcli bootstrap script
the stable branches like quincy pulls from the quay.io/ceph/ceph:v17 to
bootstrap the ceph cluster in test environments. This will cause issues
because the branches are changing constantly but the image is not. So
using the quay.ceph.io repo to bring the cluster in test environment.

Signed-off-by: Nizamudeen A <nia@redhat.com>
2023-03-09 18:00:52 +05:30
Venky Shankar
26cbd243e4 Merge PR #50177 into main
* refs/pull/50177/head:
	cephfs-top: navigate to home screen when no fs

Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
2023-03-09 17:51:17 +05:30
Nizamudeen A
dcbf81c6cc
Merge pull request #49953 from rhcs-dashboard/dashboard-rgw-multisite
mgr/dashboard: Dashboard RGW multisite configuration

Reviewed-by: aashish1412 <NOT@FOUND>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2023-03-09 17:13:45 +05:30
Aashish Sharma
3d6e9ac22b mgr/dashboard: Dashboard RGW multisite configuration
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2023-03-09 14:48:56 +05:30