Commit Graph

143847 Commits

Author SHA1 Message Date
Matan Breizman
602578935b
Merge pull request #55536 from Matan-B/wip-crimson-snapmapper-get_next
osd/SnapMapper: Crimson - fix OSDriver::get_next

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: chunmei-liu <chunmei.liu@intel.com>
2024-03-06 15:19:40 +02:00
Zhipeng Li
e0a25e59d0 rgw: add help info for radosgw-admin list object in bucket
Fixes:https://tracker.ceph.com/issues/64264

Signed-off-by: Zhipeng Li <qiuxinyidian@gmail.com>
2024-03-06 21:06:51 +08:00
Zac Dover
ae08855cf8 doc/architecture: correct typo
s/client/clients/ where necessary, and add a link to the glossary.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2024-03-06 21:40:10 +10:00
Jos Collin
59dbbb218c
cephfs_mirror, qa: fix typo in current_syncing_snap status
Signed-off-by: Jos Collin <jcollin@redhat.com>
2024-03-06 16:33:45 +05:30
Anthony D'Atri
1c7f8b2488
Merge pull request #55982 from zdover23/wip-doc-2024-03-06-dev-developer-guide-testing-integration-tests-workflow
doc/dev: edit teuthology workflow
2024-03-06 05:51:46 -05:00
Venky Shankar
f5a23d5811 qa/cephfs: add MON_DOWN' and deprecated feature inline_data' to ignorelist
Fixes: http://tracker.ceph.com/issues/64746
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2024-03-06 15:19:08 +05:30
Venky Shankar
7a6389272a qa: ignore container checkpoint/restore related selinux denials for centos9
Fixes: http://tracker.ceph.com/issues/64616
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2024-03-06 14:58:32 +05:30
Venky Shankar
71c9fe55d6 qa: use allowlist instead of whitelist for selinux denials
https://github.com/ceph/teuthology/pull/1757 introduced this change,
however, qa/distros continued to use `whitelist' keyword.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
2024-03-06 14:58:31 +05:30
Guillaume Abrioux
cc279a3dd2 mgr/node-proxy: handle 'None' statuses returned by RedFish
Looks like RedFish might return 'None' values for some attributes.

for instance:

```
[root@ceph-node-01 ~]# curl -s -k -X GET https://169.254.1.1/redfish/v1/Systems/System.Embedded.1/Storage/AHCI.SL.6-1/Drives/Disk.Direct.0-0:AHCI.SL.6-1 -H "X-Auth-Token: 3264251c28191fa5e7c9ebec49ef90fc"  | jq .Status
{
  "Health": "OK",
  "HealthRollup": "OK",
  "State": "Enabled"
}
[root@ceph-node-01 ~]# curl -s -k -X GET https://169.254.1.1/redfish/v1/Systems/System.Embedded.1/Storage/NonRAID.Slot.2-1/Drives/Disk.Bay.0:Enclosure.Internal.0-1:NonRAID.Slot.2-1 -H "X-Auth-Token: 3264251c28191fa5e7c9ebec49ef90fc" | jq .Status
{
  "Health": null,
  "HealthRollup": null,
  "State": "Enabled"
}
[root@ceph-node-01 ~]#
```

Although this seems to be a bug from RedFish, we need to handle
the case when it happens otherwise it makes the mgr orchestrator module
throw an error.

The idea here is to create a new status "unknown" when we can't fetch the
real status of a component.

Fixes: https://tracker.ceph.com/issues/64712

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
2024-03-06 09:09:34 +00:00
Matan Breizman
b420380bda
Merge pull request #52267 from xxhdx1985126/wip-crimson-pg-snapmapper-obj
crimson/osd: put snapmapper's key-value pairs into dedicated objs

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2024-03-06 11:00:59 +02:00
Matan Breizman
f2af81369d
Merge pull request #55910 from zhscn/fix-int-string
osd/SnapManager: Crimson - fix std::string constructed from 0

Reviewed-by: chunmei-liu <chunmei.liu@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radosław Zarzyński <rzarzyns@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2024-03-06 10:56:13 +02:00
Matan Breizman
e025295f27
Merge pull request #55801 from xxhdx1985126/wip-counting-semaphore
crimson/os/alienstore: use homebrew counting_semaphore

Reviewed-by: chunmei-liu <chunmei.liu@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kefu.chai@scylladb.com>
2024-03-06 10:54:49 +02:00
Venky Shankar
6dc5a60109 Merge PR #54202 into main
* refs/pull/54202/head:
	mds: fix `dump dir` indicate dir is not cached

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
2024-03-06 14:14:22 +05:30
Venky Shankar
9fd7ad53cd Merge PR #55457 into main
* refs/pull/55457/head:
	client: check for bad file handle in low level I/O APIs
	client: check for bad file handle in ll_preadv_pwritev
	client: add function to check if file handle exists
	src/test: test async I/O with invalid/closed file handle

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2024-03-06 14:11:12 +05:30
Venky Shankar
55ac8d6d0b Merge PR #55619 into main
* refs/pull/55619/head:
	client: fix leak of file handles

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
2024-03-06 14:09:23 +05:30
Aashish Sharma
d96a1805f3 mgr/dashboard: Allow the user to add the access/secret key on zone edit
and not on zone creation

Fixes: https://tracker.ceph.com/issues/64080

Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2024-03-06 13:49:08 +05:30
Zac Dover
51796e80f3 doc/dev: edit teuthology workflow
Improve a couple of sentences in
doc/dev/developer_guide_testing_integration_tests/tests-integration-testing-teutology-workflow.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2024-03-06 17:15:20 +10:00
Nizamudeen A
ea0bdc2b1e
Merge pull request #55954 from ivoalmeida/fix-cephfs-attach-command
mgr/dashboard: fixed cephfs mount command

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2024-03-06 12:11:30 +05:30
Nizamudeen A
7d58640aa8
Merge pull request #54964 from rhcs-dashboard/add-prometheus-federation-cli
mgr/dashboard: add prometheus federation config for mullti-cluster monitoring

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: frittentheke <NOT@FOUND>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2024-03-06 11:59:08 +05:30
Nizamudeen A
e87a9f15af mgr/dashboard: fix nvmeof api documentation
From Aviv:
POST /api/nvmeof/hosts - the description of the command it wrong IMO. It is not about creating a host. It is about allowing a host X to access subsystem Y.
GET /api/nvmeof/hosts/{subsystem_nqn} - also the description is not accurate. The command lists all hosts that are allowed to access this subsystem.
DELETE /api/nvmeof/hosts/{subsystem_nqn}/{host_nqn} - again the description should be changed as above.
POST /api/nvmeof/namespace - bad formatting of the description
GET /api/nvmeof/subsystem - the description is wrong, should say - "List all NVMeoF subsystems". And it shouldn't get any param.
POST /api/nvmeof/subsystem - few issues here. The serial_number, and max_namespaces are optional (we need to mention that). Also it is missing the --enable-ha argument that is also optional.
Some commands are missing: log_level, connection.i

Fixes: https://tracker.ceph.com/issues/64714
Signed-off-by: Nizamudeen A <nia@redhat.com>
2024-03-06 10:35:00 +05:30
Nizamudeen A
278daa4ba2
Merge pull request #55574 from ceph/feature-multi-cluster-management-monitoring
mgr/dashboard: introduce multi cluster management and monitoring in ceph dashboard 

Reviewed-by: Nizamudeen A <nia@redhat.com>
2024-03-06 10:26:29 +05:30
Yingxin
3db3a2efc7
Merge pull request #55946 from myoungwon/wip-less-verbose-rbm-test
crimson/os/seastore: make less verbose log when testing performance on RBM

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
2024-03-06 10:00:30 +08:00
myoungwon oh
728e564f61 crimson/os/seastore: allow to remap the dirty extent
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2024-03-06 00:47:36 +00:00
Yuri Weinstein
181a671f9d doc: reef 18.2.2 release notes
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2024-03-05 15:09:44 -08:00
Yuval Lifshitz
666e79f1fb rgw/notifications: delete persistent queue only if topic is deleted
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
2024-03-05 12:56:05 -05:00
Yuval Lifshitz
bcd79d23df rgw/notifications: handle migration state between v1 and v2
test instructions:
https://gist.github.com/yuvalif/21449e301732b719cd1ed97c3eeeabb2

* during migration all topic and notification operations must fail with HTTP error code 503
* read operations should return the values of the v1 topics and notifications
* sending notifications should continue based on v1 values

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
2024-03-05 12:56:05 -05:00
Casey Bodley
5b2ae726a9 rgw: move rgw_rest_pubsub.h out of driver/rados/
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2024-03-05 12:55:27 -05:00
Casey Bodley
9d46c21bc6 rgw/pubsub: topic apis forward post body
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2024-03-05 12:55:27 -05:00
Casey Bodley
633a5bef31 rgw/topic: metadata handler adds/removes persistent queues
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2024-03-05 12:55:27 -05:00
Casey Bodley
be9cc0c7bc rgw/topic: remove unused RGWSI_Topic_RADOS
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2024-03-05 12:55:27 -05:00
Casey Bodley
6d1d036afb rgw/topic: add rgwrados::topic interface for topic metadata
add a new interface for topic metadata that doesn't depend on metadata
backends. this low-level interface is used by both RadosStore and the
topic metadata handler

remove Driver::delete_bucket_topic_mapping() from sal because the omap
object is deleted internally by rgwrados::topic::remove()

remove the RGWRados::topics_pool_ctx member

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2024-03-05 12:55:25 -05:00
Casey Bodley
1caf954dab rgw/topic: metadata key format in rgw_pubsub.h
the format of topic metadata keys is agnostic to the backend, so the
parsing/formatting functions should be in rgw_pubsub.h

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2024-03-05 12:55:25 -05:00
Casey Bodley
db6c73a0cd rgw: paginate ListTopics
rename read_topics()/write_topics() to 'v1' and only call them from
internal v1 call paths

public get_topics() now calls read_topics_v1() for the v1 case, and does
the paginated listing with driver->meta_list_keys_next() for v2

RGWPSListTopicsOp now uses the NextToken request/response params with
the paginated get_topics(), limiting responses to 100 entries like AWS

'radosgw-admin topic list' also paginates the listing according to
--max-entries to avoid reading everything into memory at once

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2024-03-05 12:55:25 -05:00
Casey Bodley
7827b2527f rgw/mdlog: add complete_entry() for most common usage
metadata sync ignores entries other than MDLOG_STATUS_COMPLETE, so we
don't need to write separate prepare/complete entries. metadata
mutations can just call complete_entry() on success

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2024-03-05 12:55:25 -05:00
Casey Bodley
6000732706 rgw/metadata: add RGWMetadataLister abstraction
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2024-03-05 12:55:25 -05:00
Casey Bodley
1fccdad21d rgw: RGWPubSub requires SiteConfig
RGWPubSub constructor takes SiteConfig instead of zonegroup map

replace do_all_zonegroups_support_notification_v2() with a generic
function rgw::all_zonegroups_support() that handles non-realm
configurations too

remove unused sal::ZoneGroup::supports_feature()

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2024-03-05 12:55:25 -05:00
Casey Bodley
0b8b62bed0 rgw/rados: RGWTopicMetadataHandler doesn't depend on RGWPubSub
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2024-03-05 12:55:24 -05:00
Casey Bodley
0057bb7d0f rgw/rados: store SiteConfig with RGWServices
make the SiteConfig available to all of RGWRados via svc.site instead
of storing it in sal::RadosStore

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2024-03-05 12:55:24 -05:00
Casey Bodley
0f333f1cc5 radosgw-admin: make SiteConfig available to commands
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2024-03-05 12:55:24 -05:00
Yuval Lifshitz
d2db9df224 rgw/multisite-notification: allow enabling v1/v2 in tests
v1 could be enabled only in local tests. teuthology tests would run with v2

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
2024-03-05 12:55:24 -05:00
Yuval Lifshitz
7502215d70 test/rgw/notifications: support running tests in multisite environment
both locally and in teuthology

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
2024-03-05 12:55:24 -05:00
kchheda3
1ed8df24ae rgw/multisite-notification: Add integration test for bucket notifications in multisite config.
Signed-off-by: kchheda3 <kchheda3@bloomberg.net>
2024-03-05 12:55:24 -05:00
kchheda3
df668e9f65 rgw/multisite-notification: Add omap object to store the mapping between bucket and topics.
Signed-off-by: kchheda3 <kchheda3@bloomberg.net>
2024-03-05 12:55:24 -05:00
kchheda3
ab74e2cfe4 rgw/multisite-notification: Add support to replicate bucket notifications in multisite config.
Signed-off-by: kchheda3 <kchheda3@bloomberg.net>
2024-03-05 12:55:24 -05:00
kchheda3
c4a147d079 rgw/multisite-notification: Add support to replicate topic creation in multisite config
Signed-off-by: kchheda3 <kchheda3@bloomberg.net>
2024-03-05 12:55:24 -05:00
Mark Kogan
6fb4c5d0c0 rgw/beast: enablment of SSL session-id reuse speedup mechanism
Enable the OpenSSL session-id reuse acceleration mechanism that is described in:

https://www.openssl.org/docs/man1.0.2/man3/SSL_CTX_set_session_id_context.html
SSL_CTX_set_session_id_context, SSL_set_session_id_context - set context
within which session can be reused (server side only)

Fixes: https://tracker.ceph.com/issues/64719

Signed-off-by: Mark Kogan <mkogan@redhat.com>
2024-03-05 16:48:56 +02:00
Nizamudeen A
b8811c844f mgr/dashboard: introduce multi-cluster overview page
https://tracker.ceph.com/issues/64530
Signed-off-by: Nizamudeen A <nia@redhat.com>
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2024-03-05 19:05:37 +05:30
Ivo Almeida
eeb09b1958 mgr/dashboard: fixed cephfs mount command
Fixes: https://tracker.ceph.com/issues/64716

Signed-off-by: Ivo Almeida <ialmeida@redhat.com>
2024-03-05 12:39:08 +00:00
Kefu Chai
6c9d1b6484
Merge pull request #55905 from ronen-fr/wip-rf-23a
common: fix string creation from '0' in LogEntry

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2024-03-05 20:27:13 +08:00
Nizamudeen A
1d74cbc67f
Merge pull request #55464 from rhcs-dashboard/hardware-statuses
mgr/dashboard: add hardware status summary
2024-03-05 16:31:15 +05:30