Commit Graph

136359 Commits

Author SHA1 Message Date
Samuel Just
3aea58ddba
Merge pull request #50019 from rzarzynski/wip-crimson-fix-zeroed-startevent
crimson/osd: fix zeroed starting timestamp of tracked ops

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2023-02-14 19:57:02 -08:00
Yingxin
3d8b9137ff
Merge pull request #49420 from cyx1231st/wip-seastar-msgr-multi-core
crimson/net: refactors to encapsulate ProtocolV2 message read and write paths and event dispatching

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2023-02-15 09:13:06 +08:00
Zac Dover
2d45a4e5e0 doc/rgw: refine "Multi-site Config Ref" (1 of x)
Refine the syntax (and in some cases, the semantics) of the English in
"Multi-site Configuration Reference" in doc/radosgw/multisite.rst. This
is a long section, so more PRs are likely.

https://tracker.ceph.com/issues/58632

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Co-authored-by: Casey Bodley <cbodley@redhat.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-02-15 08:39:32 +10:00
Laura Flores
377738630d
Merge pull request #47555 from JoshSalomon/wip-prim-balance-score
mon, osd: Adding read (aka workload, primary) balance score
2023-02-14 15:39:09 -06:00
John Mulligan
480f7fc115 cephadm/tests: add test coverage of get_distro function
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-02-14 10:37:21 -05:00
John Mulligan
d17b92c7c8 cephadm/tests: add test coverage of check_time_sync function
This is also effectively full coverage of `check_units` as well
as check_time_sync is the only consumer.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2023-02-14 10:37:21 -05:00
Anthony D'Atri
410cb54b61
Merge pull request #50104 from cityofships/fix_docs_lvm_activate
doc/ceph-volume: fix cephadm references
2023-02-14 09:39:12 -05:00
Radoslaw Zarzynski
79e9e046e6 crimson/osd: fix FTBFS due to smart-ptr-to-bool conversion in fmt::ptr
```
../src/fmt/include/fmt/format.h: In instantiation of ‘const void* fmt::v9::ptr(T) [with T = seastar::shared_ptr<crimson::net::SocketConnection>]’:
../src/crimson/net/ProtocolV2.cc:1109:72:   required from here
../src/fmt/include/fmt/format.h:3809:37: error: static assertion failed
 3809 |   static_assert(std::is_pointer<T>::value, "");
      |                                     ^~~~~
../src/fmt/include/fmt/format.h:3809:37: note: ‘std::integral_constant<bool, false>::value’ evaluates to false
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2023-02-14 14:37:56 +00:00
Casey Bodley
cdd53d9430
Merge pull request #50102 from cbodley/wip-github-examples-rgw
examples: move rgw examples under subdirectory

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
2023-02-14 09:11:42 -05:00
kalebskeithley
8fda937e29
Merge pull request #50099 from kalebskeithley/rgw_token.gcc13
rgw: rgw_token.cc error: reference to 'formatter' is ambiguous
2023-02-14 07:56:26 -05:00
Nizamudeen A
b9e2436f8c
Merge pull request #49801 from rhcs-dashboard/service-instances-flickering
mgr/dashboard: move service_instances logic to backend

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
2023-02-14 18:14:42 +05:30
Casey Bodley
e25a7e86e9 doc/rgw: update links to examples/rgw
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-02-14 07:33:57 -05:00
Nizamudeen A
9db0d6bb87
Merge pull request #49834 from rhcs-dashboard/rgw-list-roles
mgr/dashboard: rgw role listing

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2023-02-14 18:03:28 +05:30
Nizamudeen A
dbcf28d462 mgr/dashboard: adapt nfs export code to the new changes in nfs module
when you create/edit an nfs export from dashboard it leaves this traceback and error

```
Feb 09 14:15:54 ceph-node-00 ceph-mgr[3235]: [dashboard ERROR taskexec] Error while calling Task(ns=nfs/create, md={'path': 'e2e.nfs.bucket', 'fsal': 'RGW', 'cluster_id': 'testnfs'})
                                             Traceback (most recent call last):
                                               File "/usr/share/ceph/mgr/dashboard/tools.py", line 550, in _run
                                                 val = self.task.fn(*self.task.fn_args, **self.task.fn_kwargs)  # type: ignore
                                               File "/usr/share/ceph/mgr/dashboard/controllers/nfs.py", line 148, in create
                                                 ret, _, err = export_mgr.apply_export(cluster_id, json.dumps(raw_ex))
                                             TypeError: 'AppliedExportResults' object is not iterable
Feb 09 14:15:54 ceph-node-00 ceph-mgr[3235]: [dashboard INFO taskmgr] finished Task(ns=nfs/create, md={'path': 'e2e.nfs.bucket', 'fsal': 'RGW', 'cluster_id': 'testnfs'})
Feb 09 14:15:54 ceph-node-00 ceph-mgr[3235]: [dashboard INFO request] [::ffff:192.168.100.1:43896] [POST] [500] [0.767s] [admin] [172.0B] /api/nfs-ganesha/export
```
This started after https://github.com/ceph/ceph/pull/46209, so dashboard code needs to be adapted

Fixes: https://tracker.ceph.com/issues/58681
Signed-off-by: Nizamudeen A <nia@redhat.com>
2023-02-14 16:41:09 +05:30
Nizamudeen A
bf724b505e mgr/dashboard: move service_instances logic to backend
Fixes: https://tracker.ceph.com/issues/58504
Signed-off-by: Nizamudeen A <nia@redhat.com>
2023-02-14 16:26:23 +05:30
Josh Salomon
8ed66d3087 osd, mon: add read balance score to command ceph osd pool ls detail
for replicated pools.

osd: Added workload balance score to the command
     ceph osd pool ls detail
     (different flavors for console output and json/xml output)
mon: Added command 'osd rm-primary-temp'
(developers only) rm was not working
vstart: Added osd debug messages into mon log with -d flag
        For commands that execute methonds in OSD module

Signed-off-by: Josh Salomon <jsalomon@redhat.com>
2023-02-14 12:49:56 +02:00
Nizamudeen A
e70c55cb7b
Merge pull request #50051 from rhcs-dashboard/rbd-mirroring-status-fix
mgr/dashboard: fix rbd mirroring daemon health status

Reviewed-by: Pegonzal <NOT@FOUND>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2023-02-14 15:54:54 +05:30
Pere Diaz Bou
07e07b8d8a mgr/dashboard: rgw role listing
Listing is performed using the radosgw-admin command api we have with
the mgr for now until the S3 api is fixed: https://tracker.ceph.com/issues/58547.

This commit fixes and issue with regards to the _crud.py controller
where redefining `CRUDClassMetadata` caused the users table and the
roles table to share columns. We fixed this by creating
CRUDClassMetadata dynamically for each endpoint.

The issue described above is linked to an issue with NamedTuple were
default nested lists are not a great move because it can cause
unexpected issues when 2 or more classes are created. Moreover,
NamedTuples are read-only making initialization even harder without
factory methods as with dataclasses. Therefore, let's move to the good
old __init__ :).

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
Fixes: https://tracker.ceph.com/issues/58699
2023-02-14 10:10:00 +01:00
Piotr Parczewski
2104d9ffda
doc/ceph-volume: fix cephadm references
Signed-off-by: Piotr Parczewski <piotr@stackhpc.com>
2023-02-14 09:12:30 +01:00
Samuel Just
73d4a7e44a
Merge pull request #49729 from rzarzynski/wip-crimson-obcregistry-per-pg
crimson/osd: make the ObjectContextRegistry per-PG

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2023-02-13 23:00:05 -08:00
Samuel Just
ddf898e545
Merge pull request #49020 from xxhdx1985126/wip-pg-remove
crimson/osd: implement pg removal

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2023-02-13 22:58:55 -08:00
Ronen Friedman
ffda64119f osd/scrub: create a separate chunk size conf for shallow scrubs
Using the existing common default chunk size for scrubs that are
not deep scrubs is wasteful: a high ratio of inter-OSD messages
per chunk, while the actual OSD work per chunk is minimal.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-02-14 07:58:01 +02:00
Venky Shankar
8455ca95bb
Merge pull request #50058 from rishabh-d-dave/vstart-client-debug
vstart.sh: set logging for client and MDS when -d is passed

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-02-14 10:11:34 +05:30
Casey Bodley
36747b08ff examples: move rgw examples under subdirectory
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-02-13 17:20:26 -05:00
Ilya Dryomov
b8366645c1
Merge pull request #50076 from idryomov/wip-codeowners-doc-writers-fix
CODEOWNERS: don't override doc-writers by core and rbd for some files

Reviewed-by: Neha Ojha <nojha@redhat.com>
2023-02-13 23:05:43 +01:00
Samuel Just
ea33358e18
Merge pull request #50070 from xxhdx1985126/wip-sync-peering-backfill
crimson/osd/pg: synchronize processing of peering events and backfill

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2023-02-13 12:44:51 -08:00
Samuel Just
37f1e4a85c
Merge pull request #49344 from xxhdx1985126/wip-obc-watchers-checkonload
crimson/osd/object_context_loader: check on old watchers when loading obcs

Reviewed-by: Samuel Just <sjust@redhat.com>
2023-02-13 12:44:05 -08:00
Samuel Just
561f8621fe
Merge pull request #49955 from xxhdx1985126/wip-pg-stats-read
crimson/osd/ops_executor: apply delta stats for pure read ops

Reviewed-by: Samuel Just <sjust@redhat.com>
2023-02-13 12:43:32 -08:00
Samuel Just
cd30a94e92
Merge pull request #50018 from Matan-B/wip-matanb-c-pg-pipeline
crimson/osd/osd_operations: Remove replicated_request_pg_pipeline

Reviewed-by: Samuel Just <sjust@redhat.com>
2023-02-13 12:42:45 -08:00
Ilya Dryomov
1aeae0aebd CODEOWNERS: don't override doc-writers by core and rbd for some files
This happens because core and rbd ownership is defined after
doc-writers: a later match takes precedence.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2023-02-13 19:05:03 +01:00
Kaleb S. KEITHLEY
7c84974ea2 rgw: rgw_token.cc error: reference to 'formatter' is ambiguous
compiling main branch on Fedora 38 and 39/rawhide with gcc-c++-13.0.1
fails with the error:

/builddir/build/BUILD/ceph-18.0.0-2148-g9754cafc/src/rgw/rgw_token.cc:
In function 'int main(int, char**)':
/builddir/build/BUILD/ceph-18.0.0-2148-g9754cafc/src/rgw/rgw_token.cc:125:3:
error: reference to 'formatter' is ambiguous
  125 | formatter = new JSONFormatter(true /* pretty */);
      |   ^~~~~~~~~

Similar error for src/test/test_rgw_token.cc

Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
2023-02-13 11:15:59 -05:00
Casey Bodley
a9cbbba338
Merge pull request #50073 from cbodley/wip-codeowners-rgw
github: add rgw team to CODEOWNERS

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
2023-02-13 10:53:42 -05:00
Adam King
45813ea2f3
Merge pull request #49555 from fmount/drive_group_crush
Add per OSD crush_device_class definition

Reviewed-by: Adam King <adking@redhat.com>
2023-02-13 10:09:07 -05:00
zdover23
56158d5fbd
Merge pull request #50086 from zdover23/wip-doc-2023-02-13-multisite-migrating-single-site-to-multi-site
doc/rgw: "Migrating Single Site to Multi-Site"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-02-13 22:46:05 +10:00
Zac Dover
40c82f8ef6 doc/rgw: "Migrating Single Site to Multi-Site"
Refine "Migrating Single-site System to Multi-site" in
doc/radosgw/multisite.rst.

https://tracker.ceph.com/issues/58632

Co-authored-by: Cole Mitchell <cole.mitchell@gmail.com>
Co-authored-by: Anthonh D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-02-13 22:06:33 +10:00
Ilya Dryomov
6d8fb57156
Merge pull request #50071 from idryomov/wip-58688
qa/suites/rbd: fix sporadic "rx-only direction" test failures

Reviewed-by: Ramana Raja <rraja@redhat.com>
2023-02-13 12:54:26 +01:00
Yingxin Cheng
7d54be7958 crimson/net: fix the corner case when the replacing connection doesn't have socket
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2023-02-13 14:42:13 +08:00
Yingxin Cheng
7b93616a84 test/crimson: improve messenger test coverage
* add blockings at custom_bp_t::SOCKET_CONNECTING
* adjust peer_wins to true

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2023-02-13 14:39:46 +08:00
Yingxin Cheng
19ede1aee1 test/crimson: enable blocking with custom_bp_t::SOCKET_CONNECTING in messenger test
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2023-02-13 11:35:35 +08:00
Yingxin Cheng
c30fb84d9c test/crimson: cleanup messenger test addresses
Consolidate the name/port/nonce of the addresses used in the messenger
unit tests.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2023-02-13 11:28:19 +08:00
Adam King
377b7606c1 cephadm: fix autopep8 arg parsing on newer tox versions
This was failing for me on new tox versions with

fix: commands[1]> autopep8 --max-line-length 100
incorrect number of argumentsfix: exit 99 (0.06 seconds) /home/adking/ceph/src/cephadm> autopep8 --max-line-length 100 pid=309039
fix: FAIL ✖ in 0.08 seconds

As it was considering our list of args for autopep8 to each be
one separate command. Fixed by just adding slashes to the ends
of the lines (as has already been done in the tox.ini in src/pybind/mgr
for some of the tests there)

Signed-off-by: Adam King <adking@redhat.com>
2023-02-12 22:18:25 -05:00
licc
63c7d2a44b rgw: RGWSI_RADOS::Pool::List::init() catches IoCtx::nobjects_begin() exceptions
Signed-off-by: lichaochao <lichaochao2_yewu@cmss.chinamobile.com>
2023-02-13 02:01:26 +01:00
licc
14176f00f4 rgw:log_list_init() catches IoCtx::nobjects_begin() exceptions
Signed-off-by: lichaochao <lichaochao2_yewu@cmss.chinamobile.com>
2023-02-13 02:00:49 +01:00
Casey Bodley
78c3839d89 github: add /examples to CODEOWNERS
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-02-12 11:00:59 -05:00
Anthony D'Atri
8c1d7ce1dd
Merge pull request #50077 from zdover23/wip-doc-2023-02-12-multisite-failover
doc/rgw: refine "Failover and Disaster Recovery"
2023-02-11 23:54:45 -05:00
Zac Dover
815359800c doc/rgw: refine "Failover and Disaster Recovery"
English grammar and syntax pass for "Failover and Disaster Recovery" in
doc/radosgw/multisite.rst.

https://tracker.ceph.com/issues/58632

Co-authored-by: Cole Mitchell <cole.mitchell@gmail.com>
Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2023-02-12 09:47:16 +10:00
zdover23
f21d7e0e8c
Merge pull request #50066 from zdover23/wip-doc-2023-02-10-multisite-configuring-secondary-zones
doc/rgw: refine "Configuring Secondary Zones"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-02-11 09:51:51 +10:00
Casey Bodley
41c3975c9f github: add rgw team to CODEOWNERS
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-02-10 13:17:21 -05:00
Ilya Dryomov
5cec2670be qa/suites/rbd: fix sporadic "rx-only direction" test failures
The existing

    xmlstarlet sel -t -v  '//mirror/peers/peer[1]/uuid')" = ""

test is bogus since a tx-only peer gets added after the remote
rbd-mirror daemon pings the local cluster.  It happened to pass most
of the time because xmlstarlet filter just failed on an empty peers
array, producing the wrongly expected empty string by accident.

Fixes: https://tracker.ceph.com/issues/58688
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2023-02-10 15:26:27 +01:00
Ilya Dryomov
1944588033
Merge pull request #49763 from Svelar/vstart4
vstart: check mgr status after starting mgr

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2023-02-10 15:23:19 +01:00