1
0
mirror of https://github.com/ceph/ceph synced 2024-12-25 12:54:16 +00:00
Commit Graph

142684 Commits

Author SHA1 Message Date
Venky Shankar
f155b8fb61 Merge PR into main
* refs/pull/54446/head:
	qa: bump up scrub status command timeout

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
2024-01-29 18:54:44 +05:30
Venky Shankar
527b465872 Merge PR into main
* refs/pull/54683/head:
	qa/tasks/cephfs/test_misc: switch duration to timeout
	qa/tasks/cephfs/test_misc: fix the incorrect comments

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
2024-01-29 18:52:16 +05:30
Venky Shankar
6d61a73190 Merge PR into main
* refs/pull/53578/head:
	mds: adhere to mds_health_summarize_threshold config when reporting laggy clients

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
2024-01-29 18:45:58 +05:30
Venky Shankar
320442d791 Merge PR into main
* refs/pull/53177/head:
	qa: test nfs export apply -i failure returns pseudo path in output JSON
	mgr/nfs: include pseudo in JSON output when nfs export apply -i fails

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
2024-01-29 18:44:28 +05:30
Venky Shankar
40cb741902 Merge PR into main
* refs/pull/53734/head:
	qa: refactor client upgrade yamls and other minor touchups
	qa/upgrade/nofs: upgrade pacific->reef
	qa/upgrade/upgraded_client: upgrade nautilus->pacific and pacific->reef

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2024-01-29 18:42:36 +05:30
Venky Shankar
52b20eb584 Merge PR into main
* refs/pull/55309/head:
	qa: `fs volume rename` requires `fs fail` and `refuse_client_session` set

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
2024-01-29 18:41:23 +05:30
Venky Shankar
1bf8291576 Merge PR into main
* refs/pull/54451/head:
	common/admin_socket: improvements to the RaiseHook

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2024-01-29 18:20:35 +05:30
Ronen Friedman
6f6a7376f1
Merge pull request from ronen-fr/wip-rf-reserver
osd/scrub: partial implementation of scrub reserver
Reviewed-by: Samuel Just <sjust@redhat.com>-
2024-01-29 12:43:09 +02:00
Matan Breizman
f63c485e60
Merge pull request from Matan-B/wip-matanb-crimson-pg-map-logs
crimson/osd/pg_map: Refactor

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2024-01-29 11:45:28 +02:00
Pedro Gonzalez Gomez
93c567e8ae
Merge pull request from afreen23/fix-64150
mgr/dashboard: Fix issue with Object's Overview page

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2024-01-29 08:56:55 +01:00
Nizamudeen A
c5367e9c4e
Merge pull request from ceph/dependabot-npm_and_yarn-src-pybind-mgr-dashboard-frontend-babel-traverse-7.23.2
mgr/dashboard: bump @babel/traverse from 7.23.0 to 7.23.2 in /src/pybind/mgr/dashboard/frontend

Reviewed-by: Nizamudeen A <nia@redhat.com>
2024-01-29 12:36:38 +05:30
Laura Flores
f45a39bdcd
Merge pull request from ljflores/wip-read-balancer-mgr-module 2024-01-28 23:40:01 -06:00
Nizamudeen A
2218c356a6
Merge pull request from afreen23/fix-55270
mgr/dashboard:  Code refactor rgw migrate component for using correctly the MIGRATE action verb

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2024-01-29 10:14:27 +05:30
zdover23
12febc9971
Merge pull request from zdover23/wip-doc-2024-01-29-architecture-rados
doc/architecture.rst: improve rados definition

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2024-01-29 13:33:57 +10:00
Laura Flores
0464512a17
Merge pull request from ljflores/wip-tox-mgr 2024-01-28 21:18:30 -06:00
Laura Flores
5554e565ca mgr: pin pytest to version 7.4.4
On 2024-01-27, pytest updated to 8.0.0,
which broke run-tox-mgr.

https://docs.pytest.org/en/stable/changelog.html

==================================== ERRORS ====================================
_____________________ ERROR collecting alerts/__init__.py ______________________
alerts/__init__.py:2: in <module>
    from .module import Alerts
alerts/module.py:6: in <module>
    from mgr_module import CLIReadCommand, HandleCommandResult, MgrModule, Option
mgr_module.py:1: in <module>
    import ceph_module  # noqa
E   ModuleNotFoundError: No module named 'ceph_module'
______________________ ERROR collecting alerts/module.py _______________________
alerts/module.py:6: in <module>
    from mgr_module import CLIReadCommand, HandleCommandResult, MgrModule, Option
mgr_module.py:1: in <module>
    import ceph_module  # noqa
E   ModuleNotFoundError: No module named 'ceph_module'
____________________ ERROR collecting balancer/__init__.py _____________________
balancer/__init__.py:2: in <module>
    from .module import Module
balancer/module.py:12: in <module>
    from mgr_module import CLIReadCommand, CLICommand, CommandResult, MgrModule, Option, OSDMap, CephReleases
mgr_module.py:1: in <module>
    import ceph_module  # noqa
E   ModuleNotFoundError: No module named 'ceph_module'
_____________________ ERROR collecting balancer/module.py ______________________
balancer/module.py:12: in <module>
    from mgr_module import CLIReadCommand, CLICommand, CommandResult, MgrModule, Option, OSDMap, CephReleases
mgr_module.py:1: in <module>
    import ceph_module  # noqa
E   ModuleNotFoundError: No module named 'ceph_module'

Fixes: https://tracker.ceph.com/issues/64200
Signed-off-by: Laura Flores <lflores@ibm.com>
2024-01-29 01:02:04 +00:00
Zac Dover
763f6b7a22 doc/architecture.rst: improve rados definition
Improve the definition of RADOS, and link to information about RADOS.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2024-01-29 05:33:58 +10:00
Laura Flores
cb10c0d821 osd: clear out unneeded pending pg-upmap-primary mappings
If the score did not improve, we should clear out any
pending pg-upmap-primary mappings so they don't execute
in situations where the same incremental is used to balance
multiple pools (i.e. in the balancer mgr module).

Signed-off-by: Laura Flores <lflores@ibm.com>
2024-01-28 13:15:49 -06:00
Laura Flores
33b5f3ddbb qa/suites/rados/thrash/d-balancer: add read and upmap-read modes to balancer testing
Here, we turn the balancer mgr module to read and upmap read mode
during various thrashing scenarios.

Signed-off-by: Laura Flores <lflores@ibm.com>
2024-01-28 13:15:49 -06:00
Laura Flores
f3c74a7be4 mgr/balancer: add pg_upmap_primaries to balancer status detail
Followup to 8a5553597c.
Streamlines some of the logic so pg upmap activity is properly
initalized, and updated in offline mode as well as online.

Signed-off-by: Laura Flores <lflores@ibm.com>
2024-01-28 13:15:49 -06:00
Laura Flores
61e721c9f1 PendingReleaseNotes: add note about read balancer mgr module integration
Signed-off-by: Laura Flores <lflores@ibm.com>
2024-01-28 13:15:49 -06:00
Laura Flores
e2ce8ed1ff mgr: add read balancer support inside the balancer module
Read balancing may now be managed automatically via the balancer
manager module. Users may choose between two new modes: ``upmap-read``, which
offers upmap and read optimization simultaneously, or ``read``, which may be used
to only optimize reads. Existing balancer commands have also been added to
contain more information about read balancing.

Run the following commands to test the new automatic behavior:
`ceph balancer on` (on by default)
`ceph balancer mode <read|upmap-read>`
`ceph balancer status`

Run the following commands to test the new supervised behavior:
`ceph balancer off`
`ceph balancer mode <read|upmap-read>`
`ceph balancer eval` | `ceph balancer eval <pool-name>`
`ceph balancer eval-verbose` | `ceph balancer eval-verbose <pool-name>`
`ceph balancer optimize <plan-name>`
`ceph balancer show <plan-name>`
`ceph balancer eval <plan-name>`
`ceph balancer execute <plan-name>`

In the balancer module, there is also a new "self_test" function which tests
the module's basic functionality. This test can be triggered with the following
commands:
`ceph mgr module enable selftest`
`ceph mgr self-test module balancer`

Related Trello: https://trello.com/c/sWoKctzL/859-add-read-balancer-support-inside-the-balancer-module
Signed-off-by: Laura Flores <lflores@ibm.com>
2024-01-28 13:15:38 -06:00
Ronen Friedman
5970ff6637 osd/scrub: add required sub-states to handle queued reservation requests
The scrub async reserver is not yet used. All requests are treated as
'legacy' requests, i.e. requests that expect an immediate grant/deny
reply.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2024-01-28 09:40:02 -06:00
Ronen Friedman
c61bca6d6b osd/scrub: add "queue my request" flag to replica reservation messages
Up-to-date primaries will set this flag when sending a reservation
request. The replica OSD, if too busy to handle the request immediately, will queue
it until such time that the number of concurrent reservations is below the
configured limit. The queued requests are honored in FIFO order.

Old primaries will not set this flag, and will receive the expected
grant or deny reply immediately.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2024-01-28 09:40:02 -06:00
Ronen Friedman
c6c05ab639 osd/scrub: add synchronous request to AsyncReserver API
To be used when handling replica reservation requests from "old"
primaries, that expect an immediate grant/deny reply.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2024-01-28 09:40:02 -06:00
Matan Breizman
7eb9e33f53
Merge pull request from Matan-B/wip-matanb-crimson-cyanstore-rmcoll
crimson/os/cyanstore: support OP_RMCOLL

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: chunmei-liu <chunmei.liu@intel.com>
2024-01-28 11:22:39 +02:00
zdover23
d55f4b4a8d
Merge pull request from zdover23/wip-doc-2024-01-27-radosgw-index-verb-disagreement
doc/radosgw: fix verb disagreement - index.html

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2024-01-28 18:17:52 +10:00
Zac Dover
9f271093f4 doc/radosgw: fix verb disagreement - index.html
Fix a tricky verb disagreement and rewrite a few sentences for what I
hope is greater clarity.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2024-01-28 18:04:57 +10:00
Guillaume Abrioux
327ec975e9
Merge pull request from guits/dmcrypt-optim
ceph-volume: use 'no workqueue' options with dmcrypt
2024-01-27 12:27:42 +01:00
Guillaume Abrioux
f72100bbd1 ceph-volume: fix partitions support in disk.get_devices()
The following:
```
is_part = get_file_contents(os.path.join(_sys_dev_block_path, item, 'partition')) == "1"
```
assumes any `/sys/dev/block/x:y/partition` contains '1' which is wrong.
This file actually contains the corresponding partition number.

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
2024-01-27 01:29:19 +01:00
Laura Flores
906bf69521
Merge pull request from ceph/dependabot-github_actions-gregsdennis-dependencies-action-1.3.2
.github: Bump gregsdennis/dependencies-action from 1.2.3 to 1.3.2
2024-01-26 16:46:36 -06:00
Laura Flores
15bd38eece mgr: add CephReleases class to sustainably compare releases
Changes how the upmap balancer compares min_mon_release
to account for release names eventually wrapping around the alphabet.

Signed-off-by: Laura Flores <lflores@ibm.com>
2024-01-26 22:41:23 +00:00
Laura Flores
702cb64e87
Merge pull request from ceph/revert-55096-sjust/for-review/wip-crush-msr
Revert "crush: add multistep retry rules"
2024-01-26 16:15:46 -06:00
Guillaume Abrioux
0985e20134 ceph-volume: use 'no workqueue' options with dmcrypt
CloudFlare engineers made some testing and realized that using
workqueues with encryption on flash devices has a bad effect.

See [1] for details.

With this patch it will make ceph-volume call crypsetup with
`--perf-no_read_workqueue` and `--perf-no_write_workqueue` options
when the device is not a rotational.

[1] https://blog.cloudflare.com/speeding-up-linux-disk-encryption/

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
Co-Authored-by: Stefan Kooman <stefan@kooman.org>
Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
2024-01-26 22:05:30 +01:00
Samuel Just
a5ce9c3863 Revert "crush: add multistep retry rules"
This PR was merged by accident before it was ready.
Let's revert for now and open a new PR.

Signed-off-by: Samuel Just <sjust@redhat.com>
2024-01-26 20:32:05 +00:00
Yuri Weinstein
37d5d931b0
Merge pull request from athanatos/sjust/for-review/wip-crush-msr
crush: add multistep retry rules

Reviewed-by: Laura Flores <lflores@redhat.com>
2024-01-26 11:57:53 -08:00
Casey Bodley
1112689da4
Merge pull request from cbodley/wip-63130-debug
cmake/arrow: don't treat warnings as errors

Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
2024-01-26 14:18:48 +00:00
Ilya Dryomov
2b11aa38ea
Merge pull request from ajarr/wip-64063
rbd-nbd: use netlink interface by default

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2024-01-26 12:37:52 +01:00
Kefu Chai
d813ce1923
Merge pull request from zhscn/fix-ambiguous-error
common: fix ambiguous error when using gcc 13

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2024-01-26 14:42:06 +08:00
Yingxin
3e190e5614
Merge pull request from cyx1231st/wip-crimson-save-conn-foreign-copy
crimson/osd: drop a foreign-copy to shard-0 for every pg operation

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2024-01-26 13:47:37 +08:00
Casey Bodley
ecb4eb14e5
Merge pull request from adamemerson/wip-rgw-surface-neorados
rgw: Surface neorados

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2024-01-26 02:43:44 +00:00
dependabot[bot]
5ae5925a92
.github: Bump gregsdennis/dependencies-action from 1.2.3 to 1.3.2
Bumps [gregsdennis/dependencies-action](https://github.com/gregsdennis/dependencies-action) from 1.2.3 to 1.3.2.
- [Release notes](https://github.com/gregsdennis/dependencies-action/releases)
- [Commits](80b5ffec56...f98d55eee1)

---
updated-dependencies:
- dependency-name: gregsdennis/dependencies-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-25 23:57:13 +00:00
Laura Flores
fc4ff1796e
Merge pull request from ljflores/wip-dependabot 2024-01-25 17:56:29 -06:00
zdover23
77fbe9ead3
Merge pull request from zdover23/wip-doc-2024-01-25-radosgw-admin-usage
doc/radosgw: edit "Usage" admin.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2024-01-26 09:24:13 +10:00
Zac Dover
d8df6f61e8 doc/radosgw: edit "Usage" admin.rst
Edit "Usage" in doc/radosgw/admin.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2024-01-26 09:12:59 +10:00
Casey Bodley
93d158711e
Merge pull request from cbodley/wip-moncommand-dencoder
mon: zero-initialize MonCommand::flags

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2024-01-25 17:12:07 +00:00
Ramana Raja
fcbf7367d2 rbd-nbd: map using netlink interface by default
Mapping rbd images to nbd devices using ioctl interface is not
robust. It was discovered that the device size or the md5 checksum
of the nbd device was incorrect immediately after mapping using
ioctl method. When using the nbd netlink interface to map RBD images
the issue was not encountered. Switch to using nbd netlink interface
for mapping.

Fixes: https://tracker.ceph.com/issues/64063
Signed-off-by: Ramana Raja <rraja@redhat.com>
2024-01-25 11:00:59 -05:00
Matan Breizman
6a130a7007 crimson/os/cyanstore: support OP_RMCOLL
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2024-01-25 15:33:45 +00:00
Casey Bodley
12b7352047 mon: zero-initialize MonCommand::flags
causing failures in check-generated.sh:

2 MonCommand
/tmp/typ-qmnDZ1ahR /tmp/typ-lIgJTZiUe differ: byte 100, line 6
**** MonCommand test 1 dump_json check failed ****
   ceph-dencoder type MonCommand select_test 1 dump_json > /tmp/typ-qmnDZ1ahR
   ceph-dencoder type MonCommand select_test 1 encode decode dump_json > /tmp/typ-lIgJTZiUe
6c6
<     "flags": 94661933599904
---
>     "flags": 94342467308704

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2024-01-25 09:45:10 -05:00
Ilya Dryomov
f7b52fc712
Merge pull request from ajarr/wip-64139
rbd-nbd: fix resize of images mapped using netlink

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2024-01-25 13:04:26 +01:00