Commit Graph

142912 Commits

Author SHA1 Message Date
Xuehan Xu
2226e0c942 crimson/os/seastore: add new interfaces to alloc multiple extents for a
single request

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2024-02-05 16:27:50 +08:00
Xuehan Xu
cde33fcae7 crimson/os/seastore/avlallocator: add the interface that allocate
multiple regions

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2024-02-01 15:00:09 +08:00
Casey Bodley
8e554bf639
Merge pull request #47723 from mdw-at-linuxbox/wip-master-tempurl
rgw: swift: tempurl fixes for ceph

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2024-01-31 22:15:33 +00:00
Anthony D'Atri
6760aa18bf
Merge pull request #55403 from cbodley/wip-doc-rgw-topic-attrs
doc/rgw: fix Attributes index in CreateTopic example
2024-01-31 15:08:26 -05:00
Adam King
a4452f6cc4
Merge pull request #54742 from guits/node-proxy
orch: implement hardware monitoring

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
2024-01-31 13:28:50 -05:00
Ilya Dryomov
5952230d90
Merge pull request #55397 from jagombar/fixInvalidSnapId2
librbd: return ENOENT from Snapshot::get_timestamp for nonexistent snap_id

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2024-01-31 18:32:54 +01:00
Casey Bodley
ecbd837667 doc/rgw: fix Attributes index in CreateTopic example
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2024-01-31 12:31:02 -05:00
Nizamudeen A
c8337c64b7
Merge pull request #54781 from rhcs-dashboard/nvmeof-integration
mgr/dashboard: implement APIs for nvmeof management

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2024-01-31 22:00:59 +05:30
Nizamudeen A
95455490bc
Merge pull request #54900 from ivoalmeida/snapshot-schedule-create
added snap schedule form

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2024-01-31 21:27:22 +05:30
Ronen Friedman
52a1268a05
Merge pull request #55340 from ronen-fr/wip-rf-reserver2
osd/scrub: use an AsyncReserver to handle scrub reservations on the replica side

Reviewed-by: Samuel Just <sjust@redhat.com>-
2024-01-31 17:56:43 +02:00
Josh Salomon
c43054f2f9
Merge pull request #55198 from JoshSalomon/balancer-perf
osd: Read balancer for devices with different sizes (performance)
2024-01-31 17:26:48 +02:00
Guillaume Abrioux
b7c0a6a5b0 doc: add node-proxy documentation
This commit adds some documentation about the
'hardware inventory / monitoring' feature (node-proxy agent).

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
2024-01-31 15:23:44 +01:00
Anthony D'Atri
d33af7d4eb
Merge pull request #55396 from zdover23/wip-doc-2024-01-31-arch-monitors
doc/architecture: improve some paragraphs
2024-01-31 08:49:55 -05:00
Ali Maredia
b3e0b6686e
Merge pull request #55383 from ceph/cpp-redis-cxx-standard
cpp_redis: update to a version with the CMAKE_CXX_STANDARD fixed

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2024-01-31 08:36:42 -05:00
jagombar
2a2d3d4c6b librbd: return ENOENT from Snapshot::get_timestamp for nonexistent snap_id
Fixes: https://tracker.ceph.com/issues/47287

Signed-off-by: John Agombar <agombar@uk.ibm.com>
2024-01-31 11:53:27 +00:00
Zac Dover
2f0542d669 doc/architecture: improve some paragraphs
Improve paragraphs under the heading "The Ceph Storage Cluster". Remove
a sentence that was pleonastic in its context in the paragraph.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2024-01-31 20:58:45 +10:00
Ivo Almeida
d7c9691623 mgr/dashboard: added snap schedule form
Fixes: https://tracker.ceph.com/issues/63827
Signed-off-by: Ivo Almeida <ialmeida@redhat.com>
2024-01-31 09:39:31 +00:00
Matan Breizman
46f6940c20
Merge pull request #55305 from Matan-B/wip-matanb-crimson-do_delete_work
crimson/osd/pg: do_delete_work use claim_and_reset

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>
2024-01-31 11:14:02 +02:00
Leonid Usov
7aeac65dc4 cpp_redis: update to a version with the CMAKE_CXX_STANDARD fixed
This updates the cpp_redis submodule to a version
which contains https://github.com/ceph/cpp_redis/pull/5

Signed-off-by: Leonid Usov <leonid.usov@ibm.com>
2024-01-31 09:50:37 +02:00
Ronen Friedman
6fa0fa3153 test: fix test_scrub_sched following recent scrub changes
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2024-01-31 01:29:19 -06:00
Ronen Friedman
cf9ed68c81 osd/scrub: restore 'dump_scrub_reservations' functionality
As ScrubResources is no longer involved in remote reservations, some
of the data listed by 'dump_scrub_reservations' is now collected by
OsdScrub itself (prior to this change, OsdScrub just forwarded the
request to ScrubResources).

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2024-01-31 01:29:19 -06:00
Ronen Friedman
46e18b9a2d osd/scrub: control reservation queueing using a config option
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2024-01-31 01:29:19 -06:00
Ronen Friedman
8516c0ebc5 osd/scrub: directly manage remote reservations in the FSM
The FSM now interacts with the scrub_reserver directly.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2024-01-31 01:29:19 -06:00
Nizamudeen A
538f94cf16 vstart: add nvmeof_gw to the vstart script
so that it can be later used by the dashboard to configure the nvmeof
through UI

and create rbd pool in UI

Fixes: https://tracker.ceph.com/issues/64201
Signed-off-by: Nizamudeen A <nia@redhat.com>
2024-01-31 12:46:11 +05:30
Laura Flores
60a15fc77f
Merge pull request #55379 from ljflores/wip-revert-labler-changes 2024-01-30 21:43:35 -06:00
Laura Flores
00ee28b427 tools: add 'osd-size-aware' mode to osdmaptool
Sample command:
`osdmaptool om --read out.txt --read-pool cephfs.a.data --osd-size-aware`

Signed-off-by: Laura Flores <lflores@ibm.com>
2024-01-31 00:54:28 +00:00
Laura Flores
7386d0a9d1 mgr/balancer: add new keys to balancer mgr module
Signed-off-by: Laura Flores <lflores@ibm.com>
2024-01-31 00:54:21 +00:00
zdover23
c746748162
Merge pull request #55372 from zdover23/wip-doc-2024-01-30-architecture-typo
doc/architecture.rst - fix typo

Reviewed-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>
2024-01-31 10:48:35 +10:00
Josh Salomon
6d27bbf77e osd: unit tests for read balance osd size aware score
Signed-off-by: Josh Salomon <41079547+JoshSalomon@users.noreply.github.com>
2024-01-30 19:24:43 +00:00
Josh Salomon
c1cb08689f osd: Add score for read balance osd size aware policy
This score works for pools in which the read_ratio
value is set.
Current limitations:
- This mechanism ignores osd read affinty
  - There is a plan adding support for read affinity 0
    in the next version.
- This mechanism works only when all PGs are full
- If read_ration is not set - the existing mechanism (named
  fair score) is used.

Signed-off-by: Josh Salomon <41079547+JoshSalomon@users.noreply.github.com>

Signed-off-by: Josh Salomon <41079547+JoshSalomon@users.noreply.github.com>
2024-01-30 19:24:43 +00:00
Josh Salomon
b13ffd2809 osd: Unit tests for read balancer on OSDs with different sizes
Test cases for the read balancer which takes osd sizes into account.
Some balancing code refactoring and reorg for code that is used in
multiple tests

Signed-off-by: Josh Salomon <41079547+JoshSalomon@users.noreply.github.com>
2024-01-30 19:24:43 +00:00
Josh Salomon
2953db8b58 osd: Read balancer for OSDs with different sizes
This commit adds calculation for desired primary distribution which
takes into account the osd size. This way smaller OSDs can take more
read operations (by adding more primaries) and the larger OSDs take less
primaries and the load of the cluater can increase. (This feature offset
a bit the weakest link in the chain effect under some conditions). In
order to calculate the loads correctly there is a need to know the
read/write ratio for the pool, and this commit assumes the read_ratio
parameter is available for the pool.

Signed-off-by: Josh Salomon <41079547+JoshSalomon@users.noreply.github.com>
2024-01-30 19:24:43 +00:00
Josh Salomon
8d7eb9f7e0 osd: Add 'read_ratio' pool parameterr
This parameter is used for better read balancing with non identical
devices.
- This parameter is controlled using the commands 'ceph osd pool set/get'
- This parameter is applicable only for replicated pools
- Valid values are integers in the range [0..100] and represent the
  percentage of read IOs out of all IOs in the pool
  - Value of 0 unsets this parameter and the value will be the default
    value (this is the generic behavior of the command 'ceph osd pool
    set'
  - default value can be set by config parameter
    `osd_pool_default_read_ratio`

Signed-off-by: Josh Salomon <41079547+JoshSalomon@users.noreply.github.com>
2024-01-30 19:24:43 +00:00
Laura Flores
ff6c339cfa .github/workflows: revert version bump to labler action
Signed-off-by: Laura Flores <lflores@ibm.com>
2024-01-30 11:34:22 -06:00
Laura Flores
faef8d54d8
Merge pull request #55364 from ljflores/wip-pr-triage
.github/workflows: disable sync-labels in pr-triage
2024-01-30 11:13:30 -06:00
Guillaume Abrioux
9a949f1ad7 agent/node-proxy: fix wrong host name used in data endpoint
data['cephx']['name'] will return something like:

node-proxy.hostname123

the prefix "node-proxy." has the be removed otherwise there will be
a mismatch between what is actually expected.

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
2024-01-30 17:00:26 +00:00
Guillaume Abrioux
8ebd93da1b mgr/cephadm: update node-proxy unit tests
The recent migration to a separate daemon implied
some changes which have broken these tests.
This commit fixes them.

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
2024-01-30 17:00:26 +00:00
Guillaume Abrioux
51bda540d6 mgr/cephadm: add a new config option 'oob_default_addr'
So there's a default value (169.254.1.1) which is the default
address for the 'OS to iDrac pass-through' interface.
Given that node-proxy will reach the RedFish API through this interface,
we can make users avoid to pass that addr when providing the host spec
at bootstrap time.

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
2024-01-30 17:00:06 +00:00
Adam Kupczyk
1ef493c768
Merge pull request #53610 from Patty8122/documentation-update-fsck-flagdeep
doc/man/8: Add valid options in ceph-bluestore-tool.rst
2024-01-30 15:43:49 +01:00
Venky Shankar
8518c699e7 Merge PR #54515 into main
* refs/pull/54515/head:
	qa: get_global_id() returns a string (global_id)
	qa: correct usage of DEBUGFS_META_DIR in dedent

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
2024-01-30 19:40:35 +05:30
Redouane Kachach
a44d457660
Merge pull request #55313 from rkachach/fix_issue_64183
mgr/rook: adding metrics monitoring e2e testing
2024-01-30 14:30:56 +01:00
Zac Dover
6c0417fbcb doc/architecture.rst - fix typo
s/requies/requires

Signed-off-by: Zac Dover <zac.dover@proton.me>
2024-01-30 22:24:11 +10:00
Ivo Almeida
80d6ed3cb9
Merge pull request #55284 from rhcs-dashboard/cephfs-clone-create
mgr/dashboard: create cephfs snapshot clone
2024-01-30 08:30:51 -03:00
Nizamudeen A
35e40c4e8f mgr/dashboard: adapt to new nvmeof APIs
Fixes: https://tracker.ceph.com/issues/64201
Signed-off-by: Nizamudeen A <nia@redhat.com>
2024-01-30 15:39:35 +05:30
avanthakkar
1b768332b1 mgr/dashboard: introduce APIs for NvmeOf management
Fixes: https://tracker.ceph.com/issues/64201
Signed-off-by: avanthakkar <avanjohn@gmail.com>
2024-01-30 15:39:35 +05:30
Nizamudeen A
24015310b1 mgr/dashboard: gRPC client for nvmeof communication
Created a gRPC client by utilising the protobuf file available in nvmeof
repo

Copied the file to this repo and generated its output.

Fixes: https://tracker.ceph.com/issues/64201
Signed-off-by: Nizamudeen A <nia@redhat.com>
2024-01-30 15:39:35 +05:30
Nizamudeen A
a2fae7b0e2 mgr/cephadm: configure the dashboard gateways
cephadm configures the nvmeof gateways and add the gateways to a config
store which dashboard will later on fetch to make the grpc calls.

Fixes: https://tracker.ceph.com/issues/64201
Signed-off-by: Nizamudeen A <nia@redhat.com>
2024-01-30 15:39:35 +05:30
Rongqi Sun
774ce9f98b
Merge pull request #55351 from rosinL/wip-fix-64122
script/run-make: install lvm2 for make check cephadm test
2024-01-30 17:31:52 +08:00
Nizamudeen A
17b2100ffa
Merge pull request #55228 from rhcs-dashboard/fix-64081-main
mgr/dashboard: Export rgw multisite status via the API

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2024-01-30 14:39:02 +05:30
Ronen Friedman
92bbd73c7c osd/scrub: remove remote reservation code from ScrubResources
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2024-01-30 02:56:23 -06:00