Commit Graph

142948 Commits

Author SHA1 Message Date
Kefu Chai
e74ec0b27d debian/cephadm.postinst: stop using adduser --gecos
--gecos option of adduser is deprecated in debian/bookworm, and
will be removed in debian/trixie,
see https://manpages.debian.org/bookworm/adduser/adduser.8.en.html.
so to be future-proof, let's switch to `usermod --comment`. please
note, since we still need to support ubuntu/jammy which is used in
our CI, and `adduser` shipped by ubuntu/jammy does not support
`--comment` yet, so we cannot use this option.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2024-02-03 22:36:50 +08:00
Kefu Chai
013eebf535 debian/cephadm.postinst: replace tab with spaces
for better readability, and to be more consistent with the rest
of this file, and other .postinst scripts of this project.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2024-02-03 22:36:50 +08:00
Kefu Chai
90bc036924 debian/cephadm.postinst: specify --home when adduser
quote from adduser/NEWS.Debian.gz:

>  System user home defaults to /nonexistent if --home is not specified.
>  Packages that call adduser to create system accounts should explicitly
>  specify a location for /home (see Lintian check
>  maintainer-script-lacks-home-in-adduser).

so let's follow this change in adduser. otherwise "cephadm"
would have a $HOME at `/nonexistent`.

Fixes: https://tracker.ceph.com/issues/64069
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2024-02-03 22:36:50 +08:00
Kefu Chai
c517d0b0e9 debian/cephadm.postinst: wrap command line lines which is too long
for better readability.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2024-02-03 22:36:50 +08:00
Kefu Chai
a1a857698c debian/ceph-common.postinst: set user directory using adduser
now that adduser allows us to set its home directory, we can do
this using adduser instead of using usermod. this change also
silences the warning from lintian
"maintainer-script-lacks-home-in-adduser". lintian complains if
`adduser --system` is called without passing `--home` option.

also, take this opportunity to s/-c/--comment/ in the command line
of `usermod`, for better readability.

Fixes: https://tracker.ceph.com/issues/64069
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2024-02-03 22:36:50 +08:00
Kefu Chai
2a30ddccd0 debian/control: add adduser to Depends of cephadm and ceph-common
in `debian/ceph-common.postinst` and `debian/cephadm.postinst`, we
use `adduser --system` to create the system user when configuring
the corresponding package.

before this change, the dependency is not listed in the runtime
`Depends` section of ceph-common and cephadm.

in this change, the dependency is added. this is also suggested
by Securing Debian Manual, see
https://www.debian.org/doc/manuals/securing-debian-manual/bpp-lower-privs.en.html

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2024-02-03 22:36:50 +08:00
Adam King
4103b566e4
Merge pull request #54312 from dvanders/dvanders_mcltf_true
qa/tasks/cephadm: enable mon_cluster_log_to_file

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2024-02-02 13:57:49 -05:00
Casey Bodley
ebd79592bd
Merge pull request #54837 from imtzw/urlencode
rgw: fix list obj v1 nextmarker

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2024-02-02 16:37:53 +00:00
Casey Bodley
862b2eeb5b
Merge pull request #54867 from BBoozmen/oozmen_lc_subsystem
rgw/common: add rgw lifecycle specific debug log subsystem

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Jiffin Tony Thottan <jthottan@redhat.com>
2024-02-02 15:42:00 +00:00
Casey Bodley
0359613a9f
Merge pull request #55162 from cfsnyder/wip-64014-cfsnyder-pacific
rgw: fix issue with concurrent versioned deletes leaving behind olh entries

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
2024-02-02 15:41:17 +00:00
Casey Bodley
868a8eb2d1
Merge pull request #55229 from tobias-urdin/http-error-keystone-issue-admin-token-request
rgw: fix http error check in keystone issue_admin_token_request

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2024-02-02 15:40:48 +00:00
baum
3cb0412e26
Merge pull request #55330 from baum/centos9-install-build-deps
🧹build dependencies: centos9
2024-02-02 09:54:28 +02:00
Venky Shankar
57856522a6 Merge PR #55233 into main
* refs/pull/55233/head:
	qa: use centos 9.stream for cephfs stock kernel testing

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
2024-02-02 10:35:00 +05:30
Casey Bodley
3a05c6f0f3
Merge pull request #53901 from cbodley/wip-qa-container-distros-s
qa/distros: bump container host distros from centos 8->9

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2024-02-01 19:40:57 +00:00
Ilya Dryomov
9102ec9bad
Merge pull request #55413 from petrutlucian94/flipped_tid
msg: update MOSDOp() to use ceph_tid_t instead of long

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2024-02-01 20:29:58 +01:00
Lucian Petrut
0bab1e6b69 msg: update MOSDOp() to use ceph_tid_t instead of long
The MOSDOp constructor receives the the transaction ID as a long
instead of ceph_tid_t.

The issue is that "long" uses 32b on Windows instead of 64 bits,
so it flips after about 2 billion requests. At that point, the OSD
replies are dropped because of transaction ID mismatches.

We'll solve the issue by using the correct type for the transaction
id, specifically ceph_tid_t.

Fixes: https://tracker.ceph.com/issues/64291

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2024-02-01 15:01:36 +00:00
Ivo Almeida
d0ff37be70
Merge pull request #55373 from ivoalmeida/snapshot-schedule-form-directory-typeahead
Snapshot schedule form directory typeahead

Reviewed-by: Nizamudeen A <nia@redhat.com>
2024-02-01 10:48:17 -03:00
Redouane Kachach
0a96381f32
Merge pull request #55393 from rkachach/fix_issue_64266
mgr/rook: adding deployment to ceph image built for rook e2e testing
2024-02-01 12:35:39 +01:00
Redouane Kachach
fb7221749b
mgr/rook: adding deployment to ceph image built for rook e2e testing
Fixes: https://tracker.ceph.com/issues/64266

Signed-off-by: Redouane Kachach <rkachach@redhat.com>
2024-02-01 09:39:24 +01:00
Casey Bodley
8e554bf639
Merge pull request #47723 from mdw-at-linuxbox/wip-master-tempurl
rgw: swift: tempurl fixes for ceph

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2024-01-31 22:15:33 +00:00
Anthony D'Atri
6760aa18bf
Merge pull request #55403 from cbodley/wip-doc-rgw-topic-attrs
doc/rgw: fix Attributes index in CreateTopic example
2024-01-31 15:08:26 -05:00
Adam King
a4452f6cc4
Merge pull request #54742 from guits/node-proxy
orch: implement hardware monitoring

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
2024-01-31 13:28:50 -05:00
Ilya Dryomov
5952230d90
Merge pull request #55397 from jagombar/fixInvalidSnapId2
librbd: return ENOENT from Snapshot::get_timestamp for nonexistent snap_id

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2024-01-31 18:32:54 +01:00
Casey Bodley
ecbd837667 doc/rgw: fix Attributes index in CreateTopic example
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2024-01-31 12:31:02 -05:00
Ivo Almeida
61222e11ab mgr/dashboard: added dir search to snap schdl form
Fixes: https://tracker.ceph.com/issues/64246
Signed-off-by: Ivo Almeida <ialmeida@redhat.com>
2024-01-31 16:35:47 +00:00
Nizamudeen A
c8337c64b7
Merge pull request #54781 from rhcs-dashboard/nvmeof-integration
mgr/dashboard: implement APIs for nvmeof management

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2024-01-31 22:00:59 +05:30
Nizamudeen A
95455490bc
Merge pull request #54900 from ivoalmeida/snapshot-schedule-create
added snap schedule form

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2024-01-31 21:27:22 +05:30
Ronen Friedman
52a1268a05
Merge pull request #55340 from ronen-fr/wip-rf-reserver2
osd/scrub: use an AsyncReserver to handle scrub reservations on the replica side

Reviewed-by: Samuel Just <sjust@redhat.com>-
2024-01-31 17:56:43 +02:00
Josh Salomon
c43054f2f9
Merge pull request #55198 from JoshSalomon/balancer-perf
osd: Read balancer for devices with different sizes (performance)
2024-01-31 17:26:48 +02:00
Guillaume Abrioux
b7c0a6a5b0 doc: add node-proxy documentation
This commit adds some documentation about the
'hardware inventory / monitoring' feature (node-proxy agent).

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
2024-01-31 15:23:44 +01:00
Anthony D'Atri
d33af7d4eb
Merge pull request #55396 from zdover23/wip-doc-2024-01-31-arch-monitors
doc/architecture: improve some paragraphs
2024-01-31 08:49:55 -05:00
Ali Maredia
b3e0b6686e
Merge pull request #55383 from ceph/cpp-redis-cxx-standard
cpp_redis: update to a version with the CMAKE_CXX_STANDARD fixed

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2024-01-31 08:36:42 -05:00
jagombar
2a2d3d4c6b librbd: return ENOENT from Snapshot::get_timestamp for nonexistent snap_id
Fixes: https://tracker.ceph.com/issues/47287

Signed-off-by: John Agombar <agombar@uk.ibm.com>
2024-01-31 11:53:27 +00:00
Zac Dover
2f0542d669 doc/architecture: improve some paragraphs
Improve paragraphs under the heading "The Ceph Storage Cluster". Remove
a sentence that was pleonastic in its context in the paragraph.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2024-01-31 20:58:45 +10:00
Ivo Almeida
d7c9691623 mgr/dashboard: added snap schedule form
Fixes: https://tracker.ceph.com/issues/63827
Signed-off-by: Ivo Almeida <ialmeida@redhat.com>
2024-01-31 09:39:31 +00:00
Matan Breizman
46f6940c20
Merge pull request #55305 from Matan-B/wip-matanb-crimson-do_delete_work
crimson/osd/pg: do_delete_work use claim_and_reset

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>
2024-01-31 11:14:02 +02:00
Leonid Usov
7aeac65dc4 cpp_redis: update to a version with the CMAKE_CXX_STANDARD fixed
This updates the cpp_redis submodule to a version
which contains https://github.com/ceph/cpp_redis/pull/5

Signed-off-by: Leonid Usov <leonid.usov@ibm.com>
2024-01-31 09:50:37 +02:00
Ronen Friedman
6fa0fa3153 test: fix test_scrub_sched following recent scrub changes
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2024-01-31 01:29:19 -06:00
Ronen Friedman
cf9ed68c81 osd/scrub: restore 'dump_scrub_reservations' functionality
As ScrubResources is no longer involved in remote reservations, some
of the data listed by 'dump_scrub_reservations' is now collected by
OsdScrub itself (prior to this change, OsdScrub just forwarded the
request to ScrubResources).

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2024-01-31 01:29:19 -06:00
Ronen Friedman
46e18b9a2d osd/scrub: control reservation queueing using a config option
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2024-01-31 01:29:19 -06:00
Ronen Friedman
8516c0ebc5 osd/scrub: directly manage remote reservations in the FSM
The FSM now interacts with the scrub_reserver directly.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2024-01-31 01:29:19 -06:00
Nizamudeen A
538f94cf16 vstart: add nvmeof_gw to the vstart script
so that it can be later used by the dashboard to configure the nvmeof
through UI

and create rbd pool in UI

Fixes: https://tracker.ceph.com/issues/64201
Signed-off-by: Nizamudeen A <nia@redhat.com>
2024-01-31 12:46:11 +05:30
Laura Flores
60a15fc77f
Merge pull request #55379 from ljflores/wip-revert-labler-changes 2024-01-30 21:43:35 -06:00
Laura Flores
00ee28b427 tools: add 'osd-size-aware' mode to osdmaptool
Sample command:
`osdmaptool om --read out.txt --read-pool cephfs.a.data --osd-size-aware`

Signed-off-by: Laura Flores <lflores@ibm.com>
2024-01-31 00:54:28 +00:00
Laura Flores
7386d0a9d1 mgr/balancer: add new keys to balancer mgr module
Signed-off-by: Laura Flores <lflores@ibm.com>
2024-01-31 00:54:21 +00:00
zdover23
c746748162
Merge pull request #55372 from zdover23/wip-doc-2024-01-30-architecture-typo
doc/architecture.rst - fix typo

Reviewed-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>
2024-01-31 10:48:35 +10:00
Josh Salomon
6d27bbf77e osd: unit tests for read balance osd size aware score
Signed-off-by: Josh Salomon <41079547+JoshSalomon@users.noreply.github.com>
2024-01-30 19:24:43 +00:00
Josh Salomon
c1cb08689f osd: Add score for read balance osd size aware policy
This score works for pools in which the read_ratio
value is set.
Current limitations:
- This mechanism ignores osd read affinty
  - There is a plan adding support for read affinity 0
    in the next version.
- This mechanism works only when all PGs are full
- If read_ration is not set - the existing mechanism (named
  fair score) is used.

Signed-off-by: Josh Salomon <41079547+JoshSalomon@users.noreply.github.com>

Signed-off-by: Josh Salomon <41079547+JoshSalomon@users.noreply.github.com>
2024-01-30 19:24:43 +00:00
Josh Salomon
b13ffd2809 osd: Unit tests for read balancer on OSDs with different sizes
Test cases for the read balancer which takes osd sizes into account.
Some balancing code refactoring and reorg for code that is used in
multiple tests

Signed-off-by: Josh Salomon <41079547+JoshSalomon@users.noreply.github.com>
2024-01-30 19:24:43 +00:00
Josh Salomon
2953db8b58 osd: Read balancer for OSDs with different sizes
This commit adds calculation for desired primary distribution which
takes into account the osd size. This way smaller OSDs can take more
read operations (by adding more primaries) and the larger OSDs take less
primaries and the load of the cluater can increase. (This feature offset
a bit the weakest link in the chain effect under some conditions). In
order to calculate the loads correctly there is a need to know the
read/write ratio for the pool, and this commit assumes the read_ratio
parameter is available for the pool.

Signed-off-by: Josh Salomon <41079547+JoshSalomon@users.noreply.github.com>
2024-01-30 19:24:43 +00:00