Commit Graph

143552 Commits

Author SHA1 Message Date
Xuehan Xu
b35549d714 crimson/os/seastore/object_data_handler: allow alloc multiple extents
for a single request

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2024-02-05 16:27:50 +08:00
Xuehan Xu
cf509b1933 crimson/os/seastore/transaction_manager: fix errorator mismatch
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2024-02-05 16:27:50 +08:00
Xuehan Xu
63b87f70ce test/crimson/seastore: add unittest cases for scattered allocation
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2024-02-05 16:27:50 +08:00
Xuehan Xu
e418cfd6f5 test/crimson/seastore: test avlallocator's scattered allocation
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2024-02-05 16:27:50 +08:00
Xuehan Xu
554db8d88d crimson/os/seastore/transaction_manager: TransactionManager::alloc_extent(s) may return enospc
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2024-02-05 16:27:50 +08:00
Xuehan Xu
2226e0c942 crimson/os/seastore: add new interfaces to alloc multiple extents for a
single request

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2024-02-05 16:27:50 +08:00
Kefu Chai
a3ec5d366b
Merge pull request #55089 from cbodley/wip-boost-mirrors
make-dist: update boost mirrors

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2024-02-05 10:20:07 +08:00
Josh Salomon
67464df045 osd: Fix compilation warnings for PR https://github.com/ceph/ceph/pull/55198
Signed-off-by: Josh Salomon <41079547+JoshSalomon@users.noreply.github.com>

Signed-off-by: Josh Salomon <41079547+JoshSalomon@users.noreply.github.com>
2024-02-04 19:04:07 +02:00
zdover23
3a8d63a892
Merge pull request #55434 from anthonyeleven/rgwyamlin
src/common/options: Correct typo in rgw.yaml.in

Reviewed-by: Zac Dover <zac.dover@proton.me>
2024-02-05 00:44:35 +10:00
Matan Breizman
ad212a5c83
Merge pull request #55276 from Matan-B/wip-matanb-crimson-alienstore-cores
crimson: rework CPU pinning

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2024-02-04 11:45:43 +02:00
Samuel Just
0736d5d0a1 tasks/.../test_erasure_code_profile: assertSubset in test_create_plugin
Newly added profile options may break this test otherwise.

Signed-off-by: Samuel Just <sjust@redhat.com>
2024-02-03 21:00:11 -08:00
Samuel Just
c52aabfb45 test/cli/osdmaptool/crush.t: adjust --import-crush size output
Signed-off-by: Samuel Just <sjust@redhat.com>
2024-02-03 21:00:11 -08:00
Samuel Just
39ad2e7168 test/cli/crushtool/choose-args.t: add msr related json output
Signed-off-by: Samuel Just <sjust@redhat.com>
2024-02-03 21:00:11 -08:00
Samuel Just
e56e1bb536 qa/erasure-code: modify jerasure 4/2 ec test case to use msr
Signed-off-by: Samuel Just <sjust@redhat.com>
2024-02-03 21:00:11 -08:00
Samuel Just
aa88dfac4e doc/rados/operations: add CRUSH MSR documentation
Signed-off-by: Samuel Just <sjust@redhat.com>
2024-02-03 21:00:11 -08:00
Samuel Just
b398c548fd erasure-code: add support for multiple osds in a single failure domain
Adds support for crush-osds-per-failure-domain and
crush-num-failure-domains via MSR rules.

Signed-off-by: Samuel Just <sjust@redhat.com>
2024-02-03 21:00:11 -08:00
Samuel Just
d9f463e650 vstart.sh: add --require-osd-and-client-version and --use-crush-tunables flags
Signed-off-by: Samuel Just <sjust@redhat.com>
2024-02-03 21:00:11 -08:00
Samuel Just
f58d4e8566 vstart.sh: add --osds-per-host
Signed-off-by: Samuel Just <sjust@redhat.com>
2024-02-03 21:00:11 -08:00
Samuel Just
4b4eb17d32 test/crush/crush.cc: add tests specifically for MSR
Signed-off-by: Samuel Just <sjust@redhat.com>
2024-02-03 21:00:11 -08:00
Samuel Just
0445c3347e test/crush/crush.cc: add test variants for firstn rules
Signed-off-by: Samuel Just <sjust@redhat.com>
2024-02-03 21:00:11 -08:00
Samuel Just
28989d08e7 test/crush/crush.cc: convert indep test cases to test MSR as well
Signed-off-by: Samuel Just <sjust@redhat.com>
2024-02-03 21:00:11 -08:00
Samuel Just
ab2b62c120 test/crush/crush.cc: s/NULL/nullptr/g
Signed-off-by: Samuel Just <sjust@redhat.com>
2024-02-03 21:00:11 -08:00
Samuel Just
8fba03fe42 mon/OSDMonitor: generalize rule type check for pools
Add rule_valid_for_pool_type to CrushWrapper to generalize
rule type <-> pool type mapping to include the new MSR
types.

Signed-off-by: Samuel Just <sjust@redhat.com>
2024-02-03 21:00:11 -08:00
Samuel Just
8eb6835396 doc/dev/crush-msr.rst: add developer summary of crush msr
Signed-off-by: Samuel Just <sjust@redhat.com>
2024-02-03 21:00:11 -08:00
Samuel Just
bbfa5bac97 crush/mapper: add support for MSR types
Signed-off-by: Samuel Just <sjust@redhat.com>
2024-02-03 21:00:09 -08:00
zdover23
6847404eda
Merge pull request #55419 from zdover23/wip-doc-2024-02-02-rados-config-pool-pg-config-ref
doc/rados: update config for autoscaler

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2024-02-04 08:25:50 +10:00
Kefu Chai
e74ec0b27d debian/cephadm.postinst: stop using adduser --gecos
--gecos option of adduser is deprecated in debian/bookworm, and
will be removed in debian/trixie,
see https://manpages.debian.org/bookworm/adduser/adduser.8.en.html.
so to be future-proof, let's switch to `usermod --comment`. please
note, since we still need to support ubuntu/jammy which is used in
our CI, and `adduser` shipped by ubuntu/jammy does not support
`--comment` yet, so we cannot use this option.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2024-02-03 22:36:50 +08:00
Kefu Chai
013eebf535 debian/cephadm.postinst: replace tab with spaces
for better readability, and to be more consistent with the rest
of this file, and other .postinst scripts of this project.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2024-02-03 22:36:50 +08:00
Kefu Chai
90bc036924 debian/cephadm.postinst: specify --home when adduser
quote from adduser/NEWS.Debian.gz:

>  System user home defaults to /nonexistent if --home is not specified.
>  Packages that call adduser to create system accounts should explicitly
>  specify a location for /home (see Lintian check
>  maintainer-script-lacks-home-in-adduser).

so let's follow this change in adduser. otherwise "cephadm"
would have a $HOME at `/nonexistent`.

Fixes: https://tracker.ceph.com/issues/64069
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2024-02-03 22:36:50 +08:00
Kefu Chai
c517d0b0e9 debian/cephadm.postinst: wrap command line lines which is too long
for better readability.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2024-02-03 22:36:50 +08:00
Kefu Chai
a1a857698c debian/ceph-common.postinst: set user directory using adduser
now that adduser allows us to set its home directory, we can do
this using adduser instead of using usermod. this change also
silences the warning from lintian
"maintainer-script-lacks-home-in-adduser". lintian complains if
`adduser --system` is called without passing `--home` option.

also, take this opportunity to s/-c/--comment/ in the command line
of `usermod`, for better readability.

Fixes: https://tracker.ceph.com/issues/64069
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2024-02-03 22:36:50 +08:00
Kefu Chai
2a30ddccd0 debian/control: add adduser to Depends of cephadm and ceph-common
in `debian/ceph-common.postinst` and `debian/cephadm.postinst`, we
use `adduser --system` to create the system user when configuring
the corresponding package.

before this change, the dependency is not listed in the runtime
`Depends` section of ceph-common and cephadm.

in this change, the dependency is added. this is also suggested
by Securing Debian Manual, see
https://www.debian.org/doc/manuals/securing-debian-manual/bpp-lower-privs.en.html

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2024-02-03 22:36:50 +08:00
Kotresh HR
27e9a15a53 qa: Fix fs/full suite
Fix fs/full suite. The tests in the suite were failing
in release branches, quincy and pacific because of change
in the rados bluefs pre allocation (PR 51774). The change
affected main/quincy differently than quincy/pacific.

Fixes: https://tracker.ceph.com/issues/63132
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2024-02-03 13:01:37 +05:30
Anthony D'Atri
1796022aa4 src/common/options: Correct typo in rgw.yaml.in
Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
2024-02-03 02:18:25 -05:00
Zac Dover
4dc12092be doc/rados: update config for autoscaler
Update doc/rados/configuration/pool-pg-config-ref.rst to account for the
behavior of autoscaler.

Previously, this file was last meaningfully altered in 2013, prior to
the invention of autoscaler. A recent confusion was brought to my
attention on the Ceph Slack whereby a user attempted to alter the
default values of a Quincy cluster, as suggested in this documentation.
That alteration caused Ceph to throw the error "Error ERANGE: 'pgp_num'
must be greater than 0 and lower or equal than 'pg_num', which in this
case is one" and a related "rgw_init_ioctx ERROR" reading in part
"Numerical result out of range". The user removed the
"osd_pool_default_pgp_num" configuration line from ceph.conf and the
cluster worked as expected. I presume that this is because the removal
of this configuration line allowed autoscaler to work as intended.

Fixes: https://tracker.ceph.com/issues/64259

Co-authored-by: David Orman <ormandj@corenode.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2024-02-03 12:35:19 +10:00
Casey Bodley
1183db4146 rgw: SiteConfig::load() falls back to local zonegroup
allow radosgw-admin commands like 'user create' to operate on a new zone
that hasn't been committed to the period yet. this follows similar logic
in RGWSI_Zone::do_start()

Fixes: https://tracker.ceph.com/issues/64255

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2024-02-02 20:23:39 -05:00
Adam King
4103b566e4
Merge pull request #54312 from dvanders/dvanders_mcltf_true
qa/tasks/cephadm: enable mon_cluster_log_to_file

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2024-02-02 13:57:49 -05:00
Casey Bodley
ebd79592bd
Merge pull request #54837 from imtzw/urlencode
rgw: fix list obj v1 nextmarker

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2024-02-02 16:37:53 +00:00
Casey Bodley
862b2eeb5b
Merge pull request #54867 from BBoozmen/oozmen_lc_subsystem
rgw/common: add rgw lifecycle specific debug log subsystem

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Jiffin Tony Thottan <jthottan@redhat.com>
2024-02-02 15:42:00 +00:00
Casey Bodley
0359613a9f
Merge pull request #55162 from cfsnyder/wip-64014-cfsnyder-pacific
rgw: fix issue with concurrent versioned deletes leaving behind olh entries

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
2024-02-02 15:41:17 +00:00
Casey Bodley
868a8eb2d1
Merge pull request #55229 from tobias-urdin/http-error-keystone-issue-admin-token-request
rgw: fix http error check in keystone issue_admin_token_request

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2024-02-02 15:40:48 +00:00
Redouane Kachach
d45bb4f949
mgr/rook: removing all the code related to OSDs creation/removal
Fixes: https://tracker.ceph.com/issues/64211

Signed-off-by: Redouane Kachach <rkachach@redhat.com>
2024-02-02 13:44:44 +01:00
Dhairya Parmar
e99486edc5 mgr/dashboard: allow sending back error status code fetching clients fails
Fixes: https://tracker.ceph.com/issues/64089
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
(cherry picked from commit 564dba38c4)
2024-02-02 18:46:05 +08:00
Xuehan Xu
ddac5d1cb8 crimson/osd/shard_services: make sure that only up/acting members can
create pgs

Fixes: https://tracker.ceph.com/issues/64282
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2024-02-02 17:02:28 +08:00
baum
3cb0412e26
Merge pull request #55330 from baum/centos9-install-build-deps
🧹build dependencies: centos9
2024-02-02 09:54:28 +02:00
Venky Shankar
57856522a6 Merge PR #55233 into main
* refs/pull/55233/head:
	qa: use centos 9.stream for cephfs stock kernel testing

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
2024-02-02 10:35:00 +05:30
Samuel Just
9db2e6b24c crush: add choosemsr rule
Signed-off-by: Samuel Just <sjust@redhat.com>
2024-02-01 12:01:52 -08:00
Samuel Just
3a2517f2a4 crush: add msr tunables
Signed-off-by: Samuel Just <sjust@redhat.com>
2024-02-01 12:01:49 -08:00
Samuel Just
cbfff015e4 osd/OSDMap: minor formatting nit
Signed-off-by: Samuel Just <sjust@redhat.com>
2024-02-01 11:42:35 -08:00
Samuel Just
caa9e7a45e crush: add msr rule types and CrushWrapper infrastructure
Signed-off-by: Samuel Just <sjust@redhat.com>
2024-02-01 11:42:34 -08:00