Commit Graph

139815 Commits

Author SHA1 Message Date
Xuehan Xu
04764838b5 crimson/osd/pg: check if backfill_state exists when judging objects'
degradation

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2023-08-17 14:53:09 +08:00
Lucian Petrut
28a78fa169 test/librbd/fsx: fix wnbd check
When using WNBD, the IO size must be a multiple of the sector size
(WNBD only supports 512 at the moment).

We're currently checking the "wnbd_operations" variable, however
it's only defined on Windows, leading to a compilation failure
on other platforms.

We'll switch back to a separate boolean variable called "wnbd_disk".

Fixes: https://tracker.ceph.com/issues/62475
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2023-08-17 06:19:32 +00:00
avanthakkar
014421ea16 mgr/dashboard: support cluster upgrade even if the check for upgrade fails
Fixes: https://tracker.ceph.com/issues/62457
Signed-off-by: avanthakkar <avanjohn@gmail.com>
2023-08-17 11:38:48 +05:30
Nizamudeen A
ac616eeefb
Merge pull request #51561 from rhcs-dashboard/hosts-no-data
mgr/dashboard: add not avaialable with unpresent host facts

Reviewed-by: Pegonzal <NOT@FOUND>
Reviewed-by: cloudbehl <NOT@FOUND>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2023-08-17 10:57:52 +05:30
Nizamudeen A
e5737357e3
Merge pull request #51227 from jsoref/spelling-dashboard
dashboard: Fix spelling

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2023-08-17 10:48:30 +05:30
Nizamudeen A
bee068057a
Merge pull request #51126 from rhcs-dashboard/jeje-log
mgr/prometheus: remove jeje log

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
2023-08-17 10:44:22 +05:30
myoungwon oh
a10f2855da crimson/os/seastore/journal/cbj: check if the first record in CBJ is written after rolling or not
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2023-08-17 04:52:25 +00:00
Kefu Chai
27c05edc8e cmake: update FindPython3 module
* use the one shiped by the latest CMake (ab379e5054aa792df9572078dcf95bddd75f7661)
* use the new policy to use the new find strategy.
* accomodate the vanilla FindPython3 module to Ceph by:
  - dropping the `cmake_policy()` calls which set the policy not supported
    by 3.16.
  - `include (FindPackageHandleStandardArgs)` without specifying the
    relative path.
  - dropping the `HANDLE_VERSION_RANGE` from `FindPackageHandleStandardArgs()` call.
    this option was introduced by CMake v3.19, see
    https://cmake.org/cmake/help/latest/module/FindPackageHandleStandardArgs.html
    but Ubuntu focal comes with CMake 3.16, which is our minimal required CMake version.

the new FindPython3 module from CMake:

* enables us to find the recent Python intepreter and development files up to
  CPython 3.13.
* finds Python intepreter with the new `Python_FIND_STRATEGY`. the old and
  default strategy always finds the most recent version with all specified name
  and in all locations. so, if /usr/bin/python exists, it would accept, even
  if it is a symlink to python3.9 and what we want is python3.6. the
  new policy stops at the one which satisfies the constraints. this
  helps us to address the problem of issue we have when recreating
  the building system after the CMakeLists.txt is updated.

Fixes: https://tracker.ceph.com/issues/62428
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2023-08-17 10:24:16 +08:00
Kefu Chai
fb93d6e032 cmake: update FindPython3 module
* use the one shiped by the latest CMake (ab379e5054aa792df9572078dcf95bddd75f7661)
* use the new policy to use the new find strategy.
* accomodate the vanilla FindPython3 module to Ceph by:
  - dropping the `cmake_policy()` calls which set the policy not supported
    by 3.16.
  - `include (FindPackageHandleStandardArgs)` without specifying the
    relative path.
  - dropping the `HANDLE_VERSION_RANGE` from `FindPackageHandleStandardArgs()` call.
    this option was introduced by CMake v3.19, see
    https://cmake.org/cmake/help/latest/module/FindPackageHandleStandardArgs.html
    but Ubuntu focal comes with CMake 3.16, which is our minimal required CMake version.

the new FindPython3 module from CMake:

* enables us to find the recent Python intepreter and development files up to
  CPython 3.13.
* finds intepreter with the new `Python_FIND_STRATEGY`. the old and default
  strategy always finds the most recent version with all specified name
  and in all locations. so, if /usr/bin/python exists, it would accept, even
  if it is  a symlink to python3.9 and what we want is python3.6. while
  the new policy stops at the one which satisfies the constraints.
  simpler this way and less error prone.

Fixes: https://tracker.ceph.com/issues/62428
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2023-08-17 10:16:45 +08:00
Nizamudeen A
73272c91fc
Merge pull request #53021 from rhcs-dashboard/buckets-e2e-fix
mgr/dashboard: fix rgw buckets e2e 

Reviewed-by: cloudbehl <NOT@FOUND>
2023-08-16 23:08:35 +05:30
Nizamudeen A
82654d2f02 mgr/dashboard: add e2e tests for cephfs management
Fixes: https://tracker.ceph.com/issues/62340
Signed-off-by: Nizamudeen A <nia@redhat.com>
2023-08-16 22:53:47 +05:30
Adam King
306e8437e6
Merge pull request #52740 from adk3798/ingress-vip-networks
mgr/cephadm: pick correct IPs for ingress service based on VIP

Reviewed-by: John Mulligan <jmulligan@redhat.com>
2023-08-16 13:22:50 -04:00
Adam King
149e07d633
Merge pull request #51881 from adk3798/logrotate-tcmu-runner
cephadm: add tcmu-runner to logrotate config

Reviewed-by: John Mulligan <jmulligan@redhat.com>
2023-08-16 13:05:08 -04:00
Nizamudeen A
8915205a9e mgr/dashboard: update angular to 14
Fixes: https://tracker.ceph.com/issues/61641
Signed-off-by: Nizamudeen A <nia@redhat.com>
2023-08-16 22:16:33 +05:30
Casey Bodley
7879630b45
Merge pull request #52651 from galsalomon66/trino_tpcds_qe_fixes
rgw/s3select: s3select fixes related to Trino/TPCDS benchmark and QE tests

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2023-08-16 12:36:55 -04:00
myoungwon oh
942ea6487f crimson/os/seastore/journal/cbj: introduce magic value to identify that written records are correct
The cbj would replay invalid records if the same write pattern (mkfs -> write -> mkfs -> write) repeats.
On the second mkfs(), current cbj wrongly recognizes the old records are valid beceause they have
correct seq. number at appropriate offset even though the records are expired.

To solve this, this commit introduces random value to identify the journal  during mkfs()

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2023-08-16 16:02:57 +00:00
myoungwon oh
a580e51696 crimson/os/seastore: introduce generalized scan_valid_records for RBM
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2023-08-16 16:02:46 +00:00
Ilya Dryomov
f555fc35ed
Merge pull request #50141 from petrutlucian94/rbd_tests
Enable librbd tests on Windows

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2023-08-16 17:44:37 +02:00
Ernesto Puerta
377fce0c44
Merge pull request #52988 from ceph/gh-add-to-project-fix-perm-issue
.github: fix add to dashboard failing

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2023-08-16 16:30:10 +02:00
Adam King
c53f119348
Merge pull request #52410 from phlogistonjohn/jjm-issue61852-haproxy-ips
cephadm: fix issue with limited haproxy_hosts IPs list

Reviewed-by: Adam King <adking@redhat.com>
2023-08-16 10:16:09 -04:00
Adam King
4d84503716
Merge pull request #51899 from adk3798/cephadm-signed-keys
cephadm: support for CA signed keys

Reviewed-by: John Mulligan <jmulligan@redhat.com>
2023-08-16 10:10:36 -04:00
myoungwon oh
22c747826b crimson/os/seastore: introduce RecordScanner to generalize scan_valid_records()
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2023-08-16 12:13:54 +00:00
zdover23
df8cd910bf
Merge pull request #51170 from kotreshhr/doc-mds-cap-acquisition-throttle
doc: Fix doc for mds cap acquisition throttle

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com> 
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2023-08-16 22:00:51 +10:00
Xuehan Xu
0ee9bb21be crimson/os/seastore: fix daggling reference of oid in
SeaStore::Shard::stat()

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2023-08-16 19:41:57 +08:00
Ronen Friedman
8b3101f3b8 crimson/osd: fixing fmtlib-related compilation issues
brought about by the change to fmt formatting support.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2023-08-16 05:29:09 -05:00
Nizamudeen A
fa07be72a7 mgr/dashboard: fix rgw buckets e2e
Fixes: https://tracker.ceph.com/issues/62458
Signed-off-by: Nizamudeen A <nia@redhat.com>
2023-08-16 14:46:42 +05:30
Nizamudeen A
f5c7c5fb1f
Merge pull request #53018 from rhcs-dashboard/rm-cephfs-subvolume-group
mgr/dashboard: cephfs remove subvolume group

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: cloudbehl <NOT@FOUND>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2023-08-16 14:03:44 +05:30
Lucian Petrut
9a0a715cb1 test/librbd/fsx: add rbd-wnbd tests
rbd-wnbd can be used to map rbd images as Windows block devices.

This change will allow exercising rbd-wnbd and the wnbd driver
as opposed to using librbd directly.

One thing to note here is that read and write operations must be
512B aligned. Also, wnbd does not currently support "compare
and write".

For this reason, we'll stick to the basic operations for the time
being.

Note that we're renaming krbd_read/krbd_write to generic_pread
and generic_pwrite, reusing those functions for wnbd operations.
We're also removing the duplicated ggate read/write operations,
using the generic functions instead.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2023-08-16 07:51:44 +00:00
Venky Shankar
796164cf49 Merge PR #52416 into main
* refs/pull/52416/head:
	qa/cephfs: check if re-installing this mpich fixes the issue

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2023-08-16 12:58:01 +05:30
Kotresh HR
65b0e45dd7 doc: Fix doc for mds cap acquisition throttle
The config values picks the definition from the
description. Removed the duplicate description
in the doc.

Signed-off-by: Kotresh HR <khiremat@redhat.com>
2023-08-16 12:16:24 +05:30
Pedro Gonzalez Gomez
0628c69ef7 mgr/dashboard: cephfs remove subvolume group
Fixes: https://tracker.ceph.com/issues/62452
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
2023-08-16 07:52:52 +02:00
dependabot[bot]
22b3e6b153
mgr/dashboard: bump protobufjs in /src/pybind/mgr/dashboard/frontend
Bumps [protobufjs](https://github.com/protobufjs/protobuf.js) from 6.11.3 to 6.11.4.
- [Release notes](https://github.com/protobufjs/protobuf.js/releases)
- [Changelog](https://github.com/protobufjs/protobuf.js/blob/master/CHANGELOG.md)
- [Commits](https://github.com/protobufjs/protobuf.js/commits)

---
updated-dependencies:
- dependency-name: protobufjs
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-08-16 00:04:34 +00:00
Adam King
0e90c7e097 mgr/cephadm: add utils class for tracking special host labels
Signed-off-by: Adam King <adking@redhat.com>
2023-08-15 16:17:29 -04:00
Adam King
871aefb11d mgr/cephadm: allow draining host without removing conf/keyring files
Fixes: https://tracker.ceph.com/issues/58820

Signed-off-by: Adam King <adking@redhat.com>
2023-08-15 16:17:28 -04:00
Adam King
d5d40e07ca cephadm: add tcmu-runner to logrotate config
This process could be used to set up the tcmu-runner
to log to a file much like other ceph daemons

- create /etc/tcmu directory
- create /etc/tcmu/tcmu.conf directory with default options
- change dir to /var/log
- change log level to 4
- add -v /etc/tcmu:/etc/tcmu to tcmu-runner container podman line in unit.run

In order to support this (mostly for debugging) we should
add tcmu-runner to the logrotate config

Fixes: https://tracker.ceph.com/issues/61571

Signed-off-by: Adam King <adking@redhat.com>
2023-08-15 15:59:00 -04:00
Laura Flores
93c326f949
Merge pull request #47560 from pdvian/wip-pool-app
mon, qa: issue pool application warning even if pool is empty
2023-08-15 14:53:42 -05:00
Adam King
d1a0d8d10a
Merge pull request #52674 from asm0deuz/add_networks_cli
mgr/cephadm: Add "networks" parameter to orch apply rgw

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Francesco Pantano <fpantano@redhat.com>
Reviewed-by: Guillaume Abrioux <gabrioux@ibm.com>
2023-08-15 15:52:31 -04:00
Adam King
307cc750eb
Merge pull request #52691 from adk3798/nvme-of-cleanup
mgr/cephadm: nvme-of follow up work

Reviewed-by: John Mulligan <jmulligan@redhat.com>
2023-08-15 15:37:37 -04:00
Adam King
d0b31b236a
Merge pull request #52678 from phlogistonjohn/jjm-fix-test_valid_addr
python-common: fix valid_addr on python 3.11

Reviewed-by: Adam King <adking@redhat.com>
2023-08-15 15:36:11 -04:00
Adam King
141af1c6b1 qa/cephadm: add test for ca signed keys
Test that bootstraps with a CA signed key using
the use_ca_signed_key cephadm override. Then follows
up by doing a check-host on each host which verifies
the cephadm mgr module can reach and authenticate with
the nodes using the new key setup.

This probably should really be a workunit, but
I didn't want to create a full new section for
this test and I needed a section that didn't
already run the cephadm task for every test. I could
see this being moved into some sort of
"test_special_deployment_scenarios" section in the future

Signed-off-by: Adam King <adking@redhat.com>
2023-08-15 15:34:26 -04:00
Adam King
bef9617c51 qa/cephadm: add ca signed key to cephadm task
To allow bootstrapping a cluster using a CA signed
key instead of the standard pubkey authentication.
Will allow explicit testing of this as we add support
for it

Signed-off-by: Adam King <adking@redhat.com>
2023-08-15 15:34:26 -04:00
Adam King
2c837ea9cf doc/cephadm: document setting up CA signed keys in running cluster
Signed-off-by: Adam King <adking@redhat.com>
2023-08-15 15:34:26 -04:00
Adam King
6b4d9b4427 doc/cephadm: document bootstrapping with CA signed keys
Signed-off-by: Adam King <adking@redhat.com>
2023-08-15 15:34:26 -04:00
Adam King
e09a376547 doc/cephadm: document how to pass self made SSH key pairs to bootstrap
This didn't seem to exist in the install section of
the cephadm docs. Wanted to add it in before adding
documentation for bootstrapping with CA signed keys.

Signed-off-by: Adam King <adking@redhat.com>
2023-08-15 15:34:26 -04:00
Adam King
50f74d6063 mgr/cephadm: add support for CA signed SSH keys setups
Signed-off-by: Adam King <adking@redhat.com>
2023-08-15 15:34:26 -04:00
Adam King
c5cb7e913b
Merge pull request #51785 from muahdib69/rsachere_iscsi_fixes
cephadm: Fix extra_container_args for iSCSI

Reviewed-by: Adam King <adking@redhat.com>
2023-08-15 15:25:46 -04:00
Laura Flores
a17a33f97e
Merge pull request #52776 from ljflores/wip-read-balancer-documentation
doc: clarify upmap balancer documentation
2023-08-15 12:27:48 -05:00
Nizamudeen A
6aeeb4ac40
Merge pull request #52919 from rhcs-dashboard/cluster-upgrade-progress
mgr/dashboard: cluster upgrade progress UI

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: cloudbehl <NOT@FOUND>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2023-08-15 21:38:10 +05:30
zdover23
e05b1d670e
Merge pull request #52991 from zdover23/wip-doc-2023-08-15-README-md-submodule
doc: improve submodule update command - README.md

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2023-08-16 00:28:03 +10:00
galsalomon66
87739031fd - s3select fixes, related to Trino/TPCDS benchmark
- s3select submodule
- add log-message for better tracing the flow timeline
- fix valgrind issue
- new s3select version. fix crash upon wrong initialization on JSON flow
- s3select module. a fix for a constant number of columns(128).
- s3select module. QE fixes

Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
2023-08-15 15:34:05 +03:00