Commit Graph

134340 Commits

Author SHA1 Message Date
Patrick Donnelly
efe41d2a9d
qa: ignore MDS_TRIM warnings when osd thrashing
Fixes: https://tracker.ceph.com/issues/57677
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2022-09-26 22:20:57 -04:00
Patrick Donnelly
fe60ad4436
qa: add testing for postgresql databases on CephFS
Fixes: https://tracker.ceph.com/issues/55470
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2022-09-26 22:19:44 -04:00
Patrick Donnelly
d02e41dd0a
qa: fs:workload add journal flush task
Idea here is to force the MDS to flush metadata mutations out of the
journal. This may help expose any type of corruption seen with postgres.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2022-09-26 22:19:44 -04:00
Patrick Donnelly
b5ba931f30
qa: add snap-schedule task
To exercise snapshot creation and deletion during workloads.

Fixes: https://tracker.ceph.com/issues/23724
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2022-09-26 22:19:43 -04:00
Patrick Donnelly
e9477ae421
qa: reorganize scrub/workunit tasks
So we can enforce an ordering of tasks and ensure that the fs/subvolume
is available before the task runs.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2022-09-26 22:19:43 -04:00
Patrick Donnelly
722b88328e
qa: raise error if fwd_scrub discovers damage
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2022-09-26 22:19:43 -04:00
Patrick Donnelly
483b16062d
qa: add new mntargs option for fuse
test_client_recovery was also using mntopts to specify additional
options to ceph-fuse. Because the two prior commits unify the behavior
of ceph-fuse and the kernel mount so that the "-o" option is available
for both, that changes breaks this test. Add a special set of args
available only for fuse (there is no equivalent on the kernel).

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2022-09-26 22:19:29 -04:00
Patrick Donnelly
d3295628eb
qa: allow specifying mount options for fuse
Also, fix the type default to None. A list does not work in the code but
the default is not used in QA.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2022-09-26 22:19:22 -04:00
Patrick Donnelly
ae00634a5e
qa: unify mountpoint config behavior
Only fuse would support the "mountpoint" yaml config. Move it to
mount.py so it works consistently with the kernel mount. Ditto for the
cephfs mount point.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2022-09-26 22:19:22 -04:00
Patrick Donnelly
d8ec1bd8e6
qa: allow check_counter to look at nested keys
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2022-09-26 22:19:21 -04:00
Patrick Donnelly
3d2cc1ae6b
qa: log mounts file if not mounted
For debugging.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2022-09-26 22:19:21 -04:00
Patrick Donnelly
298ef39420
qa: add missing or correct .qa links
find qa/suites/ -type d -execdir ln -sfT ../.qa/ {}/.qa \;

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2022-09-26 22:19:21 -04:00
Zack Cerza
af67a7152c tasks/cephadm: Add 'osd_method' config item
Since the default of None gets interpreted as 'lvm', the only useful
value would be 'raw'. This will enable us to use loop devices inside
containers for OSDs.

Signed-off-by: Zack Cerza <zack@redhat.com>
2022-09-26 15:35:08 -06:00
Zack Cerza
74c3446c25 cephadm: Avoid TypeError in download_cephadm()
Signed-off-by: Zack Cerza <zack@redhat.com>
2022-09-26 15:35:08 -06:00
Ken Dreyer
c2adf33ed5 Merge PR #48147 into main
* refs/pull/48147/head:
	win32: install wget when building on ubuntu
	win32: install deps with DEBIAN_FRONTEND=noninteractive

Reviewed-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
Reviewed-by: Justin Caratzas <jcaratza@redhat.com>
2022-09-26 14:44:51 -04:00
Ken Dreyer
7cfe523c7f Merge PR #47153 into main
* refs/pull/47153/head:
	build: fix atomic linking with LTO on s390x

Reviewed-by: Justin Caratzas <jcaratza@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2022-09-26 13:47:11 -04:00
Patrick Donnelly
e930a8b221
qa: ignore expected scrub error
The test_orphan_scan test deliberately removes a dentry which will cause
rstat damage. Ignore it.

Fixes: https://tracker.ceph.com/issues/57657
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2022-09-26 13:27:57 -04:00
Daniel Gryniewicz
036d034d66 RGW - Re-do admin API split
The mgr instantiation was re-added in the build re-org.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2022-09-26 12:03:39 -04:00
Adam King
c68489988c
Merge pull request #48197 from phlogistonjohn/jjm-cephadm-cov-podman
cephadm: add test cases for container engine classes

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
2022-09-26 11:43:47 -04:00
Ernesto Puerta
7f41f15745
Merge pull request #47131 from aaryanporwal/vrt-doc-update
doc/dev: mgr/dashboard: Replace broken link

Reviewed-by: Anthony D Atri <anthony.datri@gmail.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2022-09-26 13:22:11 +02:00
Tobias Urdin
d6f5e49f3b qa/tasks/barbican: dont copy policy files
In newer versions the policies is inside the code
so we don't need these files for default policy
anymore.

Signed-off-by: Tobias Urdin <tobias.urdin@binero.se>
2022-09-26 06:27:40 +00:00
Yingxin
b73279e836
Merge pull request #48217 from cyx1231st/wip-crimson-buffer
crimson: create buffer from temporary_buffer with foreign-ptr by default

Reviewed-by: Samuel Just <sjust@redhat.com>
2022-09-26 09:23:35 +08:00
zdover23
1cb996d4b1
Merge pull request #48235 from zdover23/wip-doc-2022-09-25-dev-guide-basic-workflow-fork-note-formatting
doc/dev: improve presentation of note (git remote)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2022-09-25 14:36:11 +10:00
Zac Dover
b8472b98e2 doc/dev: improve presentation of note (git remote)
This commit corrects the formatting of a note that previously
did not make console output and commands as clear as this commit
makes them.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2022-09-25 13:41:35 +10:00
Michael English
2606e649f6 python-common/ceph/deployment: filter drives by actuators when creating osds
As storage capacities grow, multi-actuator technology introduced by Seagate addresses the downward pressure on performance that comes with growing drive capacities and areal densities. Having multiple actuators enables drives to maintain the performance needs of customers with data-intensive applications.

Our testing indicates that, for certain workloads, using one OSD per actuator increases performance over using one OSD per dual-actuator drive. Bandwidth for the combined OSDs can increase over 40% and IOPS can nearly double compared to the alternative.

Extending the change introduced to ceph-volume that detects drives with two or more actuators, this feature enables the administrator to set a simple configuration that creates two OSDs on any detected dual-actuator drives using the advanced OSD service specification.

$ cat ~/osd.yaml
service_type: osd
service_id: example_osd_spec
placement:
  host_pattern: '*'
spec:
  data_devices:
    actuators: 2
  osds_per_device: 2
$ ./bin/ceph orch apply -i ~/osd.yaml
...

Copyright (c) 2022 Seagate Technology LLC and/or its Affiliates

Signed-off-by: Michael English <michael.english@seagate.com>
2022-09-23 12:15:27 -05:00
Venky Shankar
5f2e47ba21
Merge pull request #47767 from batrick/i57249
mds: correct storage of multiple damaged dentries in DamageTable

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
2022-09-23 16:00:25 +05:30
Lucian Petrut
9921c8a355
Merge pull request #48148 from ktdreyer/win-remove-curl
win32: remove libcurl
2022-09-23 10:43:44 +03:00
Venky Shankar
9016dcf085
Merge pull request #47080 from dparmar18/wip-dparmar-MDS-56522
mds/Server: Do not abort MDS on unknown messages

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2022-09-23 12:45:40 +05:30
Tobias Urdin
31e5d89377 qa/suites/rgw: blocklist object expiry tempest tests
Signed-off-by: Tobias Urdin <tobias.urdin@binero.se>
2022-09-23 06:31:06 +00:00
Tobias Urdin
7c460bd2fb qa/tasks/barbican: sync barbican db and secret stores
Signed-off-by: Tobias Urdin <tobias.urdin@binero.se>
2022-09-23 06:31:06 +00:00
Tobias Urdin
cd3313531f qa/tasks/barbican: copy barbican policy json or yaml
Signed-off-by: Tobias Urdin <tobias.urdin@binero.se>
2022-09-23 06:31:06 +00:00
Tobias Urdin
8310b179f3 qa/suites/rgw/tempest: bump openstack versions
Update the versions used for OpenStack Keystone,
Barbican and Tempest projects.

Signed-off-by: Tobias Urdin <tobias.urdin@binero.com>
2022-09-23 06:31:06 +00:00
Yingxin Cheng
6f44a21e6d crimson: create buffer from temporary_buffer with foreign-ptr by default
temporary_buffer is internally shareable with a thread-unsafe
ref-counter, we need to make sure it is released in the same core where
it is constructed.

Users that need the extra efficiency can swap to create_local as needed.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-09-23 14:03:42 +08:00
Nizamudeen A
1acdb44108
Merge pull request #47265 from s0nea/wip-dashboard-redirect-fqdn
mgr/dashboard: add option to resolve ip addr

Reviewed-by: Pegonzal <NOT@FOUND>
Reviewed-by: Anthony D Atri <anthony.datri@gmail.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2022-09-23 10:59:24 +05:30
Zack Cerza
f094103726 tasks/nvme_loop: Skip on containerized testnodes
Signed-off-by: Zack Cerza <zack@redhat.com>
2022-09-22 17:49:31 -06:00
Zack Cerza
1c8e7007ce tasks/nvme_loop: Don't modprobe if mod is loaded
Signed-off-by: Zack Cerza <zack@redhat.com>
2022-09-22 17:48:42 -06:00
Samuel Just
b4b2ab171c crimson/osd: rework OSDMap handling for multicore
OSDMaps can occupy significant space in memory.  Duplicating OSDMaps
across cores would multiply that memory usage as well as complicate
the internal structure sharing we do when applying OSDMap incrementals.

Because PeeringState and other interfaces expect efficient copying of
OSDMapRef objects we don't want to use foreign_ptr directly.

Instead, maintain a single cache and distribute local_shared_foreign_ptrs
to other cores.  ShardServices becomes the only OSDMapService.

Signed-off-by: Samuel Just <sjust@redhat.com>
2022-09-22 14:50:00 -07:00
Samuel Just
6385b86520 crimson/common: introduce local_shared_foreign_ptr
Introduces a foreign_ptr wrapper to allow core-local refcounting.

Signed-off-by: Samuel Just <sjust@redhat.com>
2022-09-22 13:16:50 -07:00
Adam King
82d3ec6a9a
Merge pull request #48198 from mgfritch/cephadm-pytest-patch-logger
cephadm: patch the cephadm.logger class 

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
2022-09-22 10:43:44 -04:00
Ali Maredia
fe9d6e9b82
Merge pull request #48095 from ivancich/wip-unify-logic
rgw: fix bool/int logic error when calling get_obj_head_ioctx
2022-09-22 10:03:45 -04:00
Adam King
2784a70271
Merge pull request #47756 from dparmar18/wip-dparmar-cephadm-after-revert
pybind/mgr/cephadm/upgrade: allow upgrades without reducing max_mds

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2022-09-22 09:54:06 -04:00
Adam King
3822cff481
Merge pull request #48118 from s0nea/container-version-fix
cephadm: consider stdout to get container version

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
2022-09-22 09:50:31 -04:00
Adam King
8d7d09e6a1
Merge pull request #48120 from mgfritch/cephadm-haproxy-nonlocal-bind
cephadm: add `ip_nonlocal_bind` to haproxy deployment

Reviewed-by: Adam King <adking@redhat.com>
2022-09-22 09:47:29 -04:00
Adam King
05f8f10324
Merge pull request #48184 from SUSE/wip-fix-57609
python-common: Add 'KB' to supported suffixes in SizeMatcher

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
2022-09-22 09:46:11 -04:00
Guillaume Abrioux
6da44bc956
Merge pull request #48200 from guits/fix-57627
ceph-volume: fix regression in activate
2022-09-22 15:28:35 +02:00
Yingxin
e3efab4919
Merge pull request #48185 from zhscn/wip-dtype
crimson/os/seastore: introduce backend_type_t

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2022-09-22 15:03:12 +08:00
Zhang Song
adff54fa72 crimson/os/seastore: make device according to device type
Signed-off-by: Zhang Song <zhangsong325@gmail.com>
2022-09-22 13:47:50 +08:00
Zhang Song
4f2e12f3e2 crimson/os/seastore: introduce get_backend_type to Device
Signed-off-by: Zhang Song <zhangsong325@gmail.com>
2022-09-22 13:45:55 +08:00
Zhang Song
a72cee838f crimson/os/seastore: make can_delay_allocation always return true
Signed-off-by: Zhang Song <zhangsong325@gmail.com>
2022-09-22 13:45:55 +08:00
Zhang Song
6a904b0922 crimson/os/seastore: rename about device_type_t and journal_type_t
Signed-off-by: Zhang Song <zhangsong325@gmail.com>
2022-09-22 13:45:55 +08:00