Commit Graph

128266 Commits

Author SHA1 Message Date
Sage Weil
124ac26c27 mgr/cephadm: record when finished with scheduled daemon action
Signed-off-by: Sage Weil <sage@newdream.net>
2021-11-23 19:32:26 -05:00
Sage Weil
0bbbe5d31d mgr/cephadm: skip osd_stats check if osd removal queue is empty
Signed-off-by: Sage Weil <sage@newdream.net>
2021-11-23 18:59:28 -05:00
Neha Ojha
1ad59c2c67
Merge pull request #44061 from Matan-B/wip-matanb-doc-teuthology
doc/dev: adding Teuthology suggested resources

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-11-23 12:07:41 -08:00
Ernesto Puerta
0a0b26cb04
Merge pull request #43996 from rhcs-dashboard/predefined-labels
mgr/dashboard: Predefine labels in create host form

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2021-11-23 20:03:22 +01:00
Sage Weil
9c2b8f557e mgr/cephadm: drop osdspec_affinity tracking
We identify which drivespec legacy OSDs belong(ed) to by metadata they
report to the mgr.  Modern cephadm does this instead by looking at the
'service' property in the unit.meta file.  Having cephadm query the osd
metadata is expensive for large clusters, so let's avoid this and rely
entirely on unit.meta.

Worst case, some upgraded clusters will show OSDs as service 'osd' instead
of service 'osd.whatever' for whatever drivespec created them.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-11-23 13:39:43 -05:00
Kefu Chai
709a77f220 cmake: test for 16-byte atomic support on mips also
it's reported that a mips64el build host is able to pass the test of
CheckCxxAtomic without linking against libatomic, while librbd.so
fails to link due to failures like

/usr/bin/ld: ../../../lib/librbd.so.1.16.0: undefined reference to `__atomic_store_16'
/usr/bin/ld: ../../../lib/librbd.so.1.16.0: undefined reference to `__atomic_load_16'
/usr/bin/ld: ../../../lib/librbd.so.1.16.0: undefined reference to `__atomic_compare_exchange_16'

so we have to check the existence of __atomic_load_16 instruction on
mips architecture.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2021-11-24 00:44:04 +08:00
Ronen Friedman
20dd022715 qa/standalone: osd-scrub-repair.sh: fix expected "not scrubbed since" warnings count
Following PR#43244, the 'ceph tell pg deep_scrub' now sets both
deep-scrub and "regular" scrub time-stamps. This necessitated a modification
to TEST_scrub_warning, as more PGs in this test are late for their regular scrubbing.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2021-11-23 16:43:21 +00:00
Radoslaw Zarzynski
82bfb2eef6 crimson/common: don't assume pointer-from-SharedLRU can't outlive it.
Initially, we were assuming that no pointer obtained from SharedLRU
can outlive the lru itself. However, since going with the interruption
concept for handling shutdowns, this is no longer valid.

The patch is supposed to deal with crashes like the following one:

```
ceph-osd: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-8898-ge57ad63c/rpm/el8/BUILD/ceph-17.0.
0-8898-ge57ad63c/src/crimson/common/shared_lru.h:46: SharedLRU<K, V>::~SharedLRU() [with K = unsigned int; V = OSDMap]: Assertion `weak_refs.empty()' failed.
Aborting on shard 0.
Backtrace:
Reactor stalled for 1162 ms on shard 0. Backtrace: 0xb14ab 0x46e57428 0x46bc450d 0x46be03bd 0x46be0782 0x46be0946 0x46be0bf6 0x12b1f 0xc8e3b 0x3fdd77e2 0x3fddccdb 0x3fdde1ee 0x3fdde8b3 0x3fdd3f2b 0x3fdd4442 0x3f
dd4c3a 0x12b1f 0x3737e 0x21db4 0x21c88 0x2fa75 0x3a5ae1b9 0x3a38c5e2 0x3a0c823d 0x3a1771f1 0x3a1796f5 0x46ff92c9 0x46ff9525 0x46ff9e93 0x46ff8eae 0x46ff8bd9 0x3a160e67 0x39f50c83 0x39f51cd0 0x46b96271 0x46bde51a
 0x46d6891b 0x46d6a8f0 0x4681a7d2 0x4681f03b 0x39fd50f2 0x23492 0x39b7a7dd
 0# gsignal in /lib64/libc.so.6
 1# abort in /lib64/libc.so.6
 2# 0x00007F9535E04C89 in /lib64/libc.so.6
 3# 0x00007F9535E12A76 in /lib64/libc.so.6
 4# crimson::osd::OSD::~OSD() in ceph-osd
 5# seastar::shared_ptr_count_for<crimson::osd::OSD>::~shared_ptr_count_for() in ceph-osd
 6# seastar::shared_ptr<crimson::osd::OSD>::~shared_ptr() in ceph-osd
 7# seastar::futurize<std::result_of<seastar::sharded<crimson::osd::OSD>::stop()::{lambda(seastar::future<void>)#2}::operator()(seastar::future<void>) const::{lambda(unsigned int)#1}::operator()(unsigned int) co
nst::{lambda()#1} ()>::type>::type seastar::smp::submit_to<seastar::sharded<crimson::osd::OSD>::stop()::{lambda(seastar::future<void>)#2}::operator()(seastar::future<void>) const::{lambda(unsigned int)#1}::opera
tor()(unsigned int) const::{lambda()#1}>(unsigned int, seastar::smp_submit_to_options, seastar::sharded<crimson::osd::OSD>::stop()::{lambda(seastar::future<void>)#2}::operator()(seastar::future<void>) const::{la
mbda(unsigned int)#1}::operator()(unsigned int) const::{lambda()#1}&&) in ceph-osd
 8# std::_Function_handler<seastar::future<void> (unsigned int), seastar::sharded<crimson::osd::OSD>::stop()::{lambda(seastar::future<void>)#2}::operator()(seastar::future<void>) const::{lambda(unsigned int)#1}>
::_M_invoke(std::_Any_data const&, unsigned int&&) in ceph-osd
 9# 0x0000562DA18162CA in ceph-osd
10# 0x0000562DA1816526 in ceph-osd
11# 0x0000562DA1816E94 in ceph-osd
12# 0x0000562DA1815EAF in ceph-osd
13# 0x0000562DA1815BDA in ceph-osd
14# seastar::noncopyable_function<seastar::future<void> (seastar::future<void>&&)>::direct_vtable_for<seastar::future<void>::then_wrapped_maybe_erase<true, seastar::future<void>, seastar::sharded<crimson::osd::OSD>::stop()::{lambda(seastar::future<void>)#2}>(seastar::sharded<crimson::osd::OSD>::stop()::{lambda(seastar::future<void>)#2}&&)::{lambda(seastar::future<void>&&)#1}>::call(seastar::noncopyable_function<seastar::future<void> (seastar::future<void>&&)> const*, seastar::future<void>&&) in ceph-osd
15# 0x0000562D9476DC84 in ceph-osd
16# 0x0000562D9476ECD1 in ceph-osd
17# 0x0000562DA13B3272 in ceph-osd
18# 0x0000562DA13FB51B in ceph-osd
19# 0x0000562DA158591C in ceph-osd
20# 0x0000562DA15878F1 in ceph-osd
21# 0x0000562DA10377D3 in ceph-osd
22# 0x0000562DA103C03C in ceph-osd
23# main in ceph-osd
24# __libc_start_main in /lib64/libc.so.6
25# _start in ceph-osd
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-11-23 15:12:52 +00:00
Arthur Outhenin-Chalandre
9385acfc25
rbd-mirror: make RemoveImmediateUpdate test synchronous
Try fixing sporadic failure linked in the tracker in
TestMockMirrorStatusUpdater.RemoveImmediateUpdate by making it
synchronous.

Fixes: https://tracker.ceph.com/issues/53375
Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@cern.ch>
2021-11-23 15:30:33 +01:00
Patrick Donnelly
23a6c25616
mds: defer messages to bootstrapping ranks
This is necessary with msgr protocol v2. The protocol no longer updates
the server "myname" for each message. It's associated for a connection.
For this reason, a newly starting rank (failover or new) updating its
messenger "myname" races with other ranks trying to connect to it. We
need those ranks to hold off on connecting until the rank reaches a
known good state.

Fixes: https://tracker.ceph.com/issues/53194
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-11-23 09:23:11 -05:00
Patrick Donnelly
402919cbe6
mds: test connections to bootstrapping MDS
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-11-23 09:23:09 -05:00
Patrick Donnelly
8dc54deed9
mds: correct whitespace
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-11-23 09:21:34 -05:00
Alfonso Martínez
3e4e29590a mgr/dashboard: upgrade Cypress to the latest stable version
- Remove unneeded dependency that was causing UI performance issues: zone.js
- Ignore 'ResizeObserver loop limit exceeded' error.
- run-frontend-e2e-tests.sh refactoring: create rgw dashboard user through
  'ceph dashboard set-rgw-credentials' and use it on rgw buckets' tests.

Fixes: https://tracker.ceph.com/issues/53357
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
2021-11-23 15:17:54 +01:00
Ronen Friedman
f600acbba1
Merge pull request #43244 from ronen-fr/wip-rf-scrub-command
osd: make 'pg deep-scrub' command initiate a scrub

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-11-23 15:19:33 +02:00
Venky Shankar
c152264b31
Merge pull request #43722 from lxbsz/caps_doc
doc: update the capabilities doc for cephfs

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2021-11-23 18:17:22 +05:30
Matan Breizman
19f6263789 doc/dev: adding Teuthology suggested resources
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2021-11-23 11:06:57 +00:00
Guillaume Abrioux
b93262f2e1
Merge pull request #43982 from guits/refactor_cv_human_readable_func
ceph-volume: human_readable_size() refactor
2021-11-23 06:12:22 +01:00
Xiubo Li
98cca5ed5a client: fix crash when iterating and deleting sessions
Introduced-by: 730dd8a87a
Fixes: https://tracker.ceph.com/issues/53082
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2021-11-23 12:21:53 +08:00
Kefu Chai
292b9a3ce2
Merge pull request #44007 from tchaikov/wip-cmake-python3.10
cmake: check for python(\d)\.(\d+) when building boost

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2021-11-23 10:59:54 +08:00
Ernesto Puerta
60171326fa
Merge pull request #43992 from rhcs-dashboard/flaky-inventory-test-fix
mgr/dashboard: fix flaky inventory e2e test

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2021-11-22 20:39:50 +01:00
Ernesto Puerta
7a5c906c9a
Merge pull request #43958 from rhcs-dashboard/daemon-event-padding
mgr/dashboard: Daemon Events listing using bootstrap class

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2021-11-22 19:43:11 +01:00
Ernesto Puerta
b81e9ecfeb
Merge pull request #43866 from rhcs-dashboard/add-hint-provisioned-images
mgr/dashboard: provisioned values is misleading in RBD image table

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2021-11-22 19:41:04 +01:00
Casey Bodley
1f8bdd2ddf
Merge pull request #43843 from cbodley/wip-test-cls-rgw-stats
test/cls/rgw: add index transaction simulator to model bucket stats

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
2021-11-22 10:59:39 -05:00
Sebastian Wagner
d5c652f07c
Merge pull request #43888 from mgfritch/cephadm-expect-hostname
cephadm: fixup expect-hostname message

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-11-22 12:00:20 +01:00
Sebastian Wagner
548f9e1930
Merge pull request #43873 from guits/add_shared_folder_shell_cmd
cephadm: add --shared_ceph_folder to shell cmd

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-11-22 11:59:19 +01:00
Sebastian Wagner
84f88eaec4
Merge pull request #43876 from sebastian-philipp/all-osd-at-once
mgr/cephadm: create osds at all hosts at once

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
2021-11-22 11:27:13 +01:00
Aashish Sharma
4def9d89c5 mgr/dashboard: dashboard does not show degraded objects if they are less than 0.5% under "Dashboard->Capacity->Objects block
This PR is intended to fix this issue

Fixes: https://tracker.ceph.com/issues/53242
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2021-11-22 15:11:31 +05:30
Ernesto Puerta
2e1c4e590a
Merge pull request #44033 from ljflores/wip-update-email-id
mailmap: add Laura Flores

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: ljflores <NOT@FOUND>
Reviewed-by: neha-ojha <NOT@FOUND>
2021-11-22 09:14:07 +01:00
Samuel Just
070e54178a
Merge pull request #43795 from myoungwon/wip-paddr-split
seastore: generalize paddr_t

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-11-21 22:17:04 -08:00
Nizamudeen A
e228bf7563 mgr/dashboard: Predfine labels in create host form
Also retains the previously created labels by user in the form

Fixes: https://tracker.ceph.com/issues/53315
Signed-off-by: Nizamudeen A <nia@redhat.com>
2021-11-22 11:21:55 +05:30
Nizamudeen A
cdce5f78ad mgr/dashboard: fix flaky inventory e2e test
When `inventory.getTableCount('total').should('be.eq', totalDiskCount);`
this line is executed the table was not loaded properly and hence the
getTableCount returns 0 on the first try but on second try it passes
since the table is loaded. But in orch e2es the retries are set to 0. I
am not sure if it makes sense to set it to 1. Anyway I am adapting the
test a bit to expect the count to be equal to totalDiskCount so that the
test will wait a bit.

Fixes: https://tracker.ceph.com/issues/53353
Signed-off-by: Nizamudeen A <nia@redhat.com>
2021-11-22 11:12:32 +05:30
Deepika Upadhyay
5b1eae6953
Merge pull request #43524 from Rethan/feat-expiration-time
rbd: when trash mv, show expiration time if it's not now

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Reviewed-by: Sunny Kumar <sunkumar@redhat.com>
2021-11-22 06:52:31 +05:30
Deepika Upadhyay
048c8e6f4a
Merge pull request #43852 from hualongfeng/show_feature
tools/rbd: make rbd info display dirty-cache feature

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
2021-11-22 06:51:51 +05:30
Deepika Upadhyay
17f4da9df2
Merge pull request #43907 from cybozu/rbd-correct-encoding-of-snap-protection-record-in-exporting
rbd: correct encoding of snap protection record in exporting image

Reviewed-by: Mykola Golub <mykola.golub@clyso.com>
2021-11-21 22:33:41 +05:30
Sage Weil
37ed8b6427 .github/pull_request_template: drop teuthology reference
It is not clear what role this has relative to the needs-qa label.

Signed-off-by: Sage Weil <sage@newdream.net>
2021-11-20 08:56:06 -06:00
Sage Weil
c68b9c0d1e .github/pull_request_template: add cleanup option
Signed-off-by: Sage Weil <sage@newdream.net>
2021-11-20 08:56:06 -06:00
myoungwon oh
1601345824 seastore: generalize paddr_t
Due to necessity to support both segment and rbm addrs,
paddr_t needs to be generlized.
To do so, this commits introduces seg_paddr_t---
seg_paddr_t inherits paddr_t, and deals with common_addr
according to the semantic of segment.

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-11-20 10:11:57 +09:00
Neha Ojha
0f9ed11e67
Merge pull request #43999 from kamoltat/wip-autoscale-profile-scale-up-default
pybind/mgr/pg_autoscale: revert to default profile scale-up

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-11-19 16:55:43 -08:00
Patrick Donnelly
860518bcb6
Merge PR #43974 into master
* refs/pull/43974/head:
	qa: disable metrics on kernel client during upgrade

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
2021-11-19 18:43:48 -05:00
Laura Flores
41b44abfe6 mailmap: add Laura Flores
Signed-off-by: Laura Flores <lflores@redhat.com>
2021-11-19 23:22:16 +00:00
Ernesto Puerta
515af762bb
Merge pull request #43987 from rhcs-dashboard/53123-dashboard-nfs-cleanup
mgr/dashboard: NFS non-existent files cleanup

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: ljflores <NOT@FOUND>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2021-11-19 20:40:41 +01:00
Ernesto Puerta
d67302fcf4
Merge pull request #43983 from rhcs-dashboard/rgw-add-realm-column
mgr/dashboard: rgw daemon list: add realm column

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: sebastian-philipp <NOT@FOUND>
2021-11-19 20:14:19 +01:00
J. Eric Ivancich
08adeae354
Merge pull request #43824 from cbodley/wip-qa-rgw-upgrade-octopus-multisite-cv
qa/upgrade: rgw multisite upgrade test excludes ceph-volume

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
2021-11-19 14:11:39 -05:00
Deepika Upadhyay
742e6cbd5f
Merge pull request #43764 from gregsfortytwo/wip-rbd-crash-consistency
doc: fix up rbd snapshot docs around crash consistency

Reviewed-by: Mykola Golub <mykola.golub@clyso.com>
Reviewed-by: Sunny Kumar <sunkumar@redhat.com>
2021-11-20 00:29:16 +05:30
Kamoltat
a9f9f7b3fd pybind/mgr/pg_autoscale: revert to default profile scale-up
pg_autoscale module will now start out all the pools
with a scale-up profile by default.

Added tests in workunits/mon/pg_autoscaler.sh
to evaluate if the default pool creation is
a scale-up profile

Updated documentation and release notes to
reflect the change in the default behavior
of the pg_autoscale profile.

Fixes: https://tracker.ceph.com/issues/53309

Signed-off-by: Kamoltat <ksirivad@redhat.com>
2021-11-19 18:55:36 +00:00
Patrick Donnelly
dcda5cb9ce
qa: disable metrics on kernel client during upgrade
v16.2.4 MDS triggers an assert from these messages.

Also: add latest pacific for extra coverage.

Fixes: https://tracker.ceph.com/issues/53293
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-11-19 13:32:04 -05:00
Casey Bodley
2ed55af056 qa/rgw: run ceph_test_cls_rgw_stats in rgw/verify suite
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-11-19 12:13:38 -05:00
Casey Bodley
5c2210d9e7 test/rgw: model multipart uploads in ceph_test_cls_rgw_stats
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-11-19 12:13:38 -05:00
Casey Bodley
b3f1c52896 test/rgw: add ceph_test_cls_rgw_stats simulator
a simulator that generates lots of create/delete operations against a
set of object names, and tracks the expected bucket stats for comparison

each operation has a 70% chance to complete successfully, a 10% chance to
cancel, 10% chance to suggest update, and 10% chance to suggest removal

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-11-19 12:13:33 -05:00
Casey Bodley
c81f2fedf2 cls/rgw: add some missing equality comparisons
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-11-19 11:40:57 -05:00