Commit Graph

127213 Commits

Author SHA1 Message Date
Ilya Dryomov
d66a7ec3ef
Merge pull request #43444 from idryomov/wip-read-from-replica-comment
librados,librbd: make it clear that replica reads are safe for general use

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-10-08 13:43:27 +02:00
Yin Congmin
dd33684733 librbd/cache/pwl: initialize number_log_entries
Using uninitialized number_log_entries cause writesame req space
calculation error. sometimes fail in TestMockCacheSSDWriteLog.writesame.

Fixes: https://tracker.ceph.com/issues/52852

Signed-off-by: Yin Congmin <congmin.yin@intel.com>
2021-10-08 10:15:48 +00:00
Ernesto Puerta
418adb88b2
Merge pull request #43137 from rhcs-dashboard/fix-52480-master
mgr/dashboard: clean-up controllers

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2021-10-08 10:56:30 +02:00
Samuel Just
0f510f05ae
Merge pull request #43459 from cyx1231st/wip-seastore-optimize-record-overhead
crimson/os/seastore: reduce write amplification from record overhead and implement placement hint

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-10-08 01:32:55 -07:00
Dai Zhiwei
eaa385f3da qa: support isal ec test for aarch64
modified:   qa/standalone/erasure-code/test-erasure-code-plugins.sh
	new file:   qa/suites/rados/thrash-erasure-code-isa/arch/aarch64.yaml

Signed-off-by: Dai Zhiwei <daizhiwei3@huawei.com>
2021-10-08 14:37:25 +08:00
Yingxin Cheng
300d27065b crimson/os/seastore: implement placement_hint HOT/COLD/REWRITE
The current strategy is to inline allocate non-REWRITE extents in order
to reduce the record overhead. The observation shows the transactions
from user is usually very small and doesn't deserve to be written in ool
records, which requires at least a block to store the record metadata.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-10-08 11:59:50 +08:00
Yingxin Cheng
05d68ddc15 crimson/os/seastore/epm: misc cleanup
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-10-08 11:59:42 +08:00
Yingxin Cheng
0f6a1571b8 crimson/os/seastore: rename ool_placement_hint_t
To placement_hint_t as the hint is not limited to ool extents.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-10-08 10:59:09 +08:00
Michael Fritch
52c20e5551
mgr/cephadm: use str type for stdin
avoid encode/decode confusion by using a str data type to both
send (stdin) and receive (stdout) data

Signed-off-by: Michael Fritch <mfritch@suse.com>
2021-10-07 17:57:12 -06:00
Michael Fritch
6c57113b18
mgr/cephadm: use asyncssh.scp to write remote files
`tee` via stdin happens to work when the file is a utf-8 byte encoded
string, but won't work if the file happens to be binary data

Signed-off-by: Michael Fritch <mfritch@suse.com>
2021-10-07 17:55:05 -06:00
Kefu Chai
4ceedec1a4
Merge pull request #43248 from inspur-wyq/wip-doc
doc/rbd/vault.rst: fix typos

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2021-10-08 06:22:54 +08:00
Gregory Farnum
251b6e243d
Merge pull request #43373 from gregsfortytwo/wip-mon-stretch-new-tiebreaker-52126
mon: Allow specifying new tiebreaker monitors

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-10-07 15:16:23 -07:00
Kefu Chai
b126538512
Merge pull request #43331 from adk3798/agent-drain
mgr/cephadm: fix host drain with agent

Reviewed-by: Nizamudeen A <nia@redhat.com>
2021-10-08 06:07:49 +08:00
Melissa Li
1ccdd941c9 mgr/cephadm: set health check warning for apply spec failures and daemon place failures in serve
Fixes: https://tracker.ceph.com/issues/44414
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>
2021-10-07 17:08:32 -04:00
Melissa Li
0f8f085058 mgr/cephadm: set health check warning in _apply_service_config for invalid config options and failures to set options
Fixes: https://tracker.ceph.com/issues/44414
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>
2021-10-07 17:08:32 -04:00
Melissa Li
f42992a1ef mgr/cephadm: refactor _check_for_strays to use remove_health_warning helper
Fixes: https://tracker.ceph.com/issues/44414
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>
2021-10-07 17:08:32 -04:00
Melissa Li
2eacd6e7e9 mgr/cephadm: refactor _refresh_hosts_and_daemons to use set_health_warning and remove_health_warning helpers
Fixes: https://tracker.ceph.com/issues/44414
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>
2021-10-07 17:08:32 -04:00
Melissa Li
cfd6ad0af2 mgr/cephadm: refactor _update_paused_health to use set_health_warning and remove_health_warning helpers
Fixes: https://tracker.ceph.com/issues/44414
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>
2021-10-07 17:08:32 -04:00
Melissa Li
c3f8fba04a mgr/cephadm: refactor _set_maintenance_health_check to use set_health_warning helper
Fixes: https://tracker.ceph.com/issues/44414
Signed-off-by: Melissa Li <li.melissa.kun@gmail.com>
2021-10-07 17:08:32 -04:00
Patrick Donnelly
d33debc643
qa: fsync dir for asynchronous creat on stray tests
Use the enhanced create_n_files to dedup code. Also split the large test
into three.

Fixes: https://tracker.ceph.com/issues/52606
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-10-07 15:08:34 -04:00
Patrick Donnelly
395d20a2b7
qa: refactor and generalize create_n_files
Few things:

- Allow calling fsync on directory (to support async create kernel).
- Allow immediately unlinking the created file (for stray testing).
- Close any file descriptors created.
- Write unique content (the i variable) to each file.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-10-07 15:08:34 -04:00
Patrick Donnelly
a5675535ba
qa: only set frag confs for workloads
Otherwise these local conf overrides prevent functional testing.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-10-07 15:08:34 -04:00
Patrick Donnelly
5ae32803e5
mds: improve debugging for fragment size check
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-10-07 15:08:31 -04:00
Matt Benjamin
103b6cc355
Merge pull request #43442 from linuxbox2/wip-rgwadmin-logfix
qa/rgw: fix ops log tests to handle non-bucket ops (which are now valid)
2021-10-07 11:04:16 -04:00
Guillaume Abrioux
edc07019c6 cephadm: bootstrap --apply-spec shouldn't enforce :z
There's no need to mount this file with the `:z` option.
read-only access is enough for this file.

Fixes: https://tracker.ceph.com/issues/52855

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-10-07 16:23:07 +02:00
Kefu Chai
263b734ed5
Merge pull request #43448 from rzarzynski/wip-crimson-monmap-moredbgs
crimson, monmap: inform about errors when interacting with DNS

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2021-10-07 22:14:03 +08:00
Adam King
b504012614 cephadm: agent: subtract average time of previous iterations off wait time
We want the agent to actually report metadata at the rate we set
it for. Before this, that rate was just being used as the wait time
between iterations so the actual time between iterations was the
given interval plus the time to gather metadata. Now the time between
reports should actually be roughly the given interval.

Signed-off-by: Adam King <adking@redhat.com>
2021-10-07 10:09:12 -04:00
Kefu Chai
8af0d76ca3 msg/async/ProtocolV2: cast usec to double when converting from nsec
to be consistent with the same logic in V1, and more importantly, it is
more correct.

the change was suggested by Ilya Dryomov <idryomov@redhat.com>.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2021-10-07 21:55:06 +08:00
Patrick Donnelly
2363078751
Merge PR #43231 into master
* refs/pull/43231/head:
	qa: fix promotion test

Reviewed-by: Ramana Raja <rraja@redhat.com>
2021-10-07 09:16:34 -04:00
Guillaume Abrioux
aeae360e25 cephadm: shell --mount shouldnt enforce ':z' option
cephadm shouldn't enforce this option.
For instance, it can be an issue when you try to bindmount a file in /usr

Fixes: https://tracker.ceph.com/issues/52853

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-10-07 14:26:06 +02:00
Cory Snyder
bce34dd686 rgw/rgw_rados: make RGW request IDs non-deterministic
Use a random number vs. incremental counter for first component of request ID.

Fixes: https://tracker.ceph.com/issues/52818
Signed-off-by: Cory Snyder <csnyder@iland.com>
2021-10-07 11:53:12 +00:00
Radoslaw Zarzynski
2bf9d7b0d0 crimson/osd: more interruptions in PeeringEvent.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-10-07 11:33:18 +00:00
Radoslaw Zarzynski
e6d10da26e crimson/osd: implement interruptions in PeeringEvent.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-10-07 11:33:18 +00:00
Radoslaw Zarzynski
28ef412f0c crimson/osd: squeeze one indentation level from PeeringEvent::start().
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-10-07 11:33:18 +00:00
Radoslaw Zarzynski
ad3d47f582 crimson, monmap: inform about errors in MonMap::init_with_dns_srv().
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-10-07 11:20:03 +00:00
Radoslaw Zarzynski
62a4638bf7 crimson, monmap: limit the scope of the seastar namespace inclusion.
More consistency that way. This also enables us to define the usual
`logger()` function without having a clash with `seastar::logger`.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-10-07 11:20:03 +00:00
Venky Shankar
6d6236dc8d mds: skip journaling blocklisted clients when in replay state
When a standby MDS is transitioning to active, it passes through
`replay` state. When the MDS is in this state, there are no journal
segments available for recording journal updates. If the MDS receives
an OSDMap update in this state, journaling blocklisted clients causes
a crash since no journal segments are available. This is a bit hard
to reproduce as it requires correct timing of an OSDMap update along
with various other factors.

Note that, when the MDS reaches `reconnect` state, it will journal
the blocklisted clients anyway.

This partially fixes tracker: https://tracker.ceph.com/issues/51589
which mentions a similar crash but in `reconnect` state. However,
that crash was seen in nautilus.

A couple of minor changes include removing hardcoded function names
and carving out reusable parts into a separate function.

Partially-fixes: https://tracker.ceph.com/issues/51589
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2021-10-07 05:33:06 -04:00
Venky Shankar
05b4772da8
Merge pull request #43384 from bosc0/master
mgr: Add check to prevent mgr from crashing

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2021-10-07 14:10:12 +05:30
Venky Shankar
ff88d7de52 qa: skip internal metadata directory when scanning ceph debugfs directory
kclient patchset

        https://patchwork.kernel.org/project/ceph-devel/list/?series=556049

introduces `meta` directory to add debugging entries. This needs to be filtered
when scanning ceph debugfs directory.

Fixes: https://tracker.ceph.com/issues/52824
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2021-10-07 00:40:13 -04:00
Kefu Chai
8613642834
Merge pull request #41445 from guits/cephadm_quick_patch
cephadm: add daemon_name in daemon description

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-07 06:48:05 +08:00
Kefu Chai
958b22e3ab
Merge pull request #43335 from liewegas/debug-51815
mon,auth: fix proposal (and mon db rebuild) of rotating secrets

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-10-07 06:45:45 +08:00
Kefu Chai
351ebd68f5
Merge pull request #43379 from myoungwon/wip-evict-range
osd: trim corresponding chunks on tier_evict

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-10-07 06:44:33 +08:00
Kefu Chai
0a0cb8eac9
Merge pull request #43378 from ivancich/wip-build-without-mgr
rados: build minimally when "WITH_MGR" is off

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2021-10-07 06:43:25 +08:00
Kefu Chai
90d2e8ad9b
Merge pull request #43360 from jianwei1216/fix-shard-threads-cannot-wakeup-bug
osd: fix shard-threads cannot wakeup bug

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2021-10-07 06:41:54 +08:00
Kefu Chai
b2e806eedb
Merge pull request #43307 from taodd/recv_stamp_fix
msg/async/ProtocolV2: Set the recv_stamp at the beginning of receiving a message

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Dan Hill <daniel.hill@canonical.com>
2021-10-07 06:33:21 +08:00
Kefu Chai
22d9a69f9c
Merge pull request #43380 from myoungwon/wip-dont-promote-dirty
osd: do not promote objects if op is is_dirty

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-10-07 06:31:32 +08:00
Kefu Chai
ea586cdcf5
Merge pull request #43435 from rzarzynski/wip-crimson-pickv2only
crimson/osd: pick only v2 addresses.

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
2021-10-07 06:19:54 +08:00
Ilya Dryomov
711c400264 librbd: clarify rbd_read_from_replica_policy vs older options
If rbd_read_from_replica_policy is set appropriately, setting
rbd_balance/localize_snap_reads or rbd_balance/localize_parent_reads
is redundant.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2021-10-06 23:14:52 +02:00
Ilya Dryomov
b6cffb9ba4 librados: OPERATION_BALANCE/LOCALIZE_READS comment is outdated
With the introduction of min_last_complete_ondisk on the OSD side,
it is safe to use these for non-immutable data.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2021-10-06 23:14:52 +02:00
Matt Benjamin
788da98cde qa/rgw: fix ops log tests to handle non-bucket ops (which are now valid)
After 3863eb89512f1698b8e56f1f1ffc78a6ca8d5826--rgw: permit logging of
list-bucket (and any other no-bucket op-- the radosgw ops-log
contains entries for ops with no associated buckets--e.g., list_buckets.
When examining such a log object in the radosgw_admin task, don't assert
that it has any bucket name.

Fixes: https://tracker.ceph.com/issues/52647

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2021-10-06 16:08:06 -04:00