Commit Graph

127018 Commits

Author SHA1 Message Date
Xuehan Xu
4554f3e3ff crimson/os/seastore: multi-device support
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-10-10 14:22:10 +08:00
Xuehan Xu
98caf058cf crimson/os/seastore: enable SegmentCleaner to hold multiple segmented devices
For now, all segmented devices are treated the same as SEGMENTED. In the future,
there may be different kinds of segmeted devices, like SEGMENTED_NVME, SEGMENTED_SSD,
and even SEGMENTED_SATA. We plan to use a dedicated segment cleaner for each kind of
those devices, and if there are multiple devices of the same kind, they share the same
segment cleaner.

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-10-10 14:21:07 +08:00
Xuehan Xu
0f3fc5af09 crimson/os/seastore: change Scanner to ExtentReader
This commit makes Scanner an extent reader that route read requests to the corresponding
backing devices according to the device ids encapsulated in the segment ids.

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-10-10 14:21:07 +08:00
Xuehan Xu
7de7f95246 crimson/os/seastore: make segment managers device-id aware
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-10-10 14:21:07 +08:00
Xuehan Xu
6f3afd83a8 crimson/os/seastore: introduce device id into segment_id_t
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-10-10 14:21:07 +08:00
Kefu Chai
55138b6d67
Merge pull request #43166 from rzarzynski/wip-crimson-no-subproc
crush, crimson: don't support crush_location_hook as crimson lacks SubProcess

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2021-10-10 09:42:35 +08:00
Kefu Chai
8f8ee5d29e
Merge pull request #43346 from hjwsm1989/fix-52765
tools/ceph-kvstore-tool: fix segfaults when repair the rocksdb

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2021-10-10 09:42:03 +08:00
Kefu Chai
a7ffc0540a
Merge pull request #43452 from adk3798/agent-perf1
cephadm: agent: subtract average time of previous iterations off wait time

Reviewed-by: Michael Fritch <mfritch@suse.com>
2021-10-10 09:41:30 +08:00
Kefu Chai
9a71badb0d
Merge pull request #43401 from romoh/master
src/cephadm: Add support for mariner

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
Reviewed-by: Varsha Rao <rvarsha016@gmail.com>
2021-10-10 09:38:22 +08:00
Kefu Chai
ae783dd81a
Merge pull request #43443 from tchaikov/wip-elapsed-in-double
msg/async/ProtocolV2: cast usec to double when converting from nsec

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: dongdong tao <dongdong.tao@canonical.com>
2021-10-10 09:36:57 +08:00
Yuri Weinstein
c2770fb650
Merge pull request #43343 from trociny/wip-crushdiff-compat
tools/crushdiff: support old format json dump

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-10-08 15:22:49 -07:00
Yuri Weinstein
3104bf46b7
Merge pull request #43324 from neha-ojha/wip-52640
common/options/global.yaml.in: add runtime flag for mon_max_pg_per_osd

Reviewed-by: Vikhyat Umrao <vikhyat@redhat.com>
2021-10-08 15:21:47 -07:00
Yuri Weinstein
cbcfc8a55b
Merge pull request #43232 from Rethan/feat-bluestore-fail-eio
os/bluestore: add option for bluestore fail eio

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2021-10-08 15:21:08 -07:00
Patrick Donnelly
c6c9734a91
Merge PR #43163 into master
* refs/pull/43163/head:
	qa: fsync dir for asynchronous creat on stray tests
	qa: refactor and generalize create_n_files
	qa: only set frag confs for workloads
	mds: improve debugging for fragment size check

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
2021-10-08 14:38:35 -04:00
Kefu Chai
ab706abeef
Merge pull request #43462 from Zhiwei-Dai/wip-test-erasure-code-for-aarch64
qa: support isal ec test for aarch64

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2021-10-08 23:24:13 +08:00
Ilya Dryomov
d66a7ec3ef
Merge pull request #43444 from idryomov/wip-read-from-replica-comment
librados,librbd: make it clear that replica reads are safe for general use

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-10-08 13:43:27 +02:00
Ernesto Puerta
418adb88b2
Merge pull request #43137 from rhcs-dashboard/fix-52480-master
mgr/dashboard: clean-up controllers

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2021-10-08 10:56:30 +02:00
Samuel Just
0f510f05ae
Merge pull request #43459 from cyx1231st/wip-seastore-optimize-record-overhead
crimson/os/seastore: reduce write amplification from record overhead and implement placement hint

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-10-08 01:32:55 -07:00
Dai Zhiwei
eaa385f3da qa: support isal ec test for aarch64
modified:   qa/standalone/erasure-code/test-erasure-code-plugins.sh
	new file:   qa/suites/rados/thrash-erasure-code-isa/arch/aarch64.yaml

Signed-off-by: Dai Zhiwei <daizhiwei3@huawei.com>
2021-10-08 14:37:25 +08:00
Yingxin Cheng
300d27065b crimson/os/seastore: implement placement_hint HOT/COLD/REWRITE
The current strategy is to inline allocate non-REWRITE extents in order
to reduce the record overhead. The observation shows the transactions
from user is usually very small and doesn't deserve to be written in ool
records, which requires at least a block to store the record metadata.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-10-08 11:59:50 +08:00
Yingxin Cheng
05d68ddc15 crimson/os/seastore/epm: misc cleanup
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-10-08 11:59:42 +08:00
Yingxin Cheng
0f6a1571b8 crimson/os/seastore: rename ool_placement_hint_t
To placement_hint_t as the hint is not limited to ool extents.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-10-08 10:59:09 +08:00
Kefu Chai
4ceedec1a4
Merge pull request #43248 from inspur-wyq/wip-doc
doc/rbd/vault.rst: fix typos

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2021-10-08 06:22:54 +08:00
Gregory Farnum
251b6e243d
Merge pull request #43373 from gregsfortytwo/wip-mon-stretch-new-tiebreaker-52126
mon: Allow specifying new tiebreaker monitors

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-10-07 15:16:23 -07:00
Kefu Chai
b126538512
Merge pull request #43331 from adk3798/agent-drain
mgr/cephadm: fix host drain with agent

Reviewed-by: Nizamudeen A <nia@redhat.com>
2021-10-08 06:07:49 +08:00
Patrick Donnelly
d33debc643
qa: fsync dir for asynchronous creat on stray tests
Use the enhanced create_n_files to dedup code. Also split the large test
into three.

Fixes: https://tracker.ceph.com/issues/52606
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-10-07 15:08:34 -04:00
Patrick Donnelly
395d20a2b7
qa: refactor and generalize create_n_files
Few things:

- Allow calling fsync on directory (to support async create kernel).
- Allow immediately unlinking the created file (for stray testing).
- Close any file descriptors created.
- Write unique content (the i variable) to each file.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-10-07 15:08:34 -04:00
Patrick Donnelly
a5675535ba
qa: only set frag confs for workloads
Otherwise these local conf overrides prevent functional testing.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-10-07 15:08:34 -04:00
Patrick Donnelly
5ae32803e5
mds: improve debugging for fragment size check
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2021-10-07 15:08:31 -04:00
Matt Benjamin
103b6cc355
Merge pull request #43442 from linuxbox2/wip-rgwadmin-logfix
qa/rgw: fix ops log tests to handle non-bucket ops (which are now valid)
2021-10-07 11:04:16 -04:00
Kefu Chai
263b734ed5
Merge pull request #43448 from rzarzynski/wip-crimson-monmap-moredbgs
crimson, monmap: inform about errors when interacting with DNS

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2021-10-07 22:14:03 +08:00
Adam King
b504012614 cephadm: agent: subtract average time of previous iterations off wait time
We want the agent to actually report metadata at the rate we set
it for. Before this, that rate was just being used as the wait time
between iterations so the actual time between iterations was the
given interval plus the time to gather metadata. Now the time between
reports should actually be roughly the given interval.

Signed-off-by: Adam King <adking@redhat.com>
2021-10-07 10:09:12 -04:00
Kefu Chai
8af0d76ca3 msg/async/ProtocolV2: cast usec to double when converting from nsec
to be consistent with the same logic in V1, and more importantly, it is
more correct.

the change was suggested by Ilya Dryomov <idryomov@redhat.com>.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2021-10-07 21:55:06 +08:00
Patrick Donnelly
2363078751
Merge PR #43231 into master
* refs/pull/43231/head:
	qa: fix promotion test

Reviewed-by: Ramana Raja <rraja@redhat.com>
2021-10-07 09:16:34 -04:00
Radoslaw Zarzynski
ad3d47f582 crimson, monmap: inform about errors in MonMap::init_with_dns_srv().
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-10-07 11:20:03 +00:00
Radoslaw Zarzynski
62a4638bf7 crimson, monmap: limit the scope of the seastar namespace inclusion.
More consistency that way. This also enables us to define the usual
`logger()` function without having a clash with `seastar::logger`.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-10-07 11:20:03 +00:00
Venky Shankar
05b4772da8
Merge pull request #43384 from bosc0/master
mgr: Add check to prevent mgr from crashing

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2021-10-07 14:10:12 +05:30
Kefu Chai
8613642834
Merge pull request #41445 from guits/cephadm_quick_patch
cephadm: add daemon_name in daemon description

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-07 06:48:05 +08:00
Kefu Chai
958b22e3ab
Merge pull request #43335 from liewegas/debug-51815
mon,auth: fix proposal (and mon db rebuild) of rotating secrets

Reviewed-by: Neha Ojha <nojha@redhat.com>
2021-10-07 06:45:45 +08:00
Kefu Chai
351ebd68f5
Merge pull request #43379 from myoungwon/wip-evict-range
osd: trim corresponding chunks on tier_evict

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-10-07 06:44:33 +08:00
Kefu Chai
0a0cb8eac9
Merge pull request #43378 from ivancich/wip-build-without-mgr
rados: build minimally when "WITH_MGR" is off

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2021-10-07 06:43:25 +08:00
Kefu Chai
90d2e8ad9b
Merge pull request #43360 from jianwei1216/fix-shard-threads-cannot-wakeup-bug
osd: fix shard-threads cannot wakeup bug

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2021-10-07 06:41:54 +08:00
Kefu Chai
b2e806eedb
Merge pull request #43307 from taodd/recv_stamp_fix
msg/async/ProtocolV2: Set the recv_stamp at the beginning of receiving a message

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Dan Hill <daniel.hill@canonical.com>
2021-10-07 06:33:21 +08:00
Kefu Chai
22d9a69f9c
Merge pull request #43380 from myoungwon/wip-dont-promote-dirty
osd: do not promote objects if op is is_dirty

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-10-07 06:31:32 +08:00
Kefu Chai
ea586cdcf5
Merge pull request #43435 from rzarzynski/wip-crimson-pickv2only
crimson/osd: pick only v2 addresses.

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
2021-10-07 06:19:54 +08:00
Ilya Dryomov
711c400264 librbd: clarify rbd_read_from_replica_policy vs older options
If rbd_read_from_replica_policy is set appropriately, setting
rbd_balance/localize_snap_reads or rbd_balance/localize_parent_reads
is redundant.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2021-10-06 23:14:52 +02:00
Ilya Dryomov
b6cffb9ba4 librados: OPERATION_BALANCE/LOCALIZE_READS comment is outdated
With the introduction of min_last_complete_ondisk on the OSD side,
it is safe to use these for non-immutable data.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2021-10-06 23:14:52 +02:00
Matt Benjamin
788da98cde qa/rgw: fix ops log tests to handle non-bucket ops (which are now valid)
After 3863eb89512f1698b8e56f1f1ffc78a6ca8d5826--rgw: permit logging of
list-bucket (and any other no-bucket op-- the radosgw ops-log
contains entries for ops with no associated buckets--e.g., list_buckets.
When examining such a log object in the radosgw_admin task, don't assert
that it has any bucket name.

Fixes: https://tracker.ceph.com/issues/52647

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2021-10-06 16:08:06 -04:00
Ernesto Puerta
df89e6a174
Merge pull request #43256 from rhcs-dashboard/fix-48845-master
qa/mgr/dashboard/test_pool: don't check HEALTH_OK

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2021-10-06 21:49:12 +02:00
Ernesto Puerta
b36766ebd8 mgr/dashboard: clean-up controllers
Fixes: https://tracker.ceph.com/issues/52589
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
2021-10-06 23:44:18 +05:30