Commit Graph

127282 Commits

Author SHA1 Message Date
Aashish Sharma
f7714de294 mgr/dashboard: monitoring: grafonnet refactoring for hosts dashboards
This PR intends to refactor hosts dashboards using grafonnet

Fixes:https://tracker.ceph.com/issues/52777
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2021-10-12 11:05:02 +05:30
Ali Maredia
8ca0012e47
Merge pull request #42052 from TRYTOBE8TME/wip-rgw-dpp-3
rgw: DPP addition 4

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2021-10-12 01:31:00 -04:00
Samuel Just
c349666838
Merge pull request #43181 from xxhdx1985126/wip-extent-placement-manager-multi-device-3
crimson/os/seastore: add multi-device support

Reviewed-by: Samuel Just <sjust@redhat.com>
2021-10-11 22:17:52 -07:00
Xuehan Xu
460f6ca047 crimson/os/seastore: set ExtentPlacementManager::allocated_to before rolling segments
There are circumstances in which a transaction that are supposed to roll the current segment
is invalidated after it finished writing and before it rolls the segment. If we don't set
ExtentPlacementManager::allocated_to in this situation, another transaction can try to write
to the old "allocated_to" position, which would cause an invalid write error

Fixes: https://tracker.ceph.com/issues/52894
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-10-12 12:43:22 +08:00
Samuel Just
c7e262fa83
Merge pull request #43449 from rzarzynski/wip-crimson-interruptible-peeringevent
crimson/osd: implement interruptions in PeeringEvent.

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2021-10-11 18:28:42 -07:00
Samuel Just
caafbf090f
Merge pull request #43483 from cyx1231st/wip-seastore-record-header-fullness
crimson/os/seastore: measure inline/ool record header fullness

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2021-10-11 18:25:35 -07:00
Adam King
1f43d8e618 mgr/cephadm: update CEPHADM_FAILED_DAEMON after receiving agent metadata
otherwise, there can be a period where a daemon will be marked
incorrectly as failed or not failed even though mgr/cephadm knows
it is or isn't in an error state

Signed-off-by: Adam King <adking@redhat.com>
2021-10-11 16:51:55 -04:00
Adam King
f9baa5c71e mgr/cephadm: update CEPHADM_AGENT_DOWN when agent reports in
Right now if an agent considered down reports in the health
warning will reamin until the serve loop runs again. This
change will give better responsiveness on setting/removing
this health warning.

Signed-off-by: Adam King <adking@redhat.com>
2021-10-11 15:07:49 -04:00
Casey Bodley
952c7c844a radosgw-admin: 'reshard list' doesn't log ENOENT errors
ENOENT errors are expected, especially in fresh clusters, before we've
written any entries to the reshard list shards. avoid logging these
non-fatal ERROR messages:

> -1 ERROR: failed to list reshard log entries, oid=reshard.0000000000 marker= (2) No such file or directory

Fixes: https://tracker.ceph.com/issues/52873

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-10-11 14:30:57 -04:00
Joseph Sawaya
1a67d3e559 mgr/rook: apply mds using placement spec and osd_pool_default_size
This commit changes the apply_mds command in the rook orchestrator
to support some placement specs and also sets the replica size according
to the osd_pool_default_size ceph option.

This commit also adds `orch apply mds` to the QA to test if the command
runs.

Signed-off-by: Joseph Sawaya <jsawaya@redhat.com>
2021-10-11 12:06:37 -05:00
Sage Weil
e517882f21 mgr/rook: factor out replica/failureDomain calc
Signed-off-by: Sage Weil <sage@newdream.net>
2021-10-11 11:58:00 -05:00
Sebastian Wagner
100a466e3f
Merge pull request #43471 from liewegas/fix-rook-1-node
qa/suites/orch/rook: replicate across osds for 1-node clusters

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-11 18:47:35 +02:00
Avan Thakkar
d388c5e958 mgr/dashboard: replace Client connections with active-stdby mgrs
Fixes: https://tracker.ceph.com/issues/52121
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
2021-10-11 21:53:23 +05:30
Sebastian Wagner
76b0657e21
Merge pull request #43376 from Daniel-Pivonka/cephadm-apply-health
mgr/cephadm: set HEALTH warnings during apply phase in serve

Reviewed-by: Melissa Li <mingkli@redhat.com>
2021-10-11 16:45:58 +02:00
Sebastian Wagner
1be6dc174f
Merge pull request #43455 from liewegas/qa-nvme-loop
qa: use nvme_loop devices for (some) cephadm tests

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-11 16:43:10 +02:00
Sebastian Wagner
b7ce9d0d6f
Merge pull request #43450 from guits/fix_selinux_apply_spec
cephadm: shell --mount shouldnt enforce ':z' option

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
2021-10-11 16:40:39 +02:00
Sebastian Wagner
146cf96af6
Merge pull request #43458 from mgfritch/cephadm-scp
mgr/cephadm: use `asyncssh.scp` to write remote files

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-10-11 16:38:35 +02:00
Sage Weil
0e7c0a81d8 mgr/rook: specify failureDomain on CephObjectStore CR
Signed-off-by: Sage Weil <sage@newdream.net>
2021-10-11 08:55:18 -05:00
Ernesto Puerta
e7cbf30387
Merge pull request #43473 from rhcs-dashboard/wip-pr_deps-master
.github: check if PR deps are merged/closed

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2021-10-11 14:52:02 +02:00
Ernesto Puerta
bbe2e6505c
Merge pull request #43485 from rhcs-dashboard/fix-pr_triage-master
.github/pr-triage: rename GH token

Reviewed-by: Nizamudeen A <nia@redhat.com>
2021-10-11 14:45:45 +02:00
Ernesto Puerta
2220646c20
.github/pr-triage: rename GH token
Repo projects use GITHUB_TOKEN instead of MY_GITHUB_TOKEN:
https://github.com/srggrs/assign-one-project-github-action/blob/master/entrypoint.sh#L19

Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
2021-10-11 13:05:34 +02:00
Kalpesh Pandya
f20055163c src/rgw: New commit to fix make check
Signed-off-by: Kalpesh Pandya <kapandya@redhat.com>
2021-10-11 15:09:16 +05:30
Kalpesh Pandya
ca4ae7cb9f src/rgw: DPP addition to rgw_kms.cc
Signed-off-by: Kalpesh Pandya <kapandya@redhat.com>
2021-10-11 15:09:16 +05:30
Kalpesh Pandya
7086ccccfb src/rgw: DPP addition to rgw_keystone.cc
Signed-off-by: Kalpesh Pandya <kapandya@redhat.com>
2021-10-11 15:09:16 +05:30
Kalpesh Pandya
e66371fad3 src/rgw: DPP addition to rgw_sts.cc
Signed-off-by: Kalpesh Pandya <kapandya@redhat.com>
2021-10-11 15:09:16 +05:30
Ernesto Puerta
0bcc9d862a
Merge pull request #43472 from rhcs-dashboard/wip-gh_action_project_dashboard-master
.github: add dashboard PRs to Dashboard project

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-10-11 11:06:39 +02:00
Yingxin Cheng
326b79f75c crimson/os/seastore: measure inline/ool record header fullness
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-10-11 16:14:01 +08:00
Yingxin Cheng
a20fa150ed crimson/os/seastore: cleanup record_size_t
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-10-11 16:14:01 +08:00
Yingxin Cheng
d9044d3b39 crimson/os/seastore: rename ool_write_stats_t::overhead_bytes
To header_bytes to be more explicit.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-10-11 16:14:01 +08:00
Anthony D'Atri
0be0772e8a ceph: document rgw_multipart_min_part_size
Signed-off-by: Anthony D'Atri <anthony.datri@gmail.com>
2021-10-10 12:37:09 -07:00
Ilya Dryomov
f5b9646108
Merge pull request #43182 from CongMinYin/fix-writesame-assert
librbd/cache/pwl: initialize number_log_entries

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2021-10-10 17:33:51 +02:00
Samuel Just
5ee7916886 crimson/os/seastore/segment_cleaner: update segment_info_set_t to use segment_map_t
Signed-off-by: Samuel Just <sjust@redhat.com>
2021-10-10 21:06:54 +08:00
Samuel Just
e6a39e218f crimson/os/seastore/segment_cleaner: rework SpaceTracker* to use a simpler segment map type
segment_info_set_t does extra things not relevant to these structures.

Signed-off-by: Samuel Just <sjust@redhat.com>
2021-10-10 14:22:10 +08:00
Samuel Just
9fdc148efe crimson/os/seastore: max_device -> DEVICE_ID_MAX, clarify users
Signed-off-by: Samuel Just <sjust@redhat.com>
2021-10-10 14:22:10 +08:00
Samuel Just
f5aa0eb7b4 crimson/os/seastore/seastore_types.h: clarify segment_id_t visibility, constants
Signed-off-by: Samuel Just <sjust@redhat.com>
2021-10-10 14:22:10 +08:00
Samuel Just
d062f6829b crimson/os/seastore/segment_cleaner: check for null in SpaceTrackerSimple constructor
Since we're passing a sparse pointer vector, we need to check for null
in every user.

Signed-off-by: Samuel Just <sjust@redhat.com>
2021-10-10 14:22:10 +08:00
Samuel Just
3b7176152d crimson/os/seastore/segment_cleaner.cc: adjust whitespace in init_segments()
Signed-off-by: Samuel Just <sjust@redhat.com>
2021-10-10 14:22:10 +08:00
Samuel Just
7677f6dcd3 crimson/os/seastore/segment_cleaner: remove unnecessary template declarations
Signed-off-by: Samuel Just <sjust@redhat.com>
2021-10-10 14:22:10 +08:00
Xuehan Xu
a615b39335 crimson/os/seastore: adjust segment cleaner's space counter calculation
Until now, segment cleaner calculate available/used spaces assuming that only the journal
does all the writes, which is not true any more. This commit make segment cleaner track
all segment managers' empty/open segments and further calculate the various space usage
based on that information

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-10-10 14:22:10 +08:00
Xuehan Xu
4554f3e3ff crimson/os/seastore: multi-device support
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-10-10 14:22:10 +08:00
Xuehan Xu
98caf058cf crimson/os/seastore: enable SegmentCleaner to hold multiple segmented devices
For now, all segmented devices are treated the same as SEGMENTED. In the future,
there may be different kinds of segmeted devices, like SEGMENTED_NVME, SEGMENTED_SSD,
and even SEGMENTED_SATA. We plan to use a dedicated segment cleaner for each kind of
those devices, and if there are multiple devices of the same kind, they share the same
segment cleaner.

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-10-10 14:21:07 +08:00
Xuehan Xu
0f3fc5af09 crimson/os/seastore: change Scanner to ExtentReader
This commit makes Scanner an extent reader that route read requests to the corresponding
backing devices according to the device ids encapsulated in the segment ids.

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-10-10 14:21:07 +08:00
Xuehan Xu
7de7f95246 crimson/os/seastore: make segment managers device-id aware
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-10-10 14:21:07 +08:00
Xuehan Xu
6f3afd83a8 crimson/os/seastore: introduce device id into segment_id_t
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-10-10 14:21:07 +08:00
Kefu Chai
55138b6d67
Merge pull request #43166 from rzarzynski/wip-crimson-no-subproc
crush, crimson: don't support crush_location_hook as crimson lacks SubProcess

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2021-10-10 09:42:35 +08:00
Kefu Chai
8f8ee5d29e
Merge pull request #43346 from hjwsm1989/fix-52765
tools/ceph-kvstore-tool: fix segfaults when repair the rocksdb

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2021-10-10 09:42:03 +08:00
Kefu Chai
a7ffc0540a
Merge pull request #43452 from adk3798/agent-perf1
cephadm: agent: subtract average time of previous iterations off wait time

Reviewed-by: Michael Fritch <mfritch@suse.com>
2021-10-10 09:41:30 +08:00
Kefu Chai
9a71badb0d
Merge pull request #43401 from romoh/master
src/cephadm: Add support for mariner

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
Reviewed-by: Varsha Rao <rvarsha016@gmail.com>
2021-10-10 09:38:22 +08:00
Kefu Chai
ae783dd81a
Merge pull request #43443 from tchaikov/wip-elapsed-in-double
msg/async/ProtocolV2: cast usec to double when converting from nsec

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: dongdong tao <dongdong.tao@canonical.com>
2021-10-10 09:36:57 +08:00
Dai Zhiwei
00e5e5d5cd qa/tasks: replace iterkeys() with keys() in Python 3
Python 2.7 reached the end of its lifetime, the pr fixes teuthology task
error in Python 3.x

Fixes: https://tracker.ceph.com/issues/52878
Signed-off-by: Dai Zhiwei <daizhiwei3@huawei.com>
2021-10-09 23:10:03 +08:00