Commit Graph

130856 Commits

Author SHA1 Message Date
Radosław Zarzyński
3e99e526c0 crimson/osd: switch a stage of ClientRequest to new blocking infra
This is a debut of the new blocking & tracking infrastructure
using the type-aware `BlockerT::BlockingEvent::Trigger<T>`.

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
2022-05-05 04:06:31 +02:00
Radosław Zarzyński
c86c231dcc crimson/osd: introduce PhasedOperationT to deduplicate PipelineHandles
This new class in the hierarchy will be handy when it will come to
introduce helpers for deailing with entering a pipeline stage.

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
2022-05-05 04:06:31 +02:00
Radosław Zarzyński
2676e395ac crimson/osd: switch all OSD operations to TrackableOperationT
Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
2022-05-05 04:06:31 +02:00
Radosław Zarzyński
ae08ac0278 crimson/osd: get rid of unnecessary private in ClientRequest
Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
2022-05-05 04:06:31 +02:00
Radosław Zarzyński
5c83589ae5 crimson/osd: switch all ClientRequest-related pipeline stages to the new infra
Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
2022-05-05 04:06:31 +02:00
Radosław Zarzyński
f5292e6f81 crimson/osd: pipeline stage classes can be bases for CRTP now
Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
2022-05-05 04:06:31 +02:00
Radosław Zarzyński
61147363c2 crimson/osd: pipeline stage classes derive from BlockerT now
Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
2022-05-05 04:06:31 +02:00
Radosław Zarzyński
2dc3770f7a crimson/osd: document OperationThrottler's behavior towards op's blocker list.
Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
2022-05-05 04:06:31 +02:00
Radosław Zarzyński
846ed08359 crimson/osd: simplify OperationThrottler::with_throttle()
Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
2022-05-05 04:06:31 +02:00
Radosław Zarzyński
f4a039278f crimson/osd: move with_blocking_future from Operation to OperationT.
Buildability of this commit proves all users of `with_blocking_future()`
have the concrete operation type which is necessary on the the way
to the compile-time defined op's blocker registry.

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
2022-05-05 04:06:31 +02:00
Radosław Zarzyński
332a0d4de0 crimson/os/seastore: migrate OrderingHandle away from blocking_future
Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
2022-05-05 04:06:31 +02:00
Radosław Zarzyński
e054d79fe9 crimson/osd: OperationThrottler derives from BlockerT instead of Blocker.
One another step closer. See the previous' commit description.

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
2022-05-05 04:06:31 +02:00
Radosław Zarzyński
c35c49908c crimson/osd: make OperationThrottler aware about exact type of operation
This is a step towards making op's blocker registry static and
compile-time checked.

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
2022-05-05 04:06:31 +02:00
Radosław Zarzyński
6bafcac240 crimson/osd: make the OperationThrottler::with_throttle private
Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
2022-05-05 04:06:31 +02:00
Radosław Zarzyński
d1885d84dd crimson/osd: add types PG and OSD-hosted OSDMapGate instances.
`ClientRequest` is going to hold all tracking events as a `std::tuple`
instance and access them through the type-related variant of `std::get`.
As two instances of `OSDMapGate` are used there, a need to differentiate
at the type level arises.

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
2022-05-05 04:06:31 +02:00
Radosław Zarzyński
3c86c04372 crimson/common: add basic infrastructure for op tracking
Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
2022-05-05 04:06:31 +02:00
Radosław Zarzyński
d7716d9d4c crimson/common: abstract from staticness of BlockerT::type_name.
Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
2022-05-04 11:38:25 +02:00
Venky Shankar
aaa3fb120a
Merge pull request #37655 from rishabh-d-dave/vr-run-multiple-cmds
vstart_runner: accepts multiple commands as str too

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-05-04 15:03:30 +05:30
Venky Shankar
abcd7ba717
Merge pull request #45473 from neesingh-rh/fix_50470
cephfs-top: Multiple filesystem support

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-05-04 15:00:15 +05:30
Venky Shankar
9682214b3f
Merge pull request #45776 from neesingh-rh/fix_48863
cephfs-shell: chmod should be allowed for changing all mode bits.

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-05-04 14:59:10 +05:30
Venky Shankar
bfe960df95
Merge pull request #45974 from rishabh-d-dave/shell-unrecognized-cmds
cephfs-shell: exit with non-zero on getting unrecognized cmd

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-05-04 14:57:40 +05:30
Samuel Just
59d52091b3
Merge pull request #46135 from athanatos/sjust/wip-vstart-asok
vstart.sh: default CEPH_ASOK_DIR to out/

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2022-05-03 20:57:51 -07:00
Samuel Just
0c19344c5b
Merge pull request #46137 from athanatos/sjust/wip-crimson-typos
crimson: misc typos

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2022-05-03 20:57:16 -07:00
Samuel Just
369ec99cc0
Merge pull request #46134 from athanatos/sjust/wip-55326
crimson/common/format: remove ceph::coarse_real_clock format speciali…

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2022-05-03 20:56:14 -07:00
Samuel Just
bbfce6c665 crimson/common/format: remove ceph::coarse_real_clock format specialization
...
..., value=..., loc=...) at /usr/include/fmt/ostream.h:114
...

Introduced: e978412524
Fixes: https://tracker.ceph.com/issues/55326
Signed-off-by: Samuel Just <sjust@redhat.com>
2022-05-04 00:20:29 +00:00
Vikhyat Umrao
1767cd4856
Merge pull request #46128 from vumrao/wip-vumrao-quincy-release-note-update
docs/releases: update quincy release notes tracker 55383 known issue
2022-05-03 10:59:35 -07:00
Vikhyat Umrao
d59e376b13 docs/releases: update quincy release notes tracker 55383 known issue
Signed-off-by: Vikhyat Umrao <vikhyat@redhat.com>
2022-05-03 09:38:21 -07:00
Samuel Just
628120df10
Merge pull request #46079 from ronen-fr/wip-rf-log-mock
common: make LogChannel mockable

Reviewed-by: Samuel Just <sjust@redhat.com>
2022-05-03 08:02:34 -07:00
Daniel Gryniewicz
94f21e5aea
Merge pull request #46115 from dang/wip-dang-split-init
RGW - Split RGWRados initialization

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Soumya Koduri <skaduri@redhat.com>
Reviewed-by: Mark Kogan <mkogan@redhat.com>
2022-05-03 08:02:51 -04:00
Anthony D'Atri
844f849148
Merge pull request #46066 from bluikko/patch-4
doc/cephadm/services: Add missing ceph command to orch apply commands
2022-05-02 22:54:06 -07:00
zdover23
42dd40365b
Merge pull request #46109 from bluikko/pgcalc_links
doc/rados: fix pgcalc links

Reviewed-by: Zac Dover <zac.dover@gmail.com>
2022-05-03 08:39:12 +10:00
Samuel Just
a91137e0c6 vstart.sh: default CEPH_ASOK_DIR to out/
Signed-off-by: Samuel Just <sjust@redhat.com>
2022-05-02 21:51:54 +00:00
Samuel Just
3571daad19 crimson/common/interruptible_future: fix typo
Signed-off-by: Samuel Just <sjust@redhat.com>
2022-05-02 21:47:12 +00:00
Samuel Just
0fabd8493f crimson/common/operation: fix comment formatting
Signed-off-by: Samuel Just <sjust@redhat.com>
2022-05-02 21:47:12 +00:00
Samuel Just
8f76ed97e8 crimson/admin/osd_admin: fix comment spacing
Signed-off-by: Samuel Just <sjust@redhat.com>
2022-05-02 21:47:12 +00:00
Ali Maredia
d634f0690b
Merge pull request #44494 from mdw-at-linuxbox/wip-master-mankey
rgw/crypt: sse-s3 changes

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2022-05-02 16:57:55 -04:00
Adam King
3f5e9454d6
Merge pull request #45843 from ajarr/wip-mgr-nfs-output-message
mgr/nfs: clarify in the output message

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
2022-05-02 13:40:36 -04:00
Yuri Weinstein
83041da20d
Merge pull request #46095 from aclamk/wip-aclamk-unbounded-wholespace-iterator
kv/RocksDBStore: Remove ability to bound WholeSpaceIterator

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Cory Snyder <csnyder@iland.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Mark Nelson <mnelson@redhat.com>
2022-05-02 10:14:33 -07:00
Daniel Gryniewicz
c9fdc3260a RGW - Split RGWRados initialization
The inter-connectedness of RadosStore and RGWRados resulted in a
segfault during RGWRados::init_complete due to the rados pointer not
being set in RadosStore yet.

Split the calls to RGWRados::initialize and RGWRados::init_complete, so
that we can set up RadosStore between them, allowing the services
created in RGWRados::init_complete to access the RadosStore.

Fixes: https://tracker.ceph.com/issues/55512

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2022-05-02 12:01:09 -04:00
Rishabh Dave
acc923c8a4 qa/vstart_runner: add a helper method to run commands
It's more convenient to not to pass stdout, stdin, and stderr every
single time.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2022-05-02 20:25:19 +05:30
Rishabh Dave
f12536fed1 qa/vstart_runner: don't use os.kill for passing SIGKILL
vstart_runner.py is written assuming that it can run commands with
superuser privileges whenever possible and vstart_runner.py is meant to
be executed without sudo.

So, it's better kill a process using "sudo kill -9 <PID>", instead of
using os.kill() because os.kill() can't kill process launched with
superuser privileges.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2022-05-02 20:25:19 +05:30
Rishabh Dave
8d4ef76923 qa/vstart_runner.py: add a note about omitting sudo
About the commit date: this commit got dropped from the patch series
during some PR branch update but is added back now.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2022-05-02 20:25:19 +05:30
Rishabh Dave
43b170ab18 qa/vstart_runner: get rid of "exec sudo" from "ceph -w" cmd args
Passing "exec sudo" to "ceph -w" caused "Ceph API test" CI job to fail.
Error was not related to this tracker issue but the code added for it
is reversed now in this commit. The tracker issue -
https://tracker.ceph.com/issues/49644.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2022-05-02 20:25:19 +05:30
Rishabh Dave
a67d500ee1 qa/vstart_runner: keep cmd args as passed by user as well
We convert all cmd args to str and pass bash functions along to override
certain arguments in those command arguments. Let's save cmd args
without those bash functions since they can be useful later (for
example, printing cmd args in logs, which is the case in this patch.)

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2022-05-02 20:25:19 +05:30
Rishabh Dave
8fc635c28c qa/vstart_runner: copy few more note points from dev guide
The intention behind copying these note points is to document the
behaviour of vstart_runner.py inside vstart_runner.py as well so that
developer don't miss it out while working on it.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2022-05-02 20:25:19 +05:30
Rishabh Dave
7d53423439 qa/vstart_runner: override adjust-ulimits, ceph-coverage & sudo
Overridding commands is much better than deleting these commands from
command argument string using Python since, unlike deleting, overridding
doesn't require parsing. A note has been added for this to
vstart_runner.py's module docstring and to Ceph Developer's Guide
document.

Since functions don't work with sh shell, to make overriding work
vstart_runner.py will use bash shell here onwards.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2022-05-02 20:25:17 +05:30
Rishabh Dave
27eaacd98a qa/vstart_runner: accept multiple commands as str
Convert all command arguments to str from list, update checks and
adjustments performed on command arguments accordingly and update
documentation to include warnings about some critical parts of
vstart_runner.py and update tasks.cephfs.mount.MountCephFS.run_shell().

Fixes: https://tracker.ceph.com/issues/47849
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2022-05-02 20:24:08 +05:30
Yuval Lifshitz
ba0bf61aac
Merge pull request #45842 from yuvalif/wip-yuval-const-notifications
rgw/notifications: make the interfaces const when possible
2022-05-02 17:01:52 +03:00
Ronen Friedman
af84a842d7
Merge pull request #46100 from ronen-fr/wip-rf-80cls-v1
osd/scrub: reformat scrub files to 80 cols

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Reviewed-by: Aishwarya Mathuria <amathuri@redhat.com>
2022-05-02 16:16:04 +03:00
Venky Shankar
ff144995a8
Merge pull request #45889 from lxbsz/wip-mds-clienup
mds: remove duplicated code

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2022-05-02 15:05:27 +05:30