Commit Graph

134340 Commits

Author SHA1 Message Date
John Mulligan
cf8c57882e cephadm: rename import from cd to _cephadm
The `cd` name is an abbreviation of an obsolete name for cephadm.
As this is not widely known it could be confusing or conflict with
other common terms like `cd` for "change dir" or "compact disc".
Therefore we rename it to something much clearer.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-09-27 14:53:34 -04:00
John Mulligan
343edeb151 cephadm: rename import from cd to _cephadm
The `cd` name is an abbreviation of an obsolete name for cephadm.
As this is not widely known it could be confusing or conflict with
other common terms like `cd` for "change dir" or "compact disc".
Therefore we rename it to something much clearer.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-09-27 14:51:36 -04:00
John Mulligan
7c8af995aa cephadm: fix some nonstandard formatting in fixtures.py
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-09-27 14:42:13 -04:00
John Mulligan
695a46e1d1 cephadm: remove trailing whitespace
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-09-27 14:42:13 -04:00
John Mulligan
fb8120d510 cephadm: lazily import cephadm in fixtures.py
We can avoid creating side-effects in the cephadm module by only importing
the module in the functions that are meant to generate side effects.
Simply importing fixtures and not calling any of the functions now can
not change cephadm by accident.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-09-27 14:42:13 -04:00
John Mulligan
bf963d5ac4 cephadm: remove container_engine argument from test helper function
The with_cephadm_ctx test helper function had an argument that *called*
mock_podman function, thus calling the function at import-time. This can
lead to confusing and unexpected side effects. Because the
container_engine argument is not used by any callers of with_cephadm_ctx
we remove the argument.
All other callers of with_cephadm_ctx that make use of container engine
are already setting `ctx.container_engine = <xyz>` so removing the
argument makes it harder to make the same mistake again in the future.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-09-27 14:42:13 -04:00
John Mulligan
9fbf1546cb cephadm: fix iscsi test_unit_run relying on how cephadm was being imported
The iscsi test_unit_run test case was relying upon a strange side-effect
of how the cephadm module was being imported in fixtures.py. In short,
it expected the podman container_engine not to be a subclass of Podman.

```
(Pdb) p ctx.container_engine
<Mock spec='Podman' id='140676342644256'>
(Pdb) p ctx.container_engine.__class__
<class 'cephadm.Podman'>
(Pdb) p isinstance(ctx.container_engine, Podman)
False
(Pdb) p id(ctx.container_engine.__class__)
94372514199712
(Pdb) p id(Podman)
94372516682112
```

In order to clean up fixtures.py and other imports of cephadm as a
module, we fix the test to expect a mock docker. We choose docker
because it only requires changes the path of the container engine. If we
changed it to a real podman subclass we'd have to change many arguments
in the expected CLI text.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-09-27 14:41:54 -04:00
John Mulligan
d0893e7558 cephadm: use import_cephadm function in test_*.py files
Any current or future chicanery needed to import cephadm is now isolated
in one place, and doesn't need to be repeated in every file.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-09-27 14:41:54 -04:00
John Mulligan
1c6f1eb598 cephadm: use a function to encapsulate loading cephadm as a module
Instead of "open coding" all the cephadm imports let's do it in a
reusable function.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2022-09-27 14:41:54 -04:00
Nizamudeen A
81da1cbfa9 mgr/dashboard: handle the cephfs permission issue in nfs exports
Fixes: https://tracker.ceph.com/issues/48686
Signed-off-by: Nizamudeen A <nia@redhat.com>
2022-09-27 23:52:04 +05:30
li3317
cff6ac5754 osd: break multi condition asserts into multiple asserts
Signed-off-by: Annabel Li <li3317@purdue.edu>

details: break multi condition asserts to improve debugging and help Telemetry group errors correctly
2022-09-27 13:10:17 -04:00
Daniel Gryniewicz
1b72b90d3e
Merge pull request #48256 from dang/wip-dang-zipper-admin-api
RGW - Re-do admin API split

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2022-09-27 12:48:11 -04:00
Igor Fedotov
2d88fdf3da
Merge pull request #48092 from rosinL/fix-57537
os/bluestore: use direct write in BlueStore::_write_bdev_label

Reviewed-by: Igor Fedotov <ifedotov@croit.io>
2022-09-27 19:17:41 +03:00
Ken Dreyer
c7838a6dd1 cmake: add -fstack-protector-strong on win32
mingw on RHEL 9 needs -fstack-protector (which adds -lssp), or it fails
with these errors:

  undefined reference to `__stack_chk_fail'
  undefined reference to `__stack_chk_guard'
  undefined reference to `__strcpy_chk'

Commit 8d35b2ecea mentions that the build
fails with -fstack-protector-strong. Perhaps that only applied to
earlier Ubuntu versions, because Ubuntu Focal can build with
-fstack-protector-strong now.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
2022-09-27 11:21:18 -04:00
Ken Dreyer
3eea1e1750 cmake: set empty CMAKE_DL_LIBS on MinGW
Upstream CMake sets CMAKE_DL_LIBS to an empty value on MINGW, but the
Fedora and RHEL packages override this and set "dl". Re-set it back to
an empty value.

We can revert this if https://bugzilla.redhat.com/2127529 is resolved
for CMake in Fedora and RHEL.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
2022-09-27 11:21:18 -04:00
Ken Dreyer
4954fe19e5 win32: add rhel option
Add an option for building the Windows client on RHEL. I've tested this
on RHEL 9.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
2022-09-27 11:21:18 -04:00
Ken Dreyer
32863d92c4 win32: parse /etc/os-release on all distros
Standardize the way that we determine Ubuntu and SUSE.

This aligns with what we do in install-deps.sh and makes it simpler to
add more distributions later.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
2022-09-27 11:21:18 -04:00
Redouane Kachach
86ba7c5eeb
changed is_empty_certificate check by a positive certs_present bool
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
2022-09-27 14:10:50 +02:00
nsedrickm
8621f6e1be mgr/dashboard: Improve level A accessibility for missing aria labels in tabs and listitems
Replace tab component ul and li elements with navs and ng-containers

Fixes: https://tracker.ceph.com/issues/55872
Signed-off-by: nsedrickm <nsedrick101@gmail.com>
2022-09-27 07:28:48 +01:00
Nizamudeen A
395daf2959
Merge pull request #47876 from torchiaf/fix/43116-2
mgr/dashboard: Add text to empty life expectancy column

Reviewed-by: Pegonzal <NOT@FOUND>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2022-09-27 10:59:40 +05:30
Samuel Just
e6d85efeaa crimson/osd/main: call OSD::stop()
Signed-off-by: Samuel Just <sjust@redhat.com>
2022-09-26 19:35:41 -07:00
Samuel Just
b12feb3d38 crimson/os/FuturizedCollection: use thread_safe_counter
This should work for now, long term we probably want to get
rid of FuturizedCollection entirely from the interface.

Signed-off-by: Samuel Just <sjust@redhat.com>
2022-09-26 19:35:41 -07:00
Samuel Just
2803358525 crimson/osd/heartbeat: drop ownership of messenger instances
Signed-off-by: Samuel Just <sjust@redhat.com>
2022-09-26 19:35:41 -07:00
Samuel Just
4e9dab8003 crimson/osd/pg_shard_manager: add shard_id asserts
Signed-off-by: Samuel Just <sjust@redhat.com>
2022-09-26 19:35:41 -07:00
Samuel Just
615ead85cc src/vstart.sh: add vstart --crimson-smp option
Signed-off-by: Samuel Just <sjust@redhat.com>
2022-09-26 19:35:41 -07:00
Samuel Just
b0b3c09962 crimson/net: Message::conn needs to be a foreign_ptr
There are two main consequences of this:
1. Messages can't be default copy constructed in crimson.  MMonCommand
   seems to be the only user, and we simply add a copy constructor that
   duplicates data portions of the message.
2. We can't casually copy-construct the conn into other structures.
   The main user here is watch/notify.  We use copy() explicitely
   to populate the object_context structures and avoid passing
   ConnectionFRef by value.

Signed-off-by: Samuel Just <sjust@redhat.com>
2022-09-26 19:35:41 -07:00
Samuel Just
5f2c40d0c8 crimson/osd/osd_operations/client_request: remove OSD reference
Capture ShardServices reference for use in put_historic directly.

Signed-off-by: Samuel Just <sjust@redhat.com>
2022-09-26 19:35:41 -07:00
Samuel Just
08758bef9e crimson/osd: move send_incremental_map to OSDSingletonState
Signed-off-by: Samuel Just <sjust@redhat.com>
2022-09-26 19:35:41 -07:00
Samuel Just
f152424631 crimson/net/Connection: let send and keepalive be called from foreign cores
Signed-off-by: Samuel Just <sjust@redhat.com>
2022-09-26 19:35:41 -07:00
Samuel Just
ea46c3e6c3 crimson/osd/shard_services: check core on each PerShardState method
Signed-off-by: Samuel Just <sjust@redhat.com>
2022-09-26 19:35:41 -07:00
Samuel Just
5451c93a27 crimson/osd/osd_operations/client_request: fix whitespace on do_process
Signed-off-by: Samuel Just <sjust@redhat.com>
2022-09-26 19:35:41 -07:00
Samuel Just
cb657b4221 crimson/osd: update_heartbeat_peers can iterate pgs syncronously
OSDSingletonState has a local inventory of all pgids.

Signed-off-by: Samuel Just <sjust@redhat.com>
2022-09-26 19:35:41 -07:00
Samuel Just
a9684d4665 crimson/osd/osd_operations: use foreign_ptr for conn
Signed-off-by: Samuel Just <sjust@redhat.com>
2022-09-26 19:35:41 -07:00
Samuel Just
b692dfbad7 vstart: echo osd arguments
Signed-off-by: Samuel Just <sjust@redhat.com>
2022-09-26 19:35:41 -07:00
Samuel Just
3c632e45bb crimson/osd/osd_operations/peering_event.cc: whitespace fix
Signed-off-by: Samuel Just <sjust@redhat.com>
2022-09-26 19:35:41 -07:00
Samuel Just
998cb8c141 crimson/osd: remove compound_peering_request
The state shared between sub events creates problems for multicore.  The
only user is MOSDPGCreate2, so the optimization isn't really worth
salvaging.

Signed-off-by: Samuel Just <sjust@redhat.com>
2022-09-26 19:35:41 -07:00
Samuel Just
6ea21eddba crimson/common/operation: release pipeline stages on the core they are on
Otherwise, tasks waiting on the stage will wake up on the wrong core.
Later, we may choose to statically enable this behavior only for stages
that can actually span cores, but this is ok for now.

Signed-off-by: Samuel Just <sjust@redhat.com>
2022-09-26 19:35:41 -07:00
Samuel Just
752e8cf8f3 crimson/osd: move perf counter ownership to OSDSingletonState
Later, we'll want to create a properly sharded implentation that avoids
barriers.

Signed-off-by: Samuel Just <sjust@redhat.com>
2022-09-26 19:35:41 -07:00
Samuel Just
ce65d7e6fa crimson/osd: shard PerShardState across cores
Signed-off-by: Samuel Just <sjust@redhat.com>
2022-09-26 19:35:41 -07:00
Samuel Just
582bbd5ee1 crimson/osd: move hb_stamps to PerShardState
We may later need to syncronize this across cores, perhaps during tick.
This should work for now, however.

Signed-off-by: Samuel Just <sjust@redhat.com>
2022-09-26 19:35:41 -07:00
Samuel Just
b52990c626 crimson/osd: replace global tid with unique core-local tids
We don't really want a global counter here if we can avoid it.  Instead,
assign tids with core-local counters prefixed with the core id.  We
continue to ensure that tids are unique within an osd boot, but lose
the property that sucessive tids on different cores are ordered.  I
don't see anything relying on that property, however, so this should be
fine.

Signed-off-by: Samuel Just <sjust@redhat.com>
2022-09-26 19:35:41 -07:00
Samuel Just
be1371762a crimson/osd: remove explicit pg counter
The main user of num_pgs was get_target_pg_log_entries(), but we
don't really want to be accessing a global counter there if we
can avoid it.  Instead, compute a core-local target and use the
core-local pg count.

Signed-off-by: Samuel Just <sjust@redhat.com>
2022-09-26 19:35:41 -07:00
Samuel Just
1297b49c93 crimson/osd: clean up PerShardState/OSDSingletonState ownership/access
- move heartbeat messenger ownership to OSD
- move OSDSingletonState/ShardServices initialization to
  PGShardManager::start() method
- add accessor OSD::get_shard_services(), remove OSD::shard_services
  reference
- accessor/forwarding cleanups within PGShardManager

Signed-off-by: Samuel Just <sjust@redhat.com>
2022-09-26 19:35:41 -07:00
Samuel Just
12042dcbc7 crimson/osd: move store responsiblity to PerShardState
Signed-off-by: Samuel Just <sjust@redhat.com>
2022-09-26 19:35:41 -07:00
Samuel Just
2b5c471913 crimson/os: consolidate context handling in FuturizedStore
Context handling is pretty uniform accross all implementations,
may as well do it in the same place.

ShardedStoreProxy would need to handle it otherwise, since
callbacks need to be handled on the core do_transaction is
invoked on.

Signed-off-by: Samuel Just <sjust@redhat.com>
2022-09-26 19:35:41 -07:00
Samuel Just
2c1f213bfa crimson/os/futurized_store: create ShardedStoreProxy
For now, FuturizedStore implementations assume that methods are invoked
on core 0.  Later, we'll adapt each implementation to intelligently
support invocation on any pg core.  Until then, this wrapper converts
the existing implementations to a safe, if not particuarly performant,
proxy behavior.

AlienStore should be safe as is.

Signed-off-by: Samuel Just <sjust@redhat.com>
2022-09-26 19:35:38 -07:00
Samuel Just
2b2f5b7009 crimson/osd: adjust get_pg_stats() and for_each_pg() to return futures
Signed-off-by: Samuel Just <sjust@redhat.com>
2022-09-26 19:33:04 -07:00
Samuel Just
a7296e5ea2 crimson/osd: refactor pg management for multicore
OSDSingletonState will now only be responsible for the spg_t->core
mapping for pgs, the individual PerShardState's will hold local
PGMap's.  PG management operations are now proxied from PGShardManager
to PerShardState.  Subsequent patches will shard PerShardState.

Signed-off-by: Samuel Just <sjust@redhat.com>
2022-09-26 19:33:01 -07:00
Samuel Just
e94a12a8e3 crimson/osd: modify start_pg_operation to return op_id
The op will actually be run generally on a different core, so we don't
want to be returning a reference to it.

Signed-off-by: Samuel Just <sjust@redhat.com>
2022-09-26 19:31:16 -07:00
Samuel Just
25a4612745 crimson/osd/pg_map: add PGShardMapping
Signed-off-by: Samuel Just <sjust@redhat.com>
2022-09-26 19:31:13 -07:00