Commit Graph

124193 Commits

Author SHA1 Message Date
Kefu Chai
c64451a940
Merge pull request #41911 from tchaikov/wip-crimson-nbd-cleanup
crimson/tools/store_nbd: better cleanup

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2021-06-18 10:22:21 +08:00
Kefu Chai
e64d8b8ca1 crimson/tools/store_nbd: replace wait_pending with seastar::gate
the inc_pending + promise<> solution is pratically identical to
seastar::gate, so let's use the prepackaged solution instead.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-18 09:05:35 +08:00
Kefu Chai
e4288b68e6 crimson/tools/store_nbd: drop unnecessary seastar::now()
the body of handle_exception() is a synchronous operation, there is no
need to return seastar::now() here.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-18 09:05:35 +08:00
Kefu Chai
a53fcc2eeb crimson/tools/store_nbd: better cleanup
* remove unix domain socket file when cleanup
  so we don't need to remove it manually after each run.
* shutdown input and output streams when cleanup
  so reactor does not watch them anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-18 09:05:35 +08:00
Kefu Chai
7f2e8c366e crimson/tools/store_nbd: s/socket/server_socket/
to prepare for the next commit, which will keep track of the
connected_socket as another member variable.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-18 09:05:35 +08:00
Amnon Hanuhov
ef8e545907
Merge pull request #41916 from AmnonHanuhov/wip-Refactor_test_messenger
test/crimson: Use crimson's make_message in test_messenger
2021-06-18 00:04:40 +03:00
Amnon Hanuhov
77fa71307c test/crimson: Use crimson's make_message in test_messenger
Signed-off-by: Amnon Hanuhov <ahanukov@redhat.com>
2021-06-17 22:49:33 +03:00
J. Eric Ivancich
dfd9f85d5f
Merge pull request #41905 from ivancich/wip-improve-cls-rgw-tracing
rgw: clean-up logging of function entering to make thorough and consistent

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Ali Maredia <amaredia@redhat.com>
2021-06-17 15:37:30 -04:00
J. Eric Ivancich
9982e15209 rgw: clean-up logging of function entering to make thorough and consistent
This provides more thorough and consistent function tracing in CLS/RGW
when logging is set to 10 or higher.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2021-06-17 13:01:17 -04:00
Sebastian Wagner
5b766a86a6
Merge pull request #41903 from liewegas/update-rook-client
rook-client-python: update to update-june-21

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-06-17 16:53:56 +02:00
Ali Maredia
0637672919
Merge pull request #41835 from TRYTOBE8TME/wip-rgw-keycloak-failure-fix
qa/tasks: Keycloak failure fix

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2021-06-17 10:30:28 -04:00
Joseph Sawaya
a4bea005db mgr/rook: comment out osd creation functions
This commit comments out the OSD creation functions in rook_cluster.py
and module.py, since the submodule update has broken them.

Signed-off-by: Joseph Sawaya <jsawaya@redhat.com>
2021-06-17 09:16:29 -04:00
Joseph Sawaya
74bde864b7 mgr/rook: fix some mypy typing errors in rook_cluster.py
This commit fixes some errors caught by mypy in rook_cluster.py. Most of the
errors were caused by the update of the rook-client-python submodule in a previous
commit.

Signed-off-by: Joseph Sawaya <jsawaya@redhat.com>
2021-06-17 09:16:05 -04:00
Joseph Sawaya
68b13c262c mgr/rook: pass zone attribute to CephObjectStore CR when creating rgw
This commit passes the zone attribute to the CephObjectStore CR when
creating a RGW instance using the Rook Orchestrator backend:
`ceph orch apply rgw <rgw-name> --realm=<realm-name> --zone=<zone-name>`

Signed-off-by: Joseph Sawaya <jsawaya@redhat.com>
2021-06-17 09:16:02 -04:00
Ernesto Puerta
4ec6306827
Merge pull request #41856 from rhcs-dashboard/maintenance-bug-fix
mgr/dashboard: Fix 500 error while exiting out of maintenance

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: sebastian-philipp <NOT@FOUND>
2021-06-17 12:44:57 +02:00
Sebastian Wagner
848fd9dc83
Merge pull request #41694 from jmolmo/kcli_cephadm_doc
doc: Add kcli utilization for development environments

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-06-17 11:37:26 +02:00
Kefu Chai
f4eba942f7
Merge pull request #41848 from xxhdx1985126/wip-errorator-parallel_for_each
crimson: errorator parallel_for_each

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-17 12:23:29 +08:00
Kefu Chai
a6d60dc4b2
Merge pull request #41894 from tchaikov/wip-crimson-sigint
crimson/{osd,store_nbd}: handle SIGINT

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-06-17 10:31:27 +08:00
Kefu Chai
647eab4088
Merge pull request #41895 from ceph/wip-cacephcom
MIRRORS: Add ca.ceph.com

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-06-17 08:13:49 +08:00
Nizamudeen A
ea89b60e3f mgr/dashboard: Fix 500 error while exiting out of maintenance
When you add a host in maintenance mode and then exit the maintenance
mode, a 500 server error will popup which will interrupt the whole
exit maintenance process and leave the host in an unknown/offline state.
It happened when I was setting the status of the host through the
HostSpec(). With this change, I am using the enter_maintenance api of
the orch to enable the maintenance.

Fixes: https://tracker.ceph.com/issues/51218
Signed-off-by: Nizamudeen A <nia@redhat.com>
2021-06-17 02:44:46 +05:30
Sage Weil
af622e8cd7 rook-client-python: update to update-june-21
Signed-off-by: Sage Weil <sage@newdream.net>
2021-06-16 16:11:34 -04:00
Sage Weil
3c870731a7 Merge PR #41890 into master
* refs/pull/41890/head:
	doc/cephadm: removing "Octopus" from procedure

Reviewed-by: Sage Weil <sage@redhat.com>
2021-06-16 14:01:26 -04:00
David Galloway
6cc502854c MIRRORS: Add ca.ceph.com
Signed-off-by: David Galloway <dgallowa@redhat.com>
2021-06-16 13:25:15 -04:00
Kefu Chai
a1d5ded993 crimson/osd: use stop_signal from seastar
and disable app_cfg.auto_handle_sigint_sigterm, otherwise app template
handles SIGINT and SIGTERM by itself, and calls app.stop(). but we don't
use this mechinary at all. we use seastar::defer() instead of
seastar::at_exit() for doing graceful shutdown and cleanup.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-17 01:13:24 +08:00
Kefu Chai
11c439d5f0 crimson/tools/store_nbd: update example usage in comment
--total-device-size is not supported any more.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-17 01:13:24 +08:00
Kefu Chai
0689b128e1 crimson/tools/store_nbd: add graceful shutdown support
we could have more sophisticate mechinary for interrupting fio job,
but so far it is able to stop itself if it idle by handling ctrl-C.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-17 01:13:24 +08:00
Yingxin Cheng
3faea72b70 crimson/seastore/nbd: destruct the store before create
Otherwise the store will register the conflicting metrics and result in
double_registration.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2021-06-17 00:23:58 +08:00
Zac Dover
1036edd69a doc/cephadm: removing "Octopus" from procedure
This PR removes "Octopus" from the curl-based installation
procedure.

After we moved on to Pacific, referring to Octopus looks wrong.
It looks wrong because it now is wrong.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2021-06-17 00:34:10 +10:00
Kefu Chai
08661e6f57
Merge pull request #41882 from tchaikov/wip-crimson-int-safty
crimson/osd: guard non-pg-op handling with with_sequencer()

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-06-16 22:09:28 +08:00
Kefu Chai
cd0bb38b2b
Merge pull request #41885 from tchaikov/wip-crimson-os-cleanups
crimson/os: cleanups and reformat

Reviewed-by: Myoungwon Oh <myoungwon.oh@samsung.com>
2021-06-16 20:36:32 +08:00
Kefu Chai
d8ee9e4f68 crimson/osd: reindent
for less indent, hence better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-16 20:30:03 +08:00
Kefu Chai
3e28e640de crimson/osd: wrap line at 80
for better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-16 20:28:10 +08:00
Kefu Chai
728fc4a0f0 crimson/osd: guard non-pg-op handling with with_sequencer()
because we should only ensure the ordering of the requests touching
the objects, the other requests like pgls should not be ordered along
with them. so as the second step, guard the non-pg-op handling with
with_sequencer().

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-16 20:28:01 +08:00
Kefu Chai
84c71bc22f crimson/os: use reference for loop variable
for better performance, also silences the warning like:

../src/crimson/os/seastore/random_block_manager/nvme_manager.cc:444:23: warning: loop variable ‘b’ creates a copy from type ‘const crimson::os::seastore::rbm_alloc_delta_t’ [-Wrange-loop-construct]
  444 |       for (const auto b : alloc_blocks) {
      |                       ^

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-16 19:58:50 +08:00
Sebastian Wagner
6ce6874bae
Merge pull request #41859 from sebastian-philipp/mypy-constrains.txt
global,tox.ini: add mypy-constrains.txt

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Patrick Seidensal <pseidensal@suse.com>
2021-06-16 13:54:31 +02:00
Kefu Chai
11216507aa crimson/os: return seastar::now() in "finally()" block
so finally() is able to identify the return is a future, and discard it
manually.
otherwise the return value will be discarded even the future is marked
[[nodiscard]], hence the C++ compiler warns.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-16 19:09:04 +08:00
Kefu Chai
2e66a657ae crimson/os: remove unnecessary now()
the previous continuation in the chain already returns a future, no need
to hook up another now().

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-16 19:09:04 +08:00
Kefu Chai
bb253d769b crimson/os: remove unnecessary parentheses
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-16 19:09:04 +08:00
Kefu Chai
44863f48aa crimson/os: fix the indent
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-16 19:09:04 +08:00
Kefu Chai
e41f7e3b41 crimson/os: do not capture unused variables
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-16 19:09:04 +08:00
Sebastian Wagner
04e586f90a
global,tox.ini: add mypy-constrains.txt
let's avoid getting new versions of those packages by accident.
Unfortunately this means we have to manually update those
packages regurarly.

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
2021-06-16 12:37:46 +02:00
Kefu Chai
1a44046339 crimson/osd: extract with_sequencer() method out
because we should only ensure the ordering of the requests touching
the objects, the other requests like pgls should not be ordered along
with them, so as the first step, let's extract with_sequencer() method
out, so we can differentiate these two cases with minimal repeatings.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-16 18:29:10 +08:00
Kefu Chai
57aefd9250
Merge pull request #41881 from tchaikov/wip-crimson-alien
crimson: adapt to the new alien API

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-06-16 18:28:33 +08:00
Kefu Chai
e53ea0886f crimson: adapt to the new alien API
since seastar now deprecates the old alien::submit_to() API, and
replaced it with a variant which requires a new parameter of "alien"
instance, let's pick it up, to amortize the cost to catch up with
the seastar master HEAD.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-06-16 17:14:40 +08:00
Xuehan Xu
b1acbb01c4 test/crimson: add test case for errorator parallel_for_each
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-06-16 17:04:10 +08:00
Xuehan Xu
0908b11daf crimson: add parallel_for_each to errorator
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-06-16 17:04:10 +08:00
Igor Fedotov
0a44cd16a9
Merge pull request #41825 from tchaikov/wip-avl-cleanup
os/bluestore/AvlAllocator: specialize _block_picker() and cleanups

Reviewed-by: Igor Fedotov <ifedotov@suse,com>
2021-06-16 12:02:38 +03:00
Sebastian Wagner
2381725447
Merge pull request #41346 from adk3798/enough-mds-upgrade
mgr/cephadm: skip ok-to-stop for mds in upgrade if not enough mds daemons

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-06-16 10:53:33 +02:00
Sebastian Wagner
deadde21af
Merge pull request #41813 from adk3798/repo
cephadm: Use gpg rather than asc key for add-repo

Reviewed-by: Sage Weil <sage@newdream.net>
2021-06-16 10:51:53 +02:00
Sebastian Wagner
c2b7b68598
Merge pull request #41820 from mgfritch/cephadm-not-a-mon-ip
cephadm: raise Error() when unable to bind to an ip

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
2021-06-16 10:51:38 +02:00