Commit Graph

119548 Commits

Author SHA1 Message Date
Neha Ojha
6a03b4cec9 qa/suites/rados/perf: pin to 18.04
Caused by a07ca4c5c0

Fixes: https://tracker.ceph.com/issues/49139
Signed-off-by: Neha Ojha <nojha@redhat.com>
2021-02-03 17:32:34 +00:00
Jan Fajerski
d8d8eb0d8a Merge PR #38687 into master
* refs/pull/38687/head:
	ceph-volume: mock VGs with string properties in tests
	ceph-volume: add some flexibility to bytes_to_extents

Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
2021-02-03 18:29:59 +01:00
Kefu Chai
73214a2536
Merge pull request #39186 from tchaikov/wip-mgr-telemetry-ann
pybind/mgr: add typing annotation

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Yaarit Hatuka <yaarit@redhat.com>
2021-02-04 00:40:07 +08:00
Kefu Chai
1533f15b52 crimson/os/seastore: do not capture unused variables
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-04 00:37:00 +08:00
Ilya Dryomov
994d77caa8
Merge pull request #39260 from idryomov/wip-expand-randomized-suite-doc
qa: expand the documentation on the '$' suite feature

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-02-03 17:14:34 +01:00
Adam King
b68675b516 mgr/orchestrator: useful message when no service type in "ceph orch apply"
Right now it just gives a traceback. A message telling users they have
not provided a proper service type would be more useful

Signed-off-by: Adam King <adking@redhat.com>
2021-02-03 11:06:39 -05:00
fullerdj
d4088d8313
Merge pull request #37954 from joscollin/wip-F46226-log-dq-throttle-limit-new
msg,mon,common: log when DispatchQueue throttle limit is reached
2021-02-03 10:42:40 -05:00
Sage Weil
b70a63a917 Merge PR #39069 into master
* refs/pull/39069/head:
	mgr/cephadm/upgrade: tolerate pre-pacific upgrade state
	mgr/cephadm/upgrade: scale down MDS cluster(s) for major version upgrades
	mgr/cephadm: fix capitalization, level; drop elipses of log msgs
	mgr/cephadm/upgrade: match against any repo_digest, not image_id
	cephadm: return repo_digests (plural) in pull/inspect output
	mgr/cephadm: include container_image_digests in inventory
	cephadm: include image_digests list in 'ls' output
	vstart.sh: only extract first container digest
	mgr/cephadm: move release -> major translation to helper
	mgr/cephadm/upgrade: tolerate old upgrade_state.target_versoin
	mgr/cephadm/upgrade: set require-osd-release when done with OSDs
	mgr: add lookup_release_name(int) to mgr interface
	mgr/cephadm: verify container image version after we pull it
	mgr/cephadm: only save version portion of version string
	cephadm: fix 'inspect' and 'pull'
	mgr/cephadm/upgrade: implement N-2 version checks on upgrade start

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2021-02-03 10:38:49 -05:00
Sebastian Wagner
f200cb26cc mgr/cephadm: cleanup daemon keyring on failed daemon deploy
Fixes: https://tracker.ceph.com/issues/48164
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2021-02-03 16:24:25 +01:00
Zac Dover
f4e053ab5f doc/rados: s/realy/really/
This corrects the spelling of "realy" to "really".
Hat-tip to IcePic, who noticed this on 03 Feb 2021.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
2021-02-04 01:04:24 +10:00
Neha Ojha
4308142b09
Merge pull request #39234 from neha-ojha/wip-49064
qa/*/test_envlibrados_for_rocksdb: use osd_client_message_cap to prevent slow requests

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-02-03 06:56:02 -08:00
Casey Bodley
3c3a6519cc cmake: partial revert of BOOST_USE_VALGRIND when ALLOCATOR=libc
the WITH_SYSTEM_BOOST binaries are not built with BOOST_USE_VALGRIND, so
it probably isn't safe to define for the headers only

this flag is needed for teuthology testing, and the shaman builds use
WITH_SYSTEM_BOOST=OFF. so the better fix is to enable WITH_BOOST_VALGRIND
so BuildBoost.cmake will build the libraries with valgrind support and add
-DBOOST_USE_VALGRIND to the necessary targets

this change was merged in https://github.com/ceph/ceph-build/pull/1736

Fixes: https://tracker.ceph.com/issues/48963

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2021-02-03 09:46:42 -05:00
Ilya Dryomov
4138938b15 qa: expand the documentation on the '$' suite feature
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2021-02-03 15:07:54 +01:00
Zac Dover
1dbeeef2dd doc/dev: Remove workbench mentions
This PR removes the "running-tests-in-cloud.rst"
file, which explains how to use ceph-workbench.
ceph-workbench is now deprecated, and the new
Teuthology documentation supplants the information
in the ceph-workbench-related documentation.

This PR also alters the "index.rst" file to remove
a link to "running-tests-in-cloud.rst".

Fixes: https://tracker.ceph.com/issues/49109 (partially)
Signed-off-by: Zac Dover <zac.dover@gmail.com>
2021-02-04 00:06:13 +10:00
Venky Shankar
bf19651d6a doc: cleanup cephfs-mirroring document
Remove references to in-progress feature, remove internal interfaces
section and minor cleanups.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
2021-02-03 08:56:29 -05:00
Sebastian Wagner
e3fa11f40e mgr/cephadm: on ssh connection error, advice chmod 0600
Fixes: https://tracker.ceph.com/issues/46551
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2021-02-03 13:56:09 +01:00
haoyixing
58406fabe8 doc/rados: remove some invalid ms option
Remove some outdated ms options.

Signed-off-by: haoyixing <haoyixing@kuaishou.com>
2021-02-03 20:53:41 +08:00
Kefu Chai
1e19226a18 pybind/mgr/hello: use f-string when appropriate
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-03 20:50:07 +08:00
Kefu Chai
ad2671c17c pybind/mgr/crash: flake8 cleanups
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-03 20:50:07 +08:00
Kefu Chai
7849cfd9b4 pybind/mgr/crash: add type annotations
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-03 20:50:07 +08:00
Kefu Chai
ef6c675ef7 pybind/mgr/crash: define options using Option
more type safe this way

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-03 20:50:07 +08:00
Kefu Chai
8c009e278a pybind/mgr/crash: define commands using CLICommand
more type safe this way

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-03 20:50:07 +08:00
Jeff Layton
615b3df1ab
Merge pull request #38955 from varshar16/wip-doc-update-nfs-ganesha-dynamic
doc/cephfs/nfs: Add rook pod restart note, export and log block example
2021-02-03 07:45:38 -05:00
Lucian Petrut
1c426e96b0 win32*.sh: move debug symbols to separate files
This patch simplifies releasing Windows binaries along with debug
symbols.

By default, we're going to provide minimum debug information (-g1).
The symbols are extracted from the binaries and placed in separate
files in the ".debug" folder, which is used by gdb implicitly.

This is more convenient than having separate versions of the binaries,
with or without debug symbols.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2021-02-03 12:07:41 +00:00
Kefu Chai
17ff5b7c26 pybind/mgr/alerts: define options using Option
more type safe this way

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-03 17:17:24 +08:00
Kefu Chai
0e2711fd64 pybind/mgr/cephadm: PEP8 cleanups
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-03 17:17:24 +08:00
Kefu Chai
db39aba626 pybind/mgr/alerts: use CLIReadCommand to define commands
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-03 17:17:24 +08:00
Kefu Chai
250f9c9e47 pybind/mgr/status: PEP8 cleanups
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-03 17:17:24 +08:00
Kefu Chai
cc960d9383 pybind/mgr/iostat: PEP8 cleanups
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-03 17:17:24 +08:00
Kefu Chai
6b7fde3b4d pybind/mgr/devicehealth: PEP8 cleanups
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-03 17:17:24 +08:00
Kefu Chai
612c218128 pybind/mgr/insights: PEP8 cleanups
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-03 17:17:24 +08:00
Kefu Chai
e69aad6c61 pybind/mgr/prometheus: PEP8 cleanups
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-03 17:17:24 +08:00
Kefu Chai
3a59752ea8 mgr/telemetry: use CLICommand to define commands
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-03 17:17:24 +08:00
Kefu Chai
37a9569e2a mgr/telemetry: silence flake8 warnings
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-03 17:17:24 +08:00
Kefu Chai
c26da642d6 pybind/mgr: define substitution for autopep8 params
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-03 17:17:24 +08:00
Kefu Chai
18da1b8f9c mgr/telemetry: remove duplicated dict entry
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-03 17:17:24 +08:00
Kefu Chai
81e46fb9a6 mgr/telemetry: remove duplicated method
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-03 17:17:24 +08:00
Kefu Chai
964dd93819 mgr/telemetry: add typing annotations
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-03 17:17:24 +08:00
Kefu Chai
25abed3c25 mgr/telemetry: use Option to define options
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-03 17:17:24 +08:00
Kefu Chai
5ca4bc8361 pybind/mgr/hello: add typing annotation
also, use Option and CLIReadCommand to define options and cli
commands. this module serves as a "hello world" example for
developers of mgr modules. so it's important to use the more
convenient and safer way to implement the module

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-03 17:17:24 +08:00
pcuzner
faa93b751d
Merge pull request #38911 from pcuzner/add-cluster-network-to-bootstrap
cephadm: Add cluster network to bootstrap
2021-02-03 21:53:54 +13:00
Lucian Petrut
8d64c7cf42 rbd: propagate rbd-wnbd errors
This change updates the "rbd device" commands, propagating
rbd-wnbd.exe exit codes.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2021-02-03 07:21:12 +00:00
Lucian Petrut
afa7b532b2 rbd: propagate WNBD start errors
This change will propagate the errors that WNBD may return when
spinning up the IO workers.

Also, we'll avoid removing the registry record for failed
non-persistent mappings. Those will be cleaned up when the service
restarts or when explicitly unmapped.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2021-02-03 07:20:49 +00:00
Lucian Petrut
14b78e4538 rbd: improve Windows remap failure handling
At the moment, if an image can't be remapped when the centralized
RBD service starts, the service will stop and already started
daemons will continue running.

This change adds a new option: "--remap-failure-fatal". If set,
when an image can't be remmaped, the service stops AND cleans up
the running daemons. By default, an error will be logged and the
service will continue running.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2021-02-03 07:20:46 +00:00
Lucian Petrut
8b08fb8028 rbd: add image map timeouts on Windows
This change adds configurable timeouts used when starting the centralized RBD
service and mapping images.

This change also fixes an issue where the service would wait indefinitely for
a failed mapping. This issue was caused by the fact that multiple child
processes were inheriting pipe handles.

Note that we can't use timeouts with Windows anonymous pipes, which is why
we're going to use unique named pipes.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2021-02-03 07:19:43 +00:00
Lucian Petrut
1744ba0568 rbd: restrict Windows service exec calls
The centralized Ceph Windows service is responsible of managing
rbd-wnbd daemons. When starting, it's respawns the daemons using the
command line saved in the Windows registry. Also, for new mappings,
the command line is passed through a named pipe.

While writing to the according named pipe and windows registry entries
requires admin privileges, it's better to avoid running arbitrary
commands.

This patch will drop the executable from the commands that the
Ceph service accepts. Instead, it will only accept arguments that
are passed to the the binary that was used to start the service
(rbd-wnbd.exe).

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2021-02-03 07:18:19 +00:00
Lucian Petrut
2455a907e6 rbd: allow non persistent Windows mappings
At the moment, all Windows RBD mappings are persistent, being
recreated when the Ceph service starts.

This change adds the "--non-persistent" flag to allow skipping
certain images.

Note that even for non-persistent mappings, we're still storing
data in the Windows registry, allowing us to retrieve image
details such as the rbd pool as well as the admin socket path.

When the daemons stop or the Ceph service starts, non-persistent
entries are cleaned up from the Windows registry.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2021-02-03 07:18:04 +00:00
Lucian Petrut
6dc45eea20 rbd: fix Windows block count overflow
rbd-wnbd uses uint32_t for storing the image total block count,
which will overflow for >2TB images that are using 512B blocks.

This patch will switch to a uint64_t.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2021-02-03 07:17:56 +00:00
Lucian Petrut
7a2f9903fe rbd: delegate map requests to the Windows service
At the moment, rbd-wnbd daemons are tied to the Windows
session. When the user logs off or the WinRM remote session
terminates, the daemons are killed.

In order to avoid such issues, we'll delegate the daemon
initialization to the centralized Ceph Windows service.

We're using a named pipe for the Ceph service communication.
We're not using Ceph admin sockets for now since unix sockets
aren't available on Windows Server 2016.

Worth mentioning that the Ceph Windows service will restart
registered mappings when the host reboots based on the command
lines saved in the Windows registries. Writing to the registry
key as well as the service named pipe requires admin privileges.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2021-02-03 07:17:52 +00:00
Jason Dillaman
8643b046fb test/rbd-mirror: fix broken ceph_test_rbd_mirror_random_write
It appears that commit 6eb8f30a23 broke the test utility and
its failure was masked by the test case that expected a failure
due to a timeout force-killing the app.

Fixes: https://tracker.ceph.com/issues/49117
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2021-02-02 23:34:18 -05:00