Commit Graph

120193 Commits

Author SHA1 Message Date
Ramana Raja
99e34b4ebf
Merge pull request #39572 from kotreshhr/version-auth-metadata-mgr
mgr/volumes: Bump up AuthMetadataManager's version

Reviewed-by: Ramana Raja <rraja@redhat.com>
2021-02-22 23:16:08 -05:00
Sage Weil
471e270674 Merge PR #39630 into master
* refs/pull/39630/head:
	mgr/cephadm/upgrade: fix typo

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2021-02-22 22:28:23 -05:00
Josh Durgin
ebff8322f5 qa: use --newest 100 to find a recent build
master and stable branches are only built a couple times a day now, so
we need to backtrack further.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2021-02-23 00:37:53 +00:00
Josh Durgin
45d7a30be3 qa: deprioritize master and upgrade runs
Keep pacific at high for now. Since teuthology-dispatcher locks nodes
before starting jobs, the upgrade jobs no longer need high priority to
run on more machines than average.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2021-02-23 00:34:45 +00:00
Josh Durgin
bab4dfaaa2 qa: schedule a wider variety of jobs
Use the day of the year instead of compressing everything into
quarters. This lets us get through more breadth of combinations in a
year.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2021-02-23 00:27:51 +00:00
Sage Weil
8e05d9759b mgr/cephadm/upgrade: fix typo
Signed-off-by: Sage Weil <sage@newdream.net>
2021-02-22 18:07:26 -05:00
Josh Durgin
01c8e8279e qa/crontab: adjust scheduled frequencies
smoke: 3x a week on master, weekly on other branches
pacific: every 2 days
master, octopus, nautilus, upgrades: weekly

This should be a net reduction in tests each week. Currently
we schedule far more than the lab can run in a given week.

Try to balance the scheduling over the course of the week.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2021-02-22 21:34:49 +00:00
Sage Weil
33dee7d7bf crush/CrushWrapper: update shadow trees on update_item()
insert_item() already does this, but update_item did not.

Fixes: https://tracker.ceph.com/issues/48065
Signed-off-by: Sage Weil <sage@newdream.net>
2021-02-22 14:21:04 -06:00
Sage Weil
791952cc01 mgr/DaemonServer: include details in 'osd ok-to-stop' output
Include specifics about which pgs are affect, which pgs prevent us from
being ok to stop, etc.

The primary downside I see here is that a success and failure output will
look more similar to a human user

Signed-off-by: Sage Weil <sage@newdream.net>
2021-02-22 13:47:52 -05:00
Patrick Donnelly
a3591378a5
Merge PR #39138 into master
* refs/pull/39138/head:
	qa: valgrind test for cephfs-mirror daemon
	cephfs-mirror: use preforker for daemonizing
	test: adjust sleep time to account for valgrind runs
	cephfs-mirror: gracefully shutdown threads, timers, etc..
	cephfs-mirror: call ceph_release() to cleanup mount alloc
	cephfs-mirror: shutdown filesystem/cluster connections on shutdown
	cephfs-mirror: set init failed flag on FSMirror::init() failure

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-02-22 10:35:32 -08:00
Patrick Donnelly
42803ecd8e
Merge PR #39618 into master
* refs/pull/39618/head:
	doc: fix broken links multimds and kcephfs

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2021-02-22 09:33:47 -08:00
Sage Weil
6f6792cf51 qa/tasks/mgr/dashboard: add wear_level to devices schema
Signed-off-by: Sage Weil <sage@newdream.net>
2021-02-22 10:32:29 -06:00
Kefu Chai
efcf35fd4d
Merge pull request #39620 from rzarzynski/wip-crimson-monc-failure-handling
crimson: multiple fixes for connection failure handling in MonClient

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-02-23 00:32:17 +08:00
Mykola Golub
3a289d43f6 rbd-mirror: reset update_status_task pointer in timer thread
To avoid a time window when m_update_status_task is invalid. If
during this time the cancel_update_mirror_image_replay_status is
called, it may cancel some other's ImageReplayer task, if it
happened to add the task with the same address.

Fixes: https://tracker.ceph.com/issues/49418
Signed-off-by: Mykola Golub <mgolub@suse.com>
2021-02-22 16:22:54 +00:00
Alfonso Martínez
44fbd04296
Merge pull request #39405 from rhcs-dashboard/cherrypy-security
mgr/dashboard: set security headers

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2021-02-22 17:01:39 +01:00
Kyr Shatskyy
1850dd508f ceph.spec.in: use lua53 for openSUSE 15.2
Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
2021-02-22 16:57:27 +01:00
Radoslaw Zarzynski
aa53bc8897 crimson/monc: drop the pending connection on ms_handle_reset().
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-02-22 15:50:58 +00:00
Radoslaw Zarzynski
dc2d4f1c39 crimson/monc: don't assume reopen_session() shall establish active_con.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2021-02-22 15:50:58 +00:00
Jos Collin
efb23dd034
doc: fix broken links multimds and kcephfs
Fixes: https://tracker.ceph.com/issues/49372
Signed-off-by: Jos Collin <jcollin@redhat.com>
2021-02-22 20:38:22 +05:30
Kefu Chai
6b7cc7ecb5 mgr/diskprediction_local: wrap line longer than 100 chars
to appease flake8 and autopep8

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-22 22:44:50 +08:00
Sebastian Wagner
0defae4e65 qa/distro/ubuntu_20.04_podman: Avoid getting asked
... for confic changes interactively

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2021-02-22 14:58:38 +01:00
Sebastian Wagner
17bf44f442 Revert "qa/suites/rados/cephadm: drop 20.04 podman"
This reverts commit dccd586173.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2021-02-22 14:56:33 +01:00
Mykola Golub
d8b02aef52
Merge pull request #39432 from dillaman/wip-49238
rbd-mirror: delay update snapshot mirror image state

Reviewed-by: Mykola Golub <mgolub@suse.com>
2021-02-22 15:16:31 +02:00
Sebastian Wagner
b8dbdb6530 mgr/orch: drop __all__ from __init__.py
`__all__` is only required if you want to modify the `form .. import *`
behavior. We don't need it.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2021-02-22 14:16:27 +01:00
Daniel Pivonka
ae83ffc789 mgr/cephadm: add DaemonDescriptionStatus
Signed-off-by: Daniel Pivonka <dpivonka@redhat.com>
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2021-02-22 14:16:27 +01:00
Mykola Golub
813728ae20 librbd: use on-disk image name when storing mirror snapshot state
Fixes: https://tracker.ceph.com/issues/49115

Signed-off-by: Mykola Golub <mgolub@suse.com>
2021-02-22 12:54:43 +00:00
Mykola Golub
2d2e04e86b test/librbd: extend TestLibRBD.RenameViaLockOwner
To cover the following case:

- Client A has image opened but does not owns the lock.
- Client B renames the image (client A is not aware of it).
- Client A becomes the lock owner.
- Client B requests rename, which is proxied to the client A.

Signed-off-by: Mykola Golub <mgolub@suse.com>
2021-02-22 12:53:38 +00:00
Kefu Chai
34024c84ec mgr/influx: handle ValueError when setting options
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-22 20:51:11 +08:00
Kefu Chai
75bbe6863f mgr: return the error if set_config() fails
in general, `ActivePyModules::set_config()` is called by mgr module when
serving user commands updating module, sometimes if the option is of the
wrong type or invalid value, monitor rejects this request sent by mgr,
but the error info is only logged in the logging message on mgr, but not
returned to user. in this change, `ceph_set_module_option()` and the
underlying methods are updated to return the error to the caller as an
python exception.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-22 20:51:11 +08:00
Nizamudeen A
7f4387d34f mgr/dashboard: Splitting tenant$user when creating rgw user
Fixes: https://tracker.ceph.com/issues/47378
Signed-off-by: Nizamudeen A <nia@redhat.com>
2021-02-22 17:56:54 +05:30
Rishabh Dave
cd760804a0 doc/ceph-fuse: add some options to man page
Add description for options --id and --client_fs to the ceph-fuse manual
and move description for -d closer to -f since both options are similar.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2021-02-22 17:25:03 +05:30
Kefu Chai
cc151a7466 mgr/BaseMgrModule: use without_gil() helper
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-22 19:49:08 +08:00
Kefu Chai
88db7b190a mgr: move GIL helpers to Gil.{h,cc}
Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-22 19:49:08 +08:00
Ilya Dryomov
f6854ac65d krbd: make sure the device node is accessible after the mapping
We have always assumed this to be the case and users' scripts and
orchestration tools have grown to depend on this.  Let's add some
enforcement, prompted by [1]:

  "I am running my Kubernetes worker node inside of an LXC container
   which doesn't benefit from the device node created by the kernel, so
   I'm using udev to create the /dev/rbd* device nodes inside of the LXC
   container."

which, through the unfortunate interaction with ceph-csi rbd plugin,
results in data loss for "volumeMode: Filesystem" PVs because it ends
up recreating the filesystem every time the PV is attached to the pod:

  "When deleting the pod and re-creating it, I can see that the RBD
   image is indeed being reformatted. This seems to be because when
   blkid is being run to check if the image is formatted, the /dev/rbd*
   device has not yet been created by udev. By the time the code gets
   down to running mkfs, the device is there and the damage is done."

[1] https://github.com/ceph/ceph-csi/issues/1820

Fixes: https://tracker.ceph.com/issues/49410
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2021-02-22 11:09:36 +01:00
wangyunqing
c36cf3edd7
doc/radosgw/: fix typos
Signed-off-by: wangyunqing wangyunqing@inspur.com
2021-02-22 17:43:18 +08:00
Ronen Friedman
717c5ddae4 tools/cephfs-mirror: fix a dangling pointer
stringify(snap_id) is ephemeral:

tools/cephfs_mirror/PeerReplayer.cc:916:62: warning: object backing the
pointer will be destroyed at the end of the full-expression

  snap_metadata snap_meta[] = {{PRIMARY_SNAP_ID_KEY.c_str(), stringify(snap_id).c_str()}};

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2021-02-22 08:22:56 +02:00
Sridhar Seshasayee
aebd9b71f2 osd: Fix the OpSchedulerItem cost scaling calculation.
Calculate the scaled cost for an OpSchedulerItem in msec based on the
overall osd capacity (across all shards). Remove logic to cache the
cost of the previous op to re-use for the next transaction as it doesn't
seem to provide any benefit in terms of performance.

Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
2021-02-22 11:26:04 +05:30
Sridhar Seshasayee
2c78cfc30a osd: Refine implementation that sets built-in profile parameters.
mclock clients are categorized into client, background_recovery and
background_best_effort. Each client category is required to specify
the mclock parameters like reservation, weight and limit. Therefore,
a simple structure (ClientAllocs) is introduced that encapsulates the
mclock parameters and is used to store those parameters for each client
category.

Other changes:
 - The "high_client_ops" profile is now set as the default profile
    instead of the "balanced" profile.
 - Replace "Stringify" with std::to_string().

Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
2021-02-22 11:25:57 +05:30
Kefu Chai
92c840475b mgr/zabbix: add desc to commands
it was an oversight in 43802146b0 which
failed to migrate the desc as docstrings.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2021-02-22 10:50:04 +08:00
Sage Weil
2a9d726fbb Merge PR #39328 into master
* refs/pull/39328/head:
	tools/rados: replace omap_key_valid with std::optional.
	tools/rados: replace omap_key_pretty with prettify().
	tools/rados: add support for binary object names.

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
2021-02-21 11:51:54 -05:00
Sage Weil
1303bf252c Merge PR #39394 into master
* refs/pull/39394/head:
	tools/kvstore-tool: implement 'dissect' command
	kv/KeyValueHistogram: make Bluestore's DBHistogram class reusable.

Reviewed-by: Adam Kupczyk <akucpzyk@redhat.com>
2021-02-21 11:51:32 -05:00
Sage Weil
f0795be246 Merge PR #39460 into master
* refs/pull/39460/head:
	msg/async/dpdk: be compatible with DPDK 19.11
	common/Tub: remove Tub.h
	cmake: use NUMA for package name in FindNUMA.cmake
	msg/async/dpdk: define MARKER in .hh
	msg/async/dpdk: use optional<> instead of Tub<>
	compressor/lz4: use optional<> instead of Tub<>
	compressor/lz4: use range-based loop

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2021-02-21 11:50:59 -05:00
Sage Weil
51f5981529 Merge PR #39495 into master
* refs/pull/39495/head:
	mgr/cephadm: remove spec from CephadmDaemonDeploySpec

Reviewed-by: Adam King <adking@redhat.com>
2021-02-21 11:50:36 -05:00
Sage Weil
df3818e77c Merge PR #39535 into master
* refs/pull/39535/head:
	osd: Don't clear m_planned_scrub until scrub finishes successfully

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2021-02-21 11:48:25 -05:00
Jeff Layton
a956c7986f client: add testcase for ceph_ll_lookup_vino
Signed-off-by: Jeff Layton <jlayton@redhat.com>
2021-02-21 10:00:52 -05:00
Jason Dillaman
0dc74ee027
Merge pull request #39478 from orozery/rbd-encryption-doc
doc/rbd: rbd encryption

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2021-02-21 09:15:00 -05:00
Sage Weil
458ea4c30c Merge PR #39504 into master
* refs/pull/39504/head:
	mgr/cephadm/upgrade: restart mgr after mons upgrade to pacific
	mgr: rename config_prefix -> mgr_store_prefix
	mgr: fix config option prefix
	mgr/MgrStandby: simplify weird code
	mgr/cephadm: use get_foreign_ceph_option() instead of 'config get' mon command
	mgr_module: get_foreign_ceph_option(entity, name)
	mgr: maintain a ConfigMap copy
	mon/ConfigMap: factor out parse_key helper
	mon/ConfigMap: fix stray option leak
	mgr: use new kv subscription for mgr/, device/, config/
	mon: allow subscription to kv/config-key data
	mon: convert ConfigKeyService -> KVMonitor

Reviewed-by: João Eduardo Luís <joao@suse.de>
2021-02-21 08:51:49 -05:00
Kefu Chai
7349cf909b
Merge pull request #39348 from xxhdx1985126/wip-crimson-client-request-sequencer
crimson/osd: client request sequencer

Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-02-21 19:53:07 +08:00
Kefu Chai
e08fcf8900
Merge pull request #39552 from smithfarm/wip-49356
rpm: openSUSE Tumbleweed has lua54 now

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2021-02-21 19:50:20 +08:00
Kefu Chai
0eefc80618 crimson/osd: log when obc lock is acquired
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
2021-02-21 19:15:14 +08:00