Commit Graph

111004 Commits

Author SHA1 Message Date
Kefu Chai
d5932cb8de rpm: BuildRequires fmt-devel on el8
it's included in EPEL8 now.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-05-09 01:54:55 +08:00
Kefu Chai
6d0b1776ea
Merge pull request #34921 from tchaikov/wip-seastar
crimson: pick up a recent seastar version and cleanups

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2020-05-07 09:30:30 +08:00
Jason Dillaman
fe8a93e4f4
Merge pull request #34851 from yison/rwl-image-writeback-cache-seq8
rbd/cache: Replicated Write Log core codes - retire entries and invalidate

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-05-06 15:56:42 -04:00
Kefu Chai
74fde3a4a0 crimson: avoid using variadic future
it is deprecated by seastar. let's use future<tuple<...>> instead.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-05-07 00:33:48 +08:00
Kefu Chai
7a383b7868 seastar: pick up a recent seastar
to alleviate the pain of keeping in sync with seastar upstream.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-05-07 00:33:48 +08:00
Kefu Chai
f4b33c7b85 crimson/common: friend with do_with_impl()
seastar changed the implementatino of do_with(), so it's do_with_impl()
who tries to up-cast erroratorized future to seastar::future<>.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-05-07 00:33:48 +08:00
Kefu Chai
720f06aaf3 crimson: drop Dispatch::ms_verify_authorizer()
as it's not used at all. also, variadic future was deprecated by
seastar, so let's avoid using it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-05-07 00:33:48 +08:00
Kefu Chai
04063715b5 crimson: s/reactor::cpu_id/seastar::this_shard_id/
the former was deprecated since seastar
02956fd436b413d6bbe562cf91533ada36553e54

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-05-07 00:33:48 +08:00
Yuan Lu
fcae953c29 librbd: add invalidate test case
Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
Signed-off-by: Lu, Yuan <yuan.y.lu@intel.com>
Signed-off-by: Chamarthy, Mahati <mahati.chamarthy@intel.com>
2020-05-06 23:30:55 +08:00
Yuan Lu
8388f1e817 librbd: add invalidate
Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
Signed-off-by: Lu, Yuan <yuan.y.lu@intel.com>
Signed-off-by: Chamarthy, Mahati <mahati.chamarthy@intel.com>
2020-05-06 23:30:48 +08:00
Yuan Lu
48ad3cc26e librbd: retire entries
Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
Signed-off-by: Lu, Yuan <yuan.y.lu@intel.com>
Signed-off-by: Chamarthy, Mahati <mahati.chamarthy@intel.com>
2020-05-06 23:30:38 +08:00
Casey Bodley
a96f9583f4
Merge pull request #34899 from smanjara/wip-dynamic-resharding
qa/rgw: add ssl option in reshard tests

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2020-05-06 11:18:39 -04:00
Sebastian Wagner
b1a572e8fe
Merge pull request #34887 from sebastian-philipp/cephadm-move-specstore
mgr/cephadm: move SpecStore and HostCache to inventoy.py

Reviewed-by: Joshua Schmid <jschmid@suse.de>
2020-05-06 17:10:17 +02:00
Sebastian Wagner
cbab073936
Merge pull request #34898 from matthewoliver/cephadm-iscsi-priv
cephadm: give ceph-iscsi permissions to configfs

Reviewed-by: Boris Ranto <branto@redhat.com>
2020-05-06 17:09:51 +02:00
Kefu Chai
99918ce403 crimson/common: make do_with() noexcept
to be consistent with seasetar change of
85aa178d9539b2c9301d4fb0815596a741fb82c4, which make
`seasetar::do_with()` noexcept

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-05-06 23:07:30 +08:00
Kefu Chai
33af821baa crimson/common: use invoke instead of apply
to be consistent with 9b69c7989da57e69ed1bfc7aa122876e67c3d1fd
in seasetar, in which, seastar::apply() is deprecated.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-05-06 23:07:30 +08:00
Jason Dillaman
d82c23c0c8
Merge pull request #34243 from trociny/wip-rbd-freeze-thaw
librbd: API for quiesce callbacks

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-05-06 10:15:40 -04:00
Jason Dillaman
a43859541b
Merge pull request #34706 from yison/rwl-image-writeback-cache-seq6
rbd/cache: Replicated Write Log core codes - aio_writesame

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-05-06 09:49:43 -04:00
Laura Paduano
cd0a702d1b
Merge pull request #34868 from tspmelo/wip-fix-e2e-role
mgr/dashboard: E2E: Fix role-mgmt.po.ts

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
2020-05-06 13:38:08 +02:00
Laura Paduano
e74468ca09
Merge pull request #34907 from tspmelo/wip-health
mgr/dashboard: Remove unnecessary call in health.component.ts

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
2020-05-06 13:36:44 +02:00
Kefu Chai
5b47191aa6 common/buffer_seastar: include used header
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-05-06 18:45:06 +08:00
Yuan Lu
cc8a8be258 librbd: add aio_writesame test case
Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
Signed-off-by: Lu, Yuan <yuan.y.lu@intel.com>
Signed-off-by: Chamarthy, Mahati <mahati.chamarthy@intel.com>
2020-05-06 13:12:08 +08:00
Yuan Lu
29266637e6 librbd: add aio_writesame
Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
Signed-off-by: Lu, Yuan <yuan.y.lu@intel.com>
Signed-off-by: Chamarthy, Mahati <mahati.chamarthy@intel.com>
2020-05-06 13:12:02 +08:00
Yuan Lu
26eca1b286 librbd: add WriteSameRequest
Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
Signed-off-by: Lu, Yuan <yuan.y.lu@intel.com>
Signed-off-by: Chamarthy, Mahati <mahati.chamarthy@intel.com>
2020-05-06 13:11:56 +08:00
Yuan Lu
4264138deb librbd: add WriteSameLogOperation
Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
Signed-off-by: Lu, Yuan <yuan.y.lu@intel.com>
Signed-off-by: Chamarthy, Mahati <mahati.chamarthy@intel.com>
2020-05-06 13:11:50 +08:00
Yuan Lu
e2764ffe34 librbd: add WriteSameLogEntry
Signed-off-by: Peterson, Scott <scott.d.peterson@intel.com>
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
Signed-off-by: Lu, Yuan <yuan.y.lu@intel.com>
Signed-off-by: Chamarthy, Mahati <mahati.chamarthy@intel.com>
2020-05-06 13:11:41 +08:00
J. Eric Ivancich
7eaa0da864
Merge pull request #34086 from ivancich/wip-deprecate-orphans-find
rgw: deprecate radosgw-admin orphans sub-commands

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2020-05-05 15:51:21 -04:00
Casey Bodley
7cbd433a00
Merge pull request #33546 from matthewoliver/rgw_swift_404_anon
rgw: anonomous swift to obj that dont exist should 401

Reviewed-by: Or Friedmann <ofriedma@redhat.com>
2020-05-05 15:02:34 -04:00
Casey Bodley
e88ecb669a
Merge pull request #34896 from cbodley/wip-rgw-multisite-py3
test/rgw: py3 fixes for multisite tests

Reviewed-by: Yuval Lifshitz <yuvalif@yahoo.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
2020-05-05 14:59:40 -04:00
J. Eric Ivancich
75483d4086 rgw: deprecate radosgw-admin orphans sub-commands
The orphans sub-commands have not been maintained for years and it's
unclear whether they're reliable. Additionally, they store logs on the
cluster, so if a user wants to run it to address a nearly full
cluster, it could push the user over the edge. As a result the
(currently experimental) `rgw-orphan-list` tool (separate executable
rather than a sub-command) has been created, which stores intermediate
results on the local filesystem.

Now all three orphans sub-commands produce a warning (on cerr) that
the command is being deprecated. Additionally all three now require
the --yes-i-really-mean-it command-line option to run.

This also documents the "bucket radoslist" sub-command to
`radosgw-admin` in its corresponding man page, a change that was
omitted from the prior commit that added "bucket radoslist"
documentation.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2020-05-05 14:33:32 -04:00
Patrick Donnelly
ab9db61a08
Merge PR #34911 into master
* refs/pull/34911/head:
	qa: move fsx deps into common cephfs install yaml

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2020-05-05 10:58:32 -07:00
Casey Bodley
d0482f3954
Merge pull request #34287 from ofriedma/wip-ofriedma-anon
rgw: fix rgw tries to fetch anonymous user

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2020-05-05 13:56:23 -04:00
Patrick Donnelly
d791daa732
qa: move fsx deps into common cephfs install yaml
The double install task causes errors during unwind.

Fixes: fe3961fc36
Fixes: https://tracker.ceph.com/issues/45387
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2020-05-05 09:35:44 -07:00
Casey Bodley
9f0ccb5f17 vstart: remove commented-out vault config options
we don't have any vstart automation for rgw encryption with vault. if
anyone wants to do local testing, they can follow instructions in
the documentation

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2020-05-05 12:10:57 -04:00
Casey Bodley
ff5ce2cf14 test/rgw: Cluster.admin() returns output as utf-8 str
fixes the TypeError:

  File "rgw_multi/tests.py", line 1107, in test_bucket_sync_disable
    check_buckets_sync_status_obj_not_exist(zone, buckets)
  File "rgw_multi/tests.py", line 438, in check_buckets_sync_status_obj_not_exist
    if log_list.find(':'+bucket+":") >= 0:
TypeError: argument should be integer or bytes-like object, not 'str'

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2020-05-05 12:10:11 -04:00
Mykola Golub
63e1433440 librbd: make "snapshot create" notification be "async"
After adding quiesce/unquiesce callbacks the "snapshot create" request
may run long time.

Signed-off-by: Mykola Golub <mgolub@suse.com>
2020-05-05 17:02:12 +01:00
Mykola Golub
269f4d233a librbd: API for quiesce callbacks
The callbacks are fired by librbd before/after creating a
snapshot. The callback users like QEMU could attempt to freeze
the FS before allowing librbd to actually perform the snapshot.

Signed-off-by: Mykola Golub <mgolub@suse.com>
2020-05-05 17:02:09 +01:00
Mykola Golub
34bde6542f librbd: don't use boost::variant for image watcher messages
We are reached 20 types limit and are not able to add new message
types.

Signed-off-by: Mykola Golub <mgolub@suse.com>
2020-05-05 17:00:43 +01:00
Yuri Weinstein
c94dc41144
Merge pull request #34894 from yuriw/wip-yuriw-distro-master
qa/tests: replaced ubuntu_latest.yaml with ubuntu 20.04
2020-05-05 08:31:32 -07:00
Tiago Melo
3cb9f0ce47 mgr/dashboard: Remove unnecessary call in health.component.ts
Fixes: https://tracker.ceph.com/issues/45386

Signed-off-by: Tiago Melo <tmelo@suse.com>
2020-05-05 14:10:36 +00:00
Sebastian Wagner
91afff19d0
Merge pull request #34856 from jschmid1/cephadm_osdspec_affinity
mgr/cephadm: add support for proper osd daemon tracking

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-05-05 16:00:12 +02:00
Shilpa Jagannath
06e6013ec1 qa/rgw: add ssl option in reshard tests
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2020-05-05 15:07:59 +05:30
Sebastian Wagner
d347db28bc
Merge pull request #34773 from ricardoasmarques/cephadm-shell-mount
cephadm: Add shell '--mount' option to mount host file or directory

Reviewed-by: Matthew Oliver <moliver@suse.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-05-05 10:57:05 +02:00
Sebastian Wagner
3924fd86b5
Merge pull request #34703 from SUSE/wip-fix-45129
cephadm: Add `chown` to unit.run for adoped simple OSDs

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-05-05 10:55:45 +02:00
Sebastian Wagner
cf60809597
Merge pull request #34778 from mgfritch/cephadm-dd-service-id
mgr/orch: service_id can contain a '.' char (mds, nfs, iscsi)

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-05-05 10:40:24 +02:00
Sebastian Wagner
f6ef61cbe8
Merge pull request #34814 from mgfritch/cephadm-iscsi-apply
mgr/orch: fix `orch apply iscsi`

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-05-05 10:38:26 +02:00
Sebastian Wagner
16c330cba9
Merge pull request #34658 from matthewoliver/cephadm_iscsi_ssl
cephadm: Make ceph-iscsi SSL aware

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-05-05 10:37:07 +02:00
Sebastian Wagner
af2dae1384
Merge pull request #34745 from cybozu/cephadm-fix-the-hang-up-on-creating-encrypted-osd
cephadm: fix the hang-up of cryptsetup on creating encrypted OSD

Reviewed-by: Sage Weil <sage@redhat.com>
2020-05-05 10:00:45 +02:00
Matthew Oliver
9b7dcb894c cephadm: give ceph-iscsi permissions to configfs
The cephadm container mounts the configfs and then bind mounts (-v) it
into the container. Currently the container is not a priviliaged
container which leads to 2 problems:

 1. The container can't insert the iscsi_target_mod kernel module; and
 2. The container can't write to the configfs as that's only writeable
    by root.

We _can_ get around 1, by preloading the kernel module. I.E add it the
systemd unit file. But that doesn't help with 2.

I've tried mounting the configfs with uid and gid options, but configfs
doesn't use them.

If we make the container a priviliged container then magically both 1
and 2 are solved. We don't need to preload the module so that's one less
workaround. But more importantly, configfs can be written to so we can
create ISCSI targets etc.

So that's what this patch does, it makes iscsi containers privileged
containers by setting the CephContainer --priviliged while creating it.

Fixes: https://tracker.ceph.com/issues/45252
Signed-off-by: Matthew Oliver <moliver@suse.com>
2020-05-05 16:17:06 +10:00
Kefu Chai
61ad4e6bb9
Merge pull request #34660 from majianpeng/bluestore-pmemdevice
os/bluestore: avoid offset overflow.

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-05-05 12:59:50 +08:00