Commit Graph

115370 Commits

Author SHA1 Message Date
Kefu Chai
0fc4ed1d91 ciommon/admin_socket: extract find_matched_hook() out
this function is complicated and isolated enough to be a separated
method.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-11 16:57:18 +08:00
Kefu Chai
3eed2d595e common/buffer: use homebrew BUF_OFFSETOF to replace offsetof()
because buffer::raw is not a standard layout type. so `offsetof()`
does not always work. let's use a homebrew macro to calculate the
address of `bptr_storage`.

this change silences the warning like:

src/common/buffer.cc:2156:15: warning: ‘offsetof’ within non-standard-layout type ‘ceph::buffer::v15_2_0::raw’ is conditionally-supported [-Winvalid-offsetof]
 2156 |      offsetof(buffer::raw, bptr_storage));

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-11 12:56:22 +08:00
Jianpeng Ma
429b5b58e5 osd/PG: remove unused function.
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
2020-09-11 11:08:17 +08:00
Adam King
351b92acda cephadm: remove z option from iSCSI dev and configfs mount
selinux complains if you try to deploy iSCSI with the z option
on the /dev and /sys/kernel/config mounts

Signed-off-by: Adam King <adking@redhat.com>
2020-09-10 20:45:28 -04:00
Sebastian Wagner
3713645324
Merge pull request #36935 from mgfritch/cephadm-ceph-service
mgr/cephadm: introduce `CephService` class

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-09-11 00:13:10 +02:00
zdover23
b22cf6da57
Merge pull request #37070 from zdover23/wip-doc-3-hacking-into-docs
doc/dev: add HACKING.rst to Dev Guide

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-09-11 08:04:24 +10:00
Casey Bodley
218932a9a8 rgw: RGWCreateBucket uses s->bucket_exists instead of EEXIST
RGWStore::create_bucket() only returns EEXIST in case of conflicts that
should be returned to the client as errors. the code that applies
changes to metadata should instead use the s->bucket_exists flag

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2020-09-10 15:37:21 -04:00
Casey Bodley
4ac040bfe9
Merge pull request #37089 from cbodley/wip-cmake-osd-fifo
cmake: add cls_fifo as a dependency of osd

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Soumya Koduri <skoduri@redhat.com>
Reviewed-by: Ali Maredia <amaredia@redhat.com>
2020-09-10 14:52:27 -04:00
Rishabh Dave
3f0284f272 qa/cephfs: add methods to read/write on CephFS mounts
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-09-10 23:57:15 +05:30
Rishabh Dave
04ed58fec2 qa/cephfs: modify delete_all_filesystems() in filesystem.py
Modify filesystem.Filesystem.delete_all_filesystems() method to make it
more succinct, move it to class MDSCluster instead and update every call
to it accordingly.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-09-10 23:56:59 +05:30
cao.leilc
e8c683221f rgw : add check for ACL when create existing bucket
Fixes: https://tracker.ceph.com/issues/47028

Signed-off-by: caolei <halei15848934852@163.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2020-09-10 14:04:00 -04:00
Sebastian Wagner
8c47db39b8
Merge pull request #36980 from adk3798/fix-iscsi
cephadm: fix iSCSI unit.run file

Reviewed-by: Matthew Oliver <moliver@suse.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-09-10 20:02:03 +02:00
Casey Bodley
a8975403a6 rgw: add comparison operators for ACL types
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2020-09-10 13:37:35 -04:00
Neha Ojha
9e81f39733
Merge pull request #36747 from varshar16/wip-doc-update-about-priority-check
doc/developer_guide: Update about '--force-priority' flag

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kyr Shatskyy <kyrylo.shatskyy@suse.de>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-09-10 10:29:29 -07:00
Casey Bodley
12c145f22b
Merge pull request #36313 from changchengx/fix_warning
rgw: fix vector index warning

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2020-09-10 12:59:05 -04:00
Casey Bodley
9fbb31d6be cmake: add cls_fifo as a dependency of osd
otherwise cls_fifo is not built by 'make vstart'

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2020-09-10 12:45:54 -04:00
Casey Bodley
701f019a76 rgw: RGWRESTOp no longer tracks separate error code
RGWOp_Bucket_Remove::execute() was storing failures from
bucket->remove_bucket() in op_ret, but left http_ret=0 so we responded
to the client with '200 OK'

to avoid bugs like this, remove the extra http_ret variable and only use
the op_ret from RGWOp

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2020-09-10 12:34:56 -04:00
Mykola Golub
f0c69761c8 qa/workunits/rbd: yet another attempt to improve rbd-nbd unmap
Previously it still could race when unmap_device returned success
because the device was not found in `rbd-nbd list-mapped` (the nbd
device was removed) but the test failed because the process was still
found in the ps table.

Fixes: https://tracker.ceph.com/issues/47394
Signed-off-by: Mykola Golub <mgolub@suse.com>
2020-09-10 17:13:26 +01:00
Kefu Chai
4ae71a068f
Merge pull request #37075 from tchaikov/wip-doc-dev
doc/dev: fix broken link and fix the codeblocks' formatting

Reviewed-by: Zac Dover <zac.dover@gmail.com>
2020-09-10 23:29:30 +08:00
Zac Dover
5760cff6e3 doc/dev: add HACKING.rst to Dev Guide
This commit exposes HACKING.rst through the
docs website. Formerly, HACKING.rst was just
a file that existed at /src/pybind/mgr/dashboard/
HACKING.rst. This commit also updates README.rst
to point to the new location of HACKING.rst.

Fixes: https://tracker.ceph.com/issues/47230
Signed-off-by: Zac Dover <zac.dover@gmail.com>
2020-09-11 01:00:48 +10:00
Sebastian Wagner
57ba572964
Merge pull request #37071 from votdev/cleanup_servicespec_code
cephadm: Remove assignment to member variable in ServiceSpecs

Reviewed-by: Joshua Schmid <jschmid@suse.de>
2020-09-10 16:08:08 +02:00
Sebastian Wagner
e79a7c8c74 Merge PR #37046 into master
* refs/pull/37046/head:
	cephadm: unit.run: create /var/run/ceph/$FSID before doing anything else

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2020-09-10 16:03:48 +02:00
Sebastian Wagner
0fd4c1e068
Merge pull request #36766 from jmolmo/cephadm_log
cephadm: log to file
2020-09-10 15:44:17 +02:00
Kefu Chai
260a702ba9 crimson/common,cmake: use seastar API Level 5
* cmake: bump Seastar_API_LEVEL to "5"
* errorator.h: add invoke() methods to futurize variants, as in seastar
  API level 5, the future's store type is now either monostate or
  a single T, instead of tuple. so it'd be simpler to integrate
  with seastar with invoke() semantic without the boxing-unboxing
  dance.
* errorator.h: drop apply() methods, as they are not used anymore.
* errorator.h: rename and revise tuple2future. as the store type
  is not tuple now, the trait's name is changed accordingly to
  match seastar::internal::future_stored_type

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-10 21:28:55 +08:00
Kefu Chai
285a05d6ea crimson/common: s/result_of/invoke_result/
std::result_of is deprecated in C++17, and removed in C++20, so let's
use std::invoke_result instead

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-10 21:28:55 +08:00
Kefu Chai
d008326706 common/dout: escape formatting string for crimson
crimson uses seastar logging facility for logging. and the latter uses
fmt::format(). currently, we collect the log message into a string and
pass it to log(fmt,...) as "fmt", but seastar/core/sstring.hh defines
the operator<<(ostream&, const vector<T>&) which is a better match than
our the operator<<(ostream&, const vector<T, Allocator>&). and seastar's
operator<<(ostream&, const vector<T>&) uses "{" and "}" to mark the
begin and end of a vector when printing it. and "{}" is also used by
libfmt to enclose its replacement fields. see
https://fmt.dev/latest/syntax.html. so when a vector is printed using
logging facility in crimson, libfmt chokes when trying to parse it as a
format string. so we have some options, like:

- disable seastar's operator<< implementation
- escape the "{}" when writing the vector to the output stream
- print the message as the args, and use "{}" as the fmt.

the 3rd one is the most straightforward, and probably more performant.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-10 21:21:39 +08:00
Kefu Chai
69260a60b7 crimson/osd: use structured binding in loop
use alias for accessing the elements for better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-10 21:21:39 +08:00
Kefu Chai
be840c59da crimson/osd: fix fmt::format() error
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-10 21:21:39 +08:00
Kefu Chai
e7ed50a718 crimson/osd: optimize the fast path of get_recovery_read()
no need to enqueue if the lock can be acquired without waiting for it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-10 21:21:39 +08:00
Kefu Chai
a838ccca8f common/config_proxy: add missing space
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-10 21:21:39 +08:00
Daniel Gryniewicz
0c37322ddb
Merge pull request #36798 from dang/wip-dang-zipper-8
Wip dang zipper 8

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2020-09-10 08:29:00 -04:00
Rishabh Dave
a7eaec9e26 qa/cephfs: modify recreate() in filesystem.py
Modify cephfs.filesystem.Filesystem.recreate() method to delete only the
FS represented by the object instead of deleting the every FS on the
Ceph cluster.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-09-10 17:10:51 +05:30
Rishabh Dave
ee6e2297a1 qa/cephfs: add a method to destroy fs in filesystem.py
And reset_obj_attrs parameter to it so that the caller of the method can
choose to destroy the Ceph FS represented by the object without
disturbing the object attributes.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-09-10 17:10:51 +05:30
Rishabh Dave
9a36a0abd0 qa/cephfs: allow not aborting execution when mount command fails
This commit adds a new argument check_status to mount methods of
KernelMount, FuseMount, LocalKernelMount and LocalFuseMount. When value
of this argument is False, these methods would catch the
CommandFailedError exception and would return a tuple consisting of the
exception itself, and stdout and stderr of the mount command. This
allows reusing these mount methods while running negative tests for
commands.

The name "check_status" is selected so since teuthology's run() and
vstart_runner's run() use a variable with same name for the very same
purpose.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-09-10 17:10:51 +05:30
Rishabh Dave
07e493ffb5 qa/cephfs: allow reusing mount objects and add remount method
This commit introduces following two set of changes -

First, make client keyring path, mountpoint on host FS and CephFS and
CephFS's name attributes of the object representing the mount
and update all the mount object creation calls accordingly. Also,
rewrite all the mount object creation to use keyword arguments instead
of positional arguments to avoid mistakes, especially since a new
argument was added in this commit.

Second, add remount method to mount.py so that it's possible to unmount
safely, modify the attributes of the object representing the mount and
mount again based on new state of the object *in a single call*. The
method is placed in mount.py to avoid duplication.

This change has two leads to two more changes: upgrading interface of
mount() and mount_wait() and upgrading testsuites to adapt to these
change.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-09-10 17:10:51 +05:30
Rishabh Dave
8301c44c70 doc/cephfs: add minor fixes for client-auth doc
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-09-10 17:10:51 +05:30
Rishabh Dave
7de4de1b71 mon/AuthMonitor: make fs authorize assign FS specific MON cap
So far running "fs authorize" subcommand sets "allow r" as MON cap;
change that to "allow <perms> fsname=<fsname>".

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-09-10 17:10:51 +05:30
Rishabh Dave
f93c4c7188 src/mds: allow passing fs names and path in same cap
Allow passing path along with fs names and capspec while creating an MDS
cap. The new syntax looks as follows -

allow rw fsname=<fsname> path=<path>

To provide caps for multiple file systems, pass the same phrase multiple
times separated by commas -

allow rw fsname=<fsname1> path=<path1>, allow rw fsname=<fsname2>
path=<path2>, ...

This commit also makes sure that the old syntax 'allow rw path=<path>'
is supported for backwards compatibility. The old syntax would imply
'allow rw fsname=* path=<path>' and would grant read-write permission for
all FSs containing the path <path>.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-09-10 17:10:51 +05:30
Douglas Fuller
91d5715be6 cephfs: add auth caps based on fs names
Add new auth caps to restrict access to clients based on fsnames. To
specify this, for example:

mds 'allow rw fsname=cephfs1'

This will restrict client access to fs name "cephfs1" only. Messages to
active MDS assigned to any other FSMap will be dropped. Standby MDS not
associated with an FSMap will accept messages from clients. To allow
multiple file systems, create MDS cap as follows -

mds 'allow rw fsname=cephfs1, allow rw fsname=cephfs2'

Fixes: http://tracker.ceph.com/issues/15070
Signed-off-by: Douglas Fuller <dfuller@redhat.com>
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-09-10 17:10:51 +05:30
Douglas Fuller
51af2346fd cephfs: store fs names associated with local MDSMap in MDSRank
Modify MMDSMap to include the name of the file system from the FSMap
when bringing up a new MDS. Store the name in the MDSRank structure for
use in security validation.

Fixes: http://tracker.ceph.com/issues/15070
Signed-off-by: Douglas Fuller <dfuller@redhat.com>
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-09-10 17:10:51 +05:30
Douglas Fuller
811e48d6ff mon/MDSMonitor: add mon auth caps for CephFS names
Add a 'fsname' clause to mon auth caps to restrict a client's view
of the FSMap. Example:

mon 'allow rw fsname=cephfs2'

This would restrict the client's view of the FSMap to the MDSMap for
cephfs2. Any MDS allocated to a different filesystem will be invisible.
Global standby daemons are always visible. To allow multiple CephFSs,
add multiple caps:

mon 'allow rw fsname=cephfs2, allow rw fsname=cephfs2'

Fixes: http://tracker.ceph.com/issues/15070
Signed-off-by: Douglas Fuller <dfuller@redhat.com>
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-09-10 17:10:51 +05:30
Jan Fajerski
4ade3e84ad
Merge pull request #37039 from jan--f/c-v-no-fstrings
ceph-volume: avoid format strings for now
2020-09-10 13:34:44 +02:00
Kefu Chai
576528d08a
Merge pull request #36855 from yaarith/fix-devid-split
mgr/telemetry: fix device id splitting when anonymizing serial

Reviewed-by: Neha Ojha <nojha@redhat.com>
2020-09-10 15:51:20 +08:00
Jan Fajerski
5baeff5ca5 ceph-volume: avoid format strings for now
Fixes: https://tracker.ceph.com/issues/47354

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2020-09-10 09:50:55 +02:00
Kefu Chai
d2ba22a2d1
Merge pull request #37005 from tchaikov/wip-buffer-hyprcombine
common/buffer: check for null before deref

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2020-09-10 15:49:55 +08:00
Kefu Chai
c80d5d47b3 doc/dev: reformat the sample for tree structure of a test suite
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-10 15:13:02 +08:00
Kefu Chai
3569244a50 doc/dev: use appropriate directive for code blocks
* use "code-block:: yaml" for qa suite samples
* use "prompt:: bash $" for shell commands using a non-root user

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-10 15:13:02 +08:00
Kefu Chai
c77382ac0e doc/dev: update supported distros of ceph
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-10 15:13:02 +08:00
Kefu Chai
549952edef doc/dev: drop unused label
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-10 15:13:02 +08:00
Kefu Chai
b17dcc1fc7 doc/dev: fix broken link
otherwise "make check" would be a link pointing to
https://wiki.sepia.ceph.com/doku.php

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-09-10 15:13:02 +08:00