Commit Graph

116063 Commits

Author SHA1 Message Date
Joao Eduardo Luis
c148a3cde5
mon: have 'mon stat' output json as well
Fixes: https://tracker.ceph.com/issues/46816

Signed-off-by: Joao Eduardo Luis <joao@suse.com>
2020-10-13 08:31:17 +00:00
Mykola Golub
c9d5c492df
Merge pull request #37581 from dillaman/wip-librbd-exclusive-lock-shutdown
librbd: avoid failing IO with -ESHUTDOWN when disabling exclusive-lock

Reviewed-by: Mykola Golub <mgolub@suse.com>
2020-10-13 10:44:58 +03:00
Dan van der Ster
6823d8fb61 mds: account for closing sessions in hit_session
While stopping an mds we can reply to a request while all client
sessions are closing. We shouldn't assert in this case.

Fixes: https://tracker.ceph.com/issues/47833
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
2020-10-13 09:08:12 +02:00
Kefu Chai
8ae611e584
Merge pull request #37613 from tchaikov/wip-cls-osd-cleanup
cls, osd: cleanups

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2020-10-13 14:23:29 +08:00
Yuval Lifshitz
9e5f319169
Merge pull request #37537 from yuvalif/missing_brackets_in_rgw_file
rgw/file: missing brackets around if statement
2020-10-13 08:06:31 +03:00
Yuval Lifshitz
2b034f2eb2
Merge pull request #37538 from yuvalif/potential_crash_in_kafka
fix potential crash in rgw_kafka
2020-10-13 08:04:18 +03:00
Yuval Lifshitz
426e6f526c
Merge pull request #37599 from yuvalif/remove_doxyfile
cmake: remove Doxyfile as doxygen run via make
2020-10-13 08:03:39 +03:00
Yuval Lifshitz
c6e8fa39cd
Merge pull request #37534 from yuvalif/fix_ownership_in_role_applier
rgw/auth: fix copy&paste bug in RoleApplier::is_owner_of
2020-10-13 08:02:40 +03:00
Venky Shankar
1ec84603d0 test: add test for setting ceph mirror virtual xattr
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2020-10-13 00:29:38 -04:00
Venky Shankar
aff632a67a tests/pybind/cephfs: cleanup xattrs before starting tests
Some xattr tests do not fully cleanup set xattrs. Subsequent
tests may expect xattrs to be absent during the test, such as
setting an xattr and then removing followed by checking if the
xattr list to be empty. This may fail if earlier tests do not
cleanup xattrs, especially for root.

So, cleanup xattrs on root before starting tests. Other directories
are removed anyway, so we do not have to bother about those.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
2020-10-13 00:29:38 -04:00
Venky Shankar
7341863a7d client: filter ceph.* xattrs from listing
Since xattr_map has entries for xattrs in ceph namespace.
Such xattrs are accessible via getxattr().

Signed-off-by: Venky Shankar <vshankar@redhat.com>
2020-10-13 00:29:38 -04:00
Venky Shankar
526e4ee5b4 client: force an attr fetch for ceph xattrs
Without this fetching a ceph.* xattr which was just set
returns -ENODATA.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
2020-10-13 00:29:38 -04:00
Venky Shankar
23a3f86598 client: changes for ceph.mirror.info xattr
Leverage the vxattr helper to handle this xattr request.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
2020-10-13 00:29:38 -04:00
Venky Shankar
dac72ff0b3 mds: restrict setting/removing certain xattrs in ceph namespace
Since all ceph.* xattrs need not be virtual (stored in inode
structure), restrict certain xattrs (ceph.mirror.info) to be
persisted in xattr_map. Other ceph.* xattrs which do not pass
the virtual xattr check are rejected.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
2020-10-13 00:29:38 -04:00
Venky Shankar
a631febb2c mds: introduce ceph.mirror.info virtual xattr
This is a compound xattr with the xattr value being fixed in
format. MDS stores this xattr as two (since the xattr value
right now just has two components or compounds) separate entries
in the xattr_map. This is done to avoid bloating the xattr value
if more "compounds" are added.

You may ask, why do it this way rather having the application
(cephfs-mirror daemon in this case) just set each xattr one
after the other? Well, we loose xattr consistency (from the
application point-of-view) -- an application crash (bug!) or
an ENOSPC in the server could leave M out of N xattrs (M < N)
on-disk.

With the compound xattr operation done on the server side,
journaling ensure that either all (N) xattrs are available or
none are available after recovering from a crash.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
2020-10-13 00:29:38 -04:00
Venky Shankar
a641e3c760 mds: customize xattr handling using dispatch handlers
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2020-10-13 00:29:38 -04:00
Venky Shankar
6b6a83aaa0 mds: introduce is_ceph_vxattr() helper
Not all ceph.* xattrs are virtual -- virtual in the sense
that such xattrs have entries in the inode structure (inode_t)
rather than in xattr_map. There could be cases where an xattr
is in ceph namespace but does not necessarily need to be stored
for each inode -- so filter such xattrs individually rather than
treating all ceph.* xattrs as virtual.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
2020-10-13 00:29:38 -04:00
Patrick Donnelly
daa2f85156
Merge PR #29951 into master
* refs/pull/29951/head:
	test: add tests for validating MDS metrics via `perf stats` module
	test: Filesystem class helpers to grow and shrink MDS cluster
	mgr/stats: mds performance stats module
	mds: support sending empty perf metrics to ceph-manager

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-10-12 18:50:33 -07:00
Changcheng Liu
89ea2d9c10 mon/Monitor: define paxos_service with fixed array size
Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
2020-10-13 09:29:51 +08:00
Changcheng Liu
324388ce66 mon/mon_types.h: define paxos services indexes using enum
Note: It's not proper to use scoped enum type here since the
enum value will be used as container index.

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
2020-10-13 09:27:40 +08:00
Changcheng Liu
47e0b49200 mon: remove unused function
Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
2020-10-13 09:22:44 +08:00
Michael Fritch
39512f9f9a
mgr/orchestrator: set lsm_data to default of Dict type
similar to handling of `sys_api`, convert a NoneType to a Dict

Fixes: https://tracker.ceph.com/issues/47841
Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-10-12 19:06:04 -06:00
Anthony D'Atri
32375cb789 doc: misc clarity and capitalization
Signed-off-by: Anthony D'Atri <anthony.datri@gmail.com>
2020-10-12 13:41:46 -07:00
zdover23
0f3bce753a
Merge pull request #37541 from anthonyeleven/osd-internals-tweaks
doc/dev: doc/dev/osd_internals capitalization, formatting, clarity

Reviewed-by: Zac Dover <zac.dover@gmail.com>
2020-10-13 06:01:05 +10:00
Jason Dillaman
8e88224d8d librbd: ignore -ENOENT error when disabling object-map
Fixes: https://tracker.ceph.com/issues/47840
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-10-12 15:28:52 -04:00
Yuval Lifshitz
270d1117bb rgw/lua: run lua scripts in s3 requests context
for more details on design and
remaining work see:
https://gist.github.com/yuvalif/60d5984c28af89ba17443ce947540c1f

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
2020-10-12 22:23:37 +03:00
Anthony D'Atri
8a4c4cb393 doc/rados/troubleshooting: clarity and modernization
Signed-off-by: Anthony D'Atri <anthony.datri@gmail.com>
2020-10-12 10:22:16 -07:00
Patrick Donnelly
473d8e9a60
Merge PR #37593 into master
* refs/pull/37593/head:
	qa/tasks/cephfs: Refactor test_volumes

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-10-12 07:42:13 -07:00
Xiubo Li
833eca15f2 qa/tasks: add a 'parallel' option support for the cram task
For the ceph-iscsi test case we need to run the tests sequentially,
because the client test will depend on the gateway ones.

Signed-off-by: Xiubo Li <xiubli@redhat.com>
2020-10-12 21:46:34 +08:00
Tatjana Dehler
1edaa17181
Merge pull request #37154 from bk201/wip-47397
mgr/dashboard: fix the error when exporting CephFS path "/" in NFS exports

Reviewed-by: Laura Paduano lpaduano@suse.com
Reviewed-by: Stephan Müller smueller@suse.com
Reviewed-by: Varsha Rao varao@redhat.com
2020-10-12 14:32:39 +02:00
Venky Shankar
a95b364e3e test: add tests for validating MDS metrics via perf stats module
Fixes: http://tracker.ceph.com/issues/24285
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2020-10-12 07:34:51 -04:00
Venky Shankar
5afdf26943 test: Filesystem class helpers to grow and shrink MDS cluster
These routines were test specific. Make them a part of Filesystem
class.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
2020-10-12 07:34:51 -04:00
Venky Shankar
7523aef6e8 mgr/stats: mds performance stats module
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2020-10-12 07:34:51 -04:00
Venky Shankar
e739b831b7 mds: support sending empty perf metrics to ceph-manager
Right now, there are no per-mds metrics that are tracked and
sent by mds. However, such metrics will get added soon. So,
send empty performance metrics to ceph-manager for now.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
2020-10-12 07:34:51 -04:00
Ramana Raja
d1d5d61884 qa/tasks/cephfs: Refactor test_volumes
... into smaller test classes. This enables breaking up the volumes yaml
into smaller yaml fragments.

Fixes: https://tracker.ceph.com/issues/47160
Signed-off-by: Ramana Raja <rraja@redhat.com>
2020-10-12 13:54:40 +05:30
Kefu Chai
130c2715d9
Merge pull request #37628 from changchengx/mail_update
mailmap: update Intel employee mail/org

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-10-12 15:43:30 +08:00
Alex Marangone
9ecdb34342 doc: update osdmaptool man page
Signed-off-by: Alex Marangone <amarangone@digitalocean.com>
2020-10-12 14:17:42 +08:00
Alex Marangone
5e6c69ed42 test: update osdmaptool tests
Signed-off-by: Alex Marangone <amarangone@digitalocean.com>
2020-10-12 14:17:42 +08:00
Alex Marangone
000300e8f9 osdmaptool: use only --save for osdmap overwrite
Signed-off-by: Alex Marangone <amarangone@digitalocean.com>
2020-10-12 14:17:42 +08:00
Alex Marangone
0c12988394 osdmaptool: Add options to change CRUSH weights
Signed-off-by: Alex Marangone <amarangone@digitalocean.com>
2020-10-12 14:17:42 +08:00
Alex Marangone
25bacbfe7f osdmaptool: add mark-up and mark-in flags
Allows to individually mark OSDs up/in. This is useful to determine
how many PGs will move when reprovisioning an OSD whose drive failed.

Signed-off-by: Alex Marangone <amarangone@digitalocean.com>
2020-10-12 14:17:41 +08:00
Alex Marangone
2aee5dc7eb osdmaptool: only change weight if unset with mark-up-in
Signed-off-by: Alex Marangone <amarangone@digitalocean.com>
2020-10-12 14:17:41 +08:00
Changcheng Liu
6ab6021ab4 msg: remove unnecessary assignment
memset(&header, 0, sizeof(header)) clear the variable to
be 0.

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
2020-10-12 12:59:09 +08:00
Changcheng Liu
08e82e2cc7 msg: remove outdated comments
Parameter "features" has been removed in commit 0dbe8fd398.

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
2020-10-12 12:58:35 +08:00
Changcheng Liu
1e8c30a1e1 mailmap: update Intel employee mail/org
1. "changcheng.liu@aliyun.com" need be classified into intel until now.
   This reverts part of commit: df07e9f3
2. add "Yuan Lu <yuan.y.lu@intel.com>" in mailmap

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
2020-10-12 10:10:06 +08:00
Matt Benjamin
118c5ee78b
Merge pull request #36942 from rosinL/wip-fix-librgw
rgw/rgw_file: Fix the incorrect lru object eviction
2020-10-11 11:14:06 -04:00
Yuval Lifshitz
fb7c5b45a7 rgw/file: add missing brackets around if statement
also some small style issues
issues were detected by pvs-studio static analyzer

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
2020-10-11 09:51:00 +03:00
lixiaoy1
5f2ffc12ee rbd/cache: save full cach path instead of folder
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
2020-10-10 10:24:31 -04:00
lixiaoy1
6b9ac3d805 rbd/cache: remove unsupported poolset
Signed-off-by: Li, Xiaoyan <xiaoyan.li@intel.com>
2020-10-10 09:48:03 -04:00
Kefu Chai
af3be4b725 include/interval_set: do not inherit from std::iterator
std::iterator is deprecated in C++17, so drop it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-10-10 20:58:04 +08:00