Commit Graph

104668 Commits

Author SHA1 Message Date
Patrick Donnelly
d7821ce8ec
Merge PR #30972 into master
* refs/pull/30972/head:
	mds: move 'traverse to auth' logic into MDCache::path_traverse
	mds: cleanup Capability initialization
	mds: cleanup code that kills session requests
	mds: cleanup Server::try_open_auth_dirfrag()
	mds: restrict path that walks into snapdir
	mds: use single map to track pinned & auth_pinned objects
	mds: define operator<(...) for MutationImpl::LockOp
	mds: cleanup Locker::acquire_locks()
	mds: introduce Locker::wrlock_try()
	mds: explictly specify if MDCache::path_traverse() needs to check null dentry
	mds: cleanup dentry non-readable check in MDCache::path_traverse()
	mds: change MDCache::path_traverse()'s 'onfail' argument to flags

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
2019-11-24 18:53:20 -08:00
Sage Weil
2c06beb5ec Merge PR #31786 into master
* refs/pull/31786/head:
	mgr: fix errors on using a reference in a Lambda function

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-11-24 12:45:26 -06:00
Kefu Chai
415172543d
Merge pull request #31761 from tchaikov/wip-cmake-include-libfmt
cmake: do not include ${CMAKE_SOURCE_DIR}/src/fmt/include

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2019-11-25 01:08:02 +08:00
Kefu Chai
2add8d1ed5
Merge pull request #31711 from p-se/wip-pse-fix-osd-full-alert
monitoring: wait before firing osd full alert

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
2019-11-25 01:06:34 +08:00
Kefu Chai
3e66ada648
Merge pull request #31692 from rzarzynski/wip-fips-zeroize-memset_bzero
FIPS: audit and switch some memset & bzero users

Reviewed-by: Marcus Watts <mwatts@redhat.com>
2019-11-25 01:05:16 +08:00
Kefu Chai
a3a5a5dc5d
Merge pull request #31705 from tchaikov/wip-mgr-orchestrator
mgr/orchestrator: do not try to iterate through None

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
2019-11-25 01:03:52 +08:00
Kefu Chai
e365514417
Merge pull request #31691 from Anjingkegui/master.monv2change1
mon: remove the restriction of address type in init_with_hosts

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Xiaoxi Chen <xiaoxchen@ebay.com>
2019-11-25 01:01:55 +08:00
Kefu Chai
8b9ffa696d
Merge pull request #31513 from ifed01/wip-ifed-check-bluefs-allocs
os/bluestore: check bluefs allocations on log replay

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
2019-11-25 01:00:54 +08:00
Kefu Chai
0847955aa9
Merge pull request #30204 from tchaikov/wip-test-aio-completion-release
test/librados: free AioCompletion using AioCompletion::release()

Reviewed-by: Sage Weil <sage@redhat.com>
2019-11-25 00:59:25 +08:00
Kefu Chai
819ccfdcd7
Merge pull request #31703 from xiexingguo/wip-42577-plus
osd/PeeringState: do not exclude up from acting_recovery_backfill

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-11-25 00:58:14 +08:00
Kefu Chai
8384a7fd8c
Merge pull request #31693 from tchaikov/wip-42782
qa/workunits/rados/test_librados_build.sh: download from current branch

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-11-25 00:56:09 +08:00
Kefu Chai
c70a16dac9
Merge pull request #31243 from lxbsz/log
log: just return if t is empty

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-11-25 00:55:24 +08:00
Kefu Chai
16c48518b1
Merge pull request #31714 from lxbsz/preforker
preforker: remove useless code

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-11-25 00:54:43 +08:00
Sage Weil
d6f5918850 Merge PR #31778 into master
* refs/pull/31778/head:
	os/bluestore: pin onodes as they are added to the cache
	Revert "Revert "Merge pull request #30964 from markhpc/wip-bs-cache-trim-pinned""

Reviewed-by: Mark Nelson <mnelson@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2019-11-23 20:30:28 -06:00
Sage Weil
130b7e75c2 Merge PR #31807 into master
* refs/pull/31807/head:
	mgr/PyModule: correctly remove config options

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
2019-11-23 20:29:52 -06:00
Sage Weil
582d713e46 Merge PR #31806 into master
* refs/pull/31806/head:
	spec,debian: ceph-mgr-ssh depends on openssh{-client{s}}

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>
2019-11-23 20:29:38 -06:00
Sage Weil
7d4675de7c Merge PR #31818 into master
* refs/pull/31818/head:
	common/options: remove unused ms_msgr2_{sign,encrypt}_messages

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-11-23 20:27:00 -06:00
Sage Weil
c30c1a6d59 Merge PR #31825 into master
* refs/pull/31825/head:
	msg/async: fix typo in Error message

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-11-23 20:26:48 -06:00
Sage Weil
d46c52c50b Merge PR #31831 into master
* refs/pull/31831/head:
	ceph-daemon: make infer_fsid behave when /var/lib/ceph dne

Reviewed-by: Michael Fritch <mfritch@suse.com>
2019-11-23 20:26:08 -06:00
Sage Weil
dcb7ce5f26 Merge PR #31832 into master
* refs/pull/31832/head:
	ceph-daemon: fix extract_uid_gid

Reviewed-by: Michael Fritch <mfritch@suse.com>
2019-11-23 20:25:52 -06:00
Sage Weil
0c9841c707 ceph-daemon: fix extract_uid_gid
This will get confused by the cephdaemon user!

Signed-off-by: Sage Weil <sage@redhat.com>
2019-11-23 09:09:02 -06:00
Sage Weil
6e19593b60 ceph-daemon: make infer_fsid behave when /var/lib/ceph dne
Signed-off-by: Sage Weil <sage@redhat.com>
2019-11-23 09:02:28 -06:00
Sage Weil
ba01e1e951 Merge PR #31636 into master
* refs/pull/31636/head:
	mgr/pg_autoscaler: default to pg_num[_min] = 16

Reviewed-by: Mark Nelson <mnelson@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-11-23 08:48:47 -06:00
Sage Weil
2890d62f85 Merge PR #31796 into master
* refs/pull/31796/head:
	PendingReleaseNotes: note about the removal of 'nvme' class
	common/blkdev: drop is_nvme() method
	os/bluestore/KernelDevice: get rid of 'nvme' type

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-11-23 08:48:32 -06:00
Alfredo Deza
681b1962ce
Merge pull request #31816 from jan--f/c-v-no-unittest-mock
ceph-volume: import mock.mock instead of unittest.mock (py2)

Reviewed-by: Alfredo Deza <adeza@redhat.com>
2019-11-23 08:53:26 -05:00
Kefu Chai
d4d31687c3
Merge pull request #30863 from matthewoliver/improve_do_cmake
do_cmake.sh: Add CEPH_GIT_DIR

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-11-23 09:13:13 +08:00
Willem Jan Withagen
ec43a7d270 msg/async: fix typo in Error message
And fix layout while there

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2019-11-22 23:38:09 +01:00
Sage Weil
1d0fd5603e PendingReleaseNotes: note about the removal of 'nvme' class
There isn't much to say here since this really shouldn't be happening in
the wild.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-11-22 15:33:13 -06:00
Sage Weil
a7542dcf6b Merge PR #31502 into master
* refs/pull/31502/head:
	qa/tasks/ceph2: get ceph-daemon from same place as ceph
	qa/tasks/ceph2: use safe_while
	qa/tasks/ceph2: pull image using sha1
	qa/tasks/ceph2: docker needs quay.io/ prefix for image name
	qa/workunits/rados/test_python: make sure rbd pool exists
	qa/suites/rados/ssh: new tests!
	qa/tasks/ceph2: pull ceph-ci/ceph:$branch
	qa/tasks/ceph2: register_daemons after pods start
	qa/tasks/ceph2: fix conf
	qa/tasks/ceph2: add restart
	qa/tasks/ceph2: pass ceph-daemon path to DaemonState
	qa/tasks/ceph2: tolerate no mdss or 1 mgr
	qa/tasks/ceph: replace wait_for_osds_up with manager.wait_for_all_osds_up
	qa/tasks/ceph: wait-until-healthy
	qa/tasks/ceph2: set up managers
	qa/tasks/ceph2: use seed ceph.conf
	qa/tasks/ceph: healthy: use manager helpers (instead of teuthology/misc ones)
	qa/tasks/ceph2: name mds daemons
	qa/tasks/ceph2: fix osd ordering
	qa/tasks/ceph2: start up mdss
	qa/tasks/ceph2: set up daemon handles and use them to stop
	qa/tasks/ceph2: make it multicluster-aware
	qa/tasks/ceph2: can bring up mon, mgr, osds!
	qa/tasks/ceph2: basic task to bring up cluster with ceph-daemon and ssh

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-11-22 15:28:17 -06:00
Neha Ojha
941807d88e
Merge pull request #31657 from liewegas/fix-merge-vs-backoff
osd: release backoffs during merge

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2019-11-22 11:47:49 -08:00
Sage Weil
69e62a0e40 qa/tasks/ceph2: get ceph-daemon from same place as ceph
Signed-off-by: Sage Weil <sage@redhat.com>
2019-11-22 19:44:45 +00:00
Ilya Dryomov
976d318b48 common/options: remove unused ms_msgr2_{sign,encrypt}_messages
These are unused since 1d29722f80 ("switch monc, daemons to use new
msgr2 auth frame exchange").  As they default to false, a confused user
might flip them to true and think that their client <-> OSD traffic is
encrypted.

The new set of options was added in c7ee66c3e5
("auth,msg/async/ProtocolV2: negotiate connection modes").

Fixes: https://tracker.ceph.com/issues/42976
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2019-11-22 19:03:09 +01:00
Sage Weil
378fed3aa0 Merge PR #31698 into master
* refs/pull/31698/head:
	rpm,deb: add cephdaemon user and sudoers file
	ceph-daemon: use mgr/ssh command to create the ssh key
	mgr/ssh: add 'ssh generate-key' and 'ssh clear-key' commands
	mgr/ssh: add mode option
	mgr/ssh: add 'ssh get-pub-key' and 'ssh get-user' commands
	mgr/ssh: convert to command decorations
	mgr/orchestrator: move command annotation helpers to orchestrator.py

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2019-11-22 11:24:27 -06:00
Sage Weil
47514d5154 Merge PR #31790 into master
* refs/pull/31790/head:
	Revert "Merge pull request #16715 from adamemerson/wip-I-Object!"

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-11-22 11:22:27 -06:00
Kefu Chai
93c2ccc474 test/librados: free AioCompletion using unique_ptr
always destroy AioCompletion pointers even if test fails

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-11-23 00:43:45 +08:00
Kefu Chai
91c7ecb490 librados: release pimpl pointer in destructor
before this change, the librados applications are responsible to call
`AioCompletion::release()` explicitly to release its internal pimpl
pointer. this is error prone and not intuitive.

after this change, the destructor of `AioCompletion` and
`PoolAsyncCompletion` will do this automatically. while
`AioCompletion::release()` and `PoolAsyncCompletion::release()` still
delete the instance as they did before. so this change is backward
compatible, as existing librados clients can still use `ptr->release()`
to free the completion instance, while new clients can just `delete
ptr`.

librados_test_stub is updated accordingly to match the new model

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-11-23 00:43:45 +08:00
Jan Fajerski
c1bd09f259 ceph-volume: import mock.mock instead of unittest.mock (py2)
Fixes: bb4de1a3fc
Fixes: https://tracker.ceph.com/issues/42970

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2019-11-22 17:27:08 +01:00
Jan Fajerski
5ae7bcab31
Merge pull request #31814 from jan--f/c-v-check-selinux-py2
ceph-volume: check if we run in an selinux environment, now also in py2
2019-11-22 17:26:45 +01:00
Sage Weil
5bb652cc2d Merge PR #31788 into master
* refs/pull/31788/head:
	ceph-daemon: append newline before public key string

Reviewed-by: Thomas Bechtold <tbechtold@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
2019-11-22 09:33:36 -06:00
Sage Weil
54ddf2c897 Revert "Merge pull request #16715 from adamemerson/wip-I-Object!"
This reverts commit 669453138d, reversing
changes made to 36f5fcbb97.

Signed-off-by: Sage Weil <sage@redhat.com>

- conflicts due to code rearrangement in 14b0db908f
2019-11-22 09:24:25 -06:00
Jan Fajerski
12c5f89ddb ceph-volume: python2 raises OSError on Popen with missing binary.
For compatibility alias FileNotFoundError to OSError.

Fixes: 72b79f71fa
Fixes: https://tracker.ceph.com/issues/42967

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2019-11-22 15:51:00 +01:00
Sage Weil
2c7aa779be Merge PR #31798 into master
* refs/pull/31798/head:
	ceph-daemon: ceph-volume works without an fsid
	ceph-daemon: several commands that can infer fsids still require them
	ceph-daemon: fix fsid inference

Reviewed-by: Michael Fritch <mfritch@suse.com>
2019-11-22 08:50:52 -06:00
Jan Fajerski
114aa57df2 ceph-volume: py2 compatibility for selinux test
Fixes: af30c7f956

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
2019-11-22 15:49:41 +01:00
Alfredo Deza
842b10180d
Merge pull request #31809 from jan--f/c-v-check-selinux
ceph-volume: check if we run in an selinux environment

Reviewed-by: Alfredo Deza <adeza@redhat.com>
2019-11-22 09:28:21 -05:00
Sage Weil
d3f51251e5 ceph-daemon: ceph-volume works without an fsid
Signed-off-by: Sage Weil <sage@redhat.com>
2019-11-22 07:44:56 -06:00
Sage Weil
80ab64b2ae ceph-daemon: several commands that can infer fsids still require them
Signed-off-by: Sage Weil <sage@redhat.com>
2019-11-22 07:44:56 -06:00
Sage Weil
82cbfbe919 ceph-daemon: fix fsid inference
All return paths should return func.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-11-22 07:44:56 -06:00
Willem Jan Withagen
67ffff38ed mgr: fix errors on using a reference in a Lambda function
This is the extra info on this matter:

https://bugs.llvm.org/show_bug.cgi?id=35984.

Fixes: https://tracker.ceph.com/issues/42905
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
2019-11-22 14:07:31 +01:00
Tim Serong
0520ff571c mgr/PyModule: correctly remove config options
Previously, incorrect parameters were being passed to "config rm",
causing it to do nothing.  This commit also ensures the correct
error message is shown for both the set and remove failure cases.
I've also moved the update of the in-memory config map to *after*
the value is persisted, to ensure the config map actually reflects
what's stored.

Fixes: https://tracker.ceph.com/issues/42958
Signed-off-by: Tim Serong <tserong@suse.com>
2019-11-22 23:25:33 +11:00
Ilya Dryomov
6c7a23b343
Merge pull request #31771 from idryomov/wip-krbd-read-only-test
qa: update krbd_blkroset.t and add krbd_get_features.t

Reviewed-by: Dongsheng Yang <dongsheng.yang@easystack.cn>
2019-11-22 12:43:13 +01:00