Commit Graph

110021 Commits

Author SHA1 Message Date
Kefu Chai
f10f865796 qa/tasks/cephfs: use StringIO for capturing str
if we don't expect non-utf8 in stdout, use StringIO instead of BytesIO.

see also d8d44ed156

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-04-07 20:33:47 +08:00
Kefu Chai
d7258ea7fd qa/tasks: use next(iter(..)) for accessing first element in a view
in python2, dict.values() and dict.keys() return lists. but in python3,
they return views, which cannot be indexed directly using an integer index.

there are three use cases when we access these views in python3:

1. get the first element
2. get all the elements and then *might* want to access them by index
3. get the first element assuming there is only a single element in
   the view
4. iterate thru the view

in the 1st case, we cannot assume the number of elements, so to be
python3 compatible, we should use `next(iter(a_dict))` instead.

in the 2nd case, in this change, the view is materialized using
`list(a_dict)`.

in the 3rd case, we can just continue using the short hand of
```py
(first_element,) = a_dict.keys()
```
to unpack the view. this works in both python2 and python3.

in the 4th case, the existing code works in both python2 and python3, as
both list and view can be iterated using `iter`, and `len` works as
well.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-04-07 20:33:47 +08:00
Jason Dillaman
16e2ccaf14
Merge pull request #33985 from zhengchengyao/clone_metadata
librbd: children should inherit parent's stripe

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
2020-04-04 09:46:27 -04:00
Jason Dillaman
448cd3fe9a
Merge pull request #34072 from shangdehao1/fix_nfs_bug
qa/tasks/qemu: fix nfs setup and teardown bug in qemu task

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-04-04 09:45:49 -04:00
Josh Durgin
7655548930
Merge pull request #34371 from smithfarm/wip-43896
doc/releases/nautilus: restart OSDs to make them bind to v2 addr

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-04-03 14:31:22 -07:00
Gregory Farnum
40fb15563d
Merge pull request #31480 from ukernel/wip-20
client: auto reconnect after blacklisted

Reviewed-by: Jeff Layton <jlayton@redhat.com>
2020-04-03 12:53:29 -07:00
Gregory Farnum
45288ad525
Merge pull request #33915 from lxbsz/client_assert_failedddd
Client: fix Finisher assert failure

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: "Yan, Zheng" <zyan@redhat.com>
2020-04-03 12:49:59 -07:00
Gregory Farnum
0e7e0d59c7
Merge pull request #34386 from gregsfortytwo/wip-33279-revert-sudo
Wip 33279 revert sudo

Reviewed-by: Douglas Fuller <dfuller@redhat.com>
2020-04-03 12:36:37 -07:00
Gregory Farnum
15b3118eda
Merge pull request #34110 from ukernel/wip-44680
mds: don't assert empty io context list when shutting down

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2020-04-03 12:33:51 -07:00
Sage Weil
004b153f13 Merge PR #34385 into master
* refs/pull/34385/head:
	doc/dev/cephadm: a few notes on developing with cephadm

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2020-04-03 14:29:43 -05:00
Gregory Farnum
13fc503b0f
Merge pull request #34281 from vshankar/wip-44677
mgr: force purge normal ceph entities from service map

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-04-03 12:29:19 -07:00
Gregory Farnum
794ce8942d
Merge pull request #34306 from ukernel/wip-44771
ceph-fuse: don't get mount options from /etc/fstab when doing remount

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2020-04-03 12:28:30 -07:00
Sage Weil
857715f7d8 Merge PR #34296 into master
* refs/pull/34296/head:
	qa/suites/rados/cephadm/smoke-roleless: add smoke test
	qa/tasks/cephadm: add 'roleless' mode

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2020-04-03 14:16:21 -05:00
Sage Weil
639762ca26 Merge PR #34384 into master
* refs/pull/34384/head:
	cephadm: create /var/run/ceph dir via unit.run, not unit file

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-04-03 14:16:00 -05:00
Yuri Weinstein
d23f2b70e8
Merge pull request #34405 from ceph/wip-yuriw-cron-master2
qa/tests: fixed typo
2020-04-03 10:19:17 -07:00
yuriw
898d9c5a60 qa/tests: fixed typo
Signed-off-by: yuriw <yuri.weinstein@gmail.com>
2020-04-03 09:28:05 -07:00
Yuri Weinstein
21e1d54e76
Merge pull request #34404 from ceph/wip-yuriw-cron-master
qa/tests: added client upgrades tests for octopus, removed mimic-x/ma…
2020-04-03 09:21:14 -07:00
yuriw
8c80a43b58 qa/tests: added client upgrades tests for octopus, removed mimic-x/masted as not needed
Signed-off-by: Yuri Weinstein <yweinstein@redhat.com>
2020-04-03 08:57:28 -07:00
Sage Weil
086acd20b9 doc/dev/cephadm: a few notes on developing with cephadm
Signed-off-by: Sage Weil <sage@redhat.com>
2020-04-03 09:42:40 -05:00
Jason Dillaman
cf7c16ca05
Merge pull request #33502 from yison/rwl-image-writeback-cache-seq3
rbd/cache: Replicated Write Log core codes part 3

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-04-03 10:39:28 -04:00
Jason Dillaman
524be11882
Merge pull request #34362 from tchaikov/wip-rbd-with-pmem
cmake, librbd: fix build with pmem and cleanups

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-04-03 10:08:08 -04:00
Kefu Chai
86c785afb4
Merge pull request #34332 from cyx1231st/wip-seastar-messenger-lossy-peer
crimson/net: enable features for lossy peer connections of heartbeat

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-04-03 21:26:04 +08:00
Sebastian Wagner
c656e598ba
Merge pull request #34319 from clyso/patch-2
doc/mgr/orchestrator: update cephadm shell proposed alias entry

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-04-03 12:48:11 +02:00
Sebastian Wagner
adedd2f531
Merge pull request #34318 from clyso/patch-1
doc/mgr/orchestrator: add path to ssh-copy-id instructions

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-04-03 12:44:55 +02:00
Sebastian Wagner
93b908a4dd
Merge pull request #34367 from sebastian-philipp/codeowners-cephadm-update
CODEOWNERS: update cephadm paths

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-04-03 12:41:34 +02:00
Kefu Chai
b6303828be
Merge pull request #34390 from tchaikov/wip-github-codeowners
.github/CODEOWNERS: add ceph/crimson

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2020-04-03 17:41:02 +08:00
Lenz Grimmer
b36491c72a
Merge pull request #34325 from tspmelo/wip-npm-update-20-03
mgr/dashboard: Update all npm packages

Reviewed-by: Sebastian Krah <skrah@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2020-04-03 11:14:50 +02:00
Lenz Grimmer
8ecc7876c0
Merge pull request #34377 from votdev/issue_44914_rgw_debug_info
mgr/dashboard: Add more debug information to Dashboard RGW backend

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2020-04-03 10:45:50 +02:00
Kefu Chai
825e1f0cdb
Merge pull request #34333 from changchengx/mail_org_update
mailmap: update mail org relationship

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-04-03 16:08:55 +08:00
clyso
4a850e1dca doc/mgr/orchestrator: update cephadm shell proposed alias entry
prior change i get following error when using alias as "ceph -v":
INFO:cephadm:Inferring fsid ecfdb924-7341-11ea-943c-020100010027
INFO:cephadm:Using recent ceph image ceph/ceph:v15
/usr/bin/docker: Error response from daemon: OCI runtime create failed: container_linux.go:349: starting container process caused "exec: "/usr/bin/ceph -v": stat /usr/bin/ceph -v: no such file or directory": unknown.
after:
INFO:cephadm:Inferring fsid ecfdb924-7341-11ea-943c-020100010027
INFO:cephadm:Using recent ceph image ceph/ceph:v15
ceph version 15.2.0 (dc6a0b5) octopus (rc)

Signed-off-by: Tobias Fischer <tobias.fischer@clyso.com>
2020-04-03 09:52:47 +02:00
clyso
87440ab4d2 doc/mgr/orchestrator: add path to ssh-copy-id instructions
by default ssh key will be placed under /etc/ceph - so it should be included in examples

Signed-off-by: Tobias Fischer <tobias.fischer@clyso.com>
2020-04-03 09:51:08 +02:00
Lenz Grimmer
ed03b37e86
Merge pull request #34227 from p-se/wip-pse-fix-alert-space-prediction
monitoring: alert for prediction of disk and pool fill up broken

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2020-04-03 09:48:33 +02:00
Kefu Chai
8717c85fc8 .github/CODEOWNERS: add ceph/crimson
so we can add reviewers in a more efficient way for crimson related
changes

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-04-03 14:59:11 +08:00
Kefu Chai
b439835cd6 cmake: specify CC when compiling pmem
make always assume that `cc` is available. but we cannot ensure this,
and furthermore, we need to use the compiler specified by user. so
specify `CC` variable when compiling pmem. and reindent the code to fix
the formatting.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-04-03 13:07:51 +08:00
Kefu Chai
0a86c8f6be librbd/cache: undefine dout_subsys before defining it
otherwise we could have following warning from compiler
```
In file included from ../src/test/librbd/cache/test_mock_ReplicatedWriteLog.cc:42:
../src/librbd/cache/ImageWriteback.cc:13: warning: "dout_subsys" redefined
   13 | #define dout_subsys ceph_subsys_rbd
      |
In file included from ../src/test/librbd/cache/test_mock_ReplicatedWriteLog.cc:39:
../src/librbd/cache/ReplicatedWriteLog.cc:23: note: this is the location of the previous definition
   23 | #define dout_subsys ceph_subsys_rbd_rwl
      |
In file included from ../src/test/librbd/cache/test_mock_ReplicatedWriteLog.cc:43:
../src/librbd/cache/rwl/ImageCacheState.cc:12: warning: "dout_subsys" redefined
   12 | #define dout_subsys ceph_subsys_rbd_rwl
      |
In file included from ../src/test/librbd/cache/test_mock_ReplicatedWriteLog.cc:42:
../src/librbd/cache/ImageWriteback.cc:13: note: this is the location of the previous definition
   13 | #define dout_subsys ceph_subsys_rbd
      |
```

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-04-03 13:07:51 +08:00
Kefu Chai
73f4f16c34 cmake: expose pmem::pmemobj from rbd_internal
because `librbd/cache/rwl/Types.h` includes `libpmemobj.h`, without this
change, `cache/test_mock_ReplicatedWriteLog.cc` will fail to compile as
`libpmemobj.h` might not exist in the default directories compile search
for header files. and hence we could have following failure
```
ceph/src/test/librbd/cache/test_mock_ReplicatedWriteLog.cc:11:
86 /root/luyuan/community/github/ceph/src/librbd/cache/rwl/Types.h:8:10: fatal error: libpmemobj.h: No such file or directory
87 8 | #include <libpmemobj.h>
88 | ^~~~~~~~~~~~~~
89 compilation terminated.
90 src/test/librbd/CMakeFiles/unittest_librbd.dir/build.make:1814: recipe for target 'src/test/librbd/CMakeFiles/unittest_librbd.dir/cache/test_mock_ReplicatedWriteLog.cc.o' failed
91 make[3]: *** [src/test/librbd/CMakeFiles/unittest_librbd.dir/cache/test_mock_ReplicatedWriteLog.cc.o] Error 1
92 make[3]: *** Waiting for unfinished jobs....
93 CMakeFiles/Makefile2:20379: recipe for target 'src/test/librbd/CMakeFiles/unittest_librbd.dir/all' failed
94 make[2]: *** [src/test/librbd/CMakeFiles/unittest_librbd.dir/all] Error 2
95 CMakeFiles/Makefile2:20391: recipe for target 'src/test/librbd/CMakeFiles/unittest_librbd.dir/rule' failed
96 make[1]: *** [src/test/librbd/CMakeFiles/unittest_librbd.dir/rule] Error 2
97 Makefile:5375: recipe for target 'unittest_librbd' failed
```

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-04-03 13:07:51 +08:00
Kefu Chai
6fca5e7d37 cmake: do not use $(MAKE) in Buildpmem.cmake
we cannot assume that we are using `make` as the cmake generatator,
for instance, if ninja is used, `$(MAKE)` won't be substituted by ninja.
so we need to check if Make is used as generator, if that's the case, we
can just use `$(MAKE)` so we can benefit from the job control of `make`,
otherwise, `make` is used, because currently, PMDK uses Makefile to
build.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-04-03 13:07:51 +08:00
Kefu Chai
8f7fae7a78 cmake: add find_make() function
it's a shorthand for finding "make" or "gmake" (for FreeBSD), and set
the path to the executable and the command to use in the generated
"Makefile" or whatever build script generated by cmake.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-04-03 13:07:51 +08:00
Greg Farnum
81ce2b240a Revert "qa/cephfs: move run_shell and related methods to mount.py"
This reverts commit 751d432a86.

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2020-04-03 02:24:48 +00:00
Greg Farnum
1902e6bfd9 Revert "vstart_runner.py: add methods for negative testing a cmd"
This reverts commit 48e0e1d703.

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2020-04-03 02:24:45 +00:00
Sage Weil
a3be5f2aca cephadm: create /var/run/ceph dir via unit.run, not unit file
The systemd unit file is shared with non-ceph daemons, which (1) don't
need the /var/run directory, and (2) are based on a uid/gid from a
different container image, which means we can't figure out the right
ceph uid/gid from them to set the ownership properly.

Instead, put it in the unit.run file... and only for ceph daemons when
we have the uid/gid we need.

Fixes: https://tracker.ceph.com/issues/44894
Signed-off-by: Sage Weil <sage@redhat.com>
2020-04-02 18:36:39 -05:00
Sage Weil
76b559598a qa/suites/rados/cephadm/smoke-roleless: add smoke test
Signed-off-by: Sage Weil <sage@redhat.com>
2020-04-02 23:16:42 +00:00
Sage Weil
9e609c9ed7 qa/tasks/cephadm: add 'roleless' mode
Allow cephadm to start up with roles like:

roles:
- - host.a
  - client.0
  - osd.0
  - osd.1
- - host.b
  - osd.2
  - osd.3

Cephadm will pick the mon names (based on host) and provision all
services by default.

The cephadm task can still provision other daemons, but it may
fight with mgr/cephadm.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-04-02 23:16:42 +00:00
Sebastian Wagner
4c2ace69cd
Merge pull request #34330 from mgfritch/cephadm-nfs-extra-args
cephadm: add `extra_args` to nfs daemon

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Varsha Rao <varao@redhat.com>
2020-04-02 20:02:46 +02:00
Sebastian Wagner
85ffae018e
Merge pull request #34361 from mgfritch/cephadm-revert-nfs-trivial
mgr/cephadm: revert trivial_completion for nfs_add

Reviewed-by: Matthew Oliver <moliver@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-04-02 18:05:19 +02:00
Sebastian Wagner
de4ca87f61
Merge pull request #34329 from liewegas/cephadm-bootstrap-typo
cephadm: fix typo

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-04-02 18:02:51 +02:00
Laura Paduano
9600b87aa6
Merge pull request #34283 from tspmelo/wip-reduce-scss
mgr/dashboard: Reduce size of component styles 

Reviewed-by: Ernesto Puertat <epuertat@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
2020-04-02 16:51:57 +02:00
Casey Bodley
e8811a0cfc
Merge pull request #33807 from ofriedma/wip-disable-range-prefetch
rgw: Disable prefetch of entire head object when GET request with ran…

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Mark Kogan <mkogan@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2020-04-02 10:46:27 -04:00
Volker Theile
4febdd6dda mgr/dashboard: Add more debug information to Dashboard RGW backend
Fixes: https://tracker.ceph.com/issues/44914
Signed-off-by: Volker Theile <vtheile@suse.com>
2020-04-02 16:01:53 +02:00
Tiago Melo
a1b2dc9ed8 mgr/dashboard: Update npm dependencies
ngx-bootstrap now requires BrowserAnimationsModule, so it has to be imported
in each unit test that imports ngx-bootstrap modules.

Fixes: https://tracker.ceph.com/issues/44854

Signed-off-by: Tiago Melo <tmelo@suse.com>
2020-04-02 13:52:43 +00:00