Commit Graph

110057 Commits

Author SHA1 Message Date
clyso
4a850e1dca doc/mgr/orchestrator: update cephadm shell proposed alias entry
prior change i get following error when using alias as "ceph -v":
INFO:cephadm:Inferring fsid ecfdb924-7341-11ea-943c-020100010027
INFO:cephadm:Using recent ceph image ceph/ceph:v15
/usr/bin/docker: Error response from daemon: OCI runtime create failed: container_linux.go:349: starting container process caused "exec: "/usr/bin/ceph -v": stat /usr/bin/ceph -v: no such file or directory": unknown.
after:
INFO:cephadm:Inferring fsid ecfdb924-7341-11ea-943c-020100010027
INFO:cephadm:Using recent ceph image ceph/ceph:v15
ceph version 15.2.0 (dc6a0b5) octopus (rc)

Signed-off-by: Tobias Fischer <tobias.fischer@clyso.com>
2020-04-03 09:52:47 +02:00
clyso
87440ab4d2 doc/mgr/orchestrator: add path to ssh-copy-id instructions
by default ssh key will be placed under /etc/ceph - so it should be included in examples

Signed-off-by: Tobias Fischer <tobias.fischer@clyso.com>
2020-04-03 09:51:08 +02:00
Lenz Grimmer
ed03b37e86
Merge pull request #34227 from p-se/wip-pse-fix-alert-space-prediction
monitoring: alert for prediction of disk and pool fill up broken

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2020-04-03 09:48:33 +02:00
Kefu Chai
8717c85fc8 .github/CODEOWNERS: add ceph/crimson
so we can add reviewers in a more efficient way for crimson related
changes

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-04-03 14:59:11 +08:00
Kefu Chai
b439835cd6 cmake: specify CC when compiling pmem
make always assume that `cc` is available. but we cannot ensure this,
and furthermore, we need to use the compiler specified by user. so
specify `CC` variable when compiling pmem. and reindent the code to fix
the formatting.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-04-03 13:07:51 +08:00
Kefu Chai
0a86c8f6be librbd/cache: undefine dout_subsys before defining it
otherwise we could have following warning from compiler
```
In file included from ../src/test/librbd/cache/test_mock_ReplicatedWriteLog.cc:42:
../src/librbd/cache/ImageWriteback.cc:13: warning: "dout_subsys" redefined
   13 | #define dout_subsys ceph_subsys_rbd
      |
In file included from ../src/test/librbd/cache/test_mock_ReplicatedWriteLog.cc:39:
../src/librbd/cache/ReplicatedWriteLog.cc:23: note: this is the location of the previous definition
   23 | #define dout_subsys ceph_subsys_rbd_rwl
      |
In file included from ../src/test/librbd/cache/test_mock_ReplicatedWriteLog.cc:43:
../src/librbd/cache/rwl/ImageCacheState.cc:12: warning: "dout_subsys" redefined
   12 | #define dout_subsys ceph_subsys_rbd_rwl
      |
In file included from ../src/test/librbd/cache/test_mock_ReplicatedWriteLog.cc:42:
../src/librbd/cache/ImageWriteback.cc:13: note: this is the location of the previous definition
   13 | #define dout_subsys ceph_subsys_rbd
      |
```

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-04-03 13:07:51 +08:00
Kefu Chai
73f4f16c34 cmake: expose pmem::pmemobj from rbd_internal
because `librbd/cache/rwl/Types.h` includes `libpmemobj.h`, without this
change, `cache/test_mock_ReplicatedWriteLog.cc` will fail to compile as
`libpmemobj.h` might not exist in the default directories compile search
for header files. and hence we could have following failure
```
ceph/src/test/librbd/cache/test_mock_ReplicatedWriteLog.cc:11:
86 /root/luyuan/community/github/ceph/src/librbd/cache/rwl/Types.h:8:10: fatal error: libpmemobj.h: No such file or directory
87 8 | #include <libpmemobj.h>
88 | ^~~~~~~~~~~~~~
89 compilation terminated.
90 src/test/librbd/CMakeFiles/unittest_librbd.dir/build.make:1814: recipe for target 'src/test/librbd/CMakeFiles/unittest_librbd.dir/cache/test_mock_ReplicatedWriteLog.cc.o' failed
91 make[3]: *** [src/test/librbd/CMakeFiles/unittest_librbd.dir/cache/test_mock_ReplicatedWriteLog.cc.o] Error 1
92 make[3]: *** Waiting for unfinished jobs....
93 CMakeFiles/Makefile2:20379: recipe for target 'src/test/librbd/CMakeFiles/unittest_librbd.dir/all' failed
94 make[2]: *** [src/test/librbd/CMakeFiles/unittest_librbd.dir/all] Error 2
95 CMakeFiles/Makefile2:20391: recipe for target 'src/test/librbd/CMakeFiles/unittest_librbd.dir/rule' failed
96 make[1]: *** [src/test/librbd/CMakeFiles/unittest_librbd.dir/rule] Error 2
97 Makefile:5375: recipe for target 'unittest_librbd' failed
```

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-04-03 13:07:51 +08:00
Kefu Chai
6fca5e7d37 cmake: do not use $(MAKE) in Buildpmem.cmake
we cannot assume that we are using `make` as the cmake generatator,
for instance, if ninja is used, `$(MAKE)` won't be substituted by ninja.
so we need to check if Make is used as generator, if that's the case, we
can just use `$(MAKE)` so we can benefit from the job control of `make`,
otherwise, `make` is used, because currently, PMDK uses Makefile to
build.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-04-03 13:07:51 +08:00
Kefu Chai
8f7fae7a78 cmake: add find_make() function
it's a shorthand for finding "make" or "gmake" (for FreeBSD), and set
the path to the executable and the command to use in the generated
"Makefile" or whatever build script generated by cmake.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-04-03 13:07:51 +08:00
Greg Farnum
81ce2b240a Revert "qa/cephfs: move run_shell and related methods to mount.py"
This reverts commit 751d432a86.

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2020-04-03 02:24:48 +00:00
Greg Farnum
1902e6bfd9 Revert "vstart_runner.py: add methods for negative testing a cmd"
This reverts commit 48e0e1d703.

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2020-04-03 02:24:45 +00:00
Sage Weil
a3be5f2aca cephadm: create /var/run/ceph dir via unit.run, not unit file
The systemd unit file is shared with non-ceph daemons, which (1) don't
need the /var/run directory, and (2) are based on a uid/gid from a
different container image, which means we can't figure out the right
ceph uid/gid from them to set the ownership properly.

Instead, put it in the unit.run file... and only for ceph daemons when
we have the uid/gid we need.

Fixes: https://tracker.ceph.com/issues/44894
Signed-off-by: Sage Weil <sage@redhat.com>
2020-04-02 18:36:39 -05:00
Matthew Oliver
4179b960a4 cephadm: ceph-iscsi first draft
This if the first draft of the ceph-iscsi in cephadm.
There are a few gotchas when running `rbd-target-api` in a container:

 1. We need both the ceph.conf and iscsi-gateway.cfg, so needed to
ability to pass extra config. This latter is based off the spec, so now
the daemon config func api allows you to return a dict of configs:

  { 'config': '<str>' # will be appended to the ceph.conf
    '<conf name>': 'str', # Will be dumped in datadir/<conf name>
    ...
  }

It will be up to cephadm to know to bind mount it to the right location.
The first 'config' isn't used by this patch, but makes it possible for
specs or config funcs to append anything? maybe it's overkill.

 2. We need the kernel's configfs in the container so we can configure
LIO. There is a chicken and egg problem, configfs isn't mounted on the
host to bind mount when the container starts. So now a check is added to
the `unit.run` and cleanup in the `unit.poststop` scripts for
daemon_type iscsi.

 3. rbd-target-api is python and hardcodes a few things, like logging
through `/dev/log` which happens to be a domain socket. So `/dev/log`
also needed to be bind mounted into the continer.

 4. The daemon expects the keyring to be in `/etc/ceph` so this needed to
be specifically bind mounted to the correct location too.

As this currently stands this is deploying and starting the api on port
5000, so seems to be "working", also gateway.conf does exist in the
pool. I have yet to set up an iscsi device, but will test that next.

The `rbd-target-api` daemon expects the ssl key and cert to be named a
certain name in the contianer. So SSL isn't working yet. However, I do
hav a PR in ceph-iscsi to look in the mon config-key store for them[0].

[0] - https://github.com/ceph/ceph-iscsi/pull/173

Signed-off-by: Matthew Oliver <moliver@suse.com>
2020-04-02 23:31:26 +00:00
Matthew Oliver
a1fd9d11e7 tox: Fix the tox.ini's to support older versions of tox
The src/cephadm/tox.ini and src/pybind/mgr/tox.ini both don't run
on older versions of tox.
When using tox 2.9.1 both fail for different reasons.

`src/cephadm/tox.ini` fails because `skipsdist=true` only works if it's
directly under the `[tox]` section.

`src/pybind/mgr/tox.ini` fails because older versions of tox can't find
the requirements.txt because they don't like whitespace between the `-r`
and `requirements.txt`.

This patch changes the tox.ini's to be backwards compatible for those
who happen to be running slightly older version of tox.

Signed-off-by: Matthew Oliver <moliver@suse.com>
2020-04-02 23:30:20 +00:00
Sage Weil
76b559598a qa/suites/rados/cephadm/smoke-roleless: add smoke test
Signed-off-by: Sage Weil <sage@redhat.com>
2020-04-02 23:16:42 +00:00
Sage Weil
9e609c9ed7 qa/tasks/cephadm: add 'roleless' mode
Allow cephadm to start up with roles like:

roles:
- - host.a
  - client.0
  - osd.0
  - osd.1
- - host.b
  - osd.2
  - osd.3

Cephadm will pick the mon names (based on host) and provision all
services by default.

The cephadm task can still provision other daemons, but it may
fight with mgr/cephadm.

Signed-off-by: Sage Weil <sage@redhat.com>
2020-04-02 23:16:42 +00:00
Sebastian Wagner
4c2ace69cd
Merge pull request #34330 from mgfritch/cephadm-nfs-extra-args
cephadm: add `extra_args` to nfs daemon

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Varsha Rao <varao@redhat.com>
2020-04-02 20:02:46 +02:00
Sebastian Wagner
85ffae018e
Merge pull request #34361 from mgfritch/cephadm-revert-nfs-trivial
mgr/cephadm: revert trivial_completion for nfs_add

Reviewed-by: Matthew Oliver <moliver@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-04-02 18:05:19 +02:00
Sebastian Wagner
de4ca87f61
Merge pull request #34329 from liewegas/cephadm-bootstrap-typo
cephadm: fix typo

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-04-02 18:02:51 +02:00
Laura Paduano
9600b87aa6
Merge pull request #34283 from tspmelo/wip-reduce-scss
mgr/dashboard: Reduce size of component styles 

Reviewed-by: Ernesto Puertat <epuertat@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
2020-04-02 16:51:57 +02:00
Casey Bodley
e8811a0cfc
Merge pull request #33807 from ofriedma/wip-disable-range-prefetch
rgw: Disable prefetch of entire head object when GET request with ran…

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>
Reviewed-by: Mark Kogan <mkogan@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2020-04-02 10:46:27 -04:00
Volker Theile
4febdd6dda mgr/dashboard: Add more debug information to Dashboard RGW backend
Fixes: https://tracker.ceph.com/issues/44914
Signed-off-by: Volker Theile <vtheile@suse.com>
2020-04-02 16:01:53 +02:00
Tiago Melo
a1b2dc9ed8 mgr/dashboard: Update npm dependencies
ngx-bootstrap now requires BrowserAnimationsModule, so it has to be imported
in each unit test that imports ngx-bootstrap modules.

Fixes: https://tracker.ceph.com/issues/44854

Signed-off-by: Tiago Melo <tmelo@suse.com>
2020-04-02 13:52:43 +00:00
Tiago Melo
e0d365e2cf mgr/dashboard: Update npm devDependencies
The following updates required code style changes:
- TSLint updated the logic of ordering imports.
- Prettier improved when to break a command chain into multiple lines.

Fixes: https://tracker.ceph.com/issues/44854

Signed-off-by: Tiago Melo <tmelo@suse.com>
2020-04-02 13:52:43 +00:00
Lenz Grimmer
d001e6e94f
Merge pull request #34312 from tspmelo/wip-improve-unit-test
mgr/dashboard: Fix ServiceDetails and PoolDetails unit tests

Reviewed-by: Stephan Müller <smueller@suse.com>
2020-04-02 15:02:50 +02:00
Sebastian Wagner
dbd6964667 pybind/mgr: Fix run_tox.sh mgr -- cephadm
Failed with a cryptic error.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-04-02 14:52:33 +02:00
Sebastian Wagner
c0e86510f6
Merge pull request #34206 from sebastian-philipp/cephadm-hostname-check-lowercase
cephadm: check hostnames case insensitive

Reviewed-by: Sage Weil <sage@redhat.com>
2020-04-02 14:27:30 +02:00
Sebastian Wagner
4e3889202c
Merge pull request #34250 from Daniel-Pivonka/cephadm_python3
mgr/cephadm: add useful error if python3 is not on remote host

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-04-02 14:26:43 +02:00
Sebastian Wagner
232900a61d
Merge pull request #33954 from jschmid1/parse_hostsspecs
python-common: fix /hosts/ parsing in servicespecs

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-04-02 14:25:54 +02:00
Lenz Grimmer
e0f3205683
Merge pull request #34033 from rhcs-dashboard/44666-fix-sso-certificate-file
mgr/dashboard: fix error when enabling SSO with cert. file

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2020-04-02 13:38:21 +02:00
Nathan Cutler
4facd9daa5 doc/releases/nautilus: restart OSDs to make them bind to v2 addr
Fixes: https://tracker.ceph.com/issues/43896
Signed-off-by: Nathan Cutler <ncutler@suse.com>
2020-04-02 13:03:31 +02:00
Sebastian Wagner
3366f6b616 CODEOWNERS: update cephadm paths
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-04-02 11:00:48 +02:00
Michael Fritch
ea263ad958
mgr/cephadm: revert trivial_completion for nfs_add
revert change that was missed in af8fa11

Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-04-01 21:14:54 -06:00
Kefu Chai
b2d160468f
Merge pull request #34324 from rzarzynski/wip-crimson-returnvec
crimson/osd: OP_CALL does support RETURNVEC now.

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-04-02 08:29:35 +08:00
Sebastian Wagner
f9c76200d4
Merge pull request #34290 from sebastian-philipp/doc-cephadm-service-spec
doc/mgr/orchestrator: Add "Service Specification"

Reviewed-by: Joshua Schmid <jschmid@suse.de>
2020-04-02 00:31:14 +02:00
Michael Fritch
f77defe551
mgr/orch: fix python3 DeprecationWarning
test_orchestrator/module.py:181: DeprecationWarning: invalid escape sequence \s
    patterns = ['-i\s(\w+)', '--id[\s=](\w+)']

Signed-off-by: Michael Fritch <mfritch@suse.com>
2020-04-01 15:54:30 -06:00
Casey Bodley
3f2023ec77
Merge pull request #34322 from cbodley/wip-44857
rgw: pubsub sync module ignores ERR_USER_EXIST

Reviewed-by: Yuval Lifshitz <yuvalif@yahoo.com>
2020-04-01 14:25:00 -04:00
Mykola Golub
3cd90d751b
Merge pull request #34134 from zhangdaolong/fix_incorrect_log_info
pybind/rbd: fix no lockers are obtained, ImageNotFound exception will be output
2020-04-01 21:01:23 +03:00
Sage Weil
94e5fc5e5f Merge PR #34320 into master
* refs/pull/34320/head:
	cephadm: ceph-volume: disallow concurrent execution

Reviewed-by: Sebastian Wagner <swagner@suse.com>
2020-04-01 10:58:57 -05:00
Casey Bodley
ba104805fc
Merge pull request #34096 from smanjara/wip-dynamic-resharding
rgw: groundwork for supporting dynamic resharding in multisite environment

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2020-04-01 11:55:06 -04:00
Radoslaw Zarzynski
3bea0ffec1 crimson/osd: OP_CALL does support RETURNVEC now.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2020-04-01 15:34:29 +02:00
Kefu Chai
a01b4df3f2
Merge pull request #34345 from tchaikov/wip-crimson-less-chatty
crimson: do not warn() under expected circumstances

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2020-04-01 20:31:52 +08:00
Kefu Chai
36448b5628 crimson/osd: do not use "warn()" when handling non-error
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-04-01 20:01:16 +08:00
Kefu Chai
a84be33461 crimson/mon: do not print error log at seeing unknown connection
it happens if a client or an peer osd drops the connection, so it's not
an error and hence we should not print this error message using
"error()".

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-04-01 20:01:15 +08:00
Jan Fajerski
748d3d444f
Merge pull request #34341 from guits/update_cv_deploy
ceph-volume: update functional testing deploy.yml playbook
2020-04-01 13:26:59 +02:00
Kefu Chai
b144ef3d1f
Merge pull request #34331 from zealoussnow/wip-fix-clone-error
test/common/unittest_blkdev: delete unused test file

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-04-01 19:23:54 +08:00
Leo Zhang
f2de1dc442 unittest_blkdev: delete unused test file
Signed-off-by: Leo Zhang <nguzcf@gmail.com>
2020-04-01 18:57:52 +08:00
Sebastian Wagner
b740ae12e5 doc/mgr/orchestrator: Update Placement by pattern matching
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-04-01 11:34:18 +02:00
Sebastian Wagner
8d4487527e doc/cephadm/drivegroup: Don't introduce DriveGroups
We already have a name for it: a Service Specification of
type `osd`. We don't need to introduce a new name for it.

Well, they are "DriveGroups", but users don't need to know it.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-04-01 11:34:18 +02:00
Sebastian Wagner
f5e3e60e0e doc/cephadm/drivegroups: Update to new yaml
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-04-01 11:34:18 +02:00