Commit Graph

111127 Commits

Author SHA1 Message Date
Volker Theile
a08dad6b18 mgr/dashboard: The max. buckets field in RGW user form should be pre-filled
Fixes: https://tracker.ceph.com/issues/45204

Signed-off-by: Volker Theile <vtheile@suse.com>
2020-05-04 09:49:44 +02:00
Kefu Chai
516785822e
Merge pull request #34874 from tchaikov/wip-crimson-thread-pool-test
test/crimson: minimize the lexical scope of thread pool

Reviewed-by: Samuel Just <sjust@redhat.com>
2020-05-02 11:52:41 +08:00
Josh Durgin
beaa4b04bc
Merge pull request #34880 from neha-ojha/wip-balancer-cleanup
qa/suites/rados/thrash*: remove upmap.yaml and off.yaml

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-05-01 13:15:58 -07:00
Gregory Farnum
294a662a86
Merge pull request #34536 from varshar16/wip-cephfs-shell-flake8-py3
cephfs-shell: Change tox testenv name to py3

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-05-01 12:30:00 -07:00
Neha Ojha
d46ea00320 qa/suites/rados/thrash*: add on.yaml
To test with balancer in upmap mode, which is the default.

Signed-off-by: Neha Ojha <nojha@redhat.com>
2020-05-01 18:49:36 +00:00
Neha Ojha
cb5611e0d5 qa/suites/rados/thrash*: remove upmap.yaml and off.yaml
The balancer is always on, so we don't need upmap.yaml.
off.yaml doesn't make sense anymore.

Signed-off-by: Neha Ojha <nojha@redhat.com>
2020-05-01 18:31:01 +00:00
Jason Dillaman
5b1e05fc25
Merge pull request #34765 from changchengx/doc
doc/rbd: explicit reminder for libvirt directory access permissions

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-05-01 12:33:00 -04:00
Casey Bodley
83cea4f3b7
Merge pull request #34877 from vumrao/wip-vumrao-45355
rgw/rgw_bucket: clear tenant string for non tenanted buckets in rgw_bucket_parse_bucket_key function.

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2020-05-01 12:31:49 -04:00
Vikhyat Umrao
6c4594a6aa rgw/rgw_bucket: clear tenant string for non tenanted buckets
in rgw_bucket_parse_bucket_key function.

Fixes: https://tracker.ceph.com/issues/45355

Signed-off-by: Vikhyat Umrao <vikhyat@redhat.com>
2020-05-01 08:28:55 -07:00
Jason Dillaman
efb6b656a1
Merge pull request #34787 from trociny/wip-45305
qa/workunits/rbd: fix list-mapped filter in unmap_device

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-05-01 11:23:02 -04:00
Changcheng Liu
f90b4997b8 doc/rbd: explicit remind the directory access right
It needs to create admin socket to use perf counter for
debug purpose. Remind the access right to directory.

Signed-off-by: Changcheng Liu <changcheng.liu@aliyun.com>
2020-05-01 22:51:18 +08:00
Jason Dillaman
384c3db1ed
Merge pull request #34756 from yison/rwl-image-writeback-cache-seq7
rbd/cache: Replicated Write Log core codes - aio_discard

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2020-05-01 10:41:28 -04:00
Yuri Weinstein
9bc92c3aac
Merge pull request #34575 from hillpd/suicide_grace_fix
rados: prevent ShardedOpWQ suicide_grace drop when waiting for work.
2020-05-01 07:33:10 -07:00
Yuri Weinstein
e10f02cbde
Merge pull request #34569 from ifed01/wip-ifed-more-bluefs-stats
os/bluestore: improve bluefs stats reporting

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
2020-05-01 07:31:52 -07:00
Casey Bodley
e68c60ac73
Merge pull request #34376 from yuvalif/fix_amqp_routable_option
rgw/amqp: fix the "routable" delivery mode

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2020-05-01 09:00:01 -04:00
Kefu Chai
0b21c597d4 qa/tasks/qemu: open file in text mode
in python3, if a file is opened in 'rb' mode, we will have bytes when
reading from it, as a result, we have following failure like:
```
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/contextutil.py", line 32, in nested
    vars.append(enter())
  File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "/home/teuthworker/src/github.com_tchaikov_ceph_nautilus-py3/qa/tasks/qemu.py", line 124, in generate_iso
    test_setup = ''.join(f.readlines())
TypeError: sequence item 0: expected str instance, bytes found
```

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-05-01 16:55:45 +08:00
Kefu Chai
4c81db2abf test/crimson: minimize the lexical scope of thread pool
to show the typical use case of thread pool in a better manner: thread
pool can be initialized *in* a seastar application.

also move `handle_exception()` out to catch all exceptions throwns in
the seastar app.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-05-01 15:05:22 +08:00
Kefu Chai
fcf222a589
Merge pull request #34657 from lxbsz/py3_com
qa/tasks/vstart_runner.py: to be python3 compatible

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-05-01 13:35:46 +08:00
Kefu Chai
d80e566c9e
Merge pull request #34824 from sebastian-philipp/python-common-fix-device-selection-type-annotation
python-common: Fix wrong type annotation

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
2020-05-01 13:11:01 +08:00
Kefu Chai
f701ed8b26
Merge pull request #34846 from tchaikov/wip-45147
mgr: do not load disabled modules

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
2020-05-01 13:08:29 +08:00
Kefu Chai
e44d861f27
Merge pull request #34786 from aclamk/wip-common-pretty-binary
common: Add pretty_binary_string conversions

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-05-01 13:06:59 +08:00
Kefu Chai
193a5b274b
Merge pull request #34772 from aclamk/wip-bs-faster-fsck-no-cache
kv/KeyValueDB, os/bluestore: Made fsck faster by not caching results

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2020-05-01 13:06:09 +08:00
Kefu Chai
c21200b614
Merge pull request #34754 from ifed01/wip-ifed-fix-45195
os/bluestore: fix improper blob usage while handling deferred big write.

Reviewed-by: Adam Kupczyk <akucpzyk@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
2020-05-01 13:04:30 +08:00
Kefu Chai
151557b850
Merge pull request #34738 from sebastian-philipp/insights-rm-tox
pybind/mgr: Remove insights/tox.ini

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-05-01 13:03:18 +08:00
Kefu Chai
a08ff8194c
Merge pull request #34690 from sebastian-philipp/cephadm-crush-keyring-erro
mgr/cephadm: raise, if crash keyring fails

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-05-01 13:02:18 +08:00
Kefu Chai
d14afb8fa9
Merge pull request #34632 from kotreshhr/bash-completion
bash_completion: Do not auto complete obsolete and hidden cmds

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-05-01 13:01:28 +08:00
Kefu Chai
befb2ab5a4
Merge pull request #34503 from ifed01/wip-ifed-fix-huge-bluefs-reads
os/bluestore: fix huge (>4GB) bluefs reads

Reviewed-by: Adam Kupczyk <akucpzyk@redhat.com>
2020-05-01 13:00:05 +08:00
Kefu Chai
60b7772e40
Merge pull request #34363 from majianpeng/osd-thread-per-shard-ssd
osd: wakeup all threads of shard rather than one thread.

Reviewed-by: Mark Nelson <mnelson@redhat.com>
2020-05-01 12:58:51 +08:00
Kefu Chai
a75d43efbf qa/tasks/mgr: always add "application" label to pool
otherwise monitor complains like
```
 health={'status': 'HEALTH_WARN', 'checks': [{'severity': 'HEALTH_WARN', 'summary': {'message': '1 pool(s) do not have an application enabled', 'count': 1}, 'detail': [{'message': "application not
enabled on pool 'dashboard_pool3'"}, {'message': "use 'ceph osd pool application enable <pool-name> <app-name>', where <app-name> is 'cephfs', 'rbd', 'rgw', or freeform for custom applications."}],
'muted': False, 'type': 'POOL_APP_NOT_ENABLED'}], 'mutes': []}
```

and if test checks for the healthy status, the test fails like:
```
2020-04-30T17:29:54.498 INFO:tasks.cephfs_test_runner:    self.assertEqual(health['status'], 'HEALTH_OK', msg='health={}'.format(health))
2020-04-30T17:29:54.498 INFO:tasks.cephfs_test_runner:AssertionError: 'HEALTH_WARN' != 'HEALTH_OK'
```

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-05-01 12:20:15 +08:00
Tiago Melo
afbdc9e45b mgr/dashboard: E2E: Fix role-mgmt.po.ts
Fixes: https://tracker.ceph.com/issues/45352

Signed-off-by: Tiago Melo <tmelo@suse.com>
2020-04-30 17:30:00 +00:00
Tiago Melo
873028a359 mgr/dashboard: Fix error when filtering table without data
Fixes: https://tracker.ceph.com/issues/45351

Signed-off-by: Tiago Melo <tmelo@suse.com>
2020-04-30 17:18:20 +00:00
Casey Bodley
eba76abdf8
Merge pull request #34641 from yuvalif/fix_zer_size_in_notifications
rgw/notifications: fix zero size in notifications

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2020-04-30 10:21:34 -04:00
Casey Bodley
6c1f3cbb24
Merge pull request #34352 from smanjara/wip-dynamic-resharding
rgw: more multisite dynamic resharding pieces.

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2020-04-30 10:02:12 -04:00
David Galloway
fee0cda092
Merge pull request #34743 from sebastian-philipp/doc-fix-getting-started
doc: fix "getting started" link
2020-04-30 08:57:47 -04:00
Kefu Chai
067adbf9a0 mgr: do not load disabled modules
an option named "mgr_disabled_modules" is added in this change to
prevent mgr from loading modules listed in this option. because mgr
loads *all* modules found in the configured path, and per
https://tracker.ceph.com/issues/45147, python subinterpreter could hang
when loading numpy, so this behavior practically creates a deadlock
in mgr.

this issue is found when mgr uses python3.8 runtime. in development
environment, it'd be inconvenient to disable the offending mgr module
without changing the source code, even if we can choose to not install
them, for instance, the enduser can workaround this issue by
uninstalling `ceph-mgr-diskprediction-local`.

an option would be useful in this case, so we can add the module to the
list before mgr tries to load it.

as this issue is found with python3.8 + diskprediction_local (numpy), so
this mgr module is disabled by default if mgr is compiled with python3.8
runtime.

Fixes: https://tracker.ceph.com/issues/45147
Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-04-30 19:11:22 +08:00
Kefu Chai
40d06ce623 mgr/PyModuleRegistry: probe modules using std::filesystem
for better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-04-30 19:10:02 +08:00
Sidharth Anupkrishnan
b98f3e388b qa/test_exports: fix TestExports failure under new python3 compability changes
self.mount_a.client_remote.sh() returns an 'str' object rather than a StringIO object. Hence the p.stdout.getvalue() produces an error. This commit fixes this and also fix str and byte mismatch as byte and string were the same object in Python2 but this is not the case in Python3.

Signed-off-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
2020-04-30 15:49:32 +05:30
Joshua Schmid
815f8e6470 ceph-volume: add osdspec_affinity flag to ceph-osd
Signed-off-by: Joshua Schmid <jschmid@suse.de>
2020-04-30 12:07:36 +02:00
Joshua Schmid
b7bc6f7607 ceph-volume: add ceph.osdspec_affinity tag
Signed-off-by: Joshua Schmid <jschmid@suse.de>
2020-04-30 12:07:35 +02:00
Kefu Chai
ea5c668fea
Merge pull request #34594 from tchaikov/wip-more-tolerant-perf-test
test/crimson/cbt: increase stddev variance to 2

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Mark Nelson <mnelson@redhat.com>
2020-04-30 17:42:47 +08:00
Kefu Chai
39b561fe03
Merge pull request #34603 from liu-chunmei/crimson_msgr_local_conf
crimson/net: remove hardcoded msgr configuration

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-04-30 17:39:26 +08:00
Sebastian Wagner
ef8a2cde4f
Merge pull request #34707 from jschmid1/fix_orch_docs
doc/orchestrator: adapt to new syntax, add more examples

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-04-30 10:36:54 +02:00
Shilpa Jagannath
71b657b9d2 minor fix: run tests in python venv
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2020-04-30 12:39:50 +05:30
Matthew Oliver
4c942d0594 cephadm: Make ceph-iscsi SSL aware
Ceph-iscsi's `rbd-target-api.py` supports listening over SSL if you
provide an SSL cert and key. Originally the script is opinionated and
requires these files to be named `/etc/ceph/iscsi-gateway.{crt,key}`.

When dealing with containers, having to place files inside a container to
enable SSL isn't very clean. To make things easier, like RGW, you can
now place the SSL cert and key data in the mon config-key store.

This will mean there are 2 ways to enable SSL in ceph-iscsi via orch/cephadm.

1. Push the SSL key and cert into the mon config-key under the keys, and
   then make sure api_secure is enabled (requires json):

  iscsi/{clientname}/iscsi-gateway.crt
  iscsi/{clientname}/iscsi-gateway.key

2. Provide the SSL key and cert in the json you pass the orchestrator and
   it'll push them up for you.

Also lockdown the caps so the container can only access iscsi ssl
key/certs.

Signed-off-by: Matthew Oliver <moliver@suse.com>
2020-04-30 05:31:52 +00:00
Yuri Weinstein
84fbf1c5cd
Merge pull request #34616 from ifed01/wip-ifed-fix-upgrade-super
os/bluestore: invoke _prepare_ondisk_format_super as the last op

Reviewed-by: Greg Farnum <gfarnum@redhat
2020-04-29 09:24:02 -07:00
Ricardo Marques
eb732dcbc5 cephadm: Infer config on "cephadm shell"
Fixes: https://tracker.ceph.com/issues/44792

Signed-off-by: Ricardo Marques <rimarques@suse.com>
2020-04-29 16:31:29 +01:00
Lenz Grimmer
dc9973a7fb
Merge pull request #33462 from s0nea/wip-dashboard-telemetry-report
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Patrick Seidensal <pnawracay@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2020-04-29 17:22:50 +02:00
Tatjana Dehler
18875ef337 mgr/dashboard: add telemetry report component
Add a telemetry component in order to give the user the
possibility to configure the telemetry module in a more
guided fashion. The component offers broader explanations,
shows a preview of the generated report and asks the user
to accept the license before enabling the module.

Fixes: https://tracker.ceph.com/issues/43956
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
2020-04-29 16:34:00 +02:00
Tatjana Dehler
d2f76c91af mgr/dashboard: handle textarea and add tooltip
Enhance `Copy2ClipboardButtonDirective` in order to
handle `textarea`s and copy a formatted input.
Add a tooltip to the button, too.

Signed-off-by: Tatjana Dehler <tdehler@suse.com>
2020-04-29 16:33:50 +02:00
Sridhar Seshasayee
5083e21192 mon/OSDMonitor: Always tune priority cache manager memory on all mons
Always call into priority cache manager (pcm) to tune the memory on the
leader and on all the followers (peons) as part of each tick(). This
ensures that the pcm on all the mons continuously tunes the tcmalloc
memory thereby ensuring that the peons don't run out of memory eventually.

Fixes: https://tracker.ceph.com/issues/45266
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
2020-04-29 19:41:14 +05:30