Commit Graph

111868 Commits

Author SHA1 Message Date
Rishabh Dave
a11b3c0177 test_cephfs_shell: catch specific exception instead of catching all
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-06-05 09:41:17 +05:30
Rishabh Dave
9b2077ff10 cephfs-shell: set return value before exiting
Code for each command has a return value set yet cephfs-shell itself
doesn't exit with proper return value. This makes cephfs-shell quit with
zero return value all the time.

Also, many tests execute cephfs-shell commands expecting the command to
fail but without expecting the method issuing the command to fail. Fix
these test accordingly by adding a new method (negtest_cephfs_shell_cmd)
that expects the command to fail and allows verifying return value and
error messages.

Fixes: https://tracker.ceph.com/issues/44113
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-06-05 09:41:16 +05:30
Rishabh Dave
c4bd10bdb9 test_cephfs_shell: re-arrange TestCephFSShell's methods
Re-arrange in the order of their usefulness and use keyword arguments
instead of positional arguments to reduce chances of errors, especially
since test_every_cephfs_shell_cmd_at_invocation overrides these methods.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-06-05 09:31:40 +05:30
Rishabh Dave
3d6feb0f26 cephfs-shell: re-arrange code under __main__
Move code from __main__ to different methods to avoid sequence of block
of code that are not strictly related to each other. This must cleanup
the code under __main__ and must improve readability.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-06-05 09:28:03 +05:30
Rishabh Dave
d0be1f9f07 test_cephfs_shell: add new test for shell cmd at invocation
Test passing each CephFS shell command at invocation of shell. The test
replaces run_cephfs_shell_cmd (and other related methods) by it's own
version of the same method that runs each shell command as
"cephfs-shell -c ceph.conf shellcmd" instead of
"cephfs-shell -c ceph.conf -- shellcmd" and then run each
test_cephfs_shell.py.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-06-05 09:28:03 +05:30
Rishabh Dave
2d5201fdba cephfs-shell: launch shell only if there are no args following
Commands like "cephfs-shell mkdir a" should just create the directory
and quit immediately with the command's return value instead of
launching a shell session. A shell session should be launched only if
there are no arguments following "cephfs-shell".

Fixes: https://tracker.ceph.com/issues/43248
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-06-05 09:26:47 +05:30
Patrick Donnelly
6be968228e
Merge PR #35263 into master
* refs/pull/35263/head:
	test_cephfs_shell: use StringIO instead of BytesIO

Reviewed-by: Varsha Rao <varao@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-06-04 16:08:58 -07:00
Rishabh Dave
b67ce004ad test_cephfs_shell: use StringIO instead of BytesIO
Code in qa/ uses both StringIO and BytesIO. Let's use StringIO
exclusively (unless necessary) for uniformity. The reason for using
StringIO over BytesIO is that tests mostly need stdout as string than
as bytes and StringIO is used more frequently used in qa/ code at this
point.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2020-06-05 00:02:36 +05:30
Sebastian Wagner
e0bbb05f56
Merge pull request #35271 from bk201/refactor-template-cephadm
mgr/cephadm: use jinja2 to render templates

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-06-04 19:50:58 +02:00
Casey Bodley
3f922a5a84
Merge pull request #33646 from ofriedma/wip-mkogan-ofriedma-add-cache-api
rgw: Adding data cache and CDN capabilities

Reviewed-by: Mark Kogan <mkogan@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2020-06-04 12:11:11 -04:00
Sebastian Wagner
3212fefde6
Merge pull request #35018 from sebastian-philipp/cephadm-rm-container
cephadm: Manually remove containers

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-06-04 17:59:48 +02:00
Kefu Chai
1166f7ae80
Merge pull request #35176 from aclamk/wip-prio-cache-name
common: PriorityCache's perf counters can have custom name

Reviewed-by: Mark Nelson <mnelson@redhat.com>
2020-06-04 23:01:46 +08:00
Lenz Grimmer
c4ee593927
Merge pull request #35299 from tspmelo/wip-ng-popover
mgr/dashboard: Use ng-bootstrap for Popover

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2020-06-04 16:55:48 +02:00
Tiago Melo
e5709c5c1a mgr/dashboard: Use ng-bootstrap for Popover
Fixes: https://tracker.ceph.com/issues/45753

Signed-off-by: Tiago Melo <tmelo@suse.com>
2020-06-04 12:53:49 +00:00
Mykola Golub
101855d467
Merge pull request #35348 from dillaman/wip-45714
rbd-mirror: stop local journal replayer first during shut down

Reviewed-by: Mykola Golub <mgolub@suse.com>
2020-06-04 13:57:01 +03:00
Mykola Golub
8bbb6ee33d
Merge pull request #35339 from dillaman/wip-45803
rbd-mirror: don't hold (stale) copy of local image journal pointer

Reviewed-by: Mykola Golub <mgolub@suse.com>
2020-06-04 13:56:10 +03:00
Lenz Grimmer
ea23de6fa6
Merge pull request #35290 from tspmelo/wip-navs
mgr/dashboard: Migrate Tabs from ngx-bootstrap to ng-bootstrap

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2020-06-04 11:17:34 +02:00
Lenz Grimmer
21f349f755
Merge pull request #34934 from tspmelo/wip-testbed-inject
mgr/dashboard: Use TestBed.inject instead of TestBed.get

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2020-06-04 11:16:47 +02:00
Kiefer Chang
5446298339 mgr/cephadm: refactor templating in services
Using Jinja2 to render configs for services:
- Grafana
- Alertmanager
- Prometheus
- NFS
- iSCSI

Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
2020-06-04 16:23:42 +08:00
Kiefer Chang
64bb27f2f6 mgr/cephadm: add template engine Jinja2
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
2020-06-04 16:23:39 +08:00
Sebastian Wagner
3333db8346
Merge pull request #35359 from ceph/docs_fix_all_available_devices
doc/cephadm: align to all-available-devices flag

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-06-04 10:06:52 +02:00
Yuri Weinstein
b8f632327f
Merge pull request #35279 from badone/wip-py2-fix-osd-scrub-repair.sh
qa/*/osd-scrub-repair.sh: Convert to python3 print syntax

Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-06-03 11:12:21 -07:00
Kefu Chai
14c44071cf
Merge pull request #35363 from tchaikov/wip-qa-cronjob
qa/tests: do not pass "-t 2" to teuthology-suite when testing nautilus

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2020-06-03 23:14:13 +08:00
Kefu Chai
398524279d
Merge pull request #35326 from idryomov/wip-read-from-replica-ec
Objecter: don't attempt to read from non-primary on EC pools

Reviewed-by: Samuel Just <sjust@redhat.com>
2020-06-03 22:44:10 +08:00
Patrick Donnelly
4729688792
Merge PR #35202 into master
* refs/pull/35202/head:
	qa: return a string via getfattr

Reviewed-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
2020-06-03 07:43:08 -07:00
Kefu Chai
775d8a9f19
Merge pull request #35342 from shinhwagk/patch-2
cephadm: fix package is not running when enabler is not none

Reviewed-by: Varsha Rao <varao@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2020-06-03 22:42:22 +08:00
Patrick Donnelly
f3d52ec619
Merge PR #35328 into master
* refs/pull/35328/head:
	qa/tasks/cephfs/test_scrub.py: use umount_wait to avoid ceph-fuse stuck

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2020-06-03 07:41:09 -07:00
Mykola Golub
f9d49cab2d
Merge pull request #35310 from dillaman/wip-librbd-io-image-dispatch
librbd: fix IO processing deadlock and race conditions in image dispatcher

Reviewed-by: Mykola Golub <mgolub@suse.com>
2020-06-03 17:40:26 +03:00
Kefu Chai
1a1811e6fb
Merge pull request #35322 from tchaikov/wip-mgr-cephadm
mgr/cephadm: drop py2 compatible bits

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
2020-06-03 22:39:31 +08:00
Kefu Chai
50a1b806dd
Merge pull request #35288 from ZhenLiu94/lznba
ceph.in: add the missing right parenthesis in the help message

Reviewed-by: Varsha Rao <varao@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-06-03 22:38:49 +08:00
Kefu Chai
6adbce280e
Merge pull request #35337 from rzarzynski/wip-crimson-pidfile-err-handling
global, osd/crimson: improve handling of the crimson-osd instance duplication

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2020-06-03 22:36:40 +08:00
Tiago Melo
fdd8ec8ca9 mgr/dashboard: Migrate Tabs from ngx-bootstrap to ng-bootstrap
Using ng-bootstrap for Tabs will allow us to easily implement some new features
like only loading 1 tab at a time (already implemented here) and
saving/restoring last opened tab.

Modified the table component to use a clone of the columns list.
Making changes directly to columns var was causing problem when the table was
loaded a second time.

Fixes: https://tracker.ceph.com/issues/45017

Signed-off-by: Tiago Melo <tmelo@suse.com>
2020-06-03 14:22:48 +00:00
Joshua Schmid
e9600666e0
doc/cephadm: align to all-available-devices flag
Signed-off-by: Joshua Schmid <jschmid@suse.de>
2020-06-03 16:01:11 +02:00
Jason Dillaman
87b2849fc2 librbd: clear pending flush contexts on shut down
The IOs are already dispatched but are not cleared from the
container. This can result in an assertion failure in the destructor.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-06-03 09:45:24 -04:00
Jason Dillaman
2aa0e11b8f librbd: dropped ImageRequestWQ temporary code from exclusive-lock dispatch
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-06-03 09:45:24 -04:00
Jason Dillaman
c62926f511 librbd: restore missing flush on write-block logic
When creating the new image dispatch layer, the original flush
upon write-block was dropped. This is resulting in some random race
conditions where object IO is still in-flight when the write-block
indicates it's complete.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-06-03 09:45:24 -04:00
Jason Dillaman
3457f0bf28 librbd: drop exclusive-lock image dispatch lock before acquiring lock
This avoids a potential race with the exclusive lock being acquired by
another thread while the image dispatch layer attempts to acquire it.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-06-03 09:45:24 -04:00
Jason Dillaman
48456f8c5f librbd: exclusive lock image dispatch should not wait on IO when setting lock
IO from later dispatch layers might have caused the need to acquire the lock
(i.e. like an image refresh). In that case, the IO will be blocked waiting for
the exclusive lock to be acquired -- but will deadlock waiting for the IO
to flush.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-06-03 09:45:23 -04:00
Jason Dillaman
975a0d253a rbd-mirror: track journal replay flush requests to prevent race
If a journal replay flush is in-progress when the ImageReplayer is stopped,
it can race and result in an assertion failure due to two attempted shutdowns
of the same journal replay state machine.

Fixes: https://tracker.ceph.com/issues/45409
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-06-03 09:40:32 -04:00
Lenz Grimmer
b6dcd15ac4
Merge pull request #35051 from Devp00l/wip-44620
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2020-06-03 15:39:06 +02:00
Kefu Chai
f4126326b9 qa/tests: do not pass "-t 2" to teuthology-suite when testing nautilus
this change partially reverts
21dacc823b

see also #34171

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-06-03 20:58:48 +08:00
Kefu Chai
8ba016e48b
Merge pull request #35360 from tchaikov/wip-rpm-cephfs-metrics
ceph.spec.in: claim ownership of include/cephfs/metrics

Reviewed-by: Nathan Cutler <ncutler@suse.com>
2020-06-03 20:57:55 +08:00
Kefu Chai
d234e5b06f ceph.spec.in: claim ownership of include/cephfs/metrics
otherwise, "osc build" fails like:

[ 5545s] libcephfs-devel-16.0.0-2172.ga59fb9ffac.x86_64.rpm: directories not owned by a package:
[ 5545s]  - /usr/include/cephfs/metrics

this addresses a regression introduced by
c669de12e0

Signed-off-by: Kefu Chai <kchai@redhat.com>
2020-06-03 20:08:33 +08:00
Jason Dillaman
e94bee3278 Revert "rbd-mirror: wait for events to replay before shut down journal replay"
This reverts commit aeccb03303.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2020-06-03 07:50:32 -04:00
Kefu Chai
bfb08d7b14
Merge pull request #35352 from tchaikov/wip-vstart-runner-create-cluster-only
qa/tasks/vstart_runner: do not teardown test_path if "create-cluster-…

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
2020-06-03 17:56:00 +08:00
Tiago Melo
ffc74cefd2 mgr/dashboard: Use TestBed.inject instead of TestBed.get
This new method has the same behavior, but is type safe.

Fixes: https://tracker.ceph.com/issues/45326

Signed-off-by: Tiago Melo <tmelo@suse.com>
2020-06-03 09:10:30 +00:00
Lenz Grimmer
a59fb9ffac
Merge pull request #35300 from tspmelo/wip-ng-typeahead
mgr/dashboard: Use ng-bootstrap for Typeahead

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2020-06-03 10:55:51 +02:00
Lenz Grimmer
a9fb9b3e57
Merge pull request #35031 from tspmelo/wip-get-current-summary
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2020-06-03 10:53:16 +02:00
Lenz Grimmer
429bd4733c
Merge pull request #35282 from bk201/wip-45735
monitoring: fixing some issues in RBD detail dashboard

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2020-06-03 10:50:37 +02:00
Sebastian Wagner
cc8d535922
Merge pull request #35320 from sebastian-philipp/cephadm-more-type-annotations
cephadm: Add a bunch of type annotations

Reviewed-by: Michael Fritch <mfritch@suse.com>
2020-06-03 09:52:04 +02:00