Commit Graph

4660 Commits

Author SHA1 Message Date
David Zafman
51a45e796e qa/test-erasure-code.sh: Don't grep entire bluestore directory
Bluestore caused grep crash with "grep: memory exhausted" due to
size of "block" storage.

Fixes: http://tracker.ceph.com/issues/38678

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-03-11 18:47:29 -07:00
David Zafman
d4915ee503 qa: Don't create rbd pool because it creates an object
This also reverts commit 10b9626ea7.

Fixes: http://tracker.ceph.com/issues/38631

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-03-11 16:57:51 -07:00
David Zafman
8114a2619b qa: Can't wait for clean when there aren't any pools/PGs.
Fixes: http://tracker.ceph.com/issues/38678

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-03-11 16:02:48 -07:00
Patrick Donnelly
e7e4eea3a6
Merge PR #26818 into master
* refs/pull/26818/head:
	qa/cephfs: relax min_caps_per_client check

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-03-11 09:21:43 -07:00
Ilya Dryomov
7615012224 Merge PR #26858 into master
* refs/pull/26858/head:
	qa: krbd deep-flatten test
	qa/suites/krbd: enable deep-flatten feature

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-03-11 14:38:01 +01:00
Patrick Donnelly
58039163e3
Merge PR #26859 into master
* refs/pull/26859/head:
	qa: ignore slow metadata io wrn during osd thrash

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-03-10 10:38:16 -07:00
Sage Weil
2ad02fbfe3 qa/standalone/erasure-code/test-erasure-eio.sh: still need to create rbd pool
Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-09 09:34:49 -06:00
Sage Weil
10b9626ea7 qa/standalone/scrub/osd-scrub-repair: fix unfound grep
It's now "1/2 unfound":

             1/2 objects unfound (50.000%)

..presumably due to the rbd pool init creating the rbd_directory.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-08 18:23:48 -06:00
Sage Weil
30fc7f5e97 qa/standalone/ceph-helpers: fix test_wait_for_clean
Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-08 18:07:10 -06:00
Sage Weil
1e2b0c7252 qa/standalone/ceph-helpers.sh: fix test_run_mon
- Only create each osd once
- forget the first osdmap dump test; it's pointless

Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-08 17:43:00 -06:00
Sage Weil
bf74c1adc4 qa/standalone/osd/osd-rep-recov-eio: fix better
- no need for the default pool size
- no initial osds or it will collide with setup_osds later
- no need for rbd pool at all

Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-08 17:41:11 -06:00
Patrick Donnelly
5abcc32ff6
qa: ignore slow metadata io wrn during osd thrash
Fixes: http://tracker.ceph.com/issues/38651
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-03-08 10:49:10 -08:00
Ilya Dryomov
6892da1c0b qa: krbd deep-flatten test
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2019-03-08 18:14:37 +01:00
Sage Weil
3e83a6e960 Merge PR #26823 into master
* refs/pull/26823/head:
	qa/suites: disable valgrind leak checks on ceph-mgr
	mgr: skip shutdown and exit

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-03-08 09:04:21 -06:00
Sage Weil
62136d381a Merge PR #26794 into master
* refs/pull/26794/head:
	mon/MgrMonitor: only try to update always_on_modules if >= NAUTILUS
	qa/standalone/mon/msgr-v2-transition: add some tests for enabling msgr v2
	mon/MonmapMonitor: add 'ceph mon set-addrs <name> <addrvec>' command
	Revert "mon/MonClient: disable ms_bind_msgr2 if NAUTILUS feature not set"
	mon/OSDMonitor: use legacy_equals to compare osd addrs
	msg/msg_types: make legacy_equals() symmetrical
	mon/MDSMonitor: stop using get_orig_source_inst()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-03-07 22:12:52 -06:00
Sage Weil
4d33b6d56a Merge PR #26770 into master
* refs/pull/26770/head:
	qa/standalone/osd/osd-force-create-pg: create more pgs
	qa/standalone: make sure an osd is running before create_rbd_pool

Reviewed-by: Mykola Golub <mgolub@suse.com>
2019-03-07 22:10:12 -06:00
Sage Weil
c939eefa16 qa/standalone/mon/msgr-v2-transition: add some tests for enabling msgr v2
Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-07 16:35:35 -06:00
Sage Weil
ec7c9976d7 Merge PR #26802 into master
* refs/pull/26802/head:
	qa/suites/upgrade/mimic-x/parallel: run master rados/test.sh

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-03-07 13:49:58 -06:00
Sage Weil
e79dc454db qa/suites: disable valgrind leak checks on ceph-mgr
We've disabled the "clean" shutdown in ceph-mgr due to
https://tracker.ceph.com/issues/38621

Until then, no valgrind leak checks!

Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-07 13:03:28 -06:00
Sage Weil
4c5ed29925 Merge PR #26764 into master
* refs/pull/26764/head:
	mgr: 'osd df' by specified class or (crush) name
	mon/OSDMonitor: add 'osd crush get-device-class' command
2019-03-07 08:52:56 -06:00
Yan, Zheng
8e81bd74c5 qa/cephfs: relax min_caps_per_client check
new kernel client proactively release caps. caps count can go below
mds_min_caps_per_client

Fixes: http://tracker.ceph.com/issues/38270
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2019-03-07 21:32:20 +08:00
Ilya Dryomov
e50aa559f6 Merge PR #26775 into master
* refs/pull/26775/head:
	qa/suites/krbd/wac: bluestore snippet is placed incorrectly

Reviewed-by: Mike Christie <mchristi@redhat.com>
2019-03-07 12:19:36 +01:00
Sage Weil
a376a151ea qa/suites/upgrade/mimic-x/parallel: run master rados/test.sh
We rename ceph_test_rados_api_tier to add _pp, so the mimic version doesn't
work.  And in any case, at this stage the client host has master installed.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-06 16:50:12 -06:00
Sage Weil
b59ff3860f qa/standalone/osd/osd-force-create-pg: create more pgs
Avoid warnings about too few pgs.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-06 16:27:56 -06:00
Sage Weil
cba0483b09 qa/standalone: make sure an osd is running before create_rbd_pool
'rbd pool init' now does IO.  Drop the pool, or change the pool size to 1.

Fixes: http://tracker.ceph.com/issues/38585
Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-06 16:27:56 -06:00
Sebastian Wagner
7ba6bece41
Merge pull request #26633 from jtlayton/wip-nfs-scale
mgr/orchestrator: Allow the orchestrator to scale the NFS server count

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2019-03-06 19:08:48 +01:00
Jeff Layton
a256735d4c mgr/orchestrator: allow scaling the NFS server count up and down
Add a new 'ceph orchestrator nfs update' command that will take the
NFS clustername and a new count as arguments. That will get translated
to a StatelessServiceSpec and passed to update_stateless_service.

Also, add the necessary stubs to the test_orchestrator and the CLI
QA test.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
2019-03-06 07:15:14 -05:00
xie xingguo
af02d1031d mgr: 'osd df' by specified class or (crush) name
For large clusters, we use device classes to isolate storage pools.
The existing 'osd df' output turns out to be too nosiy, say, if
you care about only single storage pool with osds possibly spanning over
all hosts.

With this change you are now being able to do 'osd df' by class (or by pool,
if you simply use classes to separate different pools), or by a specified
crush bucket name you are currently interested in, which is much more
convenient.

Some examples:
```
$ bin/ceph osd df tree
ID CLASS WEIGHT  REWEIGHT SIZE   RAW USE DATA    OMAP META  AVAIL   %USE  VAR  PGS STATUS TYPE NAME
-1       0.05878        - 60 GiB 6.4 GiB  23 MiB  0 B 6 GiB  54 GiB 10.60 1.00   -        root default
-3       0.02939        - 30 GiB 3.2 GiB  12 MiB  0 B 3 GiB  27 GiB 10.60 1.00   -            host ceph11
 3   aaa 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  56     up         osd.3
 4   bbb 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  58     up         osd.4
 5   ccc 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  60     up         osd.5
-5       0.02939        - 30 GiB 3.2 GiB  12 MiB  0 B 3 GiB  27 GiB 10.60 1.00   -            host ceph12
 0   aaa 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  50     up         osd.0
 1   bbb 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  61     up         osd.1
 2   ccc 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  51     up         osd.2
                    TOTAL 60 GiB 6.4 GiB  23 MiB  0 B 6 GiB  54 GiB 10.60
MIN/MAX VAR: 1.00/1.00  STDDEV: 0

$ bin/ceph osd df tree class aaa
ID CLASS WEIGHT  REWEIGHT SIZE   RAW USE DATA    OMAP META  AVAIL   %USE  VAR  PGS STATUS TYPE NAME
-1       0.05878        - 20 GiB 2.1 GiB 7.8 MiB  0 B 2 GiB  18 GiB 10.60 1.00   -        root default
-3       0.02939        - 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00   -            host ceph11
 3   aaa 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  56     up         osd.3
-5       0.02939        - 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00   -            host ceph12
 0   aaa 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  50     up         osd.0
                    TOTAL 20 GiB 2.1 GiB 7.8 MiB  0 B 2 GiB  18 GiB 10.60
MIN/MAX VAR: 1.00/1.00  STDDEV: 0

$ bin/ceph osd df tree name ceph11
ID CLASS WEIGHT  REWEIGHT SIZE   RAW USE DATA    OMAP META  AVAIL   %USE  VAR  PGS STATUS TYPE NAME
-3       0.02939        - 30 GiB 3.2 GiB  12 MiB  0 B 3 GiB  27 GiB 10.60 1.00   -            host ceph11
 3   aaa 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  56     up         osd.3
 4   bbb 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  58     up         osd.4
 5   ccc 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  60     up         osd.5
                    TOTAL 30 GiB 3.2 GiB  12 MiB  0 B 3 GiB  27 GiB 10.60
MIN/MAX VAR: 1.00/1.00  STDDEV: 0

```

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-03-06 11:10:56 +08:00
Ilya Dryomov
7ab3153902 qa/suites/krbd/wac: bluestore snippet is placed incorrectly
Instead of generating three tests, each with bluestore-bitmap.yaml, it
generates four tests: one consisting of just bluestore-bitmap.yaml and
the other three without any trace of bluestore.  This was introduced in
commit 711df71790 ("qa: objectstore snippets for krbd").

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2019-03-05 23:07:27 +01:00
Xie Xingguo
ad8e7d33b1
Merge pull request #26729 from xiexingguo/wip-recovery-priority-restrictions
mon/OSDMonitor: add boundary check for pool recovery_priority

Reviewed-by: David Zafman <dzafman@redhat.com>
2019-03-05 20:16:18 +08:00
Ilya Dryomov
b550968d8a qa/suites/krbd: enable deep-flatten feature
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2019-03-05 10:10:34 +01:00
xie xingguo
733fff22bb mon/OSDMonitor: add boundary check for pool recovery_priority
See https://github.com/ceph/ceph/pull/26705

Fixes: http://tracker.ceph.com/issues/38578
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-03-05 08:15:39 +08:00
Sage Weil
bd52f35fd0 Merge PR #26444 into master
* refs/pull/26444/head:
	ceph_features: deprecate MON_SINGLE_PAXOS
	osdmap: map versions to releases
	mon: retire some `osd set/unset` flags

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-03-04 17:26:12 -06:00
Sage Weil
01316aa7bd qa/standalone/osd/pg-split-merge: fix import_after_merge_and_gap
This test introduces a map gap.  What *should* happen is that when there is
such a gap, we cannot import.  Previously, the test didn't reliably produce
a map gap at all, and didn't check that import failed--it verified that it
passed.

Fix the test so that it reliably produces a gap *and* reports
min_last_epoch_clean to the mon so we can trim.  Then verify we fail to
import, but can with --force.  But remove the pg again, because if we
force an import with a map gap the osd will refuse to start.

Fixes: http://tracker.ceph.com/issues/38525
Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-03 10:23:27 -06:00
Joao Eduardo Luis
dfafdf3dae mon: retire some osd set/unset flags
Signed-off-by: Joao Eduardo Luis <joao@suse.de>
2019-03-02 11:36:23 +00:00
Patrick Donnelly
eac565f7b0
Merge PR #26659 into master
* refs/pull/26659/head:
	qa: remove libcephfs-java tests

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2019-03-01 06:53:20 -08:00
Patrick Donnelly
06e7b12d24
Merge PR #26699 into master
* refs/pull/26699/head:
	qa: unmount clients before deleting fs

Reviewed-by: Zheng Yan <zyan@redhat.com>
2019-03-01 06:46:46 -08:00
Patrick Donnelly
a2dac30db5
Merge PR #26700 into master
* refs/pull/26700/head:
	qa: bump fsstress timeout to 6h

Reviewed-by: Zheng Yan <zyan@redhat.com>
2019-03-01 06:44:36 -08:00
Patrick Donnelly
1f5892f820
Merge PR #26638 into master
* refs/pull/26638/head:
	doc: update documentation for standby-replay
	qa: update discontinous map test to use mds freezing
	mon: add freeze MDS command
	qa: update testing for standby-replay
	mon: add setting for fs to enable standby-replay
	ceph-mds: obsolete hot-standby option
	fs: obsolete standby_for config options
	messages/MMDSBeacon: use inline init
	mds: avoid unnecessary copy of entity_addrvec_t
	mds: use inline init for mds_info_t
	mds: use rank from MDSMap always
	mds: remove obsolete comment
	qa: use SIGTERM when stopping vstart service

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>
2019-02-28 20:37:13 -08:00
Patrick Donnelly
df62f2d91d
qa: remove libcephfs-java tests
These have bit-rotted and no longer work. No cycles from interested parties
available to fix.

Fixes: https://tracker.ceph.com/issues/38487
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-02-28 13:32:15 -08:00
Patrick Donnelly
f9146f2acc
qa: bump fsstress timeout to 6h
When run with valgrind, it takes a significant amount of time to complete.

Fixes: http://tracker.ceph.com/issues/38520
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-02-28 09:35:53 -08:00
Patrick Donnelly
a20a61f476
qa: unmount clients before deleting fs
Client unmount during test cleanup will hang if the file system was deleted.

Fixes: http://tracker.ceph.com/issues/38518
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-02-28 09:02:33 -08:00
Patrick Donnelly
4fa4eda9ee
qa: update discontinous map test to use mds freezing
and stop using standby_for_*.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-02-27 21:39:17 -08:00
Patrick Donnelly
8cbdad9f9b
qa: update testing for standby-replay
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-02-27 21:39:12 -08:00
Patrick Donnelly
2ea41574a2
qa: use SIGTERM when stopping vstart service
Otherwise MDS will not send down:dne to mons when restarting.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-02-27 21:38:24 -08:00
Mykola Golub
17daad8536
Merge pull request #26523 from dillaman/wip-38387
librbd: add missing shutdown states to managed lock helper

Reviewed-by: Mykola Golub <mgolub@suse.com>
2019-02-27 20:49:09 +02:00
Mykola Golub
3be6a53790
Merge pull request #26432 from dillaman/wip-librbd-discard
librbd: skip small, unaligned discard extents by default

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
2019-02-27 17:11:44 +02:00
Lenz Grimmer
532e16c538
Merge pull request #26116 from votdev/feature_37934
mgr/dashboard: Configure all mgr modules in UI

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2019-02-27 11:01:25 +01:00
Lenz Grimmer
aded02d065
Merge pull request #26481 from votdev/gitignore_wheelhouse
mgr: Ignore __pycache__ and wheelhouse dirs

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2019-02-26 15:44:42 +01:00
Volker Theile
c16d8f2964 mgr/dashboard: Configure all mgr modules in UI
Fixes: https://tracker.ceph.com/issues/37934

Signed-off-by: Volker Theile <vtheile@suse.com>
2019-02-26 14:12:07 +01:00