Commit Graph

4843 Commits

Author SHA1 Message Date
Kefu Chai
6f1fd44bbc
Merge pull request #27011 from liewegas/wip-redirect-mimic
qa/suites/upgrade/mimic-x/parallel: enable all classes

Reviewed-by: Myoungwon Oh <omwmw@sk.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-03-18 01:22:44 +08:00
Sage Weil
b71a766d19 qa/suitse/upgrade/luminous-x/stress-split-erasure-code: disable c-o-t tests
The stress-split thrasher already had this off, but the ec variant did
not.  We don't support ceph-objectstore-tool exports/imports between major
versions.

Fixes: http://tracker.ceph.com/issues/38294
Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-17 04:59:36 -05:00
Sage Weil
bfd0581c5f qa/suites/upgrade/mimic-x/parallel: enable all classes
This fixes the redirect tiering tests, which need the 'cas' class.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-16 15:27:29 -05:00
Brad Hubbard
db046d007c qa/ceph-ansible: Move to Nautilus
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2019-03-16 12:40:04 +10:00
Sage Weil
3ee78a9d64 Merge PR #26965 into nautilus
* refs/pull/26965/head:
	ms/async/ProtocolV2: add ms_die_on_bug and assert rxbuf/txbuf don't get big
	msg/async/ProtocolV2: do not reenable pre_auth buffering on from reset_recv_state

Reviewed-by: Ricardo Dias <rdias@suse.com>
2019-03-15 12:08:34 -05:00
Sage Weil
4f41800386 ms/async/ProtocolV2: add ms_die_on_bug and assert rxbuf/txbuf don't get big
Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-15 06:42:29 -05:00
Sage Weil
13d7c4f4ec Merge PR #26898 into nautilus
* refs/pull/26898/head:
	osd/PG: invalidate PG if merging with unexpected version
	osd,mon: include more pg merge metadata in pg_pool_t
	qa/standalone/osd/pg-split-merge.sh: reproduce pg merge problem with empty pgs
	osd: add osd_debug_no_{acting_change,purge_strays}

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-03-14 22:37:18 -05:00
Patrick Donnelly
7de8cb405c
Merge PR #26935 into nautilus
* refs/pull/26935/head:
	qa: extend MDS heartbeat grace for valgrind

Reviewed-by: Sage Weil <sage@redhat.com>
2019-03-13 20:37:03 -07:00
Patrick Donnelly
505a05f351
Merge PR #26916 into nautilus
* refs/pull/26916/head:
	qa: ignore MON_DOWN for volume-client testing

Reviewed-by: Sage Weil <sage@redhat.com>
2019-03-13 20:31:01 -07:00
Sage Weil
4bb4f7a891 Merge PR #26894 into nautilus
* refs/pull/26894/head:
	qa/standalone/erasure-code/test-erasure-code: adjust test to avoid m=0
	erasure-code: ensure m >= 1
	mon/OSDMonitor: set ec min_size to k + min(1, m - 1)

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-03-13 22:07:45 -05:00
Sage Weil
52d5797c3d qa/standalone/erasure-code/test-erasure-code: adjust test to avoid m=0
_DD is k=2 m=0, which we don't allow.  Switch it to cDD.

I confess I don't fully understand why this was _DD to begin with, but
I'm pretty sure mapping is there to control the order of results so that
it can be mapped to the CRUSH rule output sanely, and the coding portion
is not relevant to the test.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-13 12:46:50 -05:00
Patrick Donnelly
7b520755ce
qa: extend MDS heartbeat grace for valgrind
Valgrind makes the MDS slowwwww. The newish mds_heartbeat_grace config allows
us to keep sending beacons to the mons even if the internal heartbeat is slow.
This avoids the laggy messages which are useful to grep for unrelated messaging
issues.

Fixes: http://tracker.ceph.com/issues/38723
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-03-13 09:18:32 -07:00
Sage Weil
96b837830c Merge PR #26920 into master
* refs/pull/26920/head:
	qa/tasks/mgr/test_module_selftest: fix localized value test
	mgr/BaseMgrStandbyModule: parse prefix properly

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2019-03-13 08:16:20 -05:00
Volker Theile
86f47f6bfd mgr/dashboard: Add separate option to config SSL port
There is a need to introduce this new config option because the MgrModule::get_module_option() and MgrModule::get_localized_module_option() method will be refactored soon and will not support the default parameter anymore. Instead the default value must be configured in the MODULE_OPTIONS. Currently we misuse the server_port depending on if SSL is enabled or not.

Fixes: https://tracker.ceph.com/issues/38331

Signed-off-by: Volker Theile <vtheile@suse.com>
2019-03-13 13:50:14 +01:00
Sage Weil
ebdd003bf4 qa/tasks/mgr/test_module_selftest: fix localized value test
When mgr/selftest/testkey = foo and mgr/selftest/x/testkey is not set,
then get_localized() should return foo.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-13 07:11:47 -05:00
Sage Weil
0eaad2d8d8 Merge PR #26886 into master
* refs/pull/26886/head:
	crush/CrushWrapper: ensure crush_choose_arg_map.size == max_buckets

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-03-13 06:56:16 -05:00
David Zafman
3ab9f38799
Merge pull request #26899 from dzafman/wip-38678
Minor cleanups in tests and log output

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-03-12 12:41:40 -07:00
Sage Weil
ab0a652826 erasure-code: ensure m >= 1
Fixes: http://tracker.ceph.com/issues/38682
Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-12 13:12:58 -05:00
Patrick Donnelly
4f3df2cc82
Merge PR #26893 into master
* refs/pull/26893/head:
	qa: unmount clients prior to marking fs down

Reviewed-by: Zheng Yan <zyan@redhat.com>
2019-03-12 10:47:53 -07:00
Patrick Donnelly
1ceadf0f07
qa: ignore MON_DOWN for volume-client testing
The test restarts the monitors.

Fixes: http://tracker.ceph.com/issues/38704
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-03-12 10:38:55 -07:00
Patrick Donnelly
c859be5022
Merge PR #26892 into master
* refs/pull/26892/head:
	qa: stop testing simple messenger in CephFS suites

Reviewed-by: Sage Weil <sage@redhat.com>
2019-03-12 10:26:27 -07:00
Sage Weil
ccda488815 crush/CrushWrapper: ensure crush_choose_arg_map.size == max_buckets
The crush/builder.c crush_add_bucket method resizes the max_buckets array
but a power of 2 when it has to expand, but the code in CrushWrapper was
assuming that if the array grew the pos for the new bucket would be the
last position in the new array.  This led to a situation where the
crush_choose_arg_map args array size didn't match max_buckets, and
eventually caused a crash.

Fixes: http://tracker.ceph.com/issues/38664
Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-12 11:26:43 -05:00
Sage Weil
fb915c4805 osd/PG: invalidate PG if merging with unexpected version
If the source or target PG version is 0'0, we may silently take the max
of the source and target and still leave the PG complete.  This
specifically can happen with an empty PG, as seen with bug 38655.  In
theory we could encounter one of the PGs with some other last_update
that doesn't match what we expect.  If that ever happens, make sure the
result is incomplete so that backfill can clean up.

Additionally check that the pool metadata for the last merge matches the
PGs at all.  This could mismatch if we have an osdmap gap and are forced
to do some merge without merge info at all... in which case we should
definitely invalidate: there should be newer copies of the PG(s), and we
have no idea whether the PGs we are merging are what we want.  If this is
some disaster recovery situation, an operator is always free to use
ceph-objectstore-tool to re-mark a PG complete (at their own peril!).

Fixes: http://tracker.ceph.com/issues/38655
Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-12 10:08:46 -05:00
David Zafman
51a45e796e qa/test-erasure-code.sh: Don't grep entire bluestore directory
Bluestore caused grep crash with "grep: memory exhausted" due to
size of "block" storage.

Fixes: http://tracker.ceph.com/issues/38678

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-03-11 18:47:29 -07:00
David Zafman
d4915ee503 qa: Don't create rbd pool because it creates an object
This also reverts commit 10b9626ea7.

Fixes: http://tracker.ceph.com/issues/38631

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-03-11 16:57:51 -07:00
David Zafman
8114a2619b qa: Can't wait for clean when there aren't any pools/PGs.
Fixes: http://tracker.ceph.com/issues/38678

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-03-11 16:02:48 -07:00
Sage Weil
f978b27d2b qa/standalone/osd/pg-split-merge.sh: reproduce pg merge problem with empty pgs
This reproduces http://tracker.ceph.com/issues/38655

Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-11 17:10:28 -05:00
Volker Theile
bc9643657a mgr: Fix broken get_localized_module_option function
Fixes: https://tracker.ceph.com/issues/38560

Signed-off-by: Volker Theile <vtheile@suse.com>
2019-03-11 17:25:18 +01:00
Patrick Donnelly
e7e4eea3a6
Merge PR #26818 into master
* refs/pull/26818/head:
	qa/cephfs: relax min_caps_per_client check

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-03-11 09:21:43 -07:00
Patrick Donnelly
9aaf6118a4
qa: unmount clients prior to marking fs down
Evicted RHEL7.5 clients may hang.

Fixes: http://tracker.ceph.com/issues/38677
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-03-11 09:15:37 -07:00
Patrick Donnelly
897a1f7385
qa: stop testing simple messenger in CephFS suites
Simple messenger is on it's way out and it doesn't work with msgr2.

Fixes: http://tracker.ceph.com/issues/38676
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-03-11 09:06:32 -07:00
Ilya Dryomov
7615012224 Merge PR #26858 into master
* refs/pull/26858/head:
	qa: krbd deep-flatten test
	qa/suites/krbd: enable deep-flatten feature

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-03-11 14:38:01 +01:00
Patrick Donnelly
58039163e3
Merge PR #26859 into master
* refs/pull/26859/head:
	qa: ignore slow metadata io wrn during osd thrash

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-03-10 10:38:16 -07:00
Sage Weil
2ad02fbfe3 qa/standalone/erasure-code/test-erasure-eio.sh: still need to create rbd pool
Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-09 09:34:49 -06:00
Sage Weil
10b9626ea7 qa/standalone/scrub/osd-scrub-repair: fix unfound grep
It's now "1/2 unfound":

             1/2 objects unfound (50.000%)

..presumably due to the rbd pool init creating the rbd_directory.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-08 18:23:48 -06:00
Sage Weil
30fc7f5e97 qa/standalone/ceph-helpers: fix test_wait_for_clean
Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-08 18:07:10 -06:00
Sage Weil
1e2b0c7252 qa/standalone/ceph-helpers.sh: fix test_run_mon
- Only create each osd once
- forget the first osdmap dump test; it's pointless

Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-08 17:43:00 -06:00
Sage Weil
bf74c1adc4 qa/standalone/osd/osd-rep-recov-eio: fix better
- no need for the default pool size
- no initial osds or it will collide with setup_osds later
- no need for rbd pool at all

Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-08 17:41:11 -06:00
Patrick Donnelly
5abcc32ff6
qa: ignore slow metadata io wrn during osd thrash
Fixes: http://tracker.ceph.com/issues/38651
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-03-08 10:49:10 -08:00
Ilya Dryomov
6892da1c0b qa: krbd deep-flatten test
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2019-03-08 18:14:37 +01:00
Sage Weil
3e83a6e960 Merge PR #26823 into master
* refs/pull/26823/head:
	qa/suites: disable valgrind leak checks on ceph-mgr
	mgr: skip shutdown and exit

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-03-08 09:04:21 -06:00
Sage Weil
62136d381a Merge PR #26794 into master
* refs/pull/26794/head:
	mon/MgrMonitor: only try to update always_on_modules if >= NAUTILUS
	qa/standalone/mon/msgr-v2-transition: add some tests for enabling msgr v2
	mon/MonmapMonitor: add 'ceph mon set-addrs <name> <addrvec>' command
	Revert "mon/MonClient: disable ms_bind_msgr2 if NAUTILUS feature not set"
	mon/OSDMonitor: use legacy_equals to compare osd addrs
	msg/msg_types: make legacy_equals() symmetrical
	mon/MDSMonitor: stop using get_orig_source_inst()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-03-07 22:12:52 -06:00
Sage Weil
4d33b6d56a Merge PR #26770 into master
* refs/pull/26770/head:
	qa/standalone/osd/osd-force-create-pg: create more pgs
	qa/standalone: make sure an osd is running before create_rbd_pool

Reviewed-by: Mykola Golub <mgolub@suse.com>
2019-03-07 22:10:12 -06:00
Sage Weil
c939eefa16 qa/standalone/mon/msgr-v2-transition: add some tests for enabling msgr v2
Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-07 16:35:35 -06:00
Sage Weil
ec7c9976d7 Merge PR #26802 into master
* refs/pull/26802/head:
	qa/suites/upgrade/mimic-x/parallel: run master rados/test.sh

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-03-07 13:49:58 -06:00
Sage Weil
e79dc454db qa/suites: disable valgrind leak checks on ceph-mgr
We've disabled the "clean" shutdown in ceph-mgr due to
https://tracker.ceph.com/issues/38621

Until then, no valgrind leak checks!

Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-07 13:03:28 -06:00
Sage Weil
4c5ed29925 Merge PR #26764 into master
* refs/pull/26764/head:
	mgr: 'osd df' by specified class or (crush) name
	mon/OSDMonitor: add 'osd crush get-device-class' command
2019-03-07 08:52:56 -06:00
Yan, Zheng
8e81bd74c5 qa/cephfs: relax min_caps_per_client check
new kernel client proactively release caps. caps count can go below
mds_min_caps_per_client

Fixes: http://tracker.ceph.com/issues/38270
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2019-03-07 21:32:20 +08:00
Sebastian Wagner
61d53ed70e mgr/orchestrator: Add error handling to interface
Also:

* Small test_orchestrator refactorization
* Improved Docstring in MgrModule.remote
* Added `raise_if_exception` that raises Exceptions
* Added `OrchestratorError` and `OrchestratorValidationError`
* `_orchestrator_wait` no longer raises anything
* `volumes` model also calls `raise_if_exception`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2019-03-07 14:13:44 +01:00
Ilya Dryomov
e50aa559f6 Merge PR #26775 into master
* refs/pull/26775/head:
	qa/suites/krbd/wac: bluestore snippet is placed incorrectly

Reviewed-by: Mike Christie <mchristi@redhat.com>
2019-03-07 12:19:36 +01:00
Sage Weil
a376a151ea qa/suites/upgrade/mimic-x/parallel: run master rados/test.sh
We rename ceph_test_rados_api_tier to add _pp, so the mimic version doesn't
work.  And in any case, at this stage the client host has master installed.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-06 16:50:12 -06:00
Sage Weil
b59ff3860f qa/standalone/osd/osd-force-create-pg: create more pgs
Avoid warnings about too few pgs.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-06 16:27:56 -06:00
Sage Weil
cba0483b09 qa/standalone: make sure an osd is running before create_rbd_pool
'rbd pool init' now does IO.  Drop the pool, or change the pool size to 1.

Fixes: http://tracker.ceph.com/issues/38585
Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-06 16:27:56 -06:00
Sebastian Wagner
7ba6bece41
Merge pull request #26633 from jtlayton/wip-nfs-scale
mgr/orchestrator: Allow the orchestrator to scale the NFS server count

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2019-03-06 19:08:48 +01:00
Jeff Layton
a256735d4c mgr/orchestrator: allow scaling the NFS server count up and down
Add a new 'ceph orchestrator nfs update' command that will take the
NFS clustername and a new count as arguments. That will get translated
to a StatelessServiceSpec and passed to update_stateless_service.

Also, add the necessary stubs to the test_orchestrator and the CLI
QA test.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
2019-03-06 07:15:14 -05:00
xie xingguo
af02d1031d mgr: 'osd df' by specified class or (crush) name
For large clusters, we use device classes to isolate storage pools.
The existing 'osd df' output turns out to be too nosiy, say, if
you care about only single storage pool with osds possibly spanning over
all hosts.

With this change you are now being able to do 'osd df' by class (or by pool,
if you simply use classes to separate different pools), or by a specified
crush bucket name you are currently interested in, which is much more
convenient.

Some examples:
```
$ bin/ceph osd df tree
ID CLASS WEIGHT  REWEIGHT SIZE   RAW USE DATA    OMAP META  AVAIL   %USE  VAR  PGS STATUS TYPE NAME
-1       0.05878        - 60 GiB 6.4 GiB  23 MiB  0 B 6 GiB  54 GiB 10.60 1.00   -        root default
-3       0.02939        - 30 GiB 3.2 GiB  12 MiB  0 B 3 GiB  27 GiB 10.60 1.00   -            host ceph11
 3   aaa 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  56     up         osd.3
 4   bbb 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  58     up         osd.4
 5   ccc 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  60     up         osd.5
-5       0.02939        - 30 GiB 3.2 GiB  12 MiB  0 B 3 GiB  27 GiB 10.60 1.00   -            host ceph12
 0   aaa 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  50     up         osd.0
 1   bbb 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  61     up         osd.1
 2   ccc 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  51     up         osd.2
                    TOTAL 60 GiB 6.4 GiB  23 MiB  0 B 6 GiB  54 GiB 10.60
MIN/MAX VAR: 1.00/1.00  STDDEV: 0

$ bin/ceph osd df tree class aaa
ID CLASS WEIGHT  REWEIGHT SIZE   RAW USE DATA    OMAP META  AVAIL   %USE  VAR  PGS STATUS TYPE NAME
-1       0.05878        - 20 GiB 2.1 GiB 7.8 MiB  0 B 2 GiB  18 GiB 10.60 1.00   -        root default
-3       0.02939        - 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00   -            host ceph11
 3   aaa 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  56     up         osd.3
-5       0.02939        - 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00   -            host ceph12
 0   aaa 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  50     up         osd.0
                    TOTAL 20 GiB 2.1 GiB 7.8 MiB  0 B 2 GiB  18 GiB 10.60
MIN/MAX VAR: 1.00/1.00  STDDEV: 0

$ bin/ceph osd df tree name ceph11
ID CLASS WEIGHT  REWEIGHT SIZE   RAW USE DATA    OMAP META  AVAIL   %USE  VAR  PGS STATUS TYPE NAME
-3       0.02939        - 30 GiB 3.2 GiB  12 MiB  0 B 3 GiB  27 GiB 10.60 1.00   -            host ceph11
 3   aaa 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  56     up         osd.3
 4   bbb 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  58     up         osd.4
 5   ccc 0.00980  1.00000 10 GiB 1.1 GiB 3.9 MiB  0 B 1 GiB 9.0 GiB 10.60 1.00  60     up         osd.5
                    TOTAL 30 GiB 3.2 GiB  12 MiB  0 B 3 GiB  27 GiB 10.60
MIN/MAX VAR: 1.00/1.00  STDDEV: 0

```

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-03-06 11:10:56 +08:00
Ilya Dryomov
7ab3153902 qa/suites/krbd/wac: bluestore snippet is placed incorrectly
Instead of generating three tests, each with bluestore-bitmap.yaml, it
generates four tests: one consisting of just bluestore-bitmap.yaml and
the other three without any trace of bluestore.  This was introduced in
commit 711df71790 ("qa: objectstore snippets for krbd").

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2019-03-05 23:07:27 +01:00
Xie Xingguo
ad8e7d33b1
Merge pull request #26729 from xiexingguo/wip-recovery-priority-restrictions
mon/OSDMonitor: add boundary check for pool recovery_priority

Reviewed-by: David Zafman <dzafman@redhat.com>
2019-03-05 20:16:18 +08:00
Ilya Dryomov
b550968d8a qa/suites/krbd: enable deep-flatten feature
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2019-03-05 10:10:34 +01:00
xie xingguo
733fff22bb mon/OSDMonitor: add boundary check for pool recovery_priority
See https://github.com/ceph/ceph/pull/26705

Fixes: http://tracker.ceph.com/issues/38578
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-03-05 08:15:39 +08:00
Sage Weil
bd52f35fd0 Merge PR #26444 into master
* refs/pull/26444/head:
	ceph_features: deprecate MON_SINGLE_PAXOS
	osdmap: map versions to releases
	mon: retire some `osd set/unset` flags

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-03-04 17:26:12 -06:00
Sage Weil
01316aa7bd qa/standalone/osd/pg-split-merge: fix import_after_merge_and_gap
This test introduces a map gap.  What *should* happen is that when there is
such a gap, we cannot import.  Previously, the test didn't reliably produce
a map gap at all, and didn't check that import failed--it verified that it
passed.

Fix the test so that it reliably produces a gap *and* reports
min_last_epoch_clean to the mon so we can trim.  Then verify we fail to
import, but can with --force.  But remove the pg again, because if we
force an import with a map gap the osd will refuse to start.

Fixes: http://tracker.ceph.com/issues/38525
Signed-off-by: Sage Weil <sage@redhat.com>
2019-03-03 10:23:27 -06:00
Joao Eduardo Luis
dfafdf3dae mon: retire some osd set/unset flags
Signed-off-by: Joao Eduardo Luis <joao@suse.de>
2019-03-02 11:36:23 +00:00
Patrick Donnelly
eac565f7b0
Merge PR #26659 into master
* refs/pull/26659/head:
	qa: remove libcephfs-java tests

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2019-03-01 06:53:20 -08:00
Patrick Donnelly
06e7b12d24
Merge PR #26699 into master
* refs/pull/26699/head:
	qa: unmount clients before deleting fs

Reviewed-by: Zheng Yan <zyan@redhat.com>
2019-03-01 06:46:46 -08:00
Patrick Donnelly
a2dac30db5
Merge PR #26700 into master
* refs/pull/26700/head:
	qa: bump fsstress timeout to 6h

Reviewed-by: Zheng Yan <zyan@redhat.com>
2019-03-01 06:44:36 -08:00
Patrick Donnelly
1f5892f820
Merge PR #26638 into master
* refs/pull/26638/head:
	doc: update documentation for standby-replay
	qa: update discontinous map test to use mds freezing
	mon: add freeze MDS command
	qa: update testing for standby-replay
	mon: add setting for fs to enable standby-replay
	ceph-mds: obsolete hot-standby option
	fs: obsolete standby_for config options
	messages/MMDSBeacon: use inline init
	mds: avoid unnecessary copy of entity_addrvec_t
	mds: use inline init for mds_info_t
	mds: use rank from MDSMap always
	mds: remove obsolete comment
	qa: use SIGTERM when stopping vstart service

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>
2019-02-28 20:37:13 -08:00
Patrick Donnelly
df62f2d91d
qa: remove libcephfs-java tests
These have bit-rotted and no longer work. No cycles from interested parties
available to fix.

Fixes: https://tracker.ceph.com/issues/38487
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-02-28 13:32:15 -08:00
Patrick Donnelly
f9146f2acc
qa: bump fsstress timeout to 6h
When run with valgrind, it takes a significant amount of time to complete.

Fixes: http://tracker.ceph.com/issues/38520
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-02-28 09:35:53 -08:00
Patrick Donnelly
a20a61f476
qa: unmount clients before deleting fs
Client unmount during test cleanup will hang if the file system was deleted.

Fixes: http://tracker.ceph.com/issues/38518
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-02-28 09:02:33 -08:00
Patrick Donnelly
4fa4eda9ee
qa: update discontinous map test to use mds freezing
and stop using standby_for_*.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-02-27 21:39:17 -08:00
Patrick Donnelly
8cbdad9f9b
qa: update testing for standby-replay
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-02-27 21:39:12 -08:00
Patrick Donnelly
2ea41574a2
qa: use SIGTERM when stopping vstart service
Otherwise MDS will not send down:dne to mons when restarting.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-02-27 21:38:24 -08:00
Brad Hubbard
d1892aaf6b qa/workunits/rest: Better detection of rest url
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2019-02-28 07:29:25 +10:00
Mykola Golub
17daad8536
Merge pull request #26523 from dillaman/wip-38387
librbd: add missing shutdown states to managed lock helper

Reviewed-by: Mykola Golub <mgolub@suse.com>
2019-02-27 20:49:09 +02:00
Mykola Golub
3be6a53790
Merge pull request #26432 from dillaman/wip-librbd-discard
librbd: skip small, unaligned discard extents by default

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
2019-02-27 17:11:44 +02:00
Lenz Grimmer
532e16c538
Merge pull request #26116 from votdev/feature_37934
mgr/dashboard: Configure all mgr modules in UI

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2019-02-27 11:01:25 +01:00
Patrick Donnelly
3e98b5e582
qa: update kclient testing to RHEL 7.6
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-02-26 13:10:58 -08:00
Lenz Grimmer
aded02d065
Merge pull request #26481 from votdev/gitignore_wheelhouse
mgr: Ignore __pycache__ and wheelhouse dirs

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2019-02-26 15:44:42 +01:00
Volker Theile
c16d8f2964 mgr/dashboard: Configure all mgr modules in UI
Fixes: https://tracker.ceph.com/issues/37934

Signed-off-by: Volker Theile <vtheile@suse.com>
2019-02-26 14:12:07 +01:00
Jason Dillaman
8410cb05bd qa/suites/rbd: enable partial discard test
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-02-26 07:42:36 -05:00
Jason Dillaman
9d694ba351 qa/workunits/rbd: delete pools before stopping rbd-mirror
This better mimics the behavior of teuthology and tests rbd-mirror
daemon's ability to handle a pool deletion.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-02-25 13:31:43 -05:00
Mykola Golub
8cd3062323 rbd-mirror: use active/active policy by default
Fixes: http://tracker.ceph.com/issues/38453
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-02-25 11:23:25 -05:00
Lenz Grimmer
55dd5e11ce
Merge pull request #26539 from tspmelo/wip-nfs-status
mgr/dashboard: Add NFS status endpoint 

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2019-02-25 13:12:22 +01:00
Xie Xingguo
be26cb08e5
Merge pull request #26560 from xiexingguo/wip-pool-force-recovery
mgr: add per pool force-recovery/backfill commands

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
2019-02-23 15:30:14 +08:00
xie xingguo
fa98f09674 mgr: add per pool force-recovery/backfill commands
For those with multiple storage pools sharing the same devices,
I think it would make much more sense to offer per-pool
commands to bring pools with high priority, e.g., because they
are hosting data of more importance than others, back to normal
quickly.

Fixes: http://tracker.ceph.com/issues/38456
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-02-23 12:18:24 +08:00
David Zafman
99af224af2
Merge pull request #26436 from dzafman/wip-bionic-core
run-standalone.sh: Need double-quotes to handle | in core_pattern on all distributions

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-02-22 12:52:37 -08:00
Lenz Grimmer
72ef9b58ec
Merge pull request #26510 from callithea/wip-add-ganesha-qa-tests
mgr/dashboard: Added NFS Ganesha suite to QA tests

Reviewed-by: Jeff Layton <jlayton@redhat.com>
2019-02-22 10:28:41 +01:00
Patrick Donnelly
0989abe3fe
Merge PR #26468 into master
* refs/pull/26468/head:
	qa: config recall settings to test cache drop
	qa: check cache dump works without timeout
	mds: add 2nd order recall throttle
	mds: drive log flush and cache trim during recall
	mds: avoid gather assertion when subs exist
	mds: output full details for recall threshold

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2019-02-21 10:26:25 -08:00
Sage Weil
0a7077a855 Merge PR #26237 into master
* refs/pull/26237/head:
	Revert "qa: update test_envlibrados_for_rocksdb.sh for libradospp split"
	doc/librados: explicitly mention that the C++ API is not stable
	ceph.spec: force use of upgrade devtoolset-gcc under RHEL 7
	librados: add symbol versioning to the C++ API
	librados: add symbol versioning to the C API
	librados: revert librados3/libradoscc back to librados2

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-02-21 11:18:46 -06:00
Mykola Golub
a51d9188bb
Merge pull request #26521 from dillaman/wip-38385
qa/workunits: fixed mon address parsing for rbd-mirror
2019-02-21 16:46:05 +02:00
Jason Dillaman
9683869442 Revert "qa: update test_envlibrados_for_rocksdb.sh for libradospp split"
This reverts commit 0475dfdea3.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-02-21 08:28:43 -05:00
Tiago Melo
824726393b mgr/dashboard: Add NFS status endpoint
Currently each time NFS page is opened and NFS Ganesha is not configured
an error notification is thrown and no extra information is given.
Now the user will be redirected to an information page.

Removed the orchestrator information since it no longer applies.

Fixes: http://tracker.ceph.com/issues/38399

Signed-off-by: Tiago Melo <tmelo@suse.com>
2019-02-21 11:14:25 +00:00
Xie Xingguo
6c43373687
Merge pull request #26532 from xiexingguo/wip-pool-scrub
mgr: add per pool scrub commands

Reviewed-by: Sage Weil <sage@redhat.com>
2019-02-21 13:26:15 +08:00
xie xingguo
97d55fd113 mgr: add per pool scrub commands
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-02-21 09:04:41 +08:00
Jason Dillaman
65c8733b56 librados: revert librados3/libradoscc back to librados2
For backwards compatibility and upgrade reasons, the librados2
API needs to be preserved and it needs to continue to be compatible
with dependent libraries like librbd1.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-02-20 14:59:26 -05:00
Sage Weil
d198831c44 Merge PR #26262 into master
* refs/pull/26262/head:
	mgr/orch/ssh: add qa test
	qa: install/exclude ceph-mgr-ssh package
	debian: package ssh orchestrator
	spec.ceph: package ssh orchestrator
	ssh/orch: add ssh orchestrator
2019-02-20 13:05:11 -06:00
Noah Watkins
06422208ec mgr/orch/ssh: add qa test
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2019-02-20 09:45:48 -08:00
Noah Watkins
a2e8960079 qa: install/exclude ceph-mgr-ssh package
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2019-02-20 09:45:48 -08:00
Sage Weil
618c0ea5da Merge PR #26417 into master
* refs/pull/26417/head:
	qa, doc: update ec fast read setting
	qa/suites: remove mon kv backend options

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-02-20 07:03:56 -06:00
Sage Weil
dd05d8ca26 Merge PR #26489 into master
* refs/pull/26489/head:
	qa/standalone/osd/osd-markdown: disable CLI command dups

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
2019-02-20 07:03:28 -06:00
Laura Paduano
bccf11f7ef mgr/dashboard: Added NFS Ganesha suite to QA tests
Signed-off-by: Laura Paduano <lpaduano@suse.com>
2019-02-20 10:27:33 +01:00
vasukulkarni
fc22d352f5
Merge pull request #26328 from badone/wip-update-ceph_stable_release-to-mimic
qa/ceph-ansible: Set ceph_stable_release to mimic
2019-02-19 18:11:59 -08:00
Jason Dillaman
5f28a6b003
Merge pull request #25945 from trociny/wip-migration-parent
librbd: make it possible to migrate parent images

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-02-19 14:52:03 -05:00
Jason Dillaman
3e0094910a qa/workunits: fixed mon address parsing for rbd-mirror
The test extracts the mon addresses from the monmap, but with the
recent v2 format change it extracted an invalid address.

Fixes: http://tracker.ceph.com/issues/38385
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-02-19 14:22:34 -05:00
Sage Weil
6e1598bf22 Merge PR #26485 into master
* refs/pull/26485/head:
	qa/suites/upgrade/luminous-x: force clone v1 format for final rbd python test

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-02-19 08:30:41 -06:00
Lenz Grimmer
34e7f09017
Merge pull request #26450 from s0nea/wip-dashboard-readonly-config-options-2
mgr/dashboard: disallow editing read-only config options (part 2)

Reviewed-by: Volker Theile <vtheile@suse.com>
2019-02-19 12:50:30 +01:00
Lenz Grimmer
84d077a02a
Merge pull request #25233 from p-na/wip-pna-qos-pr
mgr/dashboard: Add support for managing RBD QoS

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2019-02-19 12:43:40 +01:00
Patrick Nawracay
cfbefbf169 mgr/dashboard: Add support for managing RBD QoS
Fixes: http://tracker.ceph.com/issues/36191

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>
2019-02-19 11:08:59 +01:00
Volker Theile
c9a108509b mgr: Ignore __pycache__ and wheelhouse dirs
Signed-off-by: Volker Theile <vtheile@suse.com>
2019-02-19 10:09:55 +01:00
Mykola Golub
5e951cd3ae qa/workunits/rbd: add parent migration case
Signed-off-by: Mykola Golub <mgolub@suse.com>
2019-02-19 08:58:08 +00:00
Mykola Golub
d10d40fd49 rbd: add option to list all descendant images
Signed-off-by: Mykola Golub <mgolub@suse.com>
2019-02-19 08:58:08 +00:00
Patrick Donnelly
b28dcd9d75
qa: config recall settings to test cache drop
If we use the defaults, the MDS/client will recall/release everything quickly.
We want it to take time to see things like the timeout get hit.

Fixes: http://tracker.ceph.com/issues/38348
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-02-18 16:28:37 -08:00
Patrick Donnelly
b150b22d2c
qa: check cache dump works without timeout
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-02-18 16:28:37 -08:00
Sage Weil
c6a7b2cbd1 qa/standalone/osd/osd-markdown: disable CLI command dups
The markdown test is based on marking down a specific number of times, but
the duplicate commands from the CLI may not get absorbed/batched by the
mon, breaking the test.  Override the default qa/tasks/workunit.py
behavior of sending dups.

Fixes: http://tracker.ceph.com/issues/38359
Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-18 15:02:25 -06:00
Mykola Golub
04cc884df0 qa/suites/upgrade/luminous-x: force clone v1 format for final rbd python test
Fixes: https://tracker.ceph.com/issues/38346
Signed-off-by: Mykola Golub <mgolub@suse.com>
2019-02-18 16:23:13 +00:00
Jason Dillaman
1a74204952
Merge pull request #26226 from trociny/wip-rbd-re-sparsify
rbd: online re-sparsify of images

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-02-18 09:24:21 -05:00
Patrick Nawracay
1419346886 mgr/dashboard: Implement OSD purge
Removes the `remove` functionality from the UI.

Fixes: http://tracker.ceph.com/issues/35811

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>
2019-02-18 07:18:09 +00:00
Sage Weil
a60f332c32 Merge PR #24805 into master
* refs/pull/24805/head:
	qa/suite: add dedup test
	src/tools: fix compile error (master version issue)
	src/tools: add stats (fixed objects,total objects)
	src/tools: make room for cdc
	src/tools: make enhacned stats and interface class
	src/tools: set timelimit and add signal handler to check progress
	src/tools: use the slice thing and make parallel (chunk_scrub)
	src/test: add max-thread test in test_dedup_tool.sh
	src/tools: use the slice thing and make parallel
	src/test: add chunk-scrub test in test_dedup_tool.sh
	src/tools: add chunk-scrub op in dedup tool
	src/cls/cas: add has_chunk op
	src/test: add test_dedup_tool.sh
	src/tools: initial works for dedup tool

Reviewed-by: Sage Weil <sage@redhat.com>
2019-02-17 08:18:06 -06:00
Sage Weil
cc1a7ea972 Merge PR #26455 into master
* refs/pull/26455/head:
	qa/suites/upgrade/mimic-x/stress-split: drop pglog_hardlimit test
	qa/suites/upgrade/mimic-x/stress-split: update for msgr2
	qa/suites/upgrade/mimic-x/parallel: update for msgr v2

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-02-15 18:14:05 -06:00
David Zafman
afe8859e3e
Merge pull request #26416 from dzafman/wip-fix-test-loops
test: Limit loops waiting for force-backfill/force-recovery to happen

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-02-15 14:25:59 -08:00
Sage Weil
19be155c3e qa/suites/upgrade/mimic-x/stress-split: drop pglog_hardlimit test
Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-15 16:06:52 -06:00
Sage Weil
20beafab36 qa/suites/upgrade/mimic-x/stress-split: update for msgr2
Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-15 16:06:52 -06:00
Sage Weil
6207c0b3f4 qa/suites/upgrade/mimic-x/parallel: update for msgr v2
Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-15 16:06:52 -06:00
Tatjana Dehler
35c8a05a2c mgr/dashboard: check config option updatability also for bulk create
Fixes: https://tracker.ceph.com/issues/34528
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
2019-02-15 17:00:43 +01:00
Tatjana Dehler
a2d5f983ad mgr/dashboard: add missing test case for 60bda9ae42
Fixes: https://tracker.ceph.com/issues/34528
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
2019-02-15 16:26:30 +01:00
Mykola Golub
be2cdc0bcf rbd: online re-sparsify of images
Signed-off-by: Mykola Golub <mgolub@suse.com>
2019-02-15 11:12:00 +00:00
David Zafman
bc2df8c49b run-standalone.sh: Need double-quotes to handle | in core_pattern
on all distributions

OpenSUSE does not automatically add the | back when setting
the corepattern.  I tested this on openSUSE Leap 15.0.

Fixes: http://tracker.ceph.com/issues/38325

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-02-14 16:50:14 -08:00
David Zafman
c7dd8549b6 Revert "qa/standalone: discard pipe in sysctl kernel.core_pattern"
This reverts commit 5ba6286834.

Signed-off-by: David Zafman <dzafman@redhat.com>

Conflicts:
	qa/run-standalone.sh (reseting core_pattern moved to function)
2019-02-14 16:50:14 -08:00
David Zafman
6e8a3d074e
Merge pull request #26371 from dzafman/wip-38262
test: run-standalone.sh: point LD_LIBRARY_PATH to $(pwd)/lib

Reviewed-by: Kefu Chai <kchai@redhat.com>
2019-02-14 09:28:33 -08:00
Kefu Chai
9e561e9fb4
Merge pull request #26380 from tchaikov/wip-split-dashboard-package
deb,rpm,qa: split dashboard package

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2019-02-14 16:59:58 +08:00
Lenz Grimmer
c1f6e0e867
Merge pull request #25989 from votdev/feature_36488
mgr/dashboard: Add UI to configure the telemetry mgr plugin

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
2019-02-14 09:43:15 +01:00
David Zafman
64beabc4c6 test: Limit loops waiting for force-backfill/force-recovery to happen
Fixes: http://tracker.ceph.com/issues/38309

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-02-13 17:44:53 -08:00
Sage Weil
b20862c846 qa/suites/upgrade/luminous-x/parallel: enable all classes
Otherwise it's annoying because the class list changes between luminous and nautilus,
and we don't want to futz around with changing this setting during the upgrade.

The problematic classes are 'cas' (added) and 'sdk' (not enabled by default but
included by the cls/ workunit.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-13 14:35:18 -06:00
Sage Weil
525f3f7e34 qa/suites/upgrade/luminous-x/parallel/5-final-workload/rados_mon_thrash: use x branch
The luminous version is (1) not what we want and (2) will fail because
ceph_test_rados_api_tier no longer exists in master.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-13 14:35:18 -06:00
Sage Weil
28546e065d qa/suites/upgade/luminous-x: pglog_hardlimit succeeds now on luminous due to backport
Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-13 11:41:21 -06:00
Sage Weil
5947146423 qa/suites/upgrade/luminous-x/stress-split: mons on separate hosts, enable msgr2
Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-13 11:41:21 -06:00
Sage Weil
d260ba9f84 qa/suites/upgrade/luminous-x/parallel: mon per host, msgr2
- mon per host
- enable msgr2 at the end
- include the nautilus.yaml

Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-13 11:41:21 -06:00
Kefu Chai
3d128d040e qa: exclude new packages when installing old versions
ceph-mgr-dashboard is introduced in nautilus, so exclude it when
installing old versions.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2019-02-13 21:37:30 +08:00
Sage Weil
d93abd8c53 qa/suites/upgrade/luminous-x: whitelist 'slow request'
The luminous warnings say slow request.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-13 07:20:51 -06:00
Sage Weil
96f58cd548 Merge PR #26265 into master
* refs/pull/26265/head:
	qa/suites/rados/upgrade: whitelist 'slow request'

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-02-13 06:40:41 -06:00
Sebastian Wagner
b8fe1554d3
Merge pull request #26346 from noahdesu/mgr-orch-cache-refresh
mgr/orch: refresh option for inventory query

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2019-02-13 11:13:52 +01:00
David Zafman
8bc623b949 test: run-standalone.sh set local library location so mgr can find librados.so
This caused qa/standalone/misc/test-ceph-helpers.sh to fail
    "MGR_MODULE_DEPENDENCY 8 mgr modules have failed dependencies"

Fixes: http://tracker.ceph.com/issues/38262

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-02-12 09:22:35 -08:00
Volker Theile
b9177e43c0 mgr/dashboard: Add UI to configure the telemetry mgr plugin
Fixes: tracker.ceph.com/issues/36488

Signed-off-by: Volker Theile <vtheile@suse.com>
2019-02-12 16:36:56 +01:00
Noah Watkins
96c8a48010 pybind/orch: add refresh option to inventory query
allow a user of the orchestrator interface to express that the inventory
query should not read from any cached inventory state.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2019-02-11 15:27:21 -08:00
Sage Weil
0a1ac6418d Merge PR #26282 into master
* refs/pull/26282/head:
	mon: include progress events in 'ceph status' output
	mon/MgrStatMonitor: persist progress events
	mgr: send progress events to mon
	mgr/progress: revise message syntax a bit
	mgr/progress: report events into mgr
	mgr: allow progress events to be reported from modules
	mgr/progress: make persist period/interval configurable
	mgr/progress: prune completed events

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2019-02-11 17:26:19 -06:00
Sage Weil
059844f9b9 Merge PR #26336 into master
* refs/pull/26336/head:
	qa/tasks/keystone.py: no need for notcmalloc in example
	qa/suites/rgw/tempest/tasks/rgw_tempest: no need for notcmalloc

Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
2019-02-11 15:47:29 -06:00
Ilya Dryomov
400686efd9
Merge pull request #26339 from idryomov/wip-krbd-namespaces-test
qa: krbd namespaces test

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2019-02-11 16:00:32 +01:00
Sebastian Wagner
ad7abc3995
Merge pull request #26314 from sebastian-philipp/orchestrator-host-mon-mgr
mgr/orchestrator: Add host mon mgr management to interface

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Noah Watkins <noahwatkins@gmail.com>
2019-02-11 09:52:53 +01:00
Sage Weil
b4fa47390d Merge PR #25977 into master
* refs/pull/25977/head:
	qa/suites: exclude new packages when installing old versions
	rpm: add dependency on python-kubernetes module to ceph-mgr-rook package
	rpm,deb: add rbd_support module to ceph-mgr
	packaging: split ceph-mgr diskprediction and rook plugins into own packages

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2019-02-09 13:46:51 -06:00
myoungwon oh
89f41ad9ba qa/suite: add dedup test
Signed-off-by: Myoungwon Oh <omwmw@sk.com>
2019-02-09 12:45:10 +09:00
Josh Durgin
682def2e32 qa, doc: update ec fast read setting
31dd620883 renamed it but did not update the suites or docs

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2019-02-08 17:11:34 -05:00
Josh Durgin
d45f18119b qa/suites: remove mon kv backend options
rocksdb is the default, leveldb is not recommended at this point, so drop it.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2019-02-08 16:58:44 -05:00
Sage Weil
1d305f1264 mgr/progress: revise message syntax a bit
"osd.0", not "OSD 0"

Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-08 13:50:27 -06:00
Sage Weil
5cbdf32430 qa/suites: exclude new packages when installing old versions
Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-08 12:59:22 -06:00
Sage Weil
9c729a78be qa/tasks/keystone.py: no need for notcmalloc in example
Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-08 09:12:01 -06:00
Sage Weil
b8b398e1ab qa/suites/rgw/tempest/tasks/rgw_tempest: no need for notcmalloc
Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-08 09:11:07 -06:00
Ilya Dryomov
7fdb879004 qa: krbd namespaces test
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2019-02-08 15:29:20 +01:00
David Zafman
910a95b9c8 test: osd-backfill-stats.sh Fix check of multi backfill OSDs, skip remapped test
Signed-off-by: David Zafman <dzafman@redhat.com>
2019-02-07 20:05:58 -08:00
Brad Hubbard
d74cae117c qa/ceph-ansible: Set ceph_stable_release to mimic
Fixes: http://tracker.ceph.com/issues/38231

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2019-02-08 12:10:00 +10:00
David Zafman
690ff9a21f
Merge pull request #26213 from dzafman/wip-38041
osd: Fix recovery and backfill priority handling

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2019-02-07 17:26:34 -08:00
David Zafman
ca5cf14fa8 test: Add scripts to test backfill/recovery priority handling
Signed-off-by: David Zafman <dzafman@redhat.com>
2019-02-07 15:46:23 -08:00
Sage Weil
d3766916a7 Merge PR #26059 into master
* refs/pull/26059/head:
	mon/MonClient: fix keepalive with v2 auth
	msg/async/ProtocolV2: reject peer_addrs of -
	msg/async/ProtocolV2: clean up feature management
	mon/MonClient: set up rotating_secrets, etc before msgr ready
	msg/async: let client specify preferred order of modes
	msg/async/ProtocolV2: include entity_name, features in reconnect
	msg/async/ProtocolV2: fix write_lock usage around AckFrame
	qa/suites/rados/verify/validator/valgrind: debug refs = 5
	qa/standalone/ceph-helpers: fix health_ok test
	auth/AuthRegistry: only complain about disabling cephx if cephx was enabled
	auth/AuthRegistry: fix locking for get_supported_methods()
	auth: remove AUTH_UNKNOWN weirdness, hardcoded defaults.
	msg/async/ProtocolV2: remove unused get_auth_allowed_methods
	osd: set up messener auth_* before setting dispatcher (and going 'ready')
	mon/AuthMonitor: request max_global_id increase from peon in tick
	mon: prime MgrClient only after messengers are initialized
	qa/suites/rados/workloads/rados_api_tests.yaml: debug mgrc = 20 on mon
	auth: document Auth{Client,Server} interfaces
	auth: future-proof AUTH_MODE_* a bit in case we need to change the encoding byte
	mon/MonClient: request monmap on open instead of ping
	mgr/PyModuleRegistry: add details for MGR_MODULE_{DEPENDENCY,ERROR}
	crimson: fix build
	mon/MonClient: finsih authenticate() only after we get monmap; fix 'tell mgr'
	mon: add auth_lock to protect auth_meta manipulation
	ceph-mon: set up auth before binding
	mon: defer initial connection auth attempts until initial quorum is formed
	mon/MonClient: make MonClientPinger an AuthCleint
	ceph_test_msgr: use DummyAuth
	auth/DummyAuth: dummy auth server and client for test code
	mon/Monitor: fix leak of auth_handler if we error out
	doc/dev/cephx: re-wordwrap
	doc/dev/cephx: document nautilus change to cephx
	vstart.sh: fix --msgr2 option
	msg/async/ProtocolV2: use shared_ptr to manage auth_meta
	auth/Auth{Client,Server}: pass auth_meta in explicitly
	mon/MonClient: behave if authorizer can't be built (yet)
	osd: set_auth_server on client_messenger
	common/ceph_context: get_moduel_type() for seastar cct
	auth: make connection_secret a std::string
	auth,msg/async/ProtocolV2: negotiate connection modes
	auth/AuthRegistry: refactor handling of auth_*_requred options
	osd,mgr,mds: remove unused authorize registries
	switch monc, daemons to use new msgr2 auth frame exchange
	doc/dev/msgr2: update docs to match implementation for auth frames
	auth/AuthClientHandler: add build_initial_request hook
	msg/Messenger: attach auth_client and/or auth_server to each Messenger
	auth: introduce AuthClient and AuthServer handlers
	auth: codify AUTH_MODE_AUTHORIZER
	msg/Connection: track peer_id (id portion of entity_name_t) for msgr2
	auth/AuthAuthorizeHandler: add get_supported_methods()
	auth/AuthAuthorizeHandler: fix args for verify_authorizer()
	auth: constify bufferlist arg to AuthAuthorizer::add_challenge()
	auth/cephx: share all tickets and connection_secret in initial reply
	msg/async,auth: add AuthConnectionMeta to Protocol
	auth/AuthClientHandler: pass in session_key, connection_secret pointers
	auth/AuthServiceHandler: take session_key and connection_secret as args
	auth/cephx: pass more specific type into build_session_auth_info
	mon/Session: separate session creation, peer ident, and registration
	mon/AuthMonitor: bump max_global_id from on_active() and tick()
	mon/AuthMonitor: be more careful with max_global_id
	mon: only all ms_handle_authentication() if auth method says we're done
	mon/AuthMonitor: fix "finished with auth" condition check
	auth: clean up AuthServiceHandler::handle_request() args
	auth: clean up AuthServiceHandler::start_session()
	mon/AuthMonitor: drop unused op arg to assign_global_id()
	msg/async: separate TAG_AUTH_REQUEST_MORE and TAG_AUTH_REPLY_MORE
	msg/async: consolidate authorizer checks
	msg/async: move get_auth_allowed into ProtocolV2.cc
	mon/MonClient: trivial cleanup

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2019-02-07 16:51:39 -06:00
Sage Weil
65e81e6eb4 qa/suites/rados/verify/validator/valgrind: debug refs = 5
If we detect a leak, let's include logging so we can find it.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-07 12:10:34 -06:00
Sage Weil
dcdca44aa4 qa/standalone/ceph-helpers: fix health_ok test
Stopping the osd daemon won't reliably get you HEALTH_WARN or ERR; you have
to make sure it is also marked down.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-07 12:10:34 -06:00
Sage Weil
ee59743a1a qa/suites/rados/workloads/rados_api_tests.yaml: debug mgrc = 20 on mon
Seeing some hangs when the mon is forwarding mgr commands (pg deep-scrub)
to the mgr.  This is a buggy test (it should send it to the mgr directly)
but it is helpful to verify the mon forwarding behavior works.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-07 12:10:34 -06:00
Yuri Weinstein
46e617926b
Merge pull request #26068 from badone/wip-ceph-ansible-switch-to-ansible-2.7
qa/tests: Move ceph-ansible tests to ansible version 2.7

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
2019-02-07 06:20:59 -08:00
Sebastian Wagner
1139c127d6 mgr/orchestrator: add tests to host, mgr and mon mgmt
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2019-02-07 12:23:51 +01:00
David Zafman
de98f2e0d9
Merge pull request #26290 from dzafman/wip-38027-38195
Wip 38027 38195: osd/osd-backfill-space.sh fails

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-02-06 08:41:40 -08:00
Lenz Grimmer
2275a41d09
Merge pull request #25918 from rjfd/wip-dashboard-nfs-backend
mgr/dashboard: NFS Ganesha management REST API

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
2019-02-06 16:41:49 +01:00
Sebastian Wagner
c41e6a025c
Merge pull request #26094 from sebastian-philipp/orchestrator-cli-command
mgr/orchestrator: make use of @CLICommand

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Noah Watkins <noahwatkins@gmail.com>
2019-02-06 10:57:45 +01:00
David Zafman
36e305c4b6 test: Ignore kill_daemons() error
Workaround for: http://tracker.ceph.com/issues/38195

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-02-05 11:31:32 -08:00
David Zafman
bca4fe98b1 test: Fix kill_daemon() to check after last large sleep
Signed-off-by: David Zafman <dzafman@redhat.com>
2019-02-05 11:30:04 -08:00
David Zafman
cc6339c0cd test: Increase timeouts in osd-backfill-space.sh because of failure seen
Fixes: http://tracker.ceph.com/issues/38027

Signed-off-by: David Zafman <dzafman@redhat.com>
2019-02-05 11:29:32 -08:00
Sebastian Wagner
d71a97b7fa mgr/orchestrator: make use of @CLICommand
Also Modified some command to allow multiple hosts
Also: Added more tests

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2019-02-05 11:27:47 +01:00
Ilya Dryomov
711df71790 qa: objectstore snippets for krbd
krbd was being tested with filestore, up until recently when the
default for osd_objectstore was changed to bluestore.  This broke
rbd_simple_big.yaml because bluestore_block_size defaults to 10G.
Pick up the sepia setting of 90G from bluestore-bitmap.yaml.

Run fsx subsuite with both filestore and bluestore.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2019-02-05 11:23:42 +01:00
Dan Mick
5c137e7052 packaging: split ceph-mgr diskprediction and rook plugins into own packages
There are troublesome dependencies for both, so being able to deselect
them is useful

Signed-off-by: Dan Mick <dan.mick@redhat.com>
2019-02-04 14:53:35 -08:00
Sage Weil
3cb57037b9 qa/suites/rados/upgrade: whitelist 'slow request'
failure_reason: '"2019-02-03 22:52:41.561332 osd.10 (osd.10) 790 : cluster [WRN] slow
  request 30.154662 seconds old, received at 2019-02-03 22:52:11.406639: osd_op(client.56148.0:39092
  8.9 8.70387d99 (undecoded) ondisk+retry+write+known_if_redirected e1372) currently
  waiting for peered" in cluster log'

We're restarting OSDs, and may see slow requests in the process.

Signed-off-by: Sage Weil <sage@redhat.com>
2019-02-03 23:28:54 -06:00
Ilya Dryomov
04f5b343f9 qa: update krbd tests for zeroout
Discard no longer guarantees zeroing, use BLKZEROOUT and "fallocate -z"
instead (blkdiscard(8) in xenial doesn't support -z).

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2019-02-03 10:57:07 +01:00
Ilya Dryomov
031bbea739 qa: krbd discard with alloc_size vs zeroout tests
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2019-02-02 18:33:32 +01:00
Sebastian Wagner
d361f77a78
Merge pull request #26171 from sebastian-philipp/orchestrator-merge-create-add
mgr/orchestrator: Unify `osd create` and `osd add`

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
2019-02-01 16:42:36 +01:00
Sebastian Wagner
9e926e9927 mgr/orchestrator: Unify osd create and osd add
Also:

* Added some more tests
* Better validation of drive Groups
* Simplified `TestWriteCompletion`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2019-02-01 10:10:59 +01:00
Patrick Donnelly
0d26266ccb
Merge PR #26038 into master
* refs/pull/26038/head:
	mds: simplify recall warnings
	mds: add extra details for cache drop output
	qa: test mds_max_caps_per_client conf
	mds: limit maximum number of caps held by session
	mds: adapt drop cache for incremental recall
	mds: recall caps incrementally
	mds: adapt drop cache for incremental trim
	mds: add throttle for trimming MDCache
	mds: cleanup SessionMap init
	mds: cleanup Session init

Reviewed-by: Zheng Yan <zyan@redhat.com>
2019-01-31 12:08:26 -08:00
Patrick Donnelly
c0b3a11484
mds: simplify recall warnings
Instead of a timeout and complicated decisions about whether the client is
releasing caps in an expeditious fashion, just use a DecayCounter that tracks
the number of caps we've recalled. This counter is decremented whenever the
client releases caps. If the counter passes a threshold, then we raise the
warning.

Similar reworking is done for the steady-state recall of client caps. Another
release DecayCounter is added so we can tell when the client is not releasing
any more caps.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-01-31 12:07:54 -08:00
Ricardo Dias
eabb054b1a
mgr/dashboard: ganesha: add support for sec_label_xattr CEPH fsal attribute
Signed-off-by: Ricardo Dias <rdias@suse.com>
2019-01-31 11:10:10 +00:00
Ricardo Dias
acd9fc679e
mgr/dashboard: ganesha: add 'security_label' support
Signed-off-by: Ricardo Dias <rdias@suse.com>
2019-01-31 11:10:10 +00:00
Ricardo Dias
e429afda53
qa/tasks/mgr/dashboard: NFS Ganesha tests
Signed-off-by: Ricardo Dias <rdias@suse.com>
2019-01-31 11:10:09 +00:00
Casey Bodley
aacd111e39
Merge pull request #26205 from cbodley/wip-38115
qa/mon: fix cluster support for monmap bootstrap

Reviewed-by: Sage Weil <sage@redhat.com>
2019-01-30 13:55:52 -05:00
David Zafman
70b5136208 test: Add option to wait_for_clean() to execute at every sleep
Signed-off-by: David Zafman <dzafman@redhat.com>
2019-01-30 09:35:51 -08:00
David Zafman
553d83dd24
Merge pull request #25403 from liyichao/rdigest
tools: Add clear-data-digest command to objectstore tool.

Reviewed-by: David Zafman <dzafman@redhat.com>
2019-01-30 09:30:23 -08:00
David Zafman
894bdf080e
Merge pull request #26158 from dzafman/wip-38053
Add hashinfo testing for dump command of ceph-objectstore-tool

Reviewed-by: Neha Ojha <nojha@redhat.com>
2019-01-30 09:29:01 -08:00
Casey Bodley
0aebb55af5 qa/mon: fix cluster support for monmap bootstrap
-filter out mons from other clusters
-fix parsing of mon name from role

Fixes: http://tracker.ceph.com/issues/38115

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2019-01-30 12:24:36 -05:00
Lenz Grimmer
fda70511a8
Merge pull request #26172 from rhcs-dashboard/fix-skipped-api-tests
mgr/dashboard: fix skipped backend API tests

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
2019-01-30 15:49:06 +01:00
Kefu Chai
8d5ddb5817
Merge pull request #26091 from tchaikov/wip-36737
cmake: use $CMAKE_BINARY_DIR for default $CEPH_BUILD_VIRTUALENV

Tested-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
2019-01-30 21:35:52 +08:00
Jason Dillaman
347eac67ba
Merge pull request #26168 from shyukri/qa-rbd-replace-bench-write
qa: rbd/workunits : Replace "rbd bench-write" with "rbd bench --io-type write"

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
2019-01-30 07:51:55 -05:00
alfonsomthd
c094e2d702 mgr/dashboard: fix skipped backend API tests
* When the creation of the cluster is delegated to vstart_runner.py
(--create or --create-target-only) the amount of MGRs required
is calculated by the script so there is no more skipped tests
due to insufficient amount of MGRs.

* Additionally, this issue is not reproducible anymore:
Fixes: https://tracker.ceph.com/issues/37964

* Fixed typo: TEUTHOLOFY_PY_REQS

Signed-off-by: Alfonso Martínez <almartin@redhat.com>
2019-01-30 13:05:02 +01:00
Patrick Donnelly
30aaa884bf
qa: test mds_max_caps_per_client conf
That the MDS will not let a client sit above mds_max_caps_per_client caps.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-01-29 15:16:31 -08:00
Patrick Donnelly
ef46216d8d
mds: recall caps incrementally
As with trimming, use DecayCounters to throttle the number of caps we recall,
both globally and per-session.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2019-01-29 15:16:30 -08:00
Mykola Golub
275bedc8d5
Merge pull request #26133 from dillaman/wip-37913
rbd: implement new 'rbd perf image iostat/iotop' commands

Reviewed-by: Mykola Golub <mgolub@suse.com>
2019-01-29 12:41:23 +02:00
Jason Dillaman
0474727294 rbd: simplify pool namespace positional argument handling
The pool and namespace can now be specified as in a
<pool-name>[/<namespace-name>] format as positional
arguments.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-01-28 14:36:44 -05:00