RepoMirrors/ceph
1
0
Fork 0
mirror of https://github.com/ceph/ceph synced 2025-02-24 19:47:44 +00:00
Code Issues Packages Projects Releases Wiki Activity
81,109 Commits 216 Branches 395 Tags 1.4 GiB
84660c977c
Commit Graph

3018 Commits

Author SHA1 Message Date
Patrick Donnelly
8a535d9c72
qa: get config only on running MDS 
Fixes: http://tracker.ceph.com/issues/21466

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
 2017-09-20 15:47:28 -07:00
Kefu Chai
73d4afbf8c Merge pull request #17747 from tchaikov/wip-qa 
qa/standalone: respect $TEMPDIR

Reviewed-by: David Zafman <dzafman@redhat.com>
 2017-09-20 23:08:47 +08:00
Kefu Chai
f27251432a Merge pull request #17785 from dzafman/wip-add-repair 
test: Fix ceph-objectstore-tool usage check

Reviewed-by: Kefu Chai <kchai@redhat.com>
 2017-09-20 12:35:16 +08:00
xie xingguo
b60d3498e2 osd/PrimaryLogPG: prepare_transaction - fix EDQUOT vs ENOSPC 
We now set full flag if a pool is currently running out of space and
set both full and full_no_quota flags if it is running out of quota.
Therefore the full_no_quota flag should be instead used to uniquely
identify whether we are running out of quota or not.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
 2017-09-20 08:49:42 +08:00
Vasu Kulkarni
e11a1f773d qa/tests: skip test that use public bucket landsat-pds from AWS 
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
 2017-09-19 17:12:15 -07:00
Vasu Kulkarni
184d5e6bad qa/tests: update required ceph-ansbile vars 
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
 2017-09-19 17:12:09 -07:00
Vasu Kulkarni
077904a0b5 qa/tests: Fix ceph-ansible upstream vars 
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
 2017-09-19 17:11:56 -07:00
Neha Ojha
11d8dfe591 qa/suites/rados/perf: create pool with lower pg_num 
Signed-off-by: Neha Ojha <nojha@redhat.com>
 2017-09-19 16:40:45 -07:00
Sage Weil
6767f841e5 Merge pull request #17427 from liewegas/wip-pg-num-limits 
mon/OSDMonitor: implement cluster pg limit

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
 2017-09-19 12:57:10 -05:00
huangjun
fa40add7f0 qa/ceph_manager: check pg state again before timedout 
Pg state maybe all in active+clean when no recovering going on,
so check it again before timedout.

Fixes: http://tracker.ceph.com/issues/21294

Signed-off-by: huangjun <huangjun@xsky.com>
 2017-09-20 00:04:04 +08:00
Josh Durgin
401ccd5966 Merge pull request #17786 from neha-ojha/wip-cbt-teuthology-integration-cntd 
qa/suites/rados/perf: add optimized settings

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
 2017-09-19 08:35:10 -07:00
huangjun
4d30d02e52 qa/suites/rest/basic/tasks/rest_test: more whitelisting 
Fixes: http://tracker.ceph.com/issues/21425

Signed-off-by: huangjun <huangjun@xsky.com>
 2017-09-19 16:13:34 +08:00
Neha Ojha
2635e7a591 qa/suites/rados/perf: add optimized settings 
Signed-off-by: Neha Ojha <nojha@redhat.com>
 2017-09-18 15:53:28 -07:00
David Zafman
0364ae104a test: Fix ceph-objectstore-tool usage check 
Caused by: c7b7a1f04f

Signed-off-by: David Zafman <dzafman@redhat.com>
 2017-09-18 15:29:22 -07:00
Casey Bodley
a97d5bea42 qa/rgw: hadoop suite targets centos_latest 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
 2017-09-18 13:29:40 -04:00
Casey Bodley
4a953a16d7 qa/rgw: use overrides for hadoop version 
instead of having two separate tasks, add v27.yaml and v28.yaml with
overrides for hadoop_version

Signed-off-by: Casey Bodley <cbodley@redhat.com>
 2017-09-18 13:28:24 -04:00
Nathan Cutler
2311b64025 tests: CentOS 7.4 is now the latest 
Signed-off-by: Nathan Cutler <ncutler@suse.com>
 2017-09-18 18:01:17 +02:00
xie xingguo
5dd3458eec qa: fix pool-quota related tests 
https://github.com/ceph/ceph/pull/17371 introduces support of
per-pool space-full flag, which turns out to set both
full and full_no_quota flags now if a pool is currently running out
of quota.

Actually this test is fragile as long as we keep appending new flags
at pool granularity, but let's not bother with that complexity now.

Fixes: http://tracker.ceph.com/issues/21409
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
 2017-09-16 13:50:18 +08:00
Patrick Donnelly
3c727d9a36
Merge PR #17701 into master 
* refs/remotes/upstream/pull/17701/head:
	qa/cephfs: Fix error in test_filtered_df

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
 2017-09-15 14:12:35 -07:00
Patrick Donnelly
8a54e101e5
Merge PR #17694 into master 
* refs/remotes/upstream/pull/17694/head:
	qa/cephfs: kill mount if it gets evicted by mds
	qa/cephfs: fix test_evict_client

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
 2017-09-15 14:12:33 -07:00
Patrick Donnelly
064f1dfc3c
Merge PR #17676 into master 
* refs/remotes/upstream/pull/17676/head:
	qa/tasks/cephfs: Whitelist POOL_APP_NOT_ENABLED for test_misc

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
 2017-09-15 14:12:29 -07:00
Kefu Chai
085778b80a Merge pull request #17703 from dzafman/wip-misc 
Erasure code read test and code cleanup

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
 2017-09-15 19:54:58 +08:00
Kefu Chai
279d2980fa qa/standalone/ceph-helpers.sh: pass btrfs subvolume options the right way 
with the latest btrfs-progs, it complains with

$ sudo btrfs subvolume list . -t
btrfs subvolume list: too many arguments

so, we need to pass `-t` right after `list` subcommand.

Signed-off-by: Kefu Chai <kchai@redhat.com>
 2017-09-15 12:19:50 +08:00
Kefu Chai
0c47aa8217 qa: respect $TEMPDIR 
ceph-disk and ceph-detect-init are build in $TEMPDIR if it's defined.

Signed-off-by: Kefu Chai <kchai@redhat.com>
 2017-09-15 12:19:50 +08:00
Josh Durgin
520a5a218c Merge pull request #17583 from neha-ojha/wip-cbt-teuthology-integration 
qa: add cbt task for performance testing

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
 2017-09-14 16:36:26 -07:00
Vasu Kulkarni
1f5aba674c Various whitelists for tests to pass 
See tracker http://tracker.ceph.com/issues/21376 for details

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
 2017-09-14 13:32:25 -07:00
Sage Weil
986b86fbeb mon: rename mon_pg_warn_max_per_osd -> mon_max_pg_per_osd 
Signed-off-by: Sage Weil <sage@redhat.com>
 2017-09-14 16:00:31 -04:00
Sage Weil
c9ffeeebeb qa/standalong/mon/osd-pool-create: fewer pgs in test 
This runs afoul of the new max pg per osd limit.

Signed-off-by: Sage Weil <sage@redhat.com>
 2017-09-14 12:10:13 -04:00
Xie Xingguo
b4bb7ce2da Merge pull request #17371 from xiexingguo/wip-per-pool-full-control 
mon, osd: per pool space-full flag support

Reviewed-by: Sage Weil <sage@redhat.com>
 2017-09-14 18:26:12 +08:00
Patrick Donnelly
d929dae49b
Merge PR #17657 into master 
* refs/remotes/upstream/pull/17657/head:
	mds: optimize MDCache::rejoin_scour_survivor_replicas()
	mds: fix MDSCacheObject::clear_replica_map
	mds: support limiting cache by memory
	common: refactor of lru
	mds: resolve unsigned coercion compiler warning
	common: use safer uint64_t for list size
	common: add bytes2str pretty print function
	mds: check if waiting is allocated before use
	mds: go back to compact_map for replicas
	mds: use mempool for cache objects
	mds: cleanup replica_map access
	common: add alloc_ptr smart pointer
	common: add warning on base class use of mempool
	common: use atomic uin64_t for counter

Reviewed-by: Zheng Yan <zyan@redhat.com>
 2017-09-13 20:08:51 -07:00
David Zafman
50e08b0a5d test: Add a removal test for erasure code read 
Test feature: http://tracker.ceph.com/issues/14513

Signed-off-by: David Zafman <dzafman@redhat.com>
 2017-09-13 13:15:52 -07:00
Douglas Fuller
b059cb6290 qa/cephfs: Fix error in test_filtered_df 
ceph df accounts for pool size, so there is no need to do it in the test.

Fixes: http://tracker.ceph.com/issues/21381
Signed-off-by: Douglas Fuller <dfuller@redhat.com>
 2017-09-13 14:02:24 -04:00
Casey Bodley
72b08e8f96 qa/smoke: add rgw crypto config for s3tests 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
 2017-09-13 11:22:28 -04:00
Yan, Zheng
98d86a0752 qa/cephfs: kill mount if it gets evicted by mds 
otherwise, teardown() hange at umount

Fixes: http://tracker.ceph.com/issues/21275
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
 2017-09-13 21:30:51 +08:00
Xie Xingguo
0e604b112e Merge pull request #17515 from xiexingguo/wip-data-digest 
osd/PrimaryLogPG: do not set data digest for bluestore

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
 2017-09-13 18:31:10 +08:00
Yan, Zheng
8433ced847 qa/cephfs: fix test_evict_client 
executing mount_a.kill() twice, then executing mount_b.kill_cleanup()
twice do not make sense.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
 2017-09-13 16:17:42 +08:00
xie xingguo
afcb617dc9 osd/PrimaryLogPG: do not generate data digest for BlueStore by default 
BlueStore enables CRC by default, so this is a dup and gains
no more benefits.

Turn this off by default, which is good for performance.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
 2017-09-13 12:17:16 +08:00
Patrick Donnelly
06c94de584
mds: support limiting cache by memory 
This introduces two config parameters:

    mds_cache_memory_limit: Sets the soft maximum of the cache to the given
    byte count. (Like mds_cache_size, this doesn't actually limit the maximum
    size of the cache. It just dictates the steady-state size.)

    mds_cache_reservation: This replaces mds_health_cache_threshold everywhere
    except the Beacon heartbeat sent to the mons. The idea here is to specify a
    reservation of memory (5% by default) for operations and the MDS tries to
    always maintain that reservation. So, the MDS will recall caps from clients
    when it begins dipping into its reservation of memory.

mds_cache_size still limits the cache by Inode count but is now by-default 0
(i.e. unlimited). The new preferred way of specifying cache limits is by memory
size. The default is 1GB.

Fixes: http://tracker.ceph.com/issues/20594
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1464976

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
 2017-09-12 20:02:41 -07:00
yonghengdexin735
fc5ac9ea69 common:fix error word 
Signed-off-by: yonghengdexin735 <zhang.zezhu@zte.com.cn>
 2017-09-13 10:22:08 +08:00
Patrick Donnelly
f48113f50f
Merge PR #17679 into master 
* refs/remotes/upstream/pull/17679/head:
	qa: get asok path from ceph.conf
	qa: use config_path property instead of literal

Reviewed-by: John Spray <john.spray@redhat.com>
 2017-09-12 15:56:32 -07:00
Patrick Donnelly
bc3a00af0f
qa: get asok path from ceph.conf 
The asok path was recently moved to /tmp to resolve unix path too long errors.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
 2017-09-12 15:52:54 -07:00
Patrick Donnelly
f95798b3ad
qa: use config_path property instead of literal 
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
 2017-09-12 15:50:33 -07:00
Tamilarasi Muthamizhan
4d8971e1c8 Merge pull request #17650 from ceph/wip-ceph-deploy-fix 
qa/tests: Fix get_system_type failure due to invalid remote name
 2017-09-12 15:07:24 -07:00
Douglas Fuller
eba01c48ca qa/tasks/cephfs: Whitelist POOL_APP_NOT_ENABLED for test_misc 
test_misc verifies that ceph fs new will not create a filesystem
on a pool that already contains objects. As part of the test, it
inserts a dummy object into a pool and then attempts to use it for
CephFS. This triggers POOL_APP_NOT_ENABLED. Setting the application
metadata for the pool (and having ceph fs new fail because of the
existing metadata) would then exercise a different failure case.

Signed-off-by: Douglas Fuller <dfuller@redhat.com>
 2017-09-12 13:22:09 -04:00
Neha Ojha
59531d81c5 qa: avoid using make install for fio 
Signed-off-by: Neha Ojha <nojha@redhat.com>
 2017-09-12 08:26:27 -07:00
David Zafman
44f51024cc Merge pull request #17538 from dzafman/wip-21272 
Add export and remove ceph-objectstore-tool command option

Reviewed-by: Sage Weil <sage@redhat.com>
 2017-09-11 20:12:27 -07:00
Mykola Golub
b196d38783 Merge pull request #17621 from idryomov/wip-krbd-xfstests-blockdev 
qa/suites/krbd: enable xfstests blockdev tests

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
 2017-09-11 14:09:32 +03:00
Vasu Kulkarni
9fe58d5ed6 Fix get_system_type failure due to invalid remote name 
recent changes caused the remote name to be invalid, fix the
arg passed to get_system_type

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
 2017-09-10 12:42:16 -07:00
Sage Weil
14e783c0b9 Merge pull request #17571 from jcsp/wip-21260 
mon: fix dropping mgr metadata for active mgr (#21260)

Reviewed-by: Kefu Chai <kchai@redhat.com>
 2017-09-10 14:20:30 -05:00
Mykola Golub
0bafa7e8d4 Merge pull request #17559 from dillaman/wip-21181 
qa/workunits/rbd: relax greps to support upgrade formatting change

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
 2017-09-09 11:23:57 +03:00
David Zafman
3bb20f6d75 ceph-objectstore-tool: Make pg removal require --force 
Add new export-remove to combine the 2 operations

Fixes: http://tracker.ceph.com/issues/21272

Signed-off-by: David Zafman <dzafman@redhat.com>
 2017-09-08 17:56:05 -07:00
David Zafman
49ca1fff7f ceph-objectstore-tool: Better messages for bad --journal-path 
Signed-off-by: David Zafman <dzafman@redhat.com>
 2017-09-08 17:50:46 -07:00
David Zafman
3ac219df2d test: Fix ceph-objectstore-tool test for standalone and latest code 
vstart.sh now defaults to bluestore, so specify filestore
Set environment for run-standalone.sh and cmake build
Create td/cot_dir as test directory
Crush output format change
Change dir into test directory
Give a little time after pool creation
Check for core files as ceph-helpers.sh does

Signed-off-by: David Zafman <dzafman@redhat.com>
 2017-09-08 16:53:53 -07:00
David Zafman
495c32fd31 test: Move ceph-objectstore-tool test to standalone 
Signed-off-by: David Zafman <dzafman@redhat.com>
 2017-09-08 16:53:30 -07:00
Tamilarasi Muthamizhan
ec423100dc Merge pull request #17096 from ceph/wip-ansbile-fixes 
[qa/tests]: misc ceph-ansible fixes and udpate
 2017-09-08 14:47:12 -07:00
Yuri Weinstein
0c2a139ee6 Merge pull request #17513 from Liuchang0812/wip-max-avail-in-df 
mon: incorrect MAX AVAIL in "ceph df"

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
 2017-09-08 13:41:07 -07:00
Patrick Donnelly
1b7aae6cdc
Merge PR #17533 into master 
* refs/remotes/upstream/pull/17533/head:
	qa: log ceph-fuse kill/cleanup

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
 2017-09-08 11:23:23 -07:00
Patrick Donnelly
52c79aecdd
Merge PR #17466 into master 
* refs/remotes/upstream/pull/17466/head:
	qa: whitelist expected MDS_READ_ONLY for kcephfs test

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
 2017-09-08 11:23:19 -07:00
Neha Ojha
1dfd12e852 qa/suites/rados: add perf suite 
Signed-off-by: Neha Ojha <nojha@redhat.com>
 2017-09-08 11:15:11 -07:00
Vasu Kulkarni
b1fd615e9f Stop the mgr on node which is not client, this will ensure 
the client.0's mgr is active mgr for workunit to work.

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
 2017-09-08 10:00:49 -07:00
John Spray
28cb4bb2e6 qa/tasks/mgr: regression test for 21260 
(aka http://tracker.ceph.com/issues/21260)

Fixes: http://tracker.ceph.com/issues/21260
Signed-off-by: John Spray <john.spray@redhat.com>
 2017-09-08 15:45:40 +02:00
Ilya Dryomov
131c1189d6 qa/suites/krbd: enable xfstests blockdev tests 
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
 2017-09-08 15:45:34 +02:00
xie xingguo
b4ca5ae462 mon, osd: per pool space-full flag support 
The newly introduced 'device-class' can be used to separate
different type of devices into different pools, e.g, hdd-pool
for backup data and all-flash-pool for DB applications.

However, if any osd of the cluster is currently running out
of space (exceeding the predefined 'full' threshold), Ceph
will mark the whole cluster as full and prevent writes to all pools,
which turns out to be very wrong.

This patch instead makes the space 'full' control at pool granularity,
which exactly leverages the pool quota logic but shall solve
the above problem.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
 2017-09-08 10:03:17 +08:00
Vasu Kulkarni
e5b5a1b8c8 Test ceph-mgr RESTful api 
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
 2017-09-07 17:10:04 -07:00
Vasu Kulkarni
e963bfd6ca Add additional variables as required by ceph-ansible for upstream 
installation.

possible revert if those are made default
issue: https://github.com/ceph/ceph-ansible/issues/1834

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
 2017-09-07 16:54:51 -07:00
Neha Ojha
2793ee7d7e qa: add cbt task for performance testing 
Signed-off-by: Neha Ojha <nojha@redhat.com>
 2017-09-07 15:44:48 -07:00
Tamilarasi Muthamizhan
594b736228 Merge pull request #17536 from ceph/wip-cd-rbd-pool 
qa/tests: fix rbd pool creation for systemd tests
 2017-09-07 15:13:32 -07:00
Vasu Kulkarni
5c43c5972d Add workaround for http://tracker.ceph.com/issues/20950 
mgr bootstrap key differs on disk, rewrite the new key
using auth get.

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
 2017-09-07 12:22:57 -07:00
Vasu Kulkarni
71ea4987e0 Add kraken to luminous upgrade case 
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
 2017-09-07 12:22:57 -07:00
Vasu Kulkarni
526d4aef1c Add kernel tar/untar and systemd task 
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
 2017-09-07 12:22:57 -07:00
Vasu Kulkarni
2116f76a1f Restart services after upgrade 
This seems to be an issue and should be fixed in our systemd scripts,
need to discuss more on this.

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
 2017-09-07 12:22:57 -07:00
Vasu Kulkarni
8a2b9a0809 since the roles are mapped inside ceph-deploy, store the roles that 
are mapped and use the new mapped role for upgrades during later
stage.

eg: mon.a is mapped to mon.mira002 during install, store this mapping
and durig upgrade map it back to appropriate name to find the hostname
with that role

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
 2017-09-07 12:22:57 -07:00
Vasu Kulkarni
e7a0a4f87f Add doc string for missing ceph-deploy-branch 
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
 2017-09-07 12:22:57 -07:00
Vasu Kulkarni
407ce2613d use mon.a as ceph-admin node, the previous get_first_mon returns 
a sorted(mons)[0] as first mon.

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
 2017-09-07 12:22:56 -07:00
Vasu Kulkarni
f3ec7cabf6 use elif instead of else to handle cases for jewel install 
jewel needs neither filestore or bluestore as an option, so provide none
when running with jewel branch.

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
 2017-09-07 12:22:56 -07:00
Vasu Kulkarni
9c85c76a90 Jewel to luminous upgrade scenario using ceph-deploy 
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
 2017-09-07 12:22:56 -07:00
Vasu Kulkarni
9a73127c16 Add upgrade functionality using ceph-deploy 
This is to test for customer like upgrade scenarios and to find
any issues that may be related to systemd, packaging etc

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
 2017-09-07 12:22:56 -07:00
Vasu Kulkarni
b409c3d163 Fix old flake8 issues 
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
 2017-09-07 12:22:56 -07:00
Vasu Kulkarni
a9be0c387e Add option to skip mgr install for old release 
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
 2017-09-07 12:22:56 -07:00
Sage Weil
1006b62fe6 Merge pull request #17322 from liewegas/wip-post-luminous-mon 
mon: remove pre-luminous compat cruft (2 of many)

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
 2017-09-07 13:47:43 -05:00
Vasu Kulkarni
30dbbfe4ae Remove unsupported 2-size-1-min-size config 
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
 2017-09-07 09:47:15 -07:00
Sage Weil
e715533feb qa/workunits/*: require-osd-release mimic, not luminous 
Signed-off-by: Sage Weil <sage@redhat.com>
 2017-09-07 09:18:56 -04:00
Jason Dillaman
273c84578b qa/workunits/rbd: relax greps to support upgrade formatting change 
Fixes: http://tracker.ceph.com/issues/21181
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
 2017-09-07 08:56:16 -04:00
Jason Dillaman
b0f1545d3b Merge pull request #16998 from trociny/wip-20860 
rbd-mirror: update asok hook name on image rename

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
 2017-09-07 08:34:42 -04:00
Mykola Golub
f618508624 Merge pull request #17504 from dillaman/wip-21251 
qa/suites/rbd: miscellaneous test fixes

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
 2017-09-07 11:34:01 +03:00
Vasu Kulkarni
f12127751d Add mgr create to fix single node test to reach healthy state 
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
 2017-09-06 18:13:39 -07:00
Vasu Kulkarni
49f95b3d8b create the rbd pool right after install 
rbd pool should exist for many rbd tests to work properly, create
the pool right after install is successful.

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
 2017-09-06 18:07:54 -07:00
Sage Weil
1e272575ad Merge pull request #17505 from liewegas/wip-20910 
qa/objectstore/bluestore*: less debug output

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
 2017-09-06 16:02:01 -05:00
Patrick Donnelly
b4f962a486
qa: log ceph-fuse kill/cleanup 
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
 2017-09-06 13:40:11 -07:00
Sage Weil
e2bc8883ba qa/standalone/mon/misc.sh: fix mon feature test 
Signed-off-by: Sage Weil <sage@redhat.com>
 2017-09-06 10:18:07 -04:00
Sage Weil
e21e63cb88 qa/workunits/cephtool/test.sh: drop 'pg set_full_ratio' 
Signed-off-by: Sage Weil <sage@redhat.com>
 2017-09-06 10:18:06 -04:00
liuchang0812
365558571c mon: incorrect MAX AVAIL in "ceph df" 
Fixes: http://tracker.ceph.com/issues/21243

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>
 2017-09-06 21:09:29 +08:00
Patrick Donnelly
73e927293c
Merge PR #17319 into master 
* refs/remotes/upstream/pull/17319/head:
	qa: whitelist expected rstat warning
	qa: sync whitelist with fs/basic_functional
	qa: whitelist expected MDS_CACHE_OVERSIZED

Reviewed-by: Zheng Yan <zyan@redhat.com>
 2017-09-05 20:49:54 -07:00
Patrick Donnelly
28ca766cd2
Merge PR #16305 into master 
* refs/remotes/upstream/pull/16305/head:
	qa/cephfs: test CephFS recovery pools
	qa/cephfs: support CephFS recovery pools
	qa/ceph_test_case: support CephFS recovery pools
	qa/cephfs: Allow deferred fs creation
	qa/cephfs: Refactor alternate pool test

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
 2017-09-05 20:42:30 -07:00
Sage Weil
444f5aa085 qa/objectstore/bluestore*: less debug output 
Let's see if this makes the spurious MON_DOWN failures go away?  (See
http://tracker.ceph.com/issues/20910)

Signed-off-by: Sage Weil <sage@redhat.com>
 2017-09-05 17:43:28 -04:00
Jason Dillaman
27194c30c5 qa/suites/rbd: reduce monitor requirements for mirroring tests 
Fixes: http://tracker.ceph.com/issues/21251
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
 2017-09-05 16:43:34 -04:00
Jason Dillaman
98061bb3d7 qa/suites/rbd: fixed cache override 
Fixes: http://tracker.ceph.com/issues/21251
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
 2017-09-05 16:43:34 -04:00
Jason Dillaman
763f1877ec qa/suites/rbd: reduce OSD debug log levels 
The teuthology machines are periodically running out of space
due to the aggressive log settings.

Fixes: http://tracker.ceph.com/issues/21251
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
 2017-09-05 16:43:30 -04:00
Yan, Zheng
584241285f qa: whitelist expected MDS_READ_ONLY for kcephfs test 
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
 2017-09-05 17:13:59 +08:00
Sage Weil
f47c85ed12 Merge pull request #17406 from liewegas/wip-21206 
qa/tasks/ceph_manager: revive osds before doing final rerr reset
 2017-09-02 13:35:26 -05:00
Josh Durgin
317a8bae5e Merge pull request #17160 from ceph/wip-luminous-x-master 
qa: add upgrade/luminous-x suite

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
 2017-09-01 14:19:48 -07:00
Yuri Weinstein
aff342627c Initial check in 
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
 2017-09-01 08:56:19 -07:00
Sage Weil
ec5e9c8d76 Merge pull request #17379 from liewegas/wip-div-p 
qa/suites/rados/singleton/diverget_priors*: broaden whitelist
 2017-08-31 13:56:59 -05:00
Sage Weil
21027233b2 qa/tasks/ceph_manager: revive osds before doing final rerr reset 
We assume below that rerrosd is up, but it may not be when we exit the
loop.

Fixes: http://tracker.ceph.com/issues/21206
Signed-off-by: Sage Weil <sage@redhat.com>
 2017-08-31 14:55:46 -04:00
Sage Weil
9d9c4bcbba Merge pull request #17247 from liewegas/wip-post-luminous-cleanup 
osd: clean up pre-luminous compat cruft (part 1 of many)

Reviewed-by: Kefu Chai <kchai@redhat.com>
 2017-08-30 14:15:23 -05:00
Sage Weil
39e5efbad2 qa/suites/rados/singleton/diverget_priors*: broaden whitelist 
Signed-off-by: Sage Weil <sage@redhat.com>
 2017-08-30 15:13:08 -04:00
Douglas Fuller
6af2ae80d3 qa/cephfs: test CephFS recovery pools 
Test recovering metadata in to a separate RADOS pool with
cephfs_data_scan and friends.

Signed-off-by: Douglas Fuller <dfuller@redhat.com>
 2017-08-30 09:02:44 -04:00
Douglas Fuller
8f9a252020 qa/cephfs: support CephFS recovery pools 
Add support for testing recovery of CephFS metadata into an alternate
RADOS pool, useful as a disaster recovery mechanism that avoids
modifying the metadata in-place.

Signed-off-by: Douglas Fuller <dfuller@redhat.com>
 2017-08-30 09:02:44 -04:00
Douglas Fuller
c85562c94a qa/ceph_test_case: support CephFS recovery pools 
Add support for testing recovery of CephFS metadata into an alternate
RADOS pool, useful as a disaster recovery mechanism that avoids
modifying the metadata in-place.

Signed-off-by: Douglas Fuller <dfuller@redhat.com>
 2017-08-30 09:02:44 -04:00
Douglas Fuller
5fafc03cb9 qa/cephfs: Allow deferred fs creation 
Permit Filesystem objects to be created and settings modified before
calling Filesystem.create().

Signed-off-by: Douglas Fuller <dfuller@redhat.com>
 2017-08-30 09:02:44 -04:00
Douglas Fuller
47318f8ac4 qa/cephfs: Refactor alternate pool test 
Remove the alternate pool recovery test from test_data_scan. Newer
commits will place the test in its own file.

Signed-off-by: Douglas Fuller <dfuller@redhat.com>
 2017-08-30 09:02:44 -04:00
Jason Dillaman
1b2f1358f7 Merge pull request #17346 from idryomov/wip-krbd-xfstests-update 
qa/suites/krbd: rbd_xfstests job overhaul

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
 2017-08-29 18:28:48 -04:00
Sage Weil
1ae70df392 Merge pull request #17329 from xiexingguo/wip-21162 
mon/OSDMonitor: make 'osd crush rule rename' idempotent

Reviewed-by: Sage Weil <sage@redhat.com>
 2017-08-29 15:58:21 -05:00
Kefu Chai
d24ba40be3 Merge pull request #17312 from cbodley/wip-21155 
qa: create_cache_pool no longer runs 'pool application enable'

Reviewed-by: Kefu Chai <kchai@redhat.com>
 2017-08-30 00:14:21 +08:00
Yuri Weinstein
f4c4a9d90c Merge pull request #17125 from xiexingguo/wip-add-bucket-with-loc 
mon/OSDMonitor: add location option for "crush add-bucket" command

Reviewed-by: Sage Weil <sage@redhat.com>
 2017-08-29 09:12:43 -07:00
Ilya Dryomov
6647c65ebe qa/suites/krbd: new rbd_xfstests job definition 
ext4 seems to be a better choice for our purposes -- less test churn,
rather small and reliable exclude list.

All excluded tests but generic/050 fail with no krbd in the mix, most
have popped up on linux-ext4 list at least once.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
 2017-08-29 16:09:35 +02:00
Ilya Dryomov
01914be5a3 qa/tasks/rbd.xfstests: take exclude list from yaml 
Different filesystems (and further, different configurations of the
same filesystem) need different exclude lists.  Hard coding the list in
a wrapper script is inflexible.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
 2017-08-29 16:09:35 +02:00
Ilya Dryomov
1a982fea53 qa/run_xfstests.sh: quit building xfstests on test nodes 
xfstests is a pain to build on trusty, xenial and centos7 with a single
script.  It is also very sensitive to dependencies, which again need to
be managed on all those distros -- different sets of supported commands
and switches, some versions have known bugs, etc.

Download a pre-built, statically linked tarball and use it instead.
The tarball was generated using xfstests-bld by Ted Ts'o, with a number
of tweaks by myself (mostly concerning the build environment).

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
 2017-08-29 16:08:27 +02:00
Ilya Dryomov
7a0e45046d qa/run_xfstests.sh: drop *_MKFS_OPTIONS variables 
AFAICT ./check doesn't query EXT4_MKFS_OPTIONS or BTRFS_MKFS_OPTIONS,
We don't need anything special for xfs, so remove all of them to avoid
confusion.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
 2017-08-29 16:08:27 +02:00
Ilya Dryomov
ca99f1e840 qa/run_xfstests.sh: kill FS_TYPE variable 
./check expects FSTYP, so use that throughout.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
 2017-08-29 16:08:27 +02:00
Casey Bodley
0542974504 qa: create_cache_pool no longer runs 'pool application enable' 
Fixes: http://tracker.ceph.com/issues/21155

Signed-off-by: Casey Bodley <cbodley@redhat.com>
 2017-08-29 10:01:04 -04:00
Mykola Golub
9ddb5da337 rbd-mirror: update asok hook name on image rename 
Fixes: http://tracker.ceph.com/issues/20860
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
 2017-08-29 08:30:20 +02:00
Sage Weil
7b51cedac6 qa/suites/rados/upgrade: jewel-x -> luminous-x 
Signed-off-by: Sage Weil <sage@redhat.com>
 2017-08-28 23:11:27 -04:00
Sage Weil
d8dead1aaf qa/suites/rados: remove luminous tests 
- snapdir conversion (at-end) stuff
- merge luminous-specific collections that avoided the above back
into their normal locations

Signed-off-by: Sage Weil <sage@redhat.com>
 2017-08-28 23:10:32 -04:00
xie xingguo
2ee80aead8 mon/OSDMonitor: make 'osd crush class rename' idempotent 
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
 2017-08-29 10:43:35 +08:00
xie xingguo
cd2191c3f1 mon/OSDMonitor: make 'osd crush rule rename' idempotent 
Fixes: http://tracker.ceph.com/issues/21162
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
 2017-08-29 08:29:13 +08:00
Patrick Donnelly
c53d091e51
qa: whitelist expected rstat warning 
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
 2017-08-28 11:52:36 -07:00
Patrick Donnelly
701e0910e3
qa: sync whitelist with fs/basic_functional 
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
 2017-08-28 11:49:53 -07:00
Patrick Donnelly
71f0066f6e
qa: whitelist expected MDS_CACHE_OVERSIZED 
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
 2017-08-28 11:49:33 -07:00
Sage Weil
5235f35ac8 qa/suites/powercycle/osd/whitelist_health: whitelist more 
"2017-08-26 16:09:27.704418 mon.a mon.0 172.21.15.169:6789/0 876 : cluster [WRN] MDS health message (mds.0): Behind on trimming (66/30)" in cluster log

Signed-off-by: Sage Weil <sage@redhat.com>
 2017-08-28 13:58:02 -04:00
Kefu Chai
b2d7f4f4c7 qa/suites/rados/upgrade/jewel-x-singleton: tolerate sloppy past_intervals 
See-also: d5d5d7d1
Signed-off-by: Kefu Chai <kchai@redhat.com>
 2017-08-28 15:19:41 +08:00
Xie Xingguo
3e0506a9a5 Merge pull request #17253 from hjwsm1989/fix-qa-bug 
qa: fix the potential delay of pg state change

Reviewed-by: Sage Weil <sage@redhat.com>
 2017-08-28 08:35:01 +08:00
Kefu Chai
30b5b4627c Merge pull request #16494 from asomers/bin_bash 
misc: Fix bash path in shebangs

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Kefu Chai <kchai@redhat.com>
 2017-08-27 10:14:14 +08:00
Sage Weil
d930a4e062 Merge pull request #17029 from xiexingguo/wip-crush-rule-rename 
mon: "ceph osd crush rule rename" support

Reviewed-by: Sage Weil <sage@redhat.com>
 2017-08-25 10:21:38 -05:00
Sage Weil
5db94f4786 Merge pull request #17126 from xiexingguo/wip-nicenum 
common/types: make numbers a bit nicer when displaying space usage

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
 2017-08-25 10:11:06 -05:00
Sage Weil
84465bf5a5 qa/standalone/scrub/osd-scrub-repair: fix grep pattern 
PGMap shows

    ss << pg_sum.stats.sum.num_objects_unfound
       << "/" << pg_sum.stats.sum.num_objects << " objects unfound (" << b << "%)";

but we were grepping for "1/1 unfound" instead of "1/1 objects
unfound".

Introduced by fe81b7e3a5.

Fixes: http://tracker.ceph.com/issues/21127
Signed-off-by: Sage Weil <sage@redhat.com>
 2017-08-25 11:03:44 -04:00
huangjun
3bff102d4b qa: fix the potential delay of pg state change 
If start osd process first and then mark it in, the
pg state may remain all active+clean when doing
wait_for_clean() check, which may fail the next
osd_scrub_pgs() process.
So faster pg state change by marking osd in first.

Signed-off-by: huangjun <huangjun@xsky.com>
 2017-08-25 17:07:37 +08:00
Sage Weil
805455274e qa/overrides/2-size-2-min-size: whitelist REQUEST_STUCK 
With min_size == size things can get stuck a very long time.

Signed-off-by: Sage Weil <sage@redhat.com>
 2017-08-24 15:50:45 -04:00
Sage Weil
3e7ac52a58 Merge pull request #17224 from liewegas/wip-ceph-deploy-mgr 
qa/tasks/ceph_deploy: gatherkeys before mgr deploy

Reviewed-by: Vasu Kulkarni <vasu@redhat.com>
 2017-08-24 12:41:31 -05:00
Sage Weil
893b3ac6fa Merge pull request #17227 from liewegas/wip-jewel-x 
qa/suites/upgrade/jewel-x/parallel: tolerate laggy mgr
 2017-08-24 09:30:31 -05:00
Sage Weil
bf296018ff qa/suites/upgrade/jewel-x/parallel: tolerate laggy mgr 
Signed-off-by: Sage Weil <sage@redhat.com>
 2017-08-24 10:30:01 -04:00
Sage Weil
d3632fd2f9 Merge pull request #17226 from liewegas/wip-jewel-x 
qa/suites/upgrade/jewel-x/stress-split: tolerate sloppy past_intervals
 2017-08-24 09:27:44 -05:00
Sage Weil
d5d5d7d1d2 qa/suites/upgrade/jewel-x/stress-split: tolerate sloppy past_intervals 
This is harmless in general, esp during upgrade.

Signed-off-by: Sage Weil <sage@redhat.com>
 2017-08-24 10:23:22 -04:00
Sage Weil
800fdd9953 qa/tasks/ceph_deploy: gatherkeys before mgr deploy 
Otherwise we may be missing the bootstrap-mgr key.

Signed-off-by: Sage Weil <sage@redhat.com>
 2017-08-24 09:52:17 -04:00
Sage Weil
4f1fca0483 Merge pull request #17203 from liewegas/wip-jewel-x 
qa/suites/upgarde/jewel-x/parallel: tolerate mgr warning
 2017-08-23 17:21:37 -05:00
Yuri Weinstein
304b492187 Initial check in luminous-x suite 
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
 2017-08-23 14:53:55 -07:00
Matt Benjamin
1e1731e663 Merge pull request #16612 from cbodley/wip-20668 
rgw: fixes for multisite replication of encrypted objects
 2017-08-23 15:57:02 -04:00
Sage Weil
5455f599b3 qa/suites/upgrade/jewel-x/parallel: tolerate OBJECT_MISPLACED 
Signed-off-by: Sage Weil <sage@redhat.com>
 2017-08-23 14:24:00 -04:00
Sage Weil
2504ab1675 qa/suites/upgarde/jewel-x/parallel: tolerate mgr warning 
Signed-off-by: Sage Weil <sage@redhat.com>
 2017-08-23 14:22:34 -04:00
Casey Bodley
47ffcb1caf Merge pull request #17162 from cbodley/wip-qa-rgw-pool-application 
qa/rgw: use 'ceph osd pool application enable' on created pools

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
 2017-08-23 12:52:33 -04:00
Patrick Donnelly
75967dbfe7
Merge PR #17111 into master 
* refs/remotes/upstream/pull/17111/head:
	qa: add health whitelist for kcephfs suite

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
 2017-08-23 06:49:11 -07:00
Casey Bodley
5e67c681f7 Merge pull request #16344 from rzarzynski/wip-rgwqa-tempest 
rgw, qa: integrate Tempest to verify RadosGW's compliance with Swift API

Reviewed-by: Casey Bodley <cbodley@redhat.com>
 2017-08-22 15:02:15 -04:00
Casey Bodley
568c3f3a23 qa/rgw: enable 'rgw' application on created pools 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
 2017-08-22 14:37:02 -04:00
Casey Bodley
7c865effd8 qa: add optional 'application' to pool creation helpers 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
 2017-08-22 14:37:02 -04:00
Kefu Chai
85b63670d9 Merge pull request #17039 from dzafman/wip-18206 
osd: Fixes for osd_scrub_during_recovery handling

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
 2017-08-22 22:50:24 +08:00
xie xingguo
1ea448ac75 common/types: make numbers a bit nicer when displaying space usage 
Was:
----------------------------------------------------------------------------
GLOBAL:
    SIZE       AVAIL      RAW USED     %RAW USED
    30911M     27050M        3861M         12.49
POOLS:
    NAME                  ID     USED        %USED     MAX AVAIL     OBJECTS
    rbd                   0      101216k      1.10         8913M        1178
    cephfs_data_a         1            0         0         8913M           0
    cephfs_metadata_a     2          892         0         8913M          21
----------------------------------------------------------------------------

Now:
----------------------------------------------------------------------------
GLOBAL:
    SIZE      AVAIL     RAW USED     %RAW USED
    30.2G     26.4G        3.77G         12.50
POOLS:
    NAME                  ID     USED      %USED     MAX AVAIL     OBJECTS
    rbd                   0      99.2M      1.10         8.70G        1180
    cephfs_data_a         1          0         0         8.70G           0
    cephfs_metadata_a     2        892         0         8.70G          21
----------------------------------------------------------------------------

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
 2017-08-22 12:33:10 +08:00
xie xingguo
ac36b471a3 mon/OSDMonitor: add location option for "crush add-bucket" command 
So we can combine "crush add-bucket" with "crush move" command,
and hence avoid making two separate changes to the osdmap,
and hence slow down map-epoch generation.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
 2017-08-22 10:39:03 +08:00
David Zafman
367c32c69a osd: Fixes for osd_scrub_during_recovery handling 
Fixes: http://tracker.ceph.com/issues/18206

Signed-off-by: David Zafman <dzafman@redhat.com>
 2017-08-21 17:08:14 -07:00
David Zafman
9f3d970a0d tests: osd-scrub-snaps.sh minor cleanup 
Signed-off-by: David Zafman <dzafman@redhat.com>
 2017-08-21 17:08:14 -07:00
David Zafman
9069f663f7 Merge pull request #17098 from dzafman/wip-runs-core 
tests: run-standalone.sh skip core_pattern if already set

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
 2017-08-21 08:21:13 -07:00
Yan, Zheng
b10989209f qa: add health whitelist for kcephfs suite 
Fixes: http://tracker.ceph.com/issues/20892
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
 2017-08-21 17:01:22 +08:00
Xie Xingguo
89d28a176c Merge pull request #16955 from xiexingguo/wip-app-ls 
mon/OSDMonitor: "osd pool application get" support

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
 2017-08-19 17:49:43 +08:00
Vasu Kulkarni
9cc00c5c1a Rename folders to fix task order 
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
 2017-08-18 11:35:54 -07:00
Vasu Kulkarni
1041c803f1 use bluestore with dmcrypt option 
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
 2017-08-18 11:09:50 -07:00
Vasu Kulkarni
f6de5d9f9e Add dmcrypt option 
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
 2017-08-18 11:08:00 -07:00
Vasu Kulkarni
60d00e0ead Separate the main task from options 
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
 2017-08-18 11:05:01 -07:00
Vasu Kulkarni
0395b84488 Catchup with recent changes with ceph-ansible 
Adds osd_scenario and ceph_stable_release variables

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
 2017-08-18 10:47:22 -07:00
David Zafman
599d7622dd tests: run-standalone.sh skip core_pattern if already set 
If sudo requires a password it is nice to be able to avoid
sudo sysctl by setting core_pattern ahead of time.

Signed-off-by: David Zafman <dzafman@redhat.com>
 2017-08-18 10:27:12 -07:00
Alfredo Deza
1e8e689ca7 Merge pull request #17093 from dzafman/wip-runs-message 
tests: run-standalone.sh improve error message

Reviewed-by: Alfredo Deza <adeza@redhat.com>
 2017-08-18 12:08:17 -04:00
Kefu Chai
fbfd17b2fd Merge pull request #17026 from dzafman/wip-add-trap 
tests: qa/standalone: Add trap for signals to restore the kernel core pattern

Reviewed-by: Kefu Chai <kchai@redhat.com>
 2017-08-18 23:02:04 +08:00
David Zafman
175b10795d tests: run-standalone.sh improve error message 
Signed-off-by: David Zafman <dzafman@redhat.com>
 2017-08-18 03:49:42 -07:00
David Zafman
021177b790 Merge pull request #16407 from dzafman/wip-20243 
osd: Improve size scrub error handling and ignore system attrs in xattr checking

Reviewed-by: Kefu Chai <kchai@redhat.com>
 2017-08-17 20:05:40 -07:00
Jason Dillaman
1b21476f33 Merge pull request #17062 from trociny/wip-qa-rbd-mirror-data-pool 
qa/workunits/rbd: test data pool is mirrored correctly

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
 2017-08-17 18:24:37 -04:00
David Galloway
cd50aad45a Merge pull request #17069 from yuriw/wip_add_luminous 
Decreased amount of jobs on master, kraken, luminous runs
 2017-08-17 15:27:19 -04:00
Yuri Weinstein
6b7331e038 Decreased amount of jobs on master, kraken, luminous runs 
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
 2017-08-17 09:32:09 -07:00
Mykola Golub
c4c7075029 qa/workunits/rbd: test data pool is mirrored correctly 
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
 2017-08-17 10:07:12 +02:00
xie xingguo
7c67f95201 crush, mon: "ceph osd crush rule ls-by-class" support 
This command returns all crush rules that are currently
referencing the device class specified by user.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
 2017-08-17 13:34:36 +08:00
Casey Bodley
f27ebabe55 test/rgw: add kms encryption key for teuthology 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
 2017-08-16 12:22:31 -04:00
Kefu Chai
17d6e96a6f Merge pull request #16967 from liewegas/wip-upgrade-health 
mon: fix legacy health checks in 'ceph status' during upgrade; fix jewel-x upgrade combo

Reviewed-by: Kefu Chai <kchai@redhat.com>
 2017-08-16 19:44:36 +08:00
xie xingguo
46586b2288 mon/OSDMonitor: "osd pool application ls" support 
It would be a pain if we have to call 'ceph osd dump --format=json-pretty'
to find out these each time...

Demo output:
(1) ceph osd pool application get
{
    "cephfs_data_b": {
        "cephfs": {}
    },
    "cephfs_metadata_a": {
        "cephfs": {}
    },
    "test_pool": {
        "rbd": {
            "test": "me"
        }
    }
}

(2) ceph osd pool application get test_pool
{
    "rbd": {
        "test": "me"
    }
}

(3) ceph osd pool application get test_pool rbd
{
    "test": "me"
}

(4) ceph osd pool application get test_pool rbd test
me

Fixes: http://tracker.ceph.com/issues/20976
Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
 2017-08-16 09:41:11 +08:00
xie xingguo
a5075ed253 mon: "ceph osd crush rule rename" support 
User may specify a rule with the same name of the pool that it serves.
Since a pool can be renamed, so does the rule.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
 2017-08-15 17:01:26 +08:00
David Zafman
982aef784c tests: qa/standalone: Add trap for signals to resetore the kernel core pattern 
Signed-off-by: David Zafman <dzafman@redhat.com>
 2017-08-14 19:16:30 -07:00
Radoslaw Zarzynski
ed8a6b89e4 qa/suites/rgw/tempest: use fixed-1 cluster instead of fixed-2. 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2017-08-14 01:11:22 +00:00
Radoslaw Zarzynski
43a7399720 qa/tasks/rgw: make the frontend_prefix per-client configurable. 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2017-08-14 01:11:18 +00:00
Radoslaw Zarzynski
09db786581 qa/suites/rgw: move the Tempest testing to its dedicated sub-suite. 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2017-08-13 23:14:40 +00:00
Radoslaw Zarzynski
99e1d443a0 qa/suites/rgw: freeze the Tempest version for RGW testing. 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2017-08-13 23:14:40 +00:00
Radoslaw Zarzynski
3dbacd1387 qa/tasks/keystone: make possible to deploy Keystone on CentOS as well. 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2017-08-13 23:14:40 +00:00
Radoslaw Zarzynski
afe1ad3010 qa, rgw: Keystone's instances can be now accessed via non-local network interfaces. 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2017-08-13 23:14:40 +00:00
Radoslaw Zarzynski
849f46f8cf qa/suites/rgw: integrate Tempest to verify Swift API compliance. 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2017-08-13 23:14:40 +00:00
Radoslaw Zarzynski
f81669f673 qa/tasks/keystone: handle packaged dependencies of Keystone. 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2017-08-13 23:14:40 +00:00
Radoslaw Zarzynski
1d6be97861 qa/tasks/tempest: make the Tempest blacklisting configurable. 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2017-08-13 23:14:40 +00:00
Radoslaw Zarzynski
1d8a4e5921 qa/tasks/tox: separate tox deployment from Keystone. 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2017-08-13 23:14:40 +00:00
Radoslaw Zarzynski
2a845344e7 qa/tasks/keystone: comply wih the CLI interface older openstack-client. 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2017-08-13 23:14:40 +00:00
Radoslaw Zarzynski
c1d93e80c8 qa/tasks/keystone: deploy tox. 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2017-08-13 23:14:40 +00:00
Radoslaw Zarzynski
979d936ad1 qa/tasks/keystone: the task can be now executed also on remotes. 
Initially it was compatible with local_runner only.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2017-08-13 23:14:40 +00:00
Radoslaw Zarzynski
728d642bc4 qa/tasks/tempest: properly update tempest.conf on remotes. 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2017-08-13 23:14:40 +00:00
Radoslaw Zarzynski
c1fe31d327 qa/tasks/rgw: add the "frontend_prefix" configurable. 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2017-08-13 23:14:40 +00:00
Radoslaw Zarzynski
b6a1533962 qa/tasks/tempest: integrate OpenStack Tempest for testing RadosGW's Swift API. 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2017-08-13 23:14:40 +00:00
Radoslaw Zarzynski
2ee29a7c6a qa/tasks/keystone: deploy a Keystone instance for testing RadosGW. 
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
 2017-08-13 23:14:40 +00:00
Sage Weil
929c11bf16 Merge pull request #16883 from liewegas/wip-20939 
crush,mon: fix weight-set vs crush device classes

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
 2017-08-13 13:59:43 -05:00
Sage Weil
d69f0e120b qa/suites/rados/objectstore/objectstore: less debug 
Saw an ENOSPC.

Signed-off-by: Sage Weil <sage@redhat.com>
 2017-08-13 14:41:43 -04:00
Sage Weil
196e7f317c Merge pull request #16906 from liewegas/wip-bluestore-stupid 
os/bluestore: switch default allocator to stupid; test both bitmap and stupid in qa

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
 2017-08-12 16:10:31 -05:00
Sage Weil
41e5a85308 qa/suites/rados/verify/validater/valgrind: whitelist PG_ 
Peering might be slow due to valgrind.

Signed-off-by: Sage Weil <sage@redhat.com>
 2017-08-12 14:18:59 -04:00
Sage Weil
12007044b1 qa/suites/rados/multimon/tasks/mon_lock_with_skew: whitelist PG_ 
Default pool pgs not up because mons too broken for OSDs to peer.

Signed-off-by: Sage Weil <sage@redhat.com>
 2017-08-12 14:15:15 -04:00
David Zafman
4c949b6258 osd, rados: Adding ss_attr_missing and ss_attr_corrupt errors to list-inconsistent-obj 
Signed-off-by: David Zafman <dzafman@redhat.com>
 2017-08-11 11:37:32 -07:00
David Zafman
5f58301a13 osd, rados: Improve size scrub error handling 
Fixes: http://tracker.ceph.com/issues/20243

Signed-off-by: David Zafman <dzafman@redhat.com>
 2017-08-11 11:37:32 -07:00
David Zafman
8ad4b29113 osd: Add whether shard is primary in list-inconsistent-obj 
Add new field in the client interface
Update test case

Fixes: http://tracker.ceph.com/issues/18836

Signed-off-by: David Zafman <dzafman@redhat.com>
 2017-08-11 11:37:03 -07:00
Sage Weil
ad23d7dc1f qa/suites/rados/multimon: whitelist mgr down vs clock skew test 
Clock skew might make us fail the mgr.

Signed-off-by: Sage Weil <sage@redhat.com>
 2017-08-11 13:42:02 -04:00
Sage Weil
9c42597f09 qa/workunits/mon/crush_ops.sh: test weight sets vs device classes 
Signed-off-by: Sage Weil <sage@redhat.com>
 2017-08-11 09:40:11 -04:00
Yuri Weinstein
11c57701c6 Merge pull request #16961 from xiexingguo/wip-class-rename 
crush: "osd crush class rename" support

Reviewed-by: Sage Weil <sage@redhat.com>
 2017-08-11 06:18:57 -07:00
Sage Weil
d2d9b41275 Merge pull request #16709 from dzafman/wip-standalone 
qa/standalone: misc fixes
 2017-08-10 21:33:43 -05:00
Sage Weil
22a46aab3c Merge pull request #16954 from gregsfortytwo/wip-20891-pool-metadata 
mon: fix "fs new" pool metadata update, tests

Reviewed-by: Sage Weil <sage@redhat.com>
 2017-08-10 21:29:37 -05:00
xie xingguo
d792e8d528 crush: "osd crush class rename" support 
In 076a6abd80 I killed the 'class rename' command
and thought it was totally useless but I was wrong.

Consider the following user case:
(1) randomly choose some OSDs(e.g., from different hosts) and try to make them for private use only,
    say, by grouping them into 'pool1'
(2) ceph osd crush set-device-class pool1 'OSDs from (1)'
(3) ceph osd crush rule create-replicated rule_for_pool1 default host pool1
(4) ceph osd pool rename pool1 pool2
(5) ceph osd crush class rename pool1 pool2

From the above user case, we need to safely change a pool name without worrying
any risk of data migration. That is why the 'osd crush class rename' command
is still needed here.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
 2017-08-11 08:32:39 +08:00
David Zafman
e24ac51a82 qa: Fix broken test_activate_osd() due to missing space 
Signed-off-by: David Zafman <dzafman@redhat.com>
 2017-08-10 12:37:05 -07:00
David Zafman
ae2c5331fb qa: Fix races with waiting for scrubs 
The trigger_scrub sets the last_scrub_stamp backwards to
force a scheduled scrub.  In a small window this stamp could get propagated
to the mgr.  A test failure occurred because wait_for_scrub() was confused
by seeing a backward moving date.

The most critical change is having wait_for_scrub() make sure that the
date advances past the previous in value.

A test failed because the random backoff kept delayed triggered scrub, so
set osd_scrub_backoff throughout.

Signed-off-by: David Zafman <dzafman@redhat.com>
 2017-08-10 12:37:05 -07:00
David Zafman
dddda523d1 qa: Testing of ceph-helpers.sh, teardown on fail to dump logs, save cores 
Signed-off-by: David Zafman <dzafman@redhat.com>
 2017-08-10 12:37:05 -07:00
David Zafman
1fe6cb0f02 osd: Avoid confusion over legacy snaps when head_exists corrupt 
Signed-off-by: David Zafman <dzafman@redhat.com>
 2017-08-10 12:37:05 -07:00
David Zafman
229de6b71d qa: Add support for core dumps 
Save core dumps when running tests locally
Dump logs to output whenever cores seen

Signed-off-by: David Zafman <dzafman@redhat.com>
 2017-08-10 12:37:04 -07:00
Greg Farnum
c85af7b146 qa: test that "fs new" correctly set the application_metadata 
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
 2017-08-10 11:09:38 -07:00
Sage Weil
dd2fb6c40b Merge pull request #16944 from liewegas/wip-kraken-x 
mon/Elector: force election epoch bump on start

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
 2017-08-10 11:12:43 -05:00
David Zafman
4db5124e1a qa: For FreeBSD skip osd-dup.sh because there is no bluestore 
Signed-off-by: David Zafman <dzafman@redhat.com>
 2017-08-10 08:30:47 -07:00
David Zafman
61bfd236ad qa: Raise mon-data-avail-warn to pass tests with less space 
Signed-off-by: David Zafman <dzafman@redhat.com>
 2017-08-10 08:30:47 -07:00
David Zafman
574b3cd3d4 qa: Add common generalized inject_eio() to ceph-helpers.sh 
Retry for a while to allow pool to appear

Signed-off-by: David Zafman <dzafman@redhat.com>
 2017-08-10 08:30:47 -07:00
David Zafman
3988ebab43 qa: osd-scrub-repair.sh handle older versions of jq 
Signed-off-by: David Zafman <dzafman@redhat.com>
 2017-08-10 08:30:47 -07:00
David Zafman
2a679a36de qa: Add support for specifying sub-tests with run-standalone.sh 
Fix test-ceph-helpers.sh to pass additional arguments on

Signed-off-by: David Zafman <dzafman@redhat.com>
 2017-08-10 08:30:47 -07:00
David Zafman
69413618a0 qa: ceph-helpers.sh fixes 
Add missing teardown to cleanup test directory
Fix pgid due to elimination of initial default pool
Testing could never fail because run_tests return ignored

Signed-off-by: David Zafman <dzafman@redhat.com>
 2017-08-10 08:30:47 -07:00
David Zafman
d99f2a77e9 qa: Skip ceph-helpers.sh which is tested through test-ceph-helpers.sh 
Signed-off-by: David Zafman <dzafman@redhat.com>
 2017-08-10 08:30:46 -07:00
Sage Weil
c46bdf5efd Revert "qa/suites/upgrade/jewel-x/parallel: thrash layout" 
This reverts commit 435777dbff.

This test combination is not yet stable.

Signed-off-by: Sage Weil <sage@redhat.com>
 2017-08-10 09:51:29 -04:00
Sage Weil
a0b9f37dbc qa/suites/upgrade/jewel-x/parallel: no loadgenbig 
When we do the thrashing this leads to ENOSPC on smithi.

Signed-off-by: Sage Weil <sage@redhat.com>
 2017-08-10 09:44:17 -04:00
Sage Weil
b47d975cc3 Merge pull request #16950 from liewegas/wip-20959 
osd/PG: force rebuild of missing set on jewel upgrade

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
 2017-08-10 08:32:12 -05:00
Sage Weil
c8c837f665 Merge pull request #16905 from liewegas/wip-upgrade-sloppy-flush 
qa/tasks/ceph.py: tolerate flush pg stats exception
 2017-08-09 21:21:51 -05:00
Sage Weil
435777dbff qa/suites/upgrade/jewel-x/parallel: thrash layout 
We can't kill and restart osds because that will interfere with
the upgrade process.  We can, however, thrash the layout by
tweaking osd weights and so on.  This will exercise osd recovery
paths during the upgrade that aren't normally exercised (outside
of stress-split..which doesn't upgrade individual osds while they
are non-clean).

Signed-off-by: Sage Weil <sage@redhat.com>
 2017-08-09 22:07:48 -04:00
Yuri Weinstein
c1b87c71e3 Merge pull request #16892 from xiexingguo/wip-clean-pg-temp 
mon/OSDMonitor: fix 'osd pg temp' unable to cleanup pg-temp

Reviewed-by: Sage Weil <sage@redhat.com>
 2017-08-09 16:34:38 -07:00
Matt Benjamin
0956b3aafd Merge pull request #16834 from mdw-at-linuxbox/policy 
radosgw: usage: fix bytes_sent bug.
 2017-08-09 14:24:01 -04:00
Sage Weil
b61be07d45 qa/suites/upgrade/kraken-x/stress-split: more whitelisting 
Signed-off-by: Sage Weil <sage@redhat.com>
 2017-08-09 13:58:55 -04:00
Sage Weil
1043fca076 Merge pull request #16923 from liewegas/wip-20738 
qa/suites/rados/objectstore: logs
 2017-08-09 12:45:29 -05:00
Sage Weil
34db3f8a08 Merge pull request #16947 from liewegas/wip-jewel-x 
qa/suites/upgarde/jewel-x/point-to-point-x: disable app warnings
 2017-08-09 09:56:15 -05:00
Sage Weil
bbd5fe354c qa/suites/upgarde/jewel-x/point-to-point-x: disable app warnings 
Signed-off-by: Sage Weil <sage@redhat.com>
 2017-08-09 09:18:54 -04:00
Mykola Golub
6a575136a7 qa/workunits/rbd: use command line option to specify watcher asok 
The previous method to get the watcher admin socket was fragile
and had started to fail after the recent changes to vstart ceph.conf.

Fixes: http://tracker.ceph.com/issues/20954
Signed-off-by: Mykola Golub <mgolub@mirantis.com>
 2017-08-09 09:03:00 +02:00
Marcus Watts
a45ab45f74 Test bytes_sent bugs. 
Rearrange logic to make it easier to measure accumulation.
Instrument the boto request/response loop to count bytes in and out.
Accumulate byte counts in usage like structure.
Compare actual usage reported by ceph against local usage measured.
Report and assert if there are any short-comings.
Remove zone placement rule that was newly added at end: tests should be rerunable.

Nit: the logic to wait for "delete_obj" is not quite right.

Fixes: http://tracker.ceph.com/issues/19870
Signed-off-by: Marcus Watts <mwatts@redhat.com>
 2017-08-08 21:56:01 -04:00
Sage Weil
c8d60396c7 qa/suites/rados/objectstore: logs 
Hunting http://tracker.ceph.com/issues/20738

Signed-off-by: Sage Weil <sage@redhat.com>
 2017-08-08 18:07:18 -04:00
Sage Weil
6127a4c294 Merge pull request #16546 from asomers/openstack_shebang2 
qa: Fix shebangs on openstack scripts

Reviewed-by: Amik Kumar <amitkuma@redhat.com>
 2017-08-08 15:55:38 -05:00
Patrick Donnelly
eabe662614
Merge PR #16378 into master 
* refs/remotes/upstream/pull/16378/head:
	doc: remove accidental additions to release notes
	qa/cephfs: Fix race in test_volume_client
	qa/cephfs: Test filtered df
	PendingReleaseNotes: add note about df filtering
	client: Support new, filtered MStatfs
	objecter: Support new, filtered MStatfs
	mon/PGMap stats: Support new, filtered MStatfs
	messages: Add optional data pool to MStatfs

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
 2017-08-08 09:33:52 -07:00
Sage Weil
ec60fac9c6 qa/objectstore/bluestore-bitmap: test bluestore bitmap allocator too 
Signed-off-by: Sage Weil <sage@redhat.com>
 2017-08-08 12:15:16 -04:00
Sage Weil
9da7e63c64 qa/tasks/ceph.py: tolerate flush pg stats exception 
If the OSD doesn't see IO, it won't flush more pg/osd stats when the
luminous flag is not yet set (legacy pgmonitor mode).

Signed-off-by: Sage Weil <sage@redhat.com>
 2017-08-08 12:08:31 -04:00
xie xingguo
71cef3cb74 mon/OSDMonitor: fix 'osd pg temp' unable to cleanup pg-temp 
This is not very elegant way but should work..

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
 2017-08-08 17:33:35 +08:00
Mykola Golub
b4dbfcc879 rbd-ggate: tool to map images on FreeBSD via GEOM Gate 
rbd-ggate spawns a process responsible for the creation of ggate
device and forwarding I/O requests between the GEOM Gate kernel
subsystem and RADOS.

On FreeBSD it provides functionality similar to rbd-nbd on Linux.

Signed-off-by: Mykola Golub <mgolub@mirantis.com>
 2017-08-08 11:00:30 +02:00
Sage Weil
bf29142b08 qa/suites/upgrade/kraken-x/stress-split*: whitelist 
Signed-off-by: Sage Weil <sage@redhat.com>
 2017-08-07 21:36:58 -04:00
Sage Weil
2234a0ed11 qa/suites/upgrade/kraken-x/parallel: whitelist 
Signed-off-by: Sage Weil <sage@redhat.com>
 2017-08-07 21:36:58 -04:00
Sage Weil
973772c11d Merge pull request #16871 from liewegas/wip-20920 
mon: fix commands advertised during mon cluster upgrade

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
 2017-08-07 14:48:58 -05:00