Commit Graph

369 Commits

Author SHA1 Message Date
Sage Weil
c8c837f665 Merge pull request #16905 from liewegas/wip-upgrade-sloppy-flush
qa/tasks/ceph.py: tolerate flush pg stats exception
2017-08-09 21:21:51 -05:00
Matt Benjamin
0956b3aafd Merge pull request #16834 from mdw-at-linuxbox/policy
radosgw: usage: fix bytes_sent bug.
2017-08-09 14:24:01 -04:00
Marcus Watts
a45ab45f74 Test bytes_sent bugs.
Rearrange logic to make it easier to measure accumulation.
Instrument the boto request/response loop to count bytes in and out.
Accumulate byte counts in usage like structure.
Compare actual usage reported by ceph against local usage measured.
Report and assert if there are any short-comings.
Remove zone placement rule that was newly added at end: tests should be rerunable.

Nit: the logic to wait for "delete_obj" is not quite right.

Fixes: http://tracker.ceph.com/issues/19870
Signed-off-by: Marcus Watts <mwatts@redhat.com>
2017-08-08 21:56:01 -04:00
Patrick Donnelly
eabe662614
Merge PR #16378 into master
* refs/remotes/upstream/pull/16378/head:
	doc: remove accidental additions to release notes
	qa/cephfs: Fix race in test_volume_client
	qa/cephfs: Test filtered df
	PendingReleaseNotes: add note about df filtering
	client: Support new, filtered MStatfs
	objecter: Support new, filtered MStatfs
	mon/PGMap stats: Support new, filtered MStatfs
	messages: Add optional data pool to MStatfs

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
2017-08-08 09:33:52 -07:00
Sage Weil
9da7e63c64 qa/tasks/ceph.py: tolerate flush pg stats exception
If the OSD doesn't see IO, it won't flush more pg/osd stats when the
luminous flag is not yet set (legacy pgmonitor mode).

Signed-off-by: Sage Weil <sage@redhat.com>
2017-08-08 12:08:31 -04:00
Sage Weil
d9a0145f8f Merge pull request #16824 from liewegas/wip-more-scrub-time
qa/tasks/ceph: wait longer for scrub
2017-08-05 13:35:55 -05:00
Sage Weil
62482ce82c qa/tasks/ceph: debug osd setup
I've seen a couple rbd runs that seem to skip the next block :/

Signed-off-by: Sage Weil <sage@redhat.com>
2017-08-05 13:53:26 -04:00
Douglas Fuller
552225f329 qa/cephfs: Fix race in test_volume_client
Signed-off-by: Douglas Fuller <dfuller@redhat.com>
2017-08-04 14:38:50 -04:00
Sage Weil
d09606619f qa/tasks/ceph: wait longer for scrub
Signed-off-by: Sage Weil <sage@redhat.com>
2017-08-04 12:06:27 -04:00
Sage Weil
f681d6e794 Merge pull request #16794 from liewegas/wip-rbd-app
qa/tasks/ceph: enable rbd on rbd pool
2017-08-04 08:40:45 -05:00
Sage Weil
7ed9d1e8aa qa/tasks/ceph: enable rbd on rbd pool
Signed-off-by: Sage Weil <sage@redhat.com>
2017-08-04 09:40:26 -04:00
Patrick Donnelly
d89af4a3e8
Merge PR #16802 into master
* refs/remotes/upstream/pull/16802/head:
	qa: update wait_for_health for new health json syntax

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2017-08-03 16:20:20 -07:00
John Spray
0613d411aa qa: update wait_for_health for new health json syntax
Fixes: http://tracker.ceph.com/issues/20890
Signed-off-by: John Spray <john.spray@redhat.com>
2017-08-03 23:46:41 +01:00
Patrick Donnelly
66756c4f65
Merge PR #16292 into master
* refs/remotes/upstream/pull/16292/head:
	qa: use new hex rep of inode
	qa: fix whitelist error message
	mds: refine "Scrub error" cluster log message
	mds: polish clog messages
	doc: developer logging guidance

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2017-08-03 13:55:21 -07:00
Yuri Weinstein
09fd18d031 Merge pull request #16760 from cbodley/wip-rgw-disable-lifecycle-s3tests
qa/rgw: disable lifecycle tests because of expiration failures

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
2017-08-03 13:25:39 -07:00
Douglas Fuller
b9d11af92b qa/cephfs: Test filtered df
Add a test for filtered df for file systems with single data pools.

Signed-off-by: Douglas Fuller <dfuller@redhat.com>
2017-08-03 14:11:47 -04:00
Josh Durgin
ae48c75065 Merge pull request #16797 from jdurgin/wip-upgrade-jewel-x
qa: timeout when waiting for mgr to be available in healthy()

Reviewed-by: Sage Weil <sage@redhat.com>
2017-08-03 11:11:41 -07:00
Patrick Donnelly
8d33cbbf5c
qa: use new hex rep of inode
Resolves a failure from QA:

    2017-08-02T19:23:27.489 INFO:tasks.cephfs_test_runner:======================================================================
    2017-08-02T19:23:27.489 INFO:tasks.cephfs_test_runner:FAIL: test_oversize (tasks.cephfs.test_fragment.TestFragmentation)
    2017-08-02T19:23:27.489 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
    2017-08-02T19:23:27.490 INFO:tasks.cephfs_test_runner:Traceback (most recent call last):
    2017-08-02T19:23:27.490 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20170802/qa/tasks/cephfs/test_fragment.py", line 71, in test_oversize
    2017-08-02T19:23:27.490 INFO:tasks.cephfs_test_runner:    self.assertEqual(frags[0]['dirfrag'], "10000000000.0*")
    2017-08-02T19:23:27.490 INFO:tasks.cephfs_test_runner:AssertionError: u'0x10000000000.0*' != '10000000000.0*'
    2017-08-02T19:23:27.490 INFO:tasks.cephfs_test_runner:

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2017-08-02 21:39:48 -07:00
Patrick Donnelly
6cad5be68c
Merge PR #16714 into master
* refs/remotes/upstream/pull/16714/head:
	qa: test export_pin is correct in dumped subtree
	mds: print export_pin for dumped subtree

Reviewed-by: Douglas Fuller <dfuller@redhat.com>
Reviewed-by: huanwen ren <ren.huanwen@zte.com.cn>
2017-08-02 18:41:12 -07:00
Casey Bodley
0debf4dc6e qa/rgw: disable lifecycle tests because of expiration failures
lifecycle expiration tests are too reliant on timing, and have been
failing consistently for a long time

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2017-08-02 11:06:35 -04:00
Josh Durgin
63693779fc qa: timeout when waiting for mgr to be available
Otherwise during upgrades we wait forever.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2017-08-02 02:18:28 -04:00
Kefu Chai
1ff1f836da Merge pull request #16722 from tchaikov/wip-qa-fixes
qa/suites: escape the parenthesis of the whitelist text

Reviewed-by: Sage Weil <sage@redhat.com>
2017-08-02 13:00:01 +08:00
Jason Dillaman
2589f57ecd Merge pull request #16656 from idryomov/wip-qa-newer-fio
qa/tasks/rbd_fio: bump default fio version to 2.21

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2017-08-01 10:14:46 -04:00
Kefu Chai
d12c51ca91 qa/suites: escape the parenthesis of the whitelist text
so we can avoid the warnings like

grep: Unmatched ( or \(

because we pass the whitelisted string to `egrep -v "$1"` directly.

Signed-off-by: Kefu Chai <kchai@redhat.com>
2017-08-01 21:54:44 +08:00
Patrick Donnelly
8db2c43e79
qa: test export_pin is correct in dumped subtree
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2017-07-31 15:33:49 -07:00
Patrick Donnelly
019f20ff98
Merge PR #16640 into master
* refs/remotes/upstream/pull/16640/head:
	qa: fix wait for wrong health message

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2017-07-28 09:55:49 -07:00
Patrick Donnelly
6fc2ee383f
Merge PR #16413 into master
* refs/remotes/upstream/pull/16413/head:
	qa/cephfs: lsof if umount fails

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2017-07-28 09:55:23 -07:00
Patrick Donnelly
ced01a2335
qa: fix wait for wrong health message
Fixes: http://tracker.ceph.com/issues/20805

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2017-07-27 14:40:05 -07:00
Sage Weil
41bcf2fee5 Merge pull request #16281 from badone/wip-PG-cluster-log-audit
osd: Log audit

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2017-07-27 16:25:30 -05:00
Ilya Dryomov
bd6e3e5f1f qa/tasks/rbd_fio: bump default fio version to 2.21
I'm seeing sporadic single thread deadlocks on fio stat_mutex during krbd
thrash runs:

  (gdb) info threads
    Id   Target Id         Frame
  * 1    Thread 0x7f89ee730740 (LWP 15604) 0x00007f89ed9f41bd in __lll_lock_wait () from /lib64/libpthread.so.0
  (gdb) bt
  #0  0x00007f89ed9f41bd in __lll_lock_wait () from /lib64/libpthread.so.0
  #1  0x00007f89ed9f17b2 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  #2  0x00000000004429b9 in fio_mutex_down (mutex=0x7f89ee72d000) at mutex.c:170
  #3  0x0000000000459704 in thread_main (data=<optimized out>) at backend.c:1639
  #4  0x000000000045b013 in fork_main (offset=0, shmid=<optimized out>, sk_out=0x0) at backend.c:1778
  #5  run_threads (sk_out=sk_out@entry=0x0) at backend.c:2195
  #6  0x000000000045b47f in fio_backend (sk_out=sk_out@entry=0x0) at backend.c:2400
  #7  0x000000000040cb0c in main (argc=2, argv=0x7fffad3e3888, envp=<optimized out>) at fio.c:63
  (gdb) up 2
  170                     pthread_cond_wait(&mutex->cond, &mutex->lock);
  (gdb) p mutex.lock.__data.__owner
  $1 = 15604

Upgrading to 2.21 seems to make these go away.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2017-07-27 18:57:43 +02:00
Sage Weil
a40d94b163 qa/tasks/ceph: wait for pg stats to flush in healthy check
Signed-off-by: Sage Weil <sage@redhat.com>
2017-07-27 12:10:27 -04:00
Sage Weil
80978dea8a qa/tasks/ceph_manager: wait_for_all_up -> wait_for_all_osds_up
Signed-off-by: Sage Weil <sage@redhat.com>
2017-07-27 12:10:26 -04:00
Sage Weil
7648894e55 qa/tasks/ceph_manager: expose flush_all_pg_stats
Signed-off-by: Sage Weil <sage@redhat.com>
2017-07-27 12:10:26 -04:00
David Zafman
e92c953d7b Merge pull request #16610 from dzafman/wip-fix-reg11184
test: reg11184 might not always find pg 2.0 prior to import

Reviewed-by: Sage Weil <sage@redhat.com>
2017-07-26 11:42:15 -07:00
Sage Weil
59a3a4a40e Merge pull request #16559 from hjwsm1989/dump-stuck
qa/tasks/dump_stuck: fix dump_stuck test bug

Reviewed-by: Sage Weil <sage@redhat.com>
2017-07-26 11:59:21 -05:00
David Zafman
7c43840399 test: reg11184 might not always find pg 2.0 prior to import
Signed-off-by: David Zafman <dzafman@redhat.com>
2017-07-26 09:46:15 -07:00
Sage Weil
56ffd7a727 Merge pull request #16571 from ceph/wip-cd-bluestore-2
qa/tasks/ceph-deploy: Fix bluestore options for ceph-deploy

Reviewed-by: Tamil Muthamizhan <tmuthami@redhat.com>
2017-07-26 11:43:50 -05:00
Brad Hubbard
f8acc53d82 osd: Log audit
Review current log messages for consistency, accuracy and necessesity as
part of usability initiative. First in a series.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2017-07-26 17:34:28 +10:00
Vasu Kulkarni
2fa0fae72f Add option to specify bluestore/filestore options
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
2017-07-25 12:26:03 -07:00
huangjun
daf8efee32 qa/tasks/dump_stuck: fix dump_stuck test bug
Test cluster with 2 osds, stop osd.0, if osd.1
  report the pg stats during pg peering, mon will
  record pg state to 'peering',then stop osd.1,
  finally the pg state will stuck in 'stale+peering',
  which is unexpected.

  Let's wait_for_active() after stop osd.0.

  Signed-off-by: huangjun <huangjun@xsky.com>
2017-07-25 11:14:07 +00:00
Sage Weil
2e5955212d qa/tasks/workunit: allow alt basedir
Instead of 'qa/workunits' allow something like 'qa/standalone'.

Signed-off-by: Sage Weil <sage@redhat.com>
2017-07-24 15:44:51 -04:00
Sage Weil
02c2e853d3 Merge pull request #16509 from liewegas/wip-rgw-wait
qa/suits/rados/basic/tasks/rgw_snaps: wait for pools to be created

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2017-07-24 11:55:54 -05:00
Sage Weil
29549e6834 Merge pull request #13723 from ovh/bp-forced-recovery
osd/PG: make prioritized recovery possible

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2017-07-24 09:01:03 -05:00
Sage Weil
ecd1193ab9 qa/suites/rados/basic/tasks/rgw_snaps: wait for pools to be be created
Signed-off-by: Sage Weil <sage@redhat.com>
2017-07-22 18:54:46 -04:00
Patrick Donnelly
9506789ce1
Merge PR 16379 into master
* refs/remotes/upstream/pull/16379/head:
	qa: fix MDS_CLIENT_RECALL copy error

Reviewed-by: Zheng Yan <zyan@redhat.com>
2017-07-21 13:23:07 -07:00
Patrick Donnelly
23e3d40751
Merge PR 16226 into master
* refs/remotes/upstream/pull/16226/head:
	qa: wait for OSDMap to propagate for snap purge

Reviewed-by: Zheng Yan <zyan@redhat.com>
2017-07-21 13:22:47 -07:00
Kefu Chai
4599eb7963 Merge pull request #16454 from liewegas/wip-fix-ceph-scrub
qa/tasks/ceph_manager: wait for osd to start after objectstore-tool sequence

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-07-21 19:31:19 +08:00
Sage Weil
59e3827be7 qa/tasks/reg11184: import run
Signed-off-by: Sage Weil <sage@redhat.com>
2017-07-20 17:42:59 -04:00
Sage Weil
27e8d75f61 Merge pull request #16429 from liewegas/wip-jewel-x
qa/suites/upgrade/jewel-x: misc fixes for new health checks
2017-07-20 10:47:05 -05:00
Sage Weil
3de9f22ce0 Merge pull request #16423 from liewegas/wip-ls
mon: '* list' -> '* ls'

Reviewed-by: Kefu Chai <kchai@redhat.com>
2017-07-20 10:43:34 -05:00