Commit Graph

3423 Commits

Author SHA1 Message Date
Nathan Cutler
fa5e8f9376
Merge pull request #17531 from smithfarm/wip-calamari-tests
tests: cleanup: drop calamari tasks

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Lenz Grimmer <lenz@grimmer.com>
2018-04-08 08:54:27 +02:00
Kefu Chai
9e840c4382
Merge pull request #21274 from dzafman/wip-cot-config
tools: Use --no-mon-config so ceph_objectstore_tool.py test doesn't hang

Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-04-07 11:59:28 +08:00
Sage Weil
d7bca5d19f Merge PR #20512 into wip-sage-testing-20180407.022843
* refs/pull/20512/head:
	osd: osd_mon_report_interval_min -> osd_mon_report_interval, kill _max

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-04-06 21:29:32 -05:00
David Zafman
a8d26122dc tools: Use --no-mon-config so ceph_objectstore_tool.py test doesn't hang
Signed-off-by: David Zafman <dzafman@redhat.com>
2018-04-06 11:52:10 -07:00
Gregory Farnum
6d2e4c9b7b
Merge pull request #19973 from liewegas/wip-peering-fast-dispatch
osd: fast dispatch of peering events and pg_map + osd sharded wq refactor

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2018-04-06 11:48:11 -07:00
Sage Weil
577737d007 osd: osd_mon_report_interval_min -> osd_mon_report_interval, kill _max
The _max isn't used.  Drop the _min suffix.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-06 11:00:14 -05:00
Sage Weil
8f8f65594b
Merge pull request #21250 from liewegas/wip-recovery-slow-ops
qa/suites/rados/singleton/all/recovery_preemption: whitelist SLOW_OPS
2018-04-06 10:59:23 -05:00
Sage Weil
b235a3f62a qa/suites/rados/singleton/all/ec-lost-unfound: whitelist SLOW_OPS
Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-06 10:38:45 -05:00
Joao Eduardo Luis
940dd941ef
Merge pull request #19331 from jecluis/wip-mon-osdmap-prune
mon: osdmap prune

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
2018-04-06 15:22:28 +01:00
Joao Eduardo Luis
3997eed4db qa: enable mon osdmap pruning on 'rados/' suites
Signed-off-by: Joao Eduardo Luis <joao@suse.de>
2018-04-06 04:18:23 +01:00
Joao Eduardo Luis
65e7e4a38f qa: enable mon osdmap pruning on rados/monthrash
Signed-off-by: Joao Eduardo Luis <joao@suse.de>
2018-04-06 04:18:23 +01:00
Joao Eduardo Luis
2ffed4c98f qa: mon: osdmap pruning standalone/workunit
Keep a standalone wrapper for the workunit, so we can test it locally,
leveraging the ceph-helpers to do the setup. Keep a workunit to be
exercised by teuthology.

Signed-off-by: Joao Eduardo Luis <joao@suse.de>
2018-04-06 04:18:23 +01:00
Joao Eduardo Luis
6393c7ddeb qa/standalone: change PATH to allow finding sysctl
If, by default, a user's PATH does not contain /sbin and /usr/sbin, we
may have a hard time finding sysctl to adjust the kernel core pattern.

Signed-off-by: Joao Eduardo Luis <joao@suse.de>
2018-04-06 04:18:22 +01:00
Joao Eduardo Luis
f501458967 qa/standalone: deduplicate cleanup code
Signed-off-by: Joao Eduardo Luis <joao@suse.de>
2018-04-06 04:18:22 +01:00
Joao Eduardo Luis
5ba6286834 qa/standalone: discard pipe in sysctl kernel.core_pattern
At least verified in openSUSE tumbleweed and ubuntu 16.04,
`sysctl -n kernel.core_pattern` returns a pipe at the start of the
value. We reset core_pattern to its original form once the script is
about to end, but if we do not discard the pipe the new value will
contain an extra pipe (apparently, the pipe is added automatically at
some point, possibly simply on read). We are simply stripping it on
read, as to prevent this behavior.

Additionally, we are also enclosing the reset of kernel.core_pattern in
quotes, so as to make sure patterns that include spaces (e.g., ubuntu's
apport, or tumbleweed's systemd-coredump) are properly reset.

Signed-off-by: Joao Eduardo Luis <joao@suse.de>
2018-04-06 04:18:22 +01:00
Casey Bodley
35fd4050b7 qa/rgw: verify suite tests beast with ssl
instead of adding special frontend configs like civetweb_ssl.yaml and
beast_ssl.yaml, added a new proto/ subdirectory for http.yaml and
https.yaml

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2018-04-05 15:30:22 -04:00
Kefu Chai
d440902b4d
Merge pull request #21253 from ceph/wip-yuriw-crontab
tests/qa - enabled `ceph-deploy` runs on `mira` nodes

Reviewed-by: Vasu Kulkarni <vasu@redhat.com>
2018-04-05 15:04:02 +08:00
Yuri Weinstein
3846f8445f tests/qa - enabled ceph-deploy runs on mira nodes
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2018-04-04 16:18:41 -07:00
Neha Ojha
109057c037 qa/crontab: run the perf-basic suite every day
Signed-off-by: Neha Ojha <nojha@redhat.com>
2018-04-04 15:26:21 -07:00
Sage Weil
3b3c32f643 qa/suites/rados/singleton/all/recovery_preemption: whitelist SLOW_OPS
Recovery and peering can be slow enough with all the logging enabled to
trigger a slow ops warning.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-04 16:24:31 -05:00
Josh Durgin
90600bd18d
Merge pull request #21101 from dzafman/wip-23267
scrub errors not cleared on replicas can cause inconsistent pg state when replica takes over primary

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2018-04-04 14:06:46 -07:00
vasukulkarni
806072a494
Merge pull request #21146 from zmc/wip-cephmetrics
qa/suites/cephmetrics: Updates for new version
2018-04-04 08:28:57 -07:00
Sage Weil
26f00dd67c qa/suites: mon warn on pool no app = false for api tests
Among other things, the list.cc tests set pg_num which waits for cluster
healthy.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-04 08:26:58 -05:00
Sage Weil
c2cce3bc88 qa/suites/rados/basic/tasks/rados_api_tests: debug ms = 1
Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-04 08:26:58 -05:00
Sage Weil
29a885c915 qa/suites/rados/singleton/all/recovery_preemption: make test more reliable
A 30 second run did only 7000 ops, which means ~50 log entires per pg...
not enough to trigger backfill.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-04 08:26:57 -05:00
Sage Weil
c3589df320 qa/suites/rados/singleton/all/mon-seesaw: whitelist PG_AVAILABILITY
The seesaw might delay pg creation by more than 60s.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-04 08:26:57 -05:00
Patrick Donnelly
ebd0fcd067
Merge PR #16779 into master
* refs/pull/16779/head:
	mds: cleanup MDCache::open_snaprealms()
	mds: make sure snaptable version > 0
	mds: don't consider CEPH_INO_LOST_AND_FOUND as base inode
	mds: replace MAX() with std::max()
	tools/cephfs: make cephfs-data-scan create snaprealm for base inodes
	qa/cephfs: don't run TestSnapshots.test_kill_mdstable on kclient
	qa/cephfs: adjust check of 'cephfs-table-tool all show snap' output
	mds: don't warn unconnected snaplrealms in cluster log
	mds: update CInode/CDentry's first according to global snapshot seq
	qa/cephfs: add tests for snapclient cache
	qa/cephfs: add tests for snaptable transaction
	mds: add asok command that dumps cached snap infos
	qa/cephfs: add tests for multimds snapshot
	client: don't mark snap directory complete when its dirstat is empty
	qa/workunits/snaps: add snaprealm split test
	mds: make sure mds has uptodate mdsmap before checking 'allows_snaps'
	client: fix incorrect snaprealm when adding caps
	qa/workunits/snaps: add hardlink snapshot test
	mds: add incompat feature and bump protocol for snapshot changes
	mds: detach inode with single hardlink from global snaprealm
	mds: record hardlink snaps in inode's snaprealm
	mds: attach inode with multiple hardlinks to dummy global snaprealm
	mds: cleanup rename code
	mds: ensure xlocker has uptodate lock state
	mds: simplify SnapRealm::build_snap_{set,trace}
	mds: record global last_created/last_destroyed in snaptable
	mds: pop projected snaprealm before inode's parent changes
	mds: keep isnap lock in sync state
	mds: handle mksnap vs resolve_snapname race
	mds: cleanup snaprealm past parents open check
	mds: rollback snaprealms when rolling back slave request
	mds: send updated snaprealms along with slave requests
	mds: explict notification for snap update
	mds: send snap related messages centrally during mds recovery
	mds: synchronize snaptable caches when mds recovers
	mds: introduce MDCache::maybe_finish_slave_resolve()
	mds: notify all mds about prepared snaptable update
	mds: record snaps in old snaprealm when moving inode into new snaprealm
	mds: cache snaptable in snapclient
	mds: recover snaptable client when mds enters resolve state

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-03 15:41:19 -07:00
Zack Cerza
dd5e15a33d qa/suites/cephmetrics: Add new facets
For EPEL and containerized prometheus/grafana

Signed-off-by: Zack Cerza <zack@redhat.com>
2018-04-03 15:44:33 -06:00
Zack Cerza
177b444b8d qa/suites/cephmetrics: Update ceph-ansible config
This is taken from the ceph-ansible suite.

Signed-off-by: Zack Cerza <zack@redhat.com>
2018-04-03 15:44:33 -06:00
David Zafman
5cfb8241f4 osd: Fix stale scrub stats when a primary takes over
Fixes: http://tracker.ceph.com/issues/23267

Signed-off-by: David Zafman <dzafman@redhat.com>
2018-04-03 12:51:06 -07:00
David Galloway
07186a5b87 test:qa:infra - Run update daily and use bash
bash has the path for `source` to activate the virtualenv.  sh doesn't.

[skip ci]

Signed-off-by: David Galloway <dgallowa@redhat.com>
2018-04-03 11:39:13 -04:00
Patrick Donnelly
0186795496
Merge PR #21180 into master
* refs/pull/21180/head:
	vstart_runner: examine check_status before error

Reviewed-by: John Spray <john.spray@redhat.com>
2018-04-03 06:51:18 -07:00
Jason Dillaman
3b08c0609c Merge pull request #20460 from colletj/v1_image_creation_disallow
librbd: disallow creation of v1 image format

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-04-03 09:18:39 -04:00
Yuri Weinstein
9b7b5a7673
Merge pull request #21183 from neha-ojha/wip-minor-fix-perf-suite
qa/suites/perf-basic: add desc regarding test machines

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
2018-04-02 13:56:08 -07:00
David Galloway
17f9d4e351
Merge pull request #21075 from ceph/wip-yuriw-try-crontab
test:qa:infra - teuthology crontab items as of 3/27/18
2018-04-02 16:08:02 -04:00
Neha Ojha
ca3c2156f8 qa/suites/perf-basic: add desc regarding test machines
Signed-off-by: Neha Ojha <nojha@redhat.com>
2018-04-02 13:01:30 -07:00
Patrick Donnelly
9a558c51dc
vstart_runner: examine check_status before error
Issue found by Rishabh Dave. Fix suggested by John Spray.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-02 08:08:43 -07:00
Patrick Donnelly
b7fce64601
Merge PR #20132 into master
* refs/pull/20132/head:
	qa/cephfs: update TestDamage for open file table
	mds: allow storing open file table in multiple omaps
	mds: differentiate Anchor types to clarify purpose
	mds: add perf counter for 'open ino' operation
	mds: protect open file table against partial omap update
	mds: add dirfrags whose child inodes have caps to open file table
	mds: don't try prefetching destroyed inodes
	mds: don't try opening inodes that haven't been created
	mds: don't re-requeue open files to head of log
	mds: use open file table to speed up mds recovery
	mds: introduce open file table
	mds: track how many clients/mds want caps for each inode
	mds: cleanup MDCache::opening_inodes access
	mds: cleanup CInode/CDir states definition

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-03-30 22:25:10 -07:00
Vasu Kulkarni
80f84baa94 qa/tests: use stable branch for single node tests
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
2018-03-30 09:09:37 -07:00
Jason Dillaman
6885380c24
Merge pull request #18317 from hitoshikamei/master-rbd
rbd: eager-thick provisioning support

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-03-30 08:45:15 -04:00
Jason Dillaman
285a2fdbfc
Merge pull request #20935 from trociny/wip-get-name
librbd: add API function to get image name 

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-03-30 08:44:28 -04:00
Mykola Golub
6fe8d64705 qa/suites/rbd: set qemu task time_wait param
so workloads qemu_dynamic_features.sh and qemu_rebuild_object_map.sh,
which check if qemu is finished with periodicity 60 sec, have enough
time to detect this before the rbd image is removed.

Fixes: https://tracker.ceph.com/issues/23502
Signed-off-by: Mykola Golub <mgolub@suse.com>
2018-03-29 17:44:00 +03:00
Mykola Golub
af15b2cef4 qa/tasks/qemu: add a parameter to wait for workloads detect qemu finished
In the case when a workload needs to detect qemu finished by running a
check with a periodicity of N sec it needs to set time_wait to 2 * N
in order to avoid races on finish.

Signed-off-by: Mykola Golub <mgolub@suse.com>
2018-03-29 17:43:31 +03:00
Yuri Weinstein
8c31e568e2 test:qa:infra - teuthology crontab items as of 3/27/18
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2018-03-29 06:31:03 -07:00
Mykola Golub
d392393e1f qa/workunits/rbd: remove sanity check in test_admin_socket.sh
Fixes: https://tracker.ceph.com/issues/23499
Signed-off-by: Mykola Golub <mgolub@suse.com>
2018-03-29 12:29:40 +03:00
Neha Ojha
6aceb66524
Merge pull request #21104 from neha-ojha/wip-minimal-perf-suite
qa/suites: add minimal performance suite

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Varada Kari <varada.kari@gmail.com>
2018-03-28 20:31:30 -07:00
Neha Ojha
c5484e098e qa/suites: add minimal performance suite
Signed-off-by: Neha Ojha <nojha@redhat.com>
2018-03-28 17:36:55 -07:00
David Zafman
ce9c029858 test: Eliminate use of bc (use awk) in get_timeout_delays()
Signed-off-by: David Zafman <dzafman@redhat.com>
2018-03-28 10:24:33 -07:00
Ricardo Dias
5a861f5fb1
Merge pull request #20870 from rjfd/wip-dashboard-tasks
mgr/dashboard: asynchronous task support

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2018-03-28 08:27:51 +01:00
zouaiguo
4840e899f1 [mgr/dashboard] add image id to rbd info instead of block_name_prefix
when create images use data_pool parameter, such as:
rbd create -p pool1 -s 1G --data-pool pool2  image1

then get image info from mgr use request http://192.7.7.36:7000/rbd_pool_data/pool1
we got the response:
[{"parent_name": "", "parent_pool": 18446744073709551615, "name": "image1", "num_objs": 256, "block_name_prefix": "rbd_data.14.f561643c986", "obj_size": 4194304, "features": 189, "features_name": "fast-diff, layering, exclusive-lock, deep-flatten, object-map, data-pool", "order": 22, "size": 1073741824}]

notice:
since block_name_prefix is deprecated,
from block_name_prefix we can not get image id(the real id is ''f561643c9869')
we need add image id to rbd info instead block_name_prefix

Python 2.7.5 (default, Jun 17 2014, 18:11:42)
[GCC 4.8.2 20140120 (Red Hat 4.8.2-16)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import rados
>>> import rbd
>>> RADOS_NAME = 'client.admin'
>>> cluster_name="ceph"
>>> cluster_handle = rados.Rados(name=RADOS_NAME, clustername=cluster_name, conffile='')
>>> cluster_handle.connect()
>>> ioctx = cluster_handle.open_ioctx("pool1")
>>> image_inst = rbd.Image(ioctx, "image1")
>>> image_inst.stat()
{'parent_name': '', 'parent_pool': 18446744073709551615L, 'num_objs': 256L, 'block_name_prefix': u'rbd_data.14.f561643c986', 'obj_size': 4194304L, 'order': 22, 'size': 1073741824L}
>>> image_inst.id()
u'f561643c9869'

after fix:

[root@tfg36 site-packages]# curl http://192.7.7.36:7000/rbd_pool_data/pool1
[{"parent_name": "", "parent_pool": 18446744073709551615, "name": "image1", "num_objs": 256, "block_name_prefix": "rbd_data.14.f561643c986", "obj_size": 4194304, "features": 189, "id": "f561643c9869", "features_name": "fast-diff, layering, exclusive-lock, deep-flatten, object-map, data-pool", "order": 22, "size": 1073741824}]

Signed-off-by: zouaiguo <zou.aiguo@zte.com.cn>
2018-03-28 10:14:42 +08:00