Commit Graph

660 Commits

Author SHA1 Message Date
Patrick Donnelly
dde9ee1628
qa: add max_mds thrash test
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-17 11:26:56 -07:00
Patrick Donnelly
1dc5b62557
qa: mds_thrash updates for new max_mds behavior
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-17 11:26:56 -07:00
Patrick Donnelly
88922255e9
qa: add test for cluster resizing
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-17 11:26:56 -07:00
Patrick Donnelly
5503b2014c
qa: remove use of mds deactivate
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-17 11:26:56 -07:00
Patrick Donnelly
ab4f33bd24
cephfs: add new down/joinable fs flags
Fixes: http://tracker.ceph.com/issues/20609

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-17 11:25:59 -07:00
Douglas Fuller
7d97a6e6f8
cephfs: kill allow_dirfrags
As dirfrags are now standard in CephFS, remove the machinery for
tracking and enabling this feature.

ceph fs set <fs> allow_dirfrags is now deprecated and prints a warning
message.

Signed-off-by: Douglas Fuller <dfuller@redhat.com>
2018-04-17 11:01:03 -07:00
Sage Weil
53588a41a1
Merge pull request #21334 from batrick/i23643
qa: fix typo in test_full

Reviewed-by: Sage Weil <sage@redhat.com>
2018-04-11 08:25:55 -05:00
Patrick Nawracay
e4a0b12abf mgr/dashboard: Implement RGW proxy
This implementation is basically a Rados Gateway reverse proxy.  It
additionally takes care of the authentication to the Rados Gateway, but to use
it you will have to be authenticated against the dashboards RESTful API.

The corresponding credentials can be configured using the following commands:

    dashboard set-rgw-api-secret-key <secret-key>

    dashboard set-rgw-api-access-key <access-key>

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>
2018-04-11 12:26:42 +02:00
Lenz Grimmer
34232b347b
Merge pull request #21022 from sebastian-philipp/dashboard_get_range
mgr/dashboard: Refactor multiple duplicates of `get_rate()`

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2018-04-11 12:22:00 +02:00
Lenz Grimmer
f0614a8d4e
Merge pull request #20920 from sebastian-philipp/dashboard_v2-erasure-code-profile
mgr/dashboard: Add ErasureCodeProfile controller

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Joao Eduardo Luis <joao@suse.de>
2018-04-11 12:19:40 +02:00
Patrick Donnelly
c10907af02
qa: fix typo in test_full
Cause: 577737d007

Fixes: http://tracker.ceph.com/issues/23643

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-10 11:47:10 -07:00
Sage Weil
dbdbbbfc89 Merge PR #21315 into wip-sage-testing-20180410.153007
* refs/pull/21315/head:
	qa/tasks/mgr/test_failover: fix for id -> name rename in mgr metadata

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Jan Fajerski <jfajerski@suse.com>
2018-04-10 10:30:48 -05:00
Yehuda Sadeh
63221d9a81
Merge pull request #19283 from yehudasa/wip-rgw-mfa
rgw: mfa support

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2018-04-10 07:59:10 -07:00
Sage Weil
b0fe734c21 qa/tasks/mgr/test_failover: fix for id -> name rename in mgr metadata
This was changed by 428236de8b

Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-09 21:48:40 -05:00
Vasu Kulkarni
7881a19d92 qa/tasks: wait_for_clean is called after ceph task as well after osd's are up,
the default timeout is none in that case, there are cases where it can hang  forever
due to error cases, since this dumps quite a lot of info the logs grow in GB's, with
default timeout of 1200 we can avoid such huge logs and fail sooner. Any tests needing
higher timeout can pass the required value.

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
2018-04-09 17:24:42 -07:00
Sebastian Wagner
012be1dc86 qa/tasks/mgr/dashboard: Imroved JSON validation
Refactored `OsdTest` to make use of `self.assertSchema()`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2018-04-09 16:42:33 +02:00
Yehuda Sadeh
9569cc8278 qa/tasks/s3tests: configure mfa tests
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
2018-04-09 07:02:17 -07:00
Nathan Cutler
fa5e8f9376
Merge pull request #17531 from smithfarm/wip-calamari-tests
tests: cleanup: drop calamari tasks

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Lenz Grimmer <lenz@grimmer.com>
2018-04-08 08:54:27 +02:00
Sage Weil
577737d007 osd: osd_mon_report_interval_min -> osd_mon_report_interval, kill _max
The _max isn't used.  Drop the _min suffix.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-06 11:00:14 -05:00
Stephan Müller
c60da7bb1f mgr/dashboard: Pool backend changes
As needed by the UI.

Signed-off-by: Stephan Müller <smueller@suse.com>
2018-04-06 17:41:28 +02:00
Sebastian Wagner
7b6a45cc8c mgr/dashboard: Add ErasureCodeProfile controller
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2018-04-06 17:40:03 +02:00
Patrick Donnelly
ebd0fcd067
Merge PR #16779 into master
* refs/pull/16779/head:
	mds: cleanup MDCache::open_snaprealms()
	mds: make sure snaptable version > 0
	mds: don't consider CEPH_INO_LOST_AND_FOUND as base inode
	mds: replace MAX() with std::max()
	tools/cephfs: make cephfs-data-scan create snaprealm for base inodes
	qa/cephfs: don't run TestSnapshots.test_kill_mdstable on kclient
	qa/cephfs: adjust check of 'cephfs-table-tool all show snap' output
	mds: don't warn unconnected snaplrealms in cluster log
	mds: update CInode/CDentry's first according to global snapshot seq
	qa/cephfs: add tests for snapclient cache
	qa/cephfs: add tests for snaptable transaction
	mds: add asok command that dumps cached snap infos
	qa/cephfs: add tests for multimds snapshot
	client: don't mark snap directory complete when its dirstat is empty
	qa/workunits/snaps: add snaprealm split test
	mds: make sure mds has uptodate mdsmap before checking 'allows_snaps'
	client: fix incorrect snaprealm when adding caps
	qa/workunits/snaps: add hardlink snapshot test
	mds: add incompat feature and bump protocol for snapshot changes
	mds: detach inode with single hardlink from global snaprealm
	mds: record hardlink snaps in inode's snaprealm
	mds: attach inode with multiple hardlinks to dummy global snaprealm
	mds: cleanup rename code
	mds: ensure xlocker has uptodate lock state
	mds: simplify SnapRealm::build_snap_{set,trace}
	mds: record global last_created/last_destroyed in snaptable
	mds: pop projected snaprealm before inode's parent changes
	mds: keep isnap lock in sync state
	mds: handle mksnap vs resolve_snapname race
	mds: cleanup snaprealm past parents open check
	mds: rollback snaprealms when rolling back slave request
	mds: send updated snaprealms along with slave requests
	mds: explict notification for snap update
	mds: send snap related messages centrally during mds recovery
	mds: synchronize snaptable caches when mds recovers
	mds: introduce MDCache::maybe_finish_slave_resolve()
	mds: notify all mds about prepared snaptable update
	mds: record snaps in old snaprealm when moving inode into new snaprealm
	mds: cache snaptable in snapclient
	mds: recover snaptable client when mds enters resolve state

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-03 15:41:19 -07:00
Patrick Donnelly
0186795496
Merge PR #21180 into master
* refs/pull/21180/head:
	vstart_runner: examine check_status before error

Reviewed-by: John Spray <john.spray@redhat.com>
2018-04-03 06:51:18 -07:00
Jason Dillaman
3b08c0609c Merge pull request #20460 from colletj/v1_image_creation_disallow
librbd: disallow creation of v1 image format

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
2018-04-03 09:18:39 -04:00
Patrick Donnelly
9a558c51dc
vstart_runner: examine check_status before error
Issue found by Rishabh Dave. Fix suggested by John Spray.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-02 08:08:43 -07:00
Patrick Donnelly
b7fce64601
Merge PR #20132 into master
* refs/pull/20132/head:
	qa/cephfs: update TestDamage for open file table
	mds: allow storing open file table in multiple omaps
	mds: differentiate Anchor types to clarify purpose
	mds: add perf counter for 'open ino' operation
	mds: protect open file table against partial omap update
	mds: add dirfrags whose child inodes have caps to open file table
	mds: don't try prefetching destroyed inodes
	mds: don't try opening inodes that haven't been created
	mds: don't re-requeue open files to head of log
	mds: use open file table to speed up mds recovery
	mds: introduce open file table
	mds: track how many clients/mds want caps for each inode
	mds: cleanup MDCache::opening_inodes access
	mds: cleanup CInode/CDir states definition

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-03-30 22:25:10 -07:00
Mykola Golub
af15b2cef4 qa/tasks/qemu: add a parameter to wait for workloads detect qemu finished
In the case when a workload needs to detect qemu finished by running a
check with a periodicity of N sec it needs to set time_wait to 2 * N
in order to avoid races on finish.

Signed-off-by: Mykola Golub <mgolub@suse.com>
2018-03-29 17:43:31 +03:00
Ricardo Dias
5a861f5fb1
Merge pull request #20870 from rjfd/wip-dashboard-tasks
mgr/dashboard: asynchronous task support

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2018-03-28 08:27:51 +01:00
zouaiguo
4840e899f1 [mgr/dashboard] add image id to rbd info instead of block_name_prefix
when create images use data_pool parameter, such as:
rbd create -p pool1 -s 1G --data-pool pool2  image1

then get image info from mgr use request http://192.7.7.36:7000/rbd_pool_data/pool1
we got the response:
[{"parent_name": "", "parent_pool": 18446744073709551615, "name": "image1", "num_objs": 256, "block_name_prefix": "rbd_data.14.f561643c986", "obj_size": 4194304, "features": 189, "features_name": "fast-diff, layering, exclusive-lock, deep-flatten, object-map, data-pool", "order": 22, "size": 1073741824}]

notice:
since block_name_prefix is deprecated,
from block_name_prefix we can not get image id(the real id is ''f561643c9869')
we need add image id to rbd info instead block_name_prefix

Python 2.7.5 (default, Jun 17 2014, 18:11:42)
[GCC 4.8.2 20140120 (Red Hat 4.8.2-16)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import rados
>>> import rbd
>>> RADOS_NAME = 'client.admin'
>>> cluster_name="ceph"
>>> cluster_handle = rados.Rados(name=RADOS_NAME, clustername=cluster_name, conffile='')
>>> cluster_handle.connect()
>>> ioctx = cluster_handle.open_ioctx("pool1")
>>> image_inst = rbd.Image(ioctx, "image1")
>>> image_inst.stat()
{'parent_name': '', 'parent_pool': 18446744073709551615L, 'num_objs': 256L, 'block_name_prefix': u'rbd_data.14.f561643c986', 'obj_size': 4194304L, 'order': 22, 'size': 1073741824L}
>>> image_inst.id()
u'f561643c9869'

after fix:

[root@tfg36 site-packages]# curl http://192.7.7.36:7000/rbd_pool_data/pool1
[{"parent_name": "", "parent_pool": 18446744073709551615, "name": "image1", "num_objs": 256, "block_name_prefix": "rbd_data.14.f561643c986", "obj_size": 4194304, "features": 189, "id": "f561643c9869", "features_name": "fast-diff, layering, exclusive-lock, deep-flatten, object-map, data-pool", "order": 22, "size": 1073741824}]

Signed-off-by: zouaiguo <zou.aiguo@zte.com.cn>
2018-03-28 10:14:42 +08:00
Ricardo Dias
067b86eb42
mgr/dashboard: added tasks info to summary controller
Signed-off-by: Ricardo Dias <rdias@suse.com>
2018-03-27 14:08:47 +01:00
Lenz Grimmer
d5e54fda5c
Merge pull request #21024 from s0nea/wip-dashboard-return-features-list
mgr/dashboard: Convert the RBD feature names to a list of strings

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
2018-03-27 11:45:54 +02:00
Lenz Grimmer
914faf2c1e
Merge pull request #21004 from s0nea/wip-dashboard-skip-datapool-testcase
mgr/dashboard: skip data pool testcase for none-bluestore clusters

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
2018-03-27 11:43:18 +02:00
Lenz Grimmer
43e5097c2f
Merge pull request #20873 from sebastian-philipp/dashboard_v2-simple-html-debug
mgr/dashboard: Add minimalistic browsable API

Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
2018-03-27 11:31:07 +02:00
Casey Bodley
6b59fbfcc6
Merge pull request #21027 from cbodley/wip-qa-swift-force-branch
qa/tasks/swift: add support for the "force-branch" configurable.

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
2018-03-26 14:27:46 -04:00
Sebastian Wagner
7a7a17caa9 mgr/dashboard: New API tests for perf_couters
Added OSD and MDS tests.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2018-03-26 17:16:41 +02:00
Ricardo Dias
a392e612cc
Merge pull request #21007 from rjfd/wip-dashboard-fix-py3
mgr/dashboard: fix dashboard python 3 support

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
2018-03-26 10:59:17 +01:00
Tatjana Dehler
33e15ecd39 mgr/dashboard: convert RBD features to a list of strings
Until now the API returns the features provided by an RBD as one comma
separated string. It's more usable for the frontend to receive a list
of strings instead.

Signed-off-by: Tatjana Dehler <tdehler@suse.com>
2018-03-26 09:41:56 +02:00
Radoslaw Zarzynski
c5859f25e6 qa/tasks/swift: add support for the "force-branch" configurable.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2018-03-23 12:40:05 -04:00
Sebastian Wagner
cbee371958 mgr/dashboard: Add Teuthology test for /api/pool/_info
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2018-03-23 16:40:18 +01:00
Sebastian Wagner
dd18675e4f mgr/dashboard: Minor tweak to DashboardTestCase._request
Don't fall through silently.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2018-03-23 16:40:18 +01:00
Sebastian Wagner
e6c1056ad5 mgr/dashboard: Add create() to Pool controller
Also:

* Added tests.
* Renamed `DashboardTest` to `PoolTest`.
* Added `delete()`.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2018-03-23 16:40:18 +01:00
Yuri Weinstein
6a5bab7dd4
Merge pull request #20981 from cbodley/wip-qa-fix-ssl
qa: fix overrides for openssl_keys task

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
2018-03-23 06:55:10 -07:00
Ricardo Dias
d101b33bc7 qa/tasks/mgr/dashboard: Fix login expires too soon
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
2018-03-23 11:30:16 +01:00
Tatjana Dehler
075102468b mgr/dashboard: skip 'test_create_rbd_in_data_pool' without bluestore
Skip the test case 'test_create_rbd_in_data_pool' only if the cluster
does not support bluestore, otherwise run the test. The Ceph mgr
provides the information if the cluster supports bluestore or not, so
the test will be skipped depending on the current cluster.

Signed-off-by: Tatjana Dehler <tdehler@suse.com>
2018-03-23 11:11:27 +01:00
Tatjana Dehler
25a0ed93ec mgr/dashboard: add 'osd metadata' command call
Signed-off-by: Tatjana Dehler <tdehler@suse.com>
2018-03-23 11:11:17 +01:00
Ricardo Dias
25c75a952d
mgr/dashboard: fix dashboard python 3 support
Signed-off-by: Ricardo Dias <rdias@suse.com>
2018-03-22 15:56:20 +00:00
Casey Bodley
cfe4ba6342 qa: fix overrides for openssl_keys task
teuthology looks for overrides by the class name, so overrides weren't
working after the task was renamed

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2018-03-22 10:31:08 -04:00
Yan, Zheng
ffebd0854e qa/cephfs: update TestDamage for open file table
Signed-off-by: Yan, Zheng <zyan@redhat.com>
2018-03-21 21:08:58 +08:00
Sage Weil
038afcbc2a Merge remote-tracking branch 'gh/mimic-dev2' 2018-03-18 18:39:46 -05:00
Yan, Zheng
84d7414249 qa/cephfs: don't run TestSnapshots.test_kill_mdstable on kclient
this test forcibly kills mount. Killing kernel mount requires reboot,

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2018-03-18 22:24:42 +08:00