Commit Graph

168 Commits

Author SHA1 Message Date
Yan, Zheng
0edab5fcc3
Merge pull request #21941 from ukernel/wip-24053
mds: reply session reject for open request from blacklisted client
2018-05-19 17:07:40 +08:00
Yan, Zheng
e7d0b41dea qa/tasks/cephfs: add timeout parameter to kclient umount_wait
Just make caller happy. there is no easy way to support timeout.

Signed-off-by: Yan, Zheng <zyan@redhat.com>
Fixes: https://tracker.ceph.com/issues/24053
2018-05-19 17:06:19 +08:00
Patrick Donnelly
fbc25b4486
mds: check for session import race
Credit to Yan Zheng for identifying the race condition [1].

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1572555#c7

Test-for: http://tracker.ceph.com/issues/24072

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-05-14 12:52:50 -07:00
Patrick Donnelly
3838674f65 qa: ignore version in auth metadata comp
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-05-08 20:15:24 +05:30
Ramana Raja
3a7fdb8b05 ceph_volume_client: allow volumes without namespace isolation
Fixes: https://tracker.ceph.com/issues/23695

Signed-off-by: Ramana Raja <rraja@redhat.com>
2018-05-08 20:15:15 +05:30
Patrick Donnelly
e510e29e01
Merge PR #21712 into master
* refs/pull/21712/head:
	qa/tasks/cephfs: add test for renewing stale session
	client: invalidate caps and leases when session becomes stale
	client: fix race in concurrent readdir

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-05-07 15:53:32 -07:00
Patrick Donnelly
6a788bf203
qa: add mds deactivation procedure for upgrades
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-05-03 16:51:43 -07:00
Patrick Donnelly
c44adca121
Merge PR #21374 into master
* refs/pull/21374/head:
	qa: add test for snap format upgrade
	mds: initialize SnapServer::snaprealm_v2_since after journal replay
	mds: properly distinguish cap update from snap flush
	mds: update dev document of cephfs snapshot
	doc: add release notes for cephfs snapshot
	mds: allow snapshot by default for new filesystem
	mds: close past parents after snaprealm format gets converted
	mds: automaticly allow multi-active MDS after scrubbing all inodes
	mds: don't mark primary dentry damaged if inode has been repaired
	mds: upgrade snaprealm format during scrub
	mds: allow scrubbing mdsdir
	mds: cleanup scrub code
	mds: show health warning if multimds with old format snapshots
	mds: automaticly allow multi-active MDS after removing all old snapshots
	mds: disallow multi-active MDS if snapshot was ever created by pre-mimic mds
	mds: validate SnapInfo::long_name before using it
	mds: don't bump snaptable last_snap when renaming snapshot
	mds: properly save snaptable after upgrading version

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-05-03 09:45:04 -07:00
Patrick Donnelly
91942df5a6
qa: add test for snap format upgrade
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-05-03 09:09:03 -07:00
Yan, Zheng
5688476513 qa/tasks/cephfs: add test for renewing stale session
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2018-05-03 10:50:28 +08:00
Patrick Donnelly
36b51db0e6
qa: fix blacklisted check for test_lifecycle
Caused by: 36f89c5acf

Fixes: http://tracker.ceph.com/issues/23975

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-05-02 15:13:39 -07:00
Patrick Donnelly
36f89c5acf
Merge PR #21065 into master
* refs/pull/21065/head:
	qa/cephfs: test if evicted client unmounts without hanging
	qa/tasks: allow custom timeout for umount_wait()
	client: don't hang when MDS sessions are evicted

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-05-02 09:07:31 -07:00
Rishabh Dave
b14302d1fe qa/cephfs: test if evicted client unmounts without hanging
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2018-04-30 12:02:56 +00:00
Rishabh Dave
18a9d0c491 qa/tasks: allow custom timeout for umount_wait()
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2018-04-30 12:02:56 +00:00
Patrick Donnelly
b39972d911
qa: remove racy/buggy test_purge_queue_op_rate
Fixes: https://tracker.ceph.com/issues/23829

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-24 12:48:40 -07:00
Patrick Donnelly
378a6fcfe8
qa: check old_max_mds is not wiped by marked down twice
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-19 11:57:45 -07:00
Patrick Donnelly
bb95292213
Merge PR #16608 into master
* refs/pull/16608/head:
	qa: whitelist mds down wrn during cephfs testing
	mds: add config to disable fragmentation
	qa: add max_mds thrash test
	qa: mds_thrash updates for new max_mds behavior
	doc: update upgrade procedure and release notes
	qa: add test for cluster resizing
	qa: remove use of mds deactivate
	cephfs: add new down/joinable fs flags
	mds: evict all clients if last mds shutting down
	cephfs: deprecate ceph mds deactivate
	cephfs: kill allow_dirfrags
	cephfs: Kill allow_multimds
	cephfs: Change behavior of cluster_down flag
	mon/FSCommands: Set extra MDS to standby
	cephfs: Health check changes
	mon/MDSMonitor: Remove command support for legacy syntax

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-18 14:36:54 -07:00
Patrick Donnelly
dde9ee1628
qa: add max_mds thrash test
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-17 11:26:56 -07:00
Patrick Donnelly
88922255e9
qa: add test for cluster resizing
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-17 11:26:56 -07:00
Patrick Donnelly
5503b2014c
qa: remove use of mds deactivate
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-17 11:26:56 -07:00
Patrick Donnelly
ab4f33bd24
cephfs: add new down/joinable fs flags
Fixes: http://tracker.ceph.com/issues/20609

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-17 11:25:59 -07:00
Douglas Fuller
7d97a6e6f8
cephfs: kill allow_dirfrags
As dirfrags are now standard in CephFS, remove the machinery for
tracking and enabling this feature.

ceph fs set <fs> allow_dirfrags is now deprecated and prints a warning
message.

Signed-off-by: Douglas Fuller <dfuller@redhat.com>
2018-04-17 11:01:03 -07:00
Patrick Donnelly
575c3716e3
qa: fix log typo
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-12 11:02:19 -07:00
Patrick Donnelly
c10907af02
qa: fix typo in test_full
Cause: 577737d007

Fixes: http://tracker.ceph.com/issues/23643

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-10 11:47:10 -07:00
Sage Weil
577737d007 osd: osd_mon_report_interval_min -> osd_mon_report_interval, kill _max
The _max isn't used.  Drop the _min suffix.

Signed-off-by: Sage Weil <sage@redhat.com>
2018-04-06 11:00:14 -05:00
Patrick Donnelly
ebd0fcd067
Merge PR #16779 into master
* refs/pull/16779/head:
	mds: cleanup MDCache::open_snaprealms()
	mds: make sure snaptable version > 0
	mds: don't consider CEPH_INO_LOST_AND_FOUND as base inode
	mds: replace MAX() with std::max()
	tools/cephfs: make cephfs-data-scan create snaprealm for base inodes
	qa/cephfs: don't run TestSnapshots.test_kill_mdstable on kclient
	qa/cephfs: adjust check of 'cephfs-table-tool all show snap' output
	mds: don't warn unconnected snaplrealms in cluster log
	mds: update CInode/CDentry's first according to global snapshot seq
	qa/cephfs: add tests for snapclient cache
	qa/cephfs: add tests for snaptable transaction
	mds: add asok command that dumps cached snap infos
	qa/cephfs: add tests for multimds snapshot
	client: don't mark snap directory complete when its dirstat is empty
	qa/workunits/snaps: add snaprealm split test
	mds: make sure mds has uptodate mdsmap before checking 'allows_snaps'
	client: fix incorrect snaprealm when adding caps
	qa/workunits/snaps: add hardlink snapshot test
	mds: add incompat feature and bump protocol for snapshot changes
	mds: detach inode with single hardlink from global snaprealm
	mds: record hardlink snaps in inode's snaprealm
	mds: attach inode with multiple hardlinks to dummy global snaprealm
	mds: cleanup rename code
	mds: ensure xlocker has uptodate lock state
	mds: simplify SnapRealm::build_snap_{set,trace}
	mds: record global last_created/last_destroyed in snaptable
	mds: pop projected snaprealm before inode's parent changes
	mds: keep isnap lock in sync state
	mds: handle mksnap vs resolve_snapname race
	mds: cleanup snaprealm past parents open check
	mds: rollback snaprealms when rolling back slave request
	mds: send updated snaprealms along with slave requests
	mds: explict notification for snap update
	mds: send snap related messages centrally during mds recovery
	mds: synchronize snaptable caches when mds recovers
	mds: introduce MDCache::maybe_finish_slave_resolve()
	mds: notify all mds about prepared snaptable update
	mds: record snaps in old snaprealm when moving inode into new snaprealm
	mds: cache snaptable in snapclient
	mds: recover snaptable client when mds enters resolve state

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-04-03 15:41:19 -07:00
Patrick Donnelly
b7fce64601
Merge PR #20132 into master
* refs/pull/20132/head:
	qa/cephfs: update TestDamage for open file table
	mds: allow storing open file table in multiple omaps
	mds: differentiate Anchor types to clarify purpose
	mds: add perf counter for 'open ino' operation
	mds: protect open file table against partial omap update
	mds: add dirfrags whose child inodes have caps to open file table
	mds: don't try prefetching destroyed inodes
	mds: don't try opening inodes that haven't been created
	mds: don't re-requeue open files to head of log
	mds: use open file table to speed up mds recovery
	mds: introduce open file table
	mds: track how many clients/mds want caps for each inode
	mds: cleanup MDCache::opening_inodes access
	mds: cleanup CInode/CDir states definition

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-03-30 22:25:10 -07:00
Yan, Zheng
ffebd0854e qa/cephfs: update TestDamage for open file table
Signed-off-by: Yan, Zheng <zyan@redhat.com>
2018-03-21 21:08:58 +08:00
Yan, Zheng
84d7414249 qa/cephfs: don't run TestSnapshots.test_kill_mdstable on kclient
this test forcibly kills mount. Killing kernel mount requires reboot,

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2018-03-18 22:24:42 +08:00
Yan, Zheng
338c3cf4c3 qa/cephfs: adjust check of 'cephfs-table-tool all show snap' output
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2018-03-18 18:19:08 +08:00
Patrick Donnelly
2e44b87141
Merge PR #19263 into master
* refs/pull/19263/head:
	qa: ignore bad backtrace cluster wrn
	qa/cephfs: Add tests to validate scrub functionality
	cephfs: Add option to load invalid metadata from disk
	cephfs: Reset scrub data when inodes move

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2018-02-13 14:43:32 -08:00
Douglas Fuller
07339e2d1d qa/cephfs: Add tests to validate scrub functionality
Add tests to ensure the scrub operation is not adversly affected
by certain metadata pathologies.

Signed-off-by: Douglas Fuller <dfuller@redhat.com>
2018-02-13 14:07:28 -05:00
Yan, Zheng
d532e66087 qa/cephfs: add tests for snapclient cache
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2018-02-09 18:42:29 +08:00
Yan, Zheng
79a56a7023 qa/cephfs: add tests for snaptable transaction
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2018-02-09 18:42:29 +08:00
Yan, Zheng
d46dbbebac qa/cephfs: add tests for multimds snapshot
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2018-02-09 18:42:29 +08:00
Yan, Zheng
27b1ca076e qa: adjust cephfs full test for kclient
Fixes: http://tracker.ceph.com/issues/22886
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
2018-02-05 18:00:57 +08:00
Patrick Donnelly
f6e1a797d4
Revert "Merge PR #19369 into master"
This reverts commit 3189ba19a7, reversing
changes made to b7620de020.

Despite the change in json format being positive, the unfortunate side-effect
is that it broke upgrade testing (because the QA framework must handle the
transition of mdsmap["info"] to a list from object) and the ceph-mgr.

Fixes: http://tracker.ceph.com/issues/22527
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2018-01-04 09:42:37 -08:00
Sage Weil
819a3578fa Merge tag 'v13.0.1' 2018-01-03 10:04:20 -06:00
Patrick Donnelly
3189ba19a7
Merge PR #19369 into master
* refs/pull/19369/head:
	qa: update handling of fs status format
	PendingReleaseNotes: add note for format change
	mds/MDSMap : use arrary_section for mds stat

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Xiaoxi Chen <xiaoxchen@ebay.com>
2017-12-21 20:21:18 -08:00
Patrick Donnelly
1f1a2a27ef
qa: update handling of fs status format
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2017-12-21 10:35:09 -08:00
Sage Weil
f33ab7e03a Merge remote-tracking branch 'gh/mimic-dev1' 2017-12-20 15:08:30 -06:00
Patrick Donnelly
6e046dfc90
qa: check pool full flags
Cluster-wide flag removed in b4ca5ae462.

Fixes: http://tracker.ceph.com/issues/22475

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2017-12-18 21:59:57 -08:00
Patrick Donnelly
b2284f23b8
qa: don't configure ec data pool with memstore
Fixes: http://tracker.ceph.com/issues/22436

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2017-12-18 21:12:22 -08:00
Patrick Donnelly
67ca6cd229
mds: obsolete MDSMap option configs
These configs were used for initialization but it is more appropriate to
require setting these file system attributes via `ceph fs set`. This is similar
to what was already done with max_mds. There are new variables added for `fs
set` where missing.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2017-12-13 18:30:52 -08:00
Patrick Donnelly
df43e415c6
Merge PR #18274 into master
* refs/pull/18274/head:
	mds: fold mds_revoke_cap_timeout into mds_session_timeout
	client: add new delegation testcases
	client: add delegation support for cephfs
	common: remove data_dir_option from common_preinit and global_pre_init

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2017-11-20 21:48:19 -08:00
Jeff Layton
3321cc7b37 mds: fold mds_revoke_cap_timeout into mds_session_timeout
Right now, we have two different timeout settings -- one for when the
client is just not responding at all (mds_session_timeout), and one for
when the client is otherwise responding but isn't returning caps in a
timely fashion (mds_cap_revoke_timeout).

The default settings on them are equivalent (60s), but only the
mds_session_timeout is communicated via the mdsmap. The
mds_cap_revoke_timeout is known only to the MDS. Neither timeout results
in anything other than warnings in the current codebase.

There is also a third setting (mds_session_autoclose) that is also
communicated via the MDSmap. Exceeding that value (default of 300s)
could eventually result in the client being blacklisted from the
cluster. The code to implement that doesn't exist yet, however.

The current codebase doesn't do any real sanity checking of these
timeouts, so the potential for admins to get them wrong is rather high.
It's hard to concoct a use-case where we'd want to warn about these
events at different intervals.

Simplify this by just removing the mds_cap_revoke_timeout setting, and
replace its use in the code with the mds_session_timeout. With that, the
client can at least determine when warnings might start showing up in
the MDS' logs.

Signed-off-by: Jeff Layton <jlayton@redhat.com>
2017-11-14 07:27:01 -05:00
Patrick Donnelly
2bba5d8e0f
Merge PR #18192 into master
* refs/pull/18192/head:
	qa/cephfs: test ec data pool
	qa/suites/fs/basic_functional/clusters: more osds

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2017-10-25 17:05:38 -07:00
Patrick Donnelly
c58161f25b
Merge PR #17266 into master
* refs/pull/17266/head:
	qa: update test_ceph_argparse to test fs cmds
	qa: use fs rm_data_pool
	qa: fix mdsmap lookup
	qa: remove usage of mds dump
	PendingReleaseNotes: add obsoleted mds commands
	qa: remove use of obsolete mds commands
	ceph_volume_client: remove use of obsolete mds cmd
	doc: update on obsolete mds commands
	cephfs: obsolete deprecated mds commands

Reviewed-by: Douglas Fuller <dfuller@redhat.com>
2017-10-24 16:37:14 -07:00
Patrick Donnelly
3a5f090a1e
qa: remove usage of mds dump
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2017-10-24 11:32:43 -07:00
Sage Weil
d0732fc96f qa/cephfs: test ec data pool
Signed-off-by: Sage Weil <sage@redhat.com>
2017-10-23 21:11:24 -05:00