Previously failure to stat mnt dir was interpreted
as being unmounted. For "transport endpoint no connected"
error we do want to recognise that it is mounted, albeit
with no ceph-fuse process.
Signed-off-by: John Spray <john.spray@redhat.com>
Use this during test setup to check whether
a filesystem is configured at all, before
trying to tear it down.
Signed-off-by: John Spray <john.spray@redhat.com>
So that my vstart subclass can put ./ before
all the commands.
One could set $PATH, but I like to unambiguously point
it at the local built binaries in case someone also
has some systemwide packages.
Signed-off-by: John Spray <john.spray@redhat.com>
Our ffsb and fsync tests contain so many small writes at random offsets
that it can take >10 minutes to commit all of them to disk if we get
a slower OSD cluster. 15 minutes is still a plenty-fast timeout for
this stage compared to just hanging and losing the logs!
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
Previously were defaulting to a string, which
always compared greater than elapsed, so never
timed out.
Fixes: #12820
Signed-off-by: John Spray <john.spray@redhat.com>
1) add a wait time before the mount attempt to let the cluster get set up.
By default this should be skipped, but for VMs and known-slow systems we
can give them 60 seconds.
2) Make the timeout configurable, with a 30-second default, but override it
for VM tests.
http://tracker.ceph.com/issues/12320Fixes: #12320
Signed-off-by: Loic Dachary <loic@dachary.org>
When client capabilities get released, MDS may update corresponding
inodes' client writable range and mark those inodes dirty. The auto
repair test expects MDS to trim inodes from its cache, but MDS can't
trim dirty inodes. So we should flush journal after umount.
Fixes: #12172
Signed-off-by: Yan, Zheng <zyan@redhat.com>
This is for verifying the new layout-writing behaviour. While
we're at it, test that the pre-existing backtrace behaviours
are really happening (updating old_pools)
Signed-off-by: John Spray <john.spray@redhat.com>
Sometimes mount A would get a cap revoke when mount
B did its last IO, resulting in mount A's OSD epoch
getting updated too.
Fix by making sure mount B is the last one to have
done IO before we do the barrier, so that when
it does IO again after the barrier, mount A can't
be holding any caps that B would need.
Fixes: #11913
Signed-off-by: John Spray <john.spray@redhat.com>
For cases where we have e.g. poked the fuse abort
file for a process, but it's still not dying. Because
this is a special class of error (unlike e.g. when
we force umount something because the network is gone)
raise the error instead of trying again to kill
the client.
Fixes: #11835
Signed-off-by: John Spray <john.spray@redhat.com>
To test that metadata written recently is
preserved across a client+server crash when
barriered with a directory fsync.
Signed-off-by: John Spray <john.spray@redhat.com>
The interval between writes was too short because
it was not taking account of the way OSDMap full
flags are set on tick rather than immediately.
Fixes: #11779
Signed-off-by: John Spray <john.spray@redhat.com>
flock only works properly on FUSE versions >=2.9, which is newer
than eg Ubuntu Precise. So check the version on our client mounts and
only test flock if it's at least that new.
Fixes: #9995
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
Run the same procedure as TestClusterFull, but
instead of limiting OSD memstore size, use pool
quota on the data pool.
Signed-off-by: John Spray <john.spray@redhat.com>
... s/mon_remote/admin_remote/ and allow caller to pass
in which remote they want to use for that. Enables use
with ceph_deploy task which does not give admin keys
to mons.
Signed-off-by: John Spray <john.spray@redhat.com>
This broke with recent Client changes that
do better caching of readdir results, such
that doing an ls twice is no longer sufficient
to see a fresh result after repair - we need
to remount instead.
Signed-off-by: John Spray <john.spray@redhat.com>
To track recent change in master where instead of
crashing on missing MDSTable object we'll go
into damaged state.
Instead of catching a crash, handle the rank's
transition to the damanged state. Leave the crash
handling code (unused for the moment) in the
Filesystem class in case it's needed elsewhere
soon.
Signed-off-by: John Spray <john.spray@redhat.com>
This tests the new purge file/ops throttling
in the MDS, via the new perf counters for
strays/purging.
Fixes: #10390
Signed-off-by: John Spray <john.spray@redhat.com>
...to avoid having boilerplate in each test module,
and gain the ability to run them all in one go
with a nice test-by-test pass/fail report.
Signed-off-by: John Spray <john.spray@redhat.com>
...as long as only one is active, all the ops
that default to talking to a single MDS should
be happy to talk to the active MDS, even if there
happens to be a standby lying around too.
Signed-off-by: John Spray <john.spray@redhat.com>
Where multiple MDSs were on the same node, trying
to concurrently update their firewall state was
causing an exception because the iptables command
errors out if another instance is already running.
Fixes: #10948
Signed-off-by: John Spray <john.spray@redhat.com>
teuthology helpfully escapes things for us so
the \; didn't need the backslash. The logic
was still falling over in some cases too.
Additionally, make the FUSE /sys/ abort operation
more surgical by working out the connection name
of our own mount during mount().
Signed-off-by: John Spray <john.spray@redhat.com>
This tests the new #9883 repair functionality
where we selectively scrape dentries out of
the journal while the MDS is offline.
Signed-off-by: John Spray <john.spray@redhat.com>
This was only used in get_first_mon, which doesn't actually
need the parameter itself. Makes it easier to casually
use Filesystem from any place with a ctx to hand.
Signed-off-by: John Spray <john.spray@redhat.com>
When unused clients were mounted during an fs new,
they would end up in a state where they stalled
on subsequent attempts to umount them (ceph-fuse
stalls on exit if it can't terminate its mds_session)
Signed-off-by: John Spray <john.spray@redhat.com>
Instead of blocking the whole port range (which
might make OSDs running on that node collateral
damage), read the MDS's port out of the MDS map
and just block that.
Signed-off-by: John Spray <john.spray@redhat.com>
Now that we have more of these cases, there was lots
of duplication in setup and teardown. For some tests
the "reset everything" setup/teardown is overkill,
but it's harmless.
Signed-off-by: John Spray <john.spray@redhat.com>
This tests:
* The new 'flush journal' asok command
* That the resulting on disk structures are as expected
* That cephfs-journal-tool is happy with the result
Fixes: #9881
Signed-off-by: John Spray <john.spray@redhat.com>
Previously was always using the default values of things
so querying mon instead of the appropriate service
worked fine. However, for things we might want to
update on a per-test basis we need to go ask the
correct service what the setting really is.
Needed for osd_mon_report_interval_max in the ENOSPC
testing.
Signed-off-by: John Spray <john.spray@redhat.com>
Leave the legacy handling out in cephfs_setup, move
the filesystem creation stuff into Filesystem. I
anticipate this being the right place for it if/when
we have tests that want to do 'fs rm' 'fs new' type
cycles within themselves.
Signed-off-by: John Spray <john.spray@redhat.com>
This was tripping over the recent commit 42c85e80
in Ceph master, which tightens the limits on
acceptable PG counts per OSD, and was making
teuthology runs fail due to never going clean.
Rather than put in a new hardcoded count, infer
it from config. Move some code around so that
the ceph task can get at a Filesystem object
to use in FS setup (this already has conf-getting
methods).
Signed-off-by: John Spray <john.spray@redhat.com>
New CephFS tests for MDS's auto repair functions. (So far the only
test case is verify/repair backtrace on fetch dirfrag)
Signed-off-by: Yan, Zheng <zyan@redhat.com>
Old version libfuse treats both flock and posix lock requests as posix
lock request. This is a workaround for the bug.
Fixes: #9995
Signed-off-by: Yan, Zheng <zyan@redhat.com>
'client_id' was ambiguous because in other places it
meant the '0' in client.0, whereas here it means
the runtime-generated global ID of the client.
Signed-off-by: John Spray <john.spray@redhat.com>
Some of this stuff could be even more general for embedding
unittest-style suites, but for the moment let's keep the cephfs
stuff in a walled garden.
Signed-off-by: John Spray <john.spray@redhat.com>
