Commit Graph

4596 Commits

Author SHA1 Message Date
John Spray
06891416da tasks/cephfs: stop if needed binaries are absent
Signed-off-by: John Spray <john.spray@redhat.com>
2015-10-02 17:41:17 +01:00
John Spray
23ae9815a5 tasks/cephfs: add instructions to vstart_runner
Signed-off-by: John Spray <john.spray@redhat.com>
2015-10-02 17:41:17 +01:00
John Spray
cbb85720f3 tasks/cephfs: updates for cmake environ
Signed-off-by: John Spray <john.spray@redhat.com>
2015-10-02 17:41:17 +01:00
John Spray
8b846b9f2c tasks/cephfs: add vstart runner script
This is to allow running CephFSTestCase tests
against a vstart cluster, for much faster turnaround
during development than running teuthology against
built ceph packages.

Not everything will be runnable this way, but for
certain things like filesystem repair scenarios we
have everything we need within a vstart environment.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-10-02 17:41:17 +01:00
John Spray
efb62ae1e7 tasks/cephfs: mark some tests as @needs_trimming
So that we can drop these tests when not running
client as root.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-10-02 17:41:17 +01:00
John Spray
f8a4b08702 tasks/cephfs: add needs_trimming decorator
For tests to advertise that they need the client
to be able to trim its cache (i.e. currently that
means requiring run as root)

Signed-off-by: John Spray <john.spray@redhat.com>
2015-10-02 17:41:17 +01:00
John Spray
c0e90aa5a7 tasks/cephfs: make FuseMount.teardown safer
(don't assume fuse_daemon exists)

Signed-off-by: John Spray <john.spray@redhat.com>
2015-10-02 17:41:16 +01:00
John Spray
1ed6272d93 tasks/cephfs: mark some tests as @long_running
Signed-off-by: John Spray <john.spray@redhat.com>
2015-10-02 17:41:16 +01:00
John Spray
d4ecee82e4 tasks/cephfs: add @long_running decorator
A means for test cases to mark particular methods
as long running, so that the vstart runner can skip
them when running for developers.

This is not a scientific thing, anything that takes
more than about 2 minutes due to lots of iteration
or sleeps.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-10-02 17:41:16 +01:00
John Spray
9a1ca94edc tasks/cephfs: cluster_down before fs rm
In teuthology this isn't needed because we join the
mds child processes after killing them.  In vstart
we're killing them asynchronously, so be a bit more
careful to ensure they can't re-insert themselves
to the mdsmap between our calling fail and our calling
fs rm.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-10-02 17:41:16 +01:00
John Spray
684e0deb6f tasks/cephfs: split up TestClientRecovery
...into the part that requires a network-isolated
client and the part that doesn't.

This happens to also be the part that won't work with
vstart vs. the part that will.  teuthology yaml will
still pick up and run both parts.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-10-02 17:41:16 +01:00
John Spray
6f8a35bb42 tasks/cephfs: make memstore dependency declarative
...instead of checking for it procedurally during
TestClusterFull.setUp

Signed-off-by: John Spray <john.spray@redhat.com>
2015-10-02 17:41:16 +01:00
John Spray
98da1e0d84 tasks/cephfs: refine TestClientLimits.test_client_oldest_tid
* Instead of creating files in background, create
  them in foreground (simpler).
* Instead of creating max_request*2 files, just create
  max_requests plus a litle bit.
* Set max_requests to 1000 instead of 5000 to run a bit
  faster.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-10-02 17:41:16 +01:00
John Spray
37d55663f0 tasks/cephfs: fix race in TestStrays
We weren't waiting for export dir to complete (the asok
just starts the process).  This wasn't noticeable when running
remotely due to latency between the test runner and the MDS,
but it shows up when running against a local vstart cluster.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-10-02 17:41:16 +01:00
John Spray
be749f8fde tasks/cephfs: work around fuse weirdness
I am seeing a strange thing where it seems like sometimes
a ls of /sys/fs/fuse/connections is returning empty when
connections do exist.  It is pretty easy to make this
a non-issue by waiting for "more conns than we started with"
instead of "list of conns is different", so do that.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-10-02 17:41:16 +01:00
John Spray
e51cd6bee3 tasks/cephfs: prefix override in FuseMount
For asok and invokation of ceph CLI.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-10-02 17:41:16 +01:00
John Spray
7e6f8d4905 tasks/cephfs: use stdin/stdout in Filesystem._read_data_xattr
...so that it doesn't rely on a temp local file and using
run.Raw(">") in commands.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-10-02 17:41:16 +01:00
John Spray
48cf533f0a tasks/cephfs: handle endpoint not connected
Previously failure to stat mnt dir was interpreted
as being unmounted.  For "transport endpoint no connected"
error we do want to recognise that it is mounted, albeit
with no ceph-fuse process.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-10-02 17:41:16 +01:00
John Spray
120f9d882e tasks/cephfs: use Filesystem.rados instead of direct CLI
Because it's slicker and because I can wrap it for vstart.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-10-02 17:41:16 +01:00
John Spray
bab4c7aa71 tasks/cephfs: add Filesystem.exists
Use this during test setup to check whether
a filesystem is configured at all, before
trying to tear it down.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-10-02 17:41:16 +01:00
John Spray
e2f8b48d94 tasks/cephfs: add Filesystem._prefix
So that my vstart subclass can put ./ before
all the commands.

One could set $PATH, but I like to unambiguously point
it at the local built binaries in case someone also
has some systemwide packages.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-10-02 17:41:15 +01:00
John Spray
f58c119d90 tasks/cephfs: use raw_cluster_cmd instead of manual "ceph"
It's less repetitive, and I can override it for use with
vstart.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-10-02 17:41:15 +01:00
John Spray
99a8a123da tasks/cephfs: add Filesystem.delete_all
So that we can clear down filesysteems created by
someone other than ourselves (like vstart.sh)

Signed-off-by: John Spray <john.spray@redhat.com>
2015-10-02 17:41:15 +01:00
John Spray
ad562f4b79 tasks/cephfs: make Filesystem rados command overridable
Signed-off-by: John Spray <john.spray@redhat.com>
2015-10-02 17:41:15 +01:00
John Spray
7884d5b555 tasks/cephfs: use raw_cluster_cmd in Filesystem
...instead of manually invoking 'ceph' -- so that
it's easier to override this for vstart.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-10-02 17:41:15 +01:00
Loic Dachary
aed5a6dbc6 Merge pull request #595 from ceph/wip-quota
log-whitelist 'reached quota' for librados test.sh

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-10-02 14:59:27 +02:00
Sage Weil
7bb1ceb3f0 log-whitelist 'reached quota' for librados test.sh
A new test verifies that we are stopped by the pool quota (and get
the right error messages or block).  See ceph.git
32962740ce.

Signed-off-by: Sage Weil <sage@redhat.com>
2015-10-02 08:33:46 -04:00
Gregory Farnum
6add6ba2df Merge pull request #461 from ceph/wip-client-oldest-tid
cephfs/test_client_limits.py: update test_client_oldest_tid

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2015-10-01 22:37:15 -07:00
Sage Weil
de3c9d59df rados/singleton-nomsgr/all/full-tiering: time out rados -p
We will hit a quota and block (not error out).

See #13283
Signed-off-by: Sage Weil <sage@redhat.com>
2015-10-01 15:37:22 -04:00
Loic Dachary
487ccb964a Merge pull request #592 from ceph/wip-13000-centos-activate
ceph-deploy: always zap disk before creating an osd
2015-10-01 08:49:20 +02:00
Josh Durgin
03b4a7957e Merge pull request #580 from dachary/wip-13250-ceph-deploy-branch
ceph-deploy: fix ceph-deploy-branch config options

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2015-09-30 19:45:30 -04:00
Sage Weil
f467a98a29 tasks/ceph_manager: %d -> %s
Signed-off-by: Sage Weil <sage@redhat.com>
2015-09-30 14:58:32 -04:00
Loic Dachary
01d48a270a ceph-deploy: always zap disk before creating an osd
The existing logic is to ceph-deploy osd create --zap-disk which will
zap the data device before preparing it. However it will not zap the
journal device (see http://tracker.ceph.com/issues/13291).

If ceph-deploy osd create fails, a fall back will zap both the data
device and the journal and try prepare again. This could work if
the device preparation and activation was synchronous and catch all
errors that could be caused by an unclean journal device. However,
the activation is asynchronous and it is entirely possible for a device
to be prepared successfully and fail to activate in the background.

The data and journal device are always zapped before calling ceph-deploy
osd create. The logic is simpler and the overhead is low.

http://tracker.ceph.com/issues/13000 Fixes: #13000

Signed-off-by: Loic Dachary <loic@dachary.org>
2015-09-30 20:57:24 +02:00
Sage Weil
5a450f8eeb rados/singleton-nomsgr/all/11429: blackhole osd.0 to close race
Blackhole filestore ops so that we ensure it doesn't complete
the pg deletions before the restart function does a clean shutdown
etc.

Signed-off-by: Sage Weil <sage@redhat.com>
2015-09-30 14:42:05 -04:00
Sage Weil
963f5e55a9 upgrade/hammer-x/tiering: fix upgrade sequence
Signed-off-by: Sage Weil <sage@redhat.com>
2015-09-30 09:34:26 -04:00
Sage Weil
a53a80b9f0 tasks/ceph_manager: fix logging on failed pool property
Signed-off-by: Sage Weil <sage@redhat.com>
2015-09-30 09:24:38 -04:00
Sage Weil
415a18628c Merge pull request #591 from athanatos/wip-13234
rados: add test for 13234.yaml

Reviewed-by: Sage Weil <sage@redhat.com>
2015-09-30 08:35:10 -04:00
Samuel Just
4e9f1df514 rados: add test for 13234.yaml
Signed-off-by: Samuel Just <sjust@redhat.com>
2015-09-29 21:19:10 -07:00
John Spray
a5a361e883 Merge pull request #586 from ceph/wip-mdscap
tasks/cephfs: mds allow

Reviewed-by: John Spray <john.spray@redhat.com>
2015-09-29 10:08:20 +01:00
Gregory Farnum
79cbfc86e4 Merge pull request #575 from ceph/wip-32bits-ceph-fuse
suites/fs: test ceph-fuse with 32-bits ino_t

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2015-09-28 22:19:28 -07:00
Sage Weil
8cf3d15de7 rados/singleton-nomsgr/all/11429: grep osd.0 only
Restart can be slow enough that osd.1 and 2 finish deleting the
pgs.  Verifying one osd sees the instance is sufficient.

Signed-off-by: Sage Weil <sage@redhat.com>
2015-09-28 13:10:45 -04:00
Sage Weil
a4f9bdbe54 tasks/cephfs: mds allow
Signed-off-by: Sage Weil <sage@redhat.com>
2015-09-28 12:38:04 -04:00
Sage Weil
9eea6ea6f9 Merge remote-tracking branch 'gh/wip-infernalis' 2015-09-28 10:32:04 -04:00
Loic Dachary
56e95a0ebc Merge pull request #583 from dachary/wip-buildpackages-flock
buildpackages: flock must --close
2015-09-28 02:03:54 +02:00
Loic Dachary
0b130d7f53 Merge pull request #584 from dachary/wip-buildpackages-centos-6
buildpackages: user-data must be per os-type/os-version
2015-09-28 02:01:30 +02:00
Loic Dachary
05228e1e5c buildpackages: user-data must be per os-type/os-version
CentOS 6.5 needs to install a package and reboot to grow the root file
system. Instead of assuming a common user-data.txt file can fit all
Operating Systems, make one user data per os-type/os-version combination.

Signed-off-by: Loic Dachary <loic@dachary.org>
2015-09-27 23:42:58 +02:00
Kefu Chai
d58a0b00e8 add cache tiering test for hammer-x
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2015-09-27 08:24:38 -04:00
Sage Weil
8941a8b871 rados/singleton-nomsgr/all/11439: fix greps
Signed-off-by: Sage Weil <sage@redhat.com>
2015-09-26 21:11:31 -04:00
Loic Dachary
204e27fecd buildpackages: flock must --close
The process run by flock must not inherit the file descriptor because
this will cause the lock to be held forever should the command survive
the call to flock. This is for instance the case for the ssh-agent.

Signed-off-by: Loic Dachary <loic@dachary.org>
2015-09-26 15:23:03 +02:00
Loic Dachary
8bc89b6517 Merge pull request #581 from dachary/wip-11883-git-url
buildpackages: get Ceph git url via get_ceph_git_url()
2015-09-26 13:44:56 +02:00