Commit Graph

494 Commits

Author SHA1 Message Date
John Spray
f58c119d90 tasks/cephfs: use raw_cluster_cmd instead of manual "ceph"
It's less repetitive, and I can override it for use with
vstart.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-10-02 17:41:15 +01:00
John Spray
99a8a123da tasks/cephfs: add Filesystem.delete_all
So that we can clear down filesysteems created by
someone other than ourselves (like vstart.sh)

Signed-off-by: John Spray <john.spray@redhat.com>
2015-10-02 17:41:15 +01:00
John Spray
ad562f4b79 tasks/cephfs: make Filesystem rados command overridable
Signed-off-by: John Spray <john.spray@redhat.com>
2015-10-02 17:41:15 +01:00
John Spray
7884d5b555 tasks/cephfs: use raw_cluster_cmd in Filesystem
...instead of manually invoking 'ceph' -- so that
it's easier to override this for vstart.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-10-02 17:41:15 +01:00
Gregory Farnum
6add6ba2df Merge pull request #461 from ceph/wip-client-oldest-tid
cephfs/test_client_limits.py: update test_client_oldest_tid

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2015-10-01 22:37:15 -07:00
Loic Dachary
487ccb964a Merge pull request #592 from ceph/wip-13000-centos-activate
ceph-deploy: always zap disk before creating an osd
2015-10-01 08:49:20 +02:00
Josh Durgin
03b4a7957e Merge pull request #580 from dachary/wip-13250-ceph-deploy-branch
ceph-deploy: fix ceph-deploy-branch config options

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2015-09-30 19:45:30 -04:00
Sage Weil
f467a98a29 tasks/ceph_manager: %d -> %s
Signed-off-by: Sage Weil <sage@redhat.com>
2015-09-30 14:58:32 -04:00
Loic Dachary
01d48a270a ceph-deploy: always zap disk before creating an osd
The existing logic is to ceph-deploy osd create --zap-disk which will
zap the data device before preparing it. However it will not zap the
journal device (see http://tracker.ceph.com/issues/13291).

If ceph-deploy osd create fails, a fall back will zap both the data
device and the journal and try prepare again. This could work if
the device preparation and activation was synchronous and catch all
errors that could be caused by an unclean journal device. However,
the activation is asynchronous and it is entirely possible for a device
to be prepared successfully and fail to activate in the background.

The data and journal device are always zapped before calling ceph-deploy
osd create. The logic is simpler and the overhead is low.

http://tracker.ceph.com/issues/13000 Fixes: #13000

Signed-off-by: Loic Dachary <loic@dachary.org>
2015-09-30 20:57:24 +02:00
Sage Weil
a53a80b9f0 tasks/ceph_manager: fix logging on failed pool property
Signed-off-by: Sage Weil <sage@redhat.com>
2015-09-30 09:24:38 -04:00
Samuel Just
4e9f1df514 rados: add test for 13234.yaml
Signed-off-by: Samuel Just <sjust@redhat.com>
2015-09-29 21:19:10 -07:00
Sage Weil
a4f9bdbe54 tasks/cephfs: mds allow
Signed-off-by: Sage Weil <sage@redhat.com>
2015-09-28 12:38:04 -04:00
Loic Dachary
56e95a0ebc Merge pull request #583 from dachary/wip-buildpackages-flock
buildpackages: flock must --close
2015-09-28 02:03:54 +02:00
Loic Dachary
05228e1e5c buildpackages: user-data must be per os-type/os-version
CentOS 6.5 needs to install a package and reboot to grow the root file
system. Instead of assuming a common user-data.txt file can fit all
Operating Systems, make one user data per os-type/os-version combination.

Signed-off-by: Loic Dachary <loic@dachary.org>
2015-09-27 23:42:58 +02:00
Loic Dachary
204e27fecd buildpackages: flock must --close
The process run by flock must not inherit the file descriptor because
this will cause the lock to be held forever should the command survive
the call to flock. This is for instance the case for the ssh-agent.

Signed-off-by: Loic Dachary <loic@dachary.org>
2015-09-26 15:23:03 +02:00
Loic Dachary
8bc89b6517 Merge pull request #581 from dachary/wip-11883-git-url
buildpackages: get Ceph git url via get_ceph_git_url()
2015-09-26 13:44:56 +02:00
Loic Dachary
78b2a9aa74 buildpackages: get Ceph git url via get_ceph_git_url()
Instead of relying on git_base_url, use the get_ceph_git_url() to obtain
the URL of the Ceph repository to use with git clone. This allows the
user to override it via the git_ceph_url configuration option and the
--git-ceph-url command line option to teuthology-openstack.

http://tracker.ceph.com/issues/11883 Refs: #11883

Signed-off-by: Loic Dachary <loic@dachary.org>
2015-09-26 13:11:36 +02:00
Loic Dachary
fdcfb79b3c ceph-deploy: fix ceph-deploy-branch config options
The config paramter of download_ceph_deploy does not have a ceph-deploy
item, therefore the ceph-deploy-branch parameter is always assumed to be
master.

Signed-off-by: Loic Dachary <loic@dachary.org>
2015-09-26 02:16:46 +02:00
Sage Weil
ecc504fea0 tasks/divergent_priors2: wait for osd to start before tell
Otherwise we can get

2015-09-24T19:22:15.191 INFO:teuthology.orchestra.run.mira080.stderr:Error ENXIO: problem getting command descriptions from osd.1

Signed-off-by: Sage Weil <sage@redhat.com>
2015-09-24 23:34:17 -04:00
Sage Weil
0e2814d81e tasks/ceph_manager: ignore failure getting pg_num
Otherwise, we may fail while racing with a workload that deletes a pool:

2015-09-23T15:01:52.855 INFO:tasks.workunit.client.1.vpm128.stdout:[ RUN      ] LibRadosTwoPoolsPP.PromoteSnapTrimRace
2015-09-23T15:01:53.892 INFO:teuthology.orchestra.run.vpm176:Running: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph osd pool get .rgw pg_num'
2015-09-23T15:01:54.206 INFO:teuthology.orchestra.run.vpm176:Running: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph osd pool get .rgw.gc pg_num'
2015-09-23T15:01:54.462 INFO:teuthology.orchestra.run.vpm176:Running: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph osd pool get .users.uid pg_num'
2015-09-23T15:01:54.696 INFO:teuthology.orchestra.run.vpm176:Running: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph osd pool get .users.email pg_num'
2015-09-23T15:01:55.006 INFO:teuthology.orchestra.run.vpm176:Running: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph osd pool get .users pg_num'
2015-09-23T15:01:55.296 INFO:teuthology.orchestra.run.vpm176:Running: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph osd pool get .rgw.buckets.index pg_num'
2015-09-23T15:01:55.523 INFO:teuthology.orchestra.run.vpm176:Running: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph osd pool get .log pg_num'
2015-09-23T15:01:55.752 INFO:teuthology.orchestra.run.vpm176:Running: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph osd pool get .usage pg_num'
2015-09-23T15:01:56.188 INFO:teuthology.orchestra.run.vpm176:Running: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph osd pool get .rgw.buckets.extra pg_num'
2015-09-23T15:01:56.625 INFO:teuthology.orchestra.run.vpm176:Running: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph osd pool get test-rados-api-vpm128-17360-6 pg_num'
2015-09-23T15:01:56.928 INFO:teuthology.orchestra.run.vpm176:Running: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph osd pool get test-rados-api-vpm128-17360-13 pg_num'
2015-09-23T15:01:57.193 INFO:teuthology.orchestra.run.vpm176.stderr:Error ENOENT: unrecognized pool 'test-rados-api-vpm128-17360-13'
2015-09-23T15:01:57.206 ERROR:teuthology.parallel:Exception in parallel execution
Traceback (most recent call last):
...

Signed-off-by: Sage Weil <sage@redhat.com>
2015-09-24 12:19:07 -04:00
Loic Dachary
7e3a271a89 buildpackages: make rpm packages
Signed-off-by: Loic Dachary <loic@dachary.org>
2015-09-23 21:11:26 +02:00
Gregory Farnum
e3c99476b0 Merge pull request #453 from ceph/wip-11835
tasks/cephfs: time out on ceph-fuses that don't die

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2015-09-21 16:09:48 -07:00
John Spray
eab70197a6 tasks/ceph: wait for MDS to be active when creating a cluster
This is the correct implementation of 685d76a77c,
merged while broken in ff1655cb57 and
reverted in 4cccde634f.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-09-21 16:05:51 -07:00
Sage Weil
caa9235857 Merge pull request #566 from ceph/wip-s3-branch-fix
Use old branches as is without 'ceph-' for firefly, hammer etc
2015-09-18 10:10:10 -04:00
Greg Farnum
4cccde634f Revert "Merge pull request #567 from ceph/ceph_fuse-timeout"
This reverts commit ff1655cb57, reversing
changes made to 2b25080d4f.

Since we haven't actually started the MDS daemons yet, this code is broken.

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2015-09-17 07:29:28 -07:00
Gregory Farnum
2f43e23fc6 Merge pull request #569 from dachary/wip-13031-packages
buildpackages: fix flake / pep8
2015-09-16 14:29:19 -07:00
Greg Farnum
f97fde6c26 tasks/cephfs: switch unmount timeout to 15 minutes
Our ffsb and fsync tests contain so many small writes at random offsets
that it can take >10 minutes to commit all of them to disk if we get
a slower OSD cluster. 15 minutes is still a plenty-fast timeout for
this stage compared to just hanging and losing the logs!

Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2015-09-16 14:27:14 -07:00
Loic Dachary
e38a6d69df buildpackages: build the repository and the packages
http://tracker.ceph.com/issues/13031 Fixes: #13031

Signed-off-by: Loic Dachary <loic@dachary.org>
2015-09-16 23:19:34 +02:00
Greg Farnum
685d76a77c ceph: wait for CephFS to be healthy before proceeding
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
2015-09-16 13:53:36 -07:00
Vasu Kulkarni
08bae3b9d2 Use old branches as is without 'ceph-' for firefly, hammer etc
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
2015-09-16 13:21:51 -04:00
Loic Dachary
8c618e5645 buildpackages: re-use config from install task
So that the packages built match what the install task expects.

http://tracker.ceph.com/issues/13031 Fixes: #13031

Signed-off-by: Loic Dachary <loic@dachary.org>
2015-09-15 17:37:05 +02:00
Loic Dachary
d40d8ff501 buildpackages: pep8 compliance
http://tracker.ceph.com/issues/13031 Fixes: #13031

Signed-off-by: Loic Dachary <loic@dachary.org>
2015-09-15 16:16:34 +02:00
Loic Dachary
0eeb441bb0 buildpackages: build ceph packages and upload them
Import install from task.

http://tracker.ceph.com/issues/13031 Fixes: #13031

Signed-off-by: Loic Dachary <loic@dachary.org>
2015-09-15 16:13:09 +02:00
Loic Dachary
2b25080d4f buildpackages: build ceph packages and upload them
Build Ceph packages from source for the required revision, os_type,
os_version and architecture and upload them to the gitbuilder
repository.

http://tracker.ceph.com/issues/13031 Fixes: #13031

Signed-off-by: Loic Dachary <loic@dachary.org>
2015-09-15 12:01:05 +02:00
Sage Weil
5af28cf746 tasks/radosbench: allow cleanup: false
Signed-off-by: Sage Weil <sage@redhat.com>
2015-09-11 16:52:41 -04:00
Zack Cerza
e9847570de Merge pull request #561 from ceph/wip-sudo
sudo ceph
2015-09-11 10:20:50 -06:00
Sage Weil
dad981d339 tasks: sudo ceph for cli
/var/run/ceph is 770.  This is mainly necessary for any
interaction with the daemon sockets, but it is what users do
and it may avoid log noise.

Signed-off-by: Sage Weil <sage@redhat.com>
2015-09-11 12:15:01 -04:00
Sage Weil
9b3f36f91f ceph: add option to expect valgind errors and fail if there are none
See #10328
Signed-off-by: Sage Weil <sage@redhat.com>
2015-09-11 11:53:37 -04:00
Gregory Farnum
6ac2774823 Merge pull request #550 from ceph/wip-12806
tasks/qemu: avoid adding duplicated entries to /etc/exports

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2015-09-09 14:53:45 +01:00
Sage Weil
a328e3e60d tasks/ceph_manager: dump pgs when recover times out
It is really hard to map a stuck recovery back to the pgs that
are stuck.  This will make it easy.

Signed-off-by: Sage Weil <sage@redhat.com>
2015-09-08 08:59:49 -04:00
Sage Weil
394dbc1890 Merge remote-tracking branch 'gh/wip-ceph-deploy' 2015-09-06 13:36:07 -04:00
Kefu Chai
ff4e2ad3dc Merge pull request #552 from ceph/wip-12878
tasks/scrub_test: sudo ls ...

Reviewed-by: Kefu Chai <kchai@redhat.com>
2015-09-06 18:26:56 +08:00
Sage Weil
6064443ad1 tasks/divergent_prior[2]: fix rados get syntax
Signed-off-by: Sage Weil <sage@redhat.com>
2015-09-05 09:25:24 -04:00
Sage Weil
7b3ea31efe tasks/reg11184.py: fix rados get syntax
no -o

Signed-off-by: Sage Weil <sage@redhat.com>
2015-09-05 09:23:54 -04:00
Sage Weil
26eb35d611 tasks/s3tests: use ceph-$foo branches
We need to be able to merge things into s3-tests master that
break rgw.  Create ceph-foo branches (ceph-master,
ceph-infernalis, etc.) and use those instead.

Signed-off-by: Sage Weil <sage@redhat.com>
2015-09-04 14:42:21 -04:00
Sage Weil
d46c3a53ab tasks/scrub_test: sudo ls ...
/var/lib/ceph/osd/* is owned by ceph now; ubuntu user can't read
it.

Fixes: #12878
Signed-off-by: Sage Weil <sage@redhat.com>
2015-09-02 22:01:58 -04:00
Sage Weil
0553e50401 tasks/ceph_deploy: install ceph-test
Signed-off-by: Sage Weil <sage@redhat.com>
2015-09-02 18:45:33 -04:00
John Spray
8e275049a0 tasks/cephfs: fix mount_timeout config loading
Previously were defaulting to a string, which
always compared greater than elapsed, so never
timed out.

Fixes: #12820
Signed-off-by: John Spray <john.spray@redhat.com>
2015-09-01 11:31:23 +01:00
Sage Weil
f5865d32a8 tasks/ceph_deploy: work with systemd
This is ugly, but I'm not going to rewrite this right now.

Signed-off-by: Sage Weil <sage@redhat.com>
2015-08-31 14:21:29 -04:00
Loic Dachary
fbaedbf9fa Merge pull request #546 from dachary/wip-11881-multipath
ceph-disk workunit
2015-08-31 17:30:05 +02:00