Commit Graph

618 Commits

Author SHA1 Message Date
Jason Dillaman
608fb1f0e0 suites/rbd: avoid redundant helgrind tests
Allow rbd_fsx to accept an override for which valgrind tool to use.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2016-01-12 21:08:00 -05:00
Samuel Just
7831ca1e44 Revert "pg: add test cases for pg list_missing on EC pool"
This reverts commit 6fb44db9cb.
2016-01-07 11:02:40 -08:00
Yan, Zheng
b2a304e651 tasks/cephfs: fix race in test_damage
When stray directory inodes are corrupted, MDS may go to damaged state
after becoming active. (MDCache::open_root/populate_mydir is called by
MDSRank::starting_done).

Fixes: #14196
Signed-off-by: Yan, Zheng <zyan@redhat.com>
2016-01-04 20:42:35 +08:00
Sage Weil
179ff64277 Merge pull request #755 from guangyy/wip-13441-test
pg: add test cases for pg list_missing on EC pool
2015-12-23 17:06:39 -05:00
Loic Dachary
834a38cf90 buildpackages: support CentOS 7.1 and CentOS 7.2
Signed-off-by: Loic Dachary <loic@dachary.org>
2015-12-17 10:31:37 +01:00
Loic Dachary
64d8a586f7 buildpackages: no need to sudo in bootcmd
Signed-off-by: Loic Dachary <loic@dachary.org>
2015-12-12 14:27:55 +01:00
Loic Dachary
0c1e7f1f6b buildpackages: log the output of make
Signed-off-by: Loic Dachary <loic@dachary.org>
2015-12-12 14:03:57 +01:00
Loic Dachary
f19a519b8e Merge pull request #756 from dachary/wip-13624-workunit-git
workunits: fetch from --ceph-git-url
2015-12-11 12:27:18 +01:00
Robin H. Johnson
ebf8508c08 buildpackages: pkgrepo host also needs user-data setup.
The packages repo host fails in environments where networking setup is
needed in VMs. Use the same user-data as the buildhosts to ensure this
is the case.

Signed-off-by: Robin H. Johnson <robin.johnson@dreamhost.com>
2015-12-10 22:58:09 -08:00
Robin H. Johnson
89bde0b8fe buildpackages: copy teuthology user-data.txt changes
7b27e1db7: openstack: support /etc/network/intefaces injection
2358562cf: ensure VMs always have /etc/hosts set up
4378a505d: always allow unsigned deb packages
50b2db521: openstack: encode instance name with the full IP
6e828a33b: openstack: add 8.8.8.8 as a last resort resolver

Signed-off-by: Robin H. Johnson <robin.johnson@dreamhost.com>
2015-12-10 22:58:04 -08:00
Robin H. Johnson
3c6954642a
buildpackages: catch instance creation failure.
Split the sleep from the server creation, so we catch 'server create'
failures (eg due to quota):
> Quota exceeded for cores: Requested 16, but already used 10 of 20 cores
> (HTTP 403) (Request-ID: req-6467934e-db50-4479-995c-4d44dedf553a)

Signed-off-by: Robin H. Johnson <robin.johnson@dreamhost.com>
2015-12-10 11:55:53 -08:00
Robin H. Johnson
0240f57643
buildpackages: Be robust about addresses
OpenStack could tell us the VM has multiple networks, and offers no
guarantee about the order of addresses either (the old code failed if
the v4 IP was first).

For now, take the first listed network, and the first listed IPv4
address therein. Comments contain more detailed examples of possible
output from openstack tool.

Also remove the need for using jq to parse the output.

Signed-off-by: Robin H. Johnson <robin.johnson@dreamhost.com>
2015-12-10 11:55:49 -08:00
Gregory Farnum
da8a891b74 Merge pull request #744 from ceph/wip-inode-claim
tasks/cephfs: add TestJournalRepair.test_table_tool_take_inos

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2015-12-06 21:10:10 -08:00
Gregory Farnum
5e714c69d5 Merge pull request #703 from ceph/wip-layout-restrict
tasks/cephfs: test for auth caps pool-setting restriction

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2015-12-06 21:05:55 -08:00
Loic Dachary
b0a770a3e7 workunit: fetch from --ceph-git-url
The commit from which workunits are fetched must be retrieved
from --ceph-git-url via teuth_config.get_ceph_git_url() instead of
assuming it is available via git://git.ceph.com/ceph.git.

Using git://git.ceph.com/ceph.git is convenient because it supports git
archive. In the general case, some git servers such as github do not
support git archive and a full git clone must be done instead.

Although it would be possible to

    git clone --branch=master --depth=1 --single-branch

to reduce the amount of data being retrieved, it would require a

    git fetch origin SHA1

but git version >= 1.7 do not support fetching a commit.

http://tracker.ceph.com/issues/13624 Fixes: #13624

Signed-off-by: Loic Dachary <loic@dachary.org>
2015-12-06 13:23:14 +01:00
Loic Dachary
28dc5c0c33 workunit: keep the path to the list of tests in a variable
Signed-off-by: Loic Dachary <loic@dachary.org>
2015-12-06 13:07:13 +01:00
Loic Dachary
7de33f082e workunit: allow tag to override sha1
The sha1 for the workunit task is always set by the suite.py task. The
tag must be checked before the sha1 othewise it cannot be used to
override the sha1.

Signed-off-by: Loic Dachary <loic@dachary.org>
2015-12-06 12:34:59 +01:00
Guang Yang
6fb44db9cb pg: add test cases for pg list_missing on EC pool
Reviewed-by: Guang Yang <yguang@yahoo-inc.com>
2015-12-04 23:03:03 +00:00
John Spray
1280caf7c2 tasks/cephfs: avoid using bare remote.run in scrub_checks
Use the Mount.* wrappers for filesystem operations,
so that changes like making run_shell use sudo just work.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-12-02 23:30:55 +00:00
John Spray
0cd6cb10ac tasks/cephfs: update for run_shell vs. sudo
(TestStrays.test_snapshot_remove blocked on the sudo change)

Signed-off-by: John Spray <john.spray@redhat.com>
2015-12-02 23:30:55 +00:00
John Spray
80397a5db2 tasks/cephfs: always sudo in run_shell
This was causing permissions issues when
running inside teuthology, as run_python
was using sudo and run_shell wasn't.

Would be nice to get rid of all the rootishness,
but for the moment just make it more uniform.
2015-12-02 23:30:55 +00:00
John Spray
bb924d2095 tasks/cephfs: cover snaps in test_strays
Signed-off-by: John Spray <john.spray@redhat.com>
2015-12-02 23:30:55 +00:00
John Spray
14e2cf1ef2 tasks/cephfs: add TestForwardScrub
This tests the forward scrub's ability to traverse
some metadata and tag it, and the corresponding
functionality in cephfs-data-scan to filter based
on tag and inject orphaned items.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-12-02 23:28:28 +00:00
Loic Dachary
ea6360ac26 teuthology-integration: run teuthology integration tests
Signed-off-by: Loic Dachary <loic@dachary.org>
2015-12-02 13:14:35 +01:00
Loic Dachary
ab9a3d5a88 buildpackages: create the image if it does not exist
Since buildpackages runs before target provisioning, it is possible that
the desired image does not yet exist on a newly provisionned tenant (or
region).

http://tracker.ceph.com/issues/13910 Fixes: #13910

Signed-off-by: Loic Dachary <loic@dachary.org>
2015-11-30 16:55:38 +01:00
Loic Dachary
ca64229481 Merge pull request #679 from ceph/wip-ceph-whitespace
tasks/ceph: fix up whitespace

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-11-27 18:58:10 +01:00
John Spray
b3da9d7e7c tasks/cephfs: add TestJournalRepair.test_table_tool_take_inos
Signed-off-by: John Spray <john.spray@redhat.com>
2015-11-27 17:43:28 +00:00
Loic Dachary
9ea3b26b91 Merge pull request #740 from dachary/wip-13899-rbd-fio
task/rbd_fio: allow for unsigned packages

Reviewed-by: Abhishek Varshney <abhishek.varshney@flipkart.com>
2015-11-27 17:21:13 +01:00
Loic Dachary
d14f2da437 task/rbd_fio: allow for unsigned packages
Similar to what the teuthology install.py task does, add --force-yes to
the apt-get install so that unsigned packages are successfully
installed. It is needed when the buildpackages task is used to create
packages on the fly.

There is no need to do the same for rpm packages because the
verification is controlled from the ceph-release package instead of from
the command line.

http://tracker.ceph.com/issues/13899 Fixes: #13899

Signed-off-by: Loic Dachary <loic@dachary.org>
2015-11-27 14:20:07 +01:00
Loic Dachary
8088d60524 buildpackages: wait for the build machine deletion
When the quotas are low, it matters to block until the build machine is
actually deleted. Otherwise target provisionning may fail because the
they exceed the quota. For instance the default on OVH is to have 32
cores and the build machine uses 16. The packages-repository machine
uses two, the teuthology cluster uses one and that leaves only 13 cores
for the targets which may be too low when running jobs that require
large instances.

Signed-off-by: Loic Dachary <loic@dachary.org>
2015-11-26 13:13:36 +01:00
Gregory Farnum
bb5aaac923 Merge pull request #729 from ceph/wip-scrub-tidy
tasks/cephfs: tidy test_scrub_checks

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2015-11-25 17:13:52 -05:00
Gregory Farnum
3ba38a4c29 Merge pull request #674 from ceph/wip-13698
tasks/cephfs: fix race in test_sessionmap

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2015-11-25 17:12:36 -05:00
Gregory Farnum
e774886a3c Merge pull request #663 from ceph/wip-client-mark-down
tasks/cephfs: quick test for `tell`...

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2015-11-25 17:08:58 -05:00
Gregory Farnum
079ba9ef52 Merge pull request #647 from ceph/wip-repair-dir
tasks/cephfs: include an empty dir in journal repair

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2015-11-25 17:04:59 -05:00
Gregory Farnum
d5934fc96d Merge pull request #633 from ceph/wip-test-failover
tasks/cephfs/test_failover

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2015-11-25 17:00:29 -05:00
Loic Dachary
5c8f349919 Merge pull request #718 from ceph/wip-fast-read
rados: add test for ec fast_read

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-11-24 00:38:22 +01:00
Loic Dachary
af95065c9b Merge pull request #730 from ceph/wip-repair
ceph_manager: do_pg_scrub: keep scrubbing until it's done

Reviewed-by: Loic Dachary <ldachary@redhat.com>
2015-11-24 00:37:31 +01:00
John Spray
69560a8d65 tasks/cephfs: tidy test_scrub_checks
Use named error codes instead of numbers, and
use the helper fn for getting inode number
instead of doing it by hand.

Signed-off-by: John Spray <john.spray@redhat.com>
2015-11-23 17:47:02 +00:00
Loic Dachary
f398b71e33 buildpackages: do not -jX on dumpling
http://tracker.ceph.com/issues/13845 Refs: #13845

Signed-off-by: Loic Dachary <loic@dachary.org>
2015-11-20 13:46:28 +01:00
Samuel Just
89dcc0daf3 ceph_manager: do_pg_scrub: keep scrubbing until it's done
The ceph pg scrub ... command isn't really guarranteed to
start a scrub, keep reissuing it until the scrub actually
happens.

Related: #12746
Signed-off-by: Samuel Just <sjust@redhat.com>
2015-11-19 15:07:38 -08:00
Samuel Just
5f90c35f70 rados: add test for ec fast_read
Signed-off-by: Samuel Just <sjust@redhat.com>
2015-11-19 14:44:45 -08:00
Loic Dachary
87e50d394a buildpackages: avoid duplicate packages-repository
Signed-off-by: Loic Dachary <loic@dachary.org>
2015-11-19 09:08:35 +01:00
Loic Dachary
13fb23ed69 buildpackages: implement get_pkg_type
It can be deprectated when it is implemented in teuthology.

Signed-off-by: Loic Dachary <loic@dachary.org>
2015-11-19 00:36:18 +01:00
Loic Dachary
69e9b22e4b buildpackages: refactor to not require remote
Most of the flavor, sha1, tag etc. selection logic as implemented in the
packaging module of teuthology relies on remote hosts. This is complex
to tests and inconvenient because hosts must be provisionned even before
trying to figure out which packages need to be installed.

Using remote hosts is necessary when bare metal targets are used because
teuthology must adapt to the operating system already installed. The
selection logic in the context of dynamically provisionned targets is
simpler because it is defined by the job being run.

The buildpackages is refactored to use only the job configuration to
figure out which packages must be built. It makes it specific to targets
that are dynamically provisionned. It would have to be modified to query
the remote host in the case of bare metal targets.

Signed-off-by: Loic Dachary <loic@dachary.org>
2015-11-18 22:49:59 +01:00
Loic Dachary
1877b818f6 Merge pull request #704 from dachary/wip-buildpackages-packaging
buildpackages: protect packages-repository
2015-11-14 10:27:03 +01:00
Loic Dachary
4825f64a7c buildpackages: protect packages-repository
Signed-off-by: Loic Dachary <loic@dachary.org>
2015-11-14 09:31:41 +01:00
John Spray
95538f2c06 tasks/cephfs: test for auth caps pool-setting restriction
Signed-off-by: John Spray <john.spray@redhat.com>
2015-11-13 14:16:38 +00:00
Loic Dachary
8d798c4eec buildpackages: do not fails for unpackaged files
Older versions of Ceph have that problem failing would break all upgrade suites.

Signed-off-by: Loic Dachary <loic@dachary.org>
2015-11-12 20:17:53 +01:00
Loic Dachary
9287493851 buildpackages: giant has some unpackaged files
And there really is nothing we can do about it expect ignore them.

Signed-off-by: Loic Dachary <loic@dachary.org>
2015-11-12 16:11:39 +01:00
Loic Dachary
118e919d35 Merge pull request #695 from dachary/wip-buildpackages-flavor
buildpackages: 10GB is too small most of the time
2015-11-12 15:11:10 +01:00