This takes 5 hours currently.
- Separate out filestore and memstore into separate task (~1 hr)
- Split bluestore into -a and -b (a tests exclude SynethicMatrixC,
b tests include it)
Signed-off-by: Sage Weil <sage@newdream.net>
instead of using the default / existing distro, specify the distro for
testing. this change prevents us from using bionic for testing the
builds. this has two consequences:
* so we are one step closer to a non-bionic world.
* avoid building packages with PPA repo which *might* introduce runtime
dependencies on 3rd party runtimes provided by PPA repo.
see also: https://tracker.ceph.com/issues/50218
Signed-off-by: Kefu Chai <kchai@redhat.com>
This PR includes 2 things:
1. Changing force-branch to master and removing the git-remote. This change was forgetten for PR #39139.
2. Proper cleanup/removal after completion of commands more precisely removing the kafka logs directory.
Signed-off-by: Kalpesh Pandya <kapandya@redhat.com>
query the python version before trying to test diskprediction_local
Fixes: https://tracker.ceph.com/issues/50196
Signed-off-by: Kefu Chai <kchai@redhat.com>
If this is a fresh pacific cluster, let's assume that there won't be
legacy clients connecting. (And if there are, let's put the burden on
the user to enable them to do so insecurely.)
This is in contrast to upgrades, where our focus is on not breaking
anything.
Signed-off-by: Sage Weil <sage@newdream.net>
* refs/pull/40418/head:
test: unmount when finished ino_release_cb
test: wait a time for inode release
qa: move ino_release_cb to libcephfs sub-suite
qa: simplify recall triggers for bug
qa: fix name for qa task referencing tracker issue
Reviewed-by: Jeff Layton <jlayton@redhat.com>
this was added in https://github.com/ceph/ceph/pull/39820 to
avoid repeating the override in rgw subsuites, but teuthology
was picking up suites/rgw/ignore-pg-availability.yaml as a job
of its own, and led to 'list index out of range' failures
Fixes: https://tracker.ceph.com/issues/50147
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Starting with pacific, CEPHX_V2 is required by default but
pre-single-major.yaml kernel doesn't support it.
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
It is not testing a bug. Also, fix cap recall configs so this test is
properly triggered.
Fixes: https://tracker.ceph.com/issues/49500
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
The issue number was for an rbd backport. It should be [1].
[1] https://tracker.ceph.com/issues/24137
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
The 'orch daemon add ...' command is not idempotent and can cause
duplicate (and failing) attempts to add the same mon.
Maintain the ability to add mons the old way for the benefit of testing
upgrades from early octopus versions of cephadm.
Signed-off-by: Sage Weil <sage@newdream.net>
* refs/pull/40160/head:
qa/suites/rados/cephadm/orchestrator_cli: random-distro$ -> 0-random-distro$
qa/suites/rados/cephadm/smoke-roleless: distro -> 0-distro
qa/distros/podman: install kubic once per host, in parallel
qa/suites/fs/multiclient: use clients: not all: for pexec
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
centos currently has the semi-broken podman 2.2.1 (no json function
for inspect), but the latest octopus (15.2.10) works around it.
Signed-off-by: Sage Weil <sage@newdream.net>
* refs/pull/40034/head:
qa/suites/rados/cephadm/upgrade: start at 15.2.9 to test iscsi upgrade
qa/tasks/cephadm.py: don't set mgr count to +1
Reviewed-by: Sebastian Wagner <swagner@suse.com>
* refs/pull/39845/head:
doc: clarify mirror daemon user capability requirements
doc: doc changes for additional mirroring interfaces
pybind/mirroring: add interface to list file system mirror peers
pybind/mirroring: set libcephfs handle root uid/gid as 0:0
test: add tests for mirroring bootstrap interfaces
pybind/mirroring: introduce peer_bootstrap {create|import} commands
cephfs-mirror: use peer cluster monitor address (and key) if available
mon: peer_add should accept Ceph file system UUID
mon: introduce "profile cephfs-mirror" cap constrained to "config-get cephfs/mirror/peer"
test: add test for failed filesystem mirror instances
cephfs-mirror: restart failed mirror filesystem instances
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
mgr/cephadm: simplify handling for rgw
Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Or Friedmann <ofriedma@redhat.com>
Note that cephadm.py will no longer do anything with rgw realms and
zones. That means that the setup of rgw roles here is only useful
for the default zone and a non-multisite config.
Signed-off-by: Sage Weil <sage@newdream.net>
Test both "old" and "new" transit logic with s3tests. Does not test
migration - that will need to be done separately. I've added
a "flavor" parameter so the test logic can tell the difference
between the "old" engine and the "new" engine. The vault
keys creation logic now has options to determine whether
the keys created are exportable (needed for the old transit
engine), or not (should be the case going forward with the
new transit engine.)
Fixes: http://tracker.ceph.com/issues/48746
Signed-off-by: Marcus Watts <mwatts@redhat.com>
* refs/pull/39817/head:
qa/suites/rados/cephadm: drop centos/rhel cephadm tests for the moment
qa/sites/rados/cephadm/thrash: rename 3-tasks.yaml/ -> 3-tasks/
qa/suites/rados/cephadm: adjust distros
qa/suites/upgrade: use kubic; test all distros
qa/suites/rados/cephadm/upgrade: use kubic on centos
qa: new kubic distro files; use kubic podman for centos/rhel
Reviewed-by: Sebastian Wagner <swagner@suse.com>
* refs/pull/39855/head:
qa/suites/rgw/singleton: add trivial cephadm case
Revert "rgw: register daemon in service map with more details"
Reviewed-by: Sebastian Wagner <swagner@suse.com>
The current centos/rhel version of podman (2.2.1) is broken.
- create new qa/distros/podman/* files that install kubic podman
- include centos/rhel variants
- adjust cephadm jobs to use new yaml files
- remove old qa/distros/all/*_podman.yaml files
Signed-off-by: Sage Weil <sage@newdream.net>
* refs/pull/39787/head:
qa: Update featureful_client suite to use octopus instead of nautilus
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Actually add kmip to the kms crypt suite.
This also makes some ssl certs which is required for use of kmip.
Signed-off-by: Marcus Watts <mwatts@redhat.com>
This will retain the debug log settings for all RADOS suites
that were previously symlinked to the 'objectstore'
directory. The next commit will reduce the debug log level
for the original 'objectstore' directory for the remainder
of tests.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
valgrind can't handle execve of /proc/self/exe:
2021-02-27T05:52:37.813 INFO:tasks.ceph.mds.d.smithi073.stderr:==00:01:03:20.556 41218== execve(0x18546740(/proc/self/exe), 0x18546670, 0x133ef310) failed, errno 2
2021-02-27T05:52:37.813 INFO:tasks.ceph.mds.d.smithi073.stderr:==00:01:03:20.556 41218== EXEC FAILED: I can't recover from execve() failing, so I'm dying.
2021-02-27T05:52:37.813 INFO:tasks.ceph.mds.d.smithi073.stderr:==00:01:03:20.556 41218== Add more stringent tests in PRE(sys_execve), or work out how to recover.
So configure the MDS to just exit so it can be restarted by QA infra (the
daemon watchdog).
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
This commit consists of 3 things:
1. Files required for setting up new directory (in order to run the task in teuthology)
2. Kafka task file
3. The new files conataining tests and it's infrastructure for separation of bucket notification tests from pubsub tests
Signed-off-by: Kalpesh Pandya <kapandya@redhat.com>
* refs/pull/39710/head:
qa: run fs:verify on all distros
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
ext4/002 exercises obsolete EXT4_EOFBLOCKS_FL feature that was removed
in kernel 5.7 and therefore always fails.
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
* refs/pull/39709/head:
qa: use tcmalloc with valgrind in fs:valgrind
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
It's believed this is no longer a problem now that we use tcmalloc.
Fixes: https://tracker.ceph.com/issues/49391
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
1. removed symlink to `.qa/objectstore`
2. added `random-objectstore&` to `.qa/objectstore`
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
* refs/pull/39138/head:
qa: valgrind test for cephfs-mirror daemon
cephfs-mirror: use preforker for daemonizing
test: adjust sleep time to account for valgrind runs
cephfs-mirror: gracefully shutdown threads, timers, etc..
cephfs-mirror: call ceph_release() to cleanup mount alloc
cephfs-mirror: shutdown filesystem/cluster connections on shutdown
cephfs-mirror: set init failed flag on FSMirror::init() failure
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
* refs/pull/39561/head:
qa/valgrind.supp: suppress free[] warnings
qa/suites: do not use notcmalloc flavor
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
centos/rhel have podman 2, which does not like conflicting --cap-add and
--privileged arguments. cephadm versions prior to 15.2.5 use both args,
however, which means the rhel/centos upgrade test has to start at 15.2.5
to work at all on those distros (with the updated podman).
Fixes: https://tracker.ceph.com/issues/48142
Signed-off-by: Sage Weil <sage@newdream.net>
* refs/pull/39147/head:
qa/tasks/ceph_fuse: do not createfs
qa/tasks/cephfs/fuse_mount: pass admin_socket path
qa/suites/fs/cephadm/multivolume: add basic multivolume test
mgr/mds_autoscaler: some fixes and cleanup
mgr/volumes: deploy MDSs when creating fs
Reviewed-by: Milind Changire <mchangir@redhat.com>
Teuthology already defaults to quincy now and results in a failure
when trying to set to pacific. Additionally, drop the LUKS readbalance
test since it's unnecessary to duplicate that test.
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
EnvLibradosMutipoolTest.DBBulkLoadKeysInRandomOrder can overload OSDs and cause
heartbeat timeouts. Tests in test_envlibrados_for_rocksdb also generate slow
requests on OSDs. Use osd_client_message_cap to prevent this.
Since this option is disabled by default, this may be a good way to exercise it.
Fixes: https://tracker.ceph.com/issues/49064
Signed-off-by: Neha Ojha <nojha@redhat.com>
* refs/pull/39039/head:
src/test/cli/monmaptool: adjust for new monmap features
qa/tasks/cephadm: allow custom git_url for cephadm_branch pull
qa/suites/rados/upgrade: include pacific-x
qa/suites/upgrade/pacific-x/parallel
qa/suites: some clean up for quincy
mon: updates for quincy
mon: update for quincy ondisk features
script: add pacific
doc/dev/release-checklist: we tagged v17.0.0
ceph-volume: change to quincy
include/ceph_features: retire MON_SINGLE_PAXOS
include/ceph_features: define FEATURE_SERVER_QUINCY
mon/MgrMonitor: add always_on_modules for quincy
add feature/release name quincy
kickoff v17 quincy
doc/dev/release-checklists: uncheck everything!
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Normally the git_url is git://git.ceph.com/ceph-ci.git, which mirrors
upstream ceph-ci.git. However, the release branches aren't present there.
Allow a custom git_url so we can pull these from the main ceph.git.
Signed-off-by: Sage Weil <sage@newdream.net>
thrash_cache_writeback_proxy_none tests have been failing consistently. Some investigation
shows that the writeback overlay tests are reponsible for it. Instead of removing these
cache tiering tests entirely, we'll disable them for now.
Related to: https://tracker.ceph.com/issues/46323
Signed-off-by: Neha Ojha <nojha@redhat.com>
- remove upgrades from nautilus
- stubs for completing upgrade to quincy
Still missing the pacific-x upgrade tests.
Signed-off-by: Sage Weil <sage@newdream.net>
The current bionice version triggers a podman/conmon bug that
truncates output, affecting both cephadm bootstrap when 'mgr dump' is
large, and teuthology 'pg dump' when it is large.
See https://tracker.ceph.com/issues/48993
Signed-off-by: Sage Weil <sage@newdream.net>
basic, rbd and rbd-nomount subsuites are expanded to run with each
of ms_mode=legacy, ms_mode=crc and ms_mode=secure. This increases
the total number of jobs in the suite from 100 to 220.
fsx, singleton and thrash subsuites choose ms_mode at random (from
the above plus ms_mode=prefer-crc).
unmap and wac subsuites remain msgr1-only.
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Link directly to the distro version... no need to use _latest here since
it obscures the podman vs docker difference.
Signed-off-by: Sage Weil <sage@newdream.net>
Older cephadm is not smart enough to not combine --cap-add=SYS_PTRACE
and --privileged, which some version of podman cannot handle.
For upgrades, leave off the allow_ptrace behavior since we may be starting
on one of those old versions.
See also https://tracker.ceph.com/issues/46429
Fixes: https://tracker.ceph.com/issues/48142
Signed-off-by: Sage Weil <sage@newdream.net>
This commit adds new qemu xfstests workloads that run on top of librbd luks1/luks2 encryption.
This is currently done via nbd, instead of the qemu rbd driver.
Signed-off-by: Or Ozeri <oro@il.ibm.com>
this will provide a more detailed output, like
```yaml
...snip...
service_type: node-exporter
service_name: node-exporter
placement:
host_pattern: '*'
status:
created: '2021-01-18T11:21:56.024810Z'
last_refresh: '2021-01-18T11:23:24.477672Z'
running: 0
size: 1
events:
- "2021-01-18T11:23:09.602644Z service:node-exporter [ERROR] \"Failed while placing\
\ node-exporter.ubuntuon ubuntu: cephadm exited with an error code: 1, stderr:Deploy\
\ daemon node-exporter.ubuntu ...\nVerifying port 9100 ...\nTraceback (most recent\
\ call last):\n File \"<stdin>\", line 7274, in <module>\n File \"<stdin>\", line\
\ 1563, in _default_image\n File \"<stdin>\", line 3698, in command_deploy\n File\
\ \"<stdin>\", line 2338, in deploy_daemon\n File \"<stdin>\", line 1961, in create_daemon_dirs\n\
AssertionError\""
...snip...
```
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
