In 8b95c4b7c5 we set log_to_journald=false
in the cephadm config. However, that's not present in pre-quincy builds,
which means that when we upgrade the new daemons start spamming the
teuthology.log. Set this (with --force, since it's not valid pre-quincy)
in the config before we start the ugprade.
Signed-off-by: Sage Weil <sage@newdream.net>
this cephadm task was merged without testing in
https://github.com/ceph/ceph/pull/39855/ and fails consistently with an
error in kernel.py. the teuthology issue
https://tracker.ceph.com/issues/50338 has gone unfixed for months, so
removing rgw_cephadm.yaml to clean up the rgw suite
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Add a workunit for testing the rgw object cache
by using s3cmd to write objects and then
verify the objects in the cache.
Also move the 0-install.yaml file out of tasks and
into the main dir for the rgw/verify subsuite.
Signed-off-by: Ali Maredia <amaredia@redhat.com>
qa: d3n: add debug logs
Signed-off-by: Ali Maredia <amaredia@redhat.com>
rgw: s3n: qa: fix netstat search for rgw process
Signed-off-by: Mark Kogan <mkogan@redhat.com>
it's a regression introduced by the restrcuture of the test suites,
let's pin the test to CentOS8.
See-also: https://tracker.ceph.com/issues/49638
Signed-off-by: Kefu Chai <kchai@redhat.com>
* refs/pull/41574/head:
qa/tasks/vstart_runner: add LocalCluster.run
qa/tasks/cephfs/test_nfs: fiddle with sudo
mgr/nfs/export: some cleanup, minor refactoring
mgr/nfs/cluster: remove unused @cluster_setter
nfs/mgr: fix help message case
doc/cephfs/fs-nfs-export: add note about export update behavior
mgr/nfs: move user create/delete into helper
mgr/nfs: refactor _delete_user helper
mgr/nfs: refactor create_export_from_dict() helper
mgr/nfs: keep 'nfs export get' around for backward-compat
mgr/nfs: rename method
qa/tasks/cephfs/test_nfs: test new export via apply
doc/cephfs/fs-nfs-export: be consistent with cluster_id and _ vs -
mgr/nfs: addr -> client_addr for 'nfs export create ...'
mgr/nfs: fix tests
mgr/nfs: 'nfs export get' -> 'nfs export info'
mgr/nfs: binding -> pseudo_path
mgr/nfs: more revisions based on review
mgr/nfs: adjust NFSExceptoin errno arg
doc/cephfs: update 'nfs export {get,apply}' docs
mgr/nfs: merge FSExport back into ExportMgr
doc/radosgw/nfs: document mgr/nfs way to add/remove rgw exports
mgr/nfs: merge 'nfs export {update,import}' -> 'nfs export apply'
mgr/nfs: test export creation and list
mgr/nfs: test export_update (+ fixes)
mgr/nfs: test Export.validate(); several fixes
mgr/nfs: test that export <-> block+dict conversions go both ways
mgr/nfs: clean up test a bit
mgr/nfs/export: fix export validation
mgr/nfs/export: fix tests
mgr/nfs: handle option addr/client block in create_export()
mgr/nfs: allow multiple addrs for new exports
mgr/nfs: fix/finish rgw export
mgr/nfs/module: clusterid -> cluster_id
mgr/nfs/export: fix export_update_1 to type check
mgr/nfs/cluster: fix type error
mgr/nfs/export: wrap long lines
mgr/nfs: ExportMgr._delete_export only works for cephfs for now
mgr/nfs: Remove pool_ns from NFSCluster
mgr/nfs: Remove ExportMgr.rados_namespace
mgr/nfs: flake8
mgr/nfs: Add type checking
mgr/nfs: Add __eq__ method to Export
mgr/nfs: Add some compatibility to mgr/dashboard
mgr/nfs: Fix whitespace handling
mgr/nfs: Copy unit tests from mgr/dashboard
mgr/nfs: partially implement rgw export support
mgr/nfs: abstract FSAL; add RGWFSAL
mgr/nfs: refactor to merge 'update' and 'import' code
mgr/nfs: add 'nfs export import' command
mgr/nfs: refactor 'nfs export update' and export validation
mgr/nfs: fix _fetch_export to distinguish between clusters
mgr/nfs: move export ganesha conf translation into caller
mgr/nfs: name nfs cephfs client key 'nfs.{cluster_id}.{export_id}'
mgr/nfs: add --addr to 'nfs export create'
mgr/nfs: add --squash to 'nfs export create'
mgr/nfs/export_utils: include false but non-None items in config
vstart.sh: enable nfs module
mgr/cephadm: nfs: drop attr_expiration_time from top-level config
mgr/cephadm: remove Dir_Chunk = 0
Reviewed-by: Michael Fritch <mfritch@suse.com>
This is mostly for testing: a lot of tests assume that there are no
existing pools. These tests relied on a config to turn off creating the
"device_health_metrics" pool which generally exists for any new Ceph
cluster. It would be better to make these tests tolerant of the new .mgr
pool but clearly there's a lot of these. So just convert the config to
make it work.
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
* refs/pull/39505/head:
qa: test nowsync option in kernel client workflows
qa: deep merge top level overrides for fuse/kclient
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
even rook does not really install ceph packages in the host directly, it
uses the ceph container image. but teuthology insists on checking the
existence of debian packages by querying shaman server when it sees a
teuthology facet file which includes:
os_type: ubuntu
os_version: "18.04"
but since we've stopped building ubuntu/bionic packages, teuthology
just complains when we are scheduling test suites which are composed
from facets in qa/suites/orch/rook/smoke.
in this change, the ubuntu_18.04.yaml is dropped because ubuntu/bionic
does not really increase the test coverage of ceph. it helps to test
the rook and container runtime though.
Signed-off-by: Kefu Chai <kchai@redhat.com>
* refs/pull/39910/head:
test: Add test for mgr hang when osd is full
mgr: Set client_check_pool_perm to false
mds: Add full caps to avoid osd full check
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
"get_heap_property *" asock commands are exposed to operators
to check the tcmalloc internals for understanding the performance
of the memory subsystem. but crimson uses the builtin seastar allocator
which is not backed by tcmalloc. but we can dump the metrics using
the "dump_metrics" asock command which is only available from
crimson-osd.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
This commit majorly consists of the RabbitMQ task which is a required and supported endpoint in bucket notification tests.
And some related changes in the AMQP tests. Major changes are:
1. Addition of RabbitMQ task
2. Documentation update for the steps to execute AMQP tests
3. Addition of attributes to the tests
4. Tox dependency removal from kafka.py
Signed-off-by: Kalpesh Pandya <kapandya@redhat.com>
Also, have a fs:mirror specific whitelist override since
the current one is a symlink to a common YAML probably used
by other tests.
Fixes: http://tracker.ceph.com/issues/50867
Signed-off-by: Venky Shankar <vshankar@redhat.com>
qa/tests - removed ref to 18.04 distro as it's not supported on master+
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
use centos_latest instead of bionic because this is only common
distro for which we build packages for nautilus and above.
Signed-off-by: Neha Ojha <nojha@redhat.com>
Add fs suite for tests requiring one node as well.
Fixes: https://tracker.ceph.com/issues/50532
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Signed-off-by: gal salomon <gal.salomon@gmail.com>
force-branch to s3test/master
Signed-off-by: gal salomon <gal.salomon@gmail.com>
update of s3select (== -> =)
Signed-off-by: gal salomon <gal.salomon@gmail.com>
update of s3select (== -> =)
Signed-off-by: gal salomon <gal.salomon@gmail.com>
closing documentation gaps between previous and current functionalities(WIP)
Signed-off-by: gal salomon <gal.salomon@gmail.com>
Update s3select.rst
additional features
Update s3select.rst
Signed-off-by: gal salomon <gal.salomon@gmail.com>
Update s3select.rst
editorial
Signed-off-by: gal salomon <gal.salomon@gmail.com>
Update s3select.rst
editorial
Signed-off-by: gal salomon <gal.salomon@gmail.com>
Update s3select.rst
adding cast to bool
Signed-off-by: gal salomon <gal.salomon@gmail.com>
editorial; more description for substring and trim
Signed-off-by: gal salomon <gal.salomon@gmail.com>
This just to make sure we don't break mgr/orchestrator.
Note that we already symlink ../orch/cephadm, so this makes rados
include all of orch/.
Signed-off-by: Sage Weil <sage@newdream.net>
* refs/pull/41286/head:
qa/suites/orch/rook: disable centos for now
qa/suites/orch/rook/smoke: initial smoke suite
qa/tasks/rook: ROOK_HOSTPATH_REQUIRES_PRIVILEGED=true on centos
qa/tasks/rook: simplify shutdown
qa/tasks/rook: archive logs
qa/tasks/rook: more orderly cluster teardown
qa/tasks/rook: deploy ceph via rook on top of kubernetes
qa/tasks/kubeadm: install kubernetes with kubeadm
qa/suites: move rados/cephadm -> orch/cephadm; symlink
qa/tasks/cephadm: add whitespace between functions
qa/tasks/cephadm: clean up ctx.manager setup
Reviewed-by: Sébastien Han <seb@redhat.com>
use mclock_scheduler as the default scheduler
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Sridhar Seshasayee <sseshasa@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Sunny Kumar <sunkumar@redhat.com>
The standalone tests need parameters to be passed as ceph_args to
override defaults.
This was just doubling the number of standalone tests being run in each rados
run with no effect!
Signed-off-by: Neha Ojha <nojha@redhat.com>
There already is a test to verify the mempool sharding works, in the sense that
it uses at least half of the variables available to count the number of
allocated objects and their total size. This new test verifies that, with
sharding, object counting is at least twice faster than without sharding. It
also collects cacheline contention data with the perf c2c tool. The manual
analysis of this data shows the optimization gain is indeed related to cacheline
contention.
Fixes: https://tracker.ceph.com/issues/49896
Signed-off-by: Loïc Dachary <loic@dachary.org>
* refs/pull/40941/head:
qa/suites/rados/cephadm/smoke-roleless: test client-keyring
qa/tasks/cephadm.py: adjust client.admin key mode; place on all hosts
cephadm: distribute client.admin keyring+conf to label:_admin on bootstrap
doc/cephadm: document the default 'admin' label
mgr/cephadm: 'ceph orch client-keyring ...' commands to manage keyring files
mgr/cephadm: reimplement ceph.conf pushing
mgr/cephadm: use _write_remote_file for ceph.conf
mgr/cephadm: _write_remote_file helper
mgr/cephadm: add placementspec for which hosts get ceph.conf
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Adam King <adking@redhat.com>
these parameters have proven to catch some of the uncaught bugs such as:
https://tracker.ceph.com/issues/48417, adopting them will help in
preventing more such hard to debug bugs.
Signed-off-by: Deepika Upadhyay <dupadhya@redhat.com>
This is the longest of the thrash workloads; reducing it will bring
this test in line with the others (<= 45 min).
Signed-off-by: Sage Weil <sage@newdream.net>
Sync up with master up to commit 3d8e73b266 ("Merge pull request
#40731 from tchaikov/wip-yamlize-options"). Specifically, bring in
src/common/options.cc yamlization and move new auth-related options
into src/common/options/global.yaml.in.
Conflicts:
src/common/options.cc
src/common/options/global.yaml.in
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
This takes 5 hours currently.
- Separate out filestore and memstore into separate task (~1 hr)
- Split bluestore into -a and -b (a tests exclude SynethicMatrixC,
b tests include it)
Signed-off-by: Sage Weil <sage@newdream.net>
instead of using the default / existing distro, specify the distro for
testing. this change prevents us from using bionic for testing the
builds. this has two consequences:
* so we are one step closer to a non-bionic world.
* avoid building packages with PPA repo which *might* introduce runtime
dependencies on 3rd party runtimes provided by PPA repo.
see also: https://tracker.ceph.com/issues/50218
Signed-off-by: Kefu Chai <kchai@redhat.com>
This PR includes 2 things:
1. Changing force-branch to master and removing the git-remote. This change was forgetten for PR #39139.
2. Proper cleanup/removal after completion of commands more precisely removing the kafka logs directory.
Signed-off-by: Kalpesh Pandya <kapandya@redhat.com>
query the python version before trying to test diskprediction_local
Fixes: https://tracker.ceph.com/issues/50196
Signed-off-by: Kefu Chai <kchai@redhat.com>
If this is a fresh pacific cluster, let's assume that there won't be
legacy clients connecting. (And if there are, let's put the burden on
the user to enable them to do so insecurely.)
This is in contrast to upgrades, where our focus is on not breaking
anything.
Signed-off-by: Sage Weil <sage@newdream.net>
* refs/pull/40418/head:
test: unmount when finished ino_release_cb
test: wait a time for inode release
qa: move ino_release_cb to libcephfs sub-suite
qa: simplify recall triggers for bug
qa: fix name for qa task referencing tracker issue
Reviewed-by: Jeff Layton <jlayton@redhat.com>
this was added in https://github.com/ceph/ceph/pull/39820 to
avoid repeating the override in rgw subsuites, but teuthology
was picking up suites/rgw/ignore-pg-availability.yaml as a job
of its own, and led to 'list index out of range' failures
Fixes: https://tracker.ceph.com/issues/50147
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Starting with pacific, CEPHX_V2 is required by default but
pre-single-major.yaml kernel doesn't support it.
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
It is not testing a bug. Also, fix cap recall configs so this test is
properly triggered.
Fixes: https://tracker.ceph.com/issues/49500
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
The issue number was for an rbd backport. It should be [1].
[1] https://tracker.ceph.com/issues/24137
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
The 'orch daemon add ...' command is not idempotent and can cause
duplicate (and failing) attempts to add the same mon.
Maintain the ability to add mons the old way for the benefit of testing
upgrades from early octopus versions of cephadm.
Signed-off-by: Sage Weil <sage@newdream.net>
* refs/pull/40160/head:
qa/suites/rados/cephadm/orchestrator_cli: random-distro$ -> 0-random-distro$
qa/suites/rados/cephadm/smoke-roleless: distro -> 0-distro
qa/distros/podman: install kubic once per host, in parallel
qa/suites/fs/multiclient: use clients: not all: for pexec
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
