ceph/qa/tasks
Sridhar Seshasayee 328271d587 qa/tasks: Enhance wait_until_true() to check & retry recovery progress
With mclock scheduler enabled, the recovery throughput is throttled based
on factors like the type of mclock profile enabled, the OSD capacity among
others. Due to this the recovery times may vary and therefore the existing
timeout of 120 secs may not be sufficient.

To address the above, a new method called _is_inprogress_or_complete() is
introduced in the TestProgress Class that checks if the event with the
specified 'id' is in progress by checking the 'progress' key of the
progress command response. This method also handles the corner case where
the event completes just before it's called.

The existing wait_until_true() method in the CephTestCase Class is
modified to accept another function argument called "check_fn". This is
set to the _is_inprogress_or_complete() function described earlier in the
"test_turn_off_module" test that has been observed to fail due to the
reasons already described above. A retry mechanism of a maximum of 5
attempts is introduced after the first timeout is hit. This means that
the wait can extend up to a maximum of 600 secs (120 secs * 5) as long as
there is recovery progress reported by the 'ceph progress' command result.

Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
2021-06-02 14:19:48 +05:30
..
cephadm_cases qa/tasks/cephadm_cases: longer wait for osd to start 2021-05-05 11:03:02 -05:00
cephfs Merge pull request #40831 from vshankar/wip-cephfs-mirror-incremental-sync 2021-05-27 13:39:23 +05:30
mgr qa/tasks: Enhance wait_until_true() to check & retry recovery progress 2021-06-02 14:19:48 +05:30
tests qa/tasks: drop python2 support from rgw related tests 2020-07-22 11:55:49 +08:00
util qa/tasks: drop python2 support from rgw related tests 2020-07-22 11:55:49 +08:00
__init__.py
admin_socket.py qa/tasks/admin_socket: get rid of cStringIO for py3 2020-03-04 13:09:17 +08:00
autotest.py qa/tasks/autotest: use remote.write_file 2020-09-04 00:02:15 +02:00
aver.py
barbican.py qa/tasks/barbican.py: fix year2021 problem 2021-01-21 13:31:27 -05:00
blktrace.py qa: get rid of iteritems for python3 compatibility 2019-10-15 11:27:39 +02:00
boto.cfg.template
cbt.py qa/tasks/cbt: user remote.write_file 2020-09-04 00:02:15 +02:00
ceph_client.py qa: get rid of iteritems for python3 compatibility 2019-10-15 11:27:39 +02:00
ceph_deploy.py qa/tasks/ceph_deploy: use remote.read_file instead of misc.get_file 2020-09-04 00:02:15 +02:00
ceph_fuse.py qa: move get_valgrind_args to qa 2021-03-03 09:30:08 -08:00
ceph_iscsi.py qa: add ceph iscsi test suite support 2020-12-17 20:49:21 +08:00
ceph_manager.py qa/tasks/rook: deploy ceph via rook on top of kubernetes 2021-05-18 15:19:04 -05:00
ceph_objectstore_tool.py qa/tasks/ceph_objectstore_tool: use remote.write_file 2020-09-04 00:02:15 +02:00
ceph_test_case.py qa/tasks: Enhance wait_until_true() to check & retry recovery progress 2021-06-02 14:19:48 +05:30
ceph.conf.template qa/tasks/ceph[adm].conf[.template]: disable insecure global_id reclaim health alerts 2021-04-06 17:29:21 -04:00
ceph.py qa/tasks: Add wait_for_clean() check prior to initiating scrubbing. 2021-03-25 22:01:19 +05:30
cephadm.conf qa/tasks/cephadm.conf: log_to_journald=false 2021-05-21 09:54:46 -04:00
cephadm.py Merge PR #41007 into master 2021-05-25 16:17:44 -04:00
cephfs_mirror.py qa: move get_valgrind_args to qa 2021-03-03 09:30:08 -08:00
cephfs_test_runner.py qa/tasks/cephfs_test_runner: setattr to class not instance 2020-01-09 20:48:23 +08:00
cephfs_upgrade_snap.py qa: add run_scrub helper in filesystem class 2021-01-09 14:56:09 +08:00
check_counter.py qa/tasks: use next(iter(..)) for accessing first element in a view 2020-04-07 20:33:47 +08:00
cifs_mount.py
cram.py qa/tasks: get the correct clone_dir for each client 2020-12-17 20:38:48 +08:00
create_verify_lfn_objects.py qa: Enable flake8 tox and fix failures 2019-12-12 10:21:01 +01:00
daemonwatchdog.py qa: check mounts attribute in ctx 2021-03-29 18:30:39 +05:30
devstack.py doc/dev: use http://docs.ceph.com/en/latest/ instead of /docs/master/ for docs 2020-11-24 12:49:47 +08:00
die_on_err.py qa: import with full path 2020-03-24 18:27:55 +08:00
divergent_priors2.py qa: import with full path 2020-03-24 18:27:55 +08:00
divergent_priors.py qa: import with full path 2020-03-24 18:27:55 +08:00
dnsmasq.py qa/tasks/dnsmasq: use remote.write_file 2020-09-04 00:02:15 +02:00
dump_stuck.py qa: import with full path 2020-03-24 18:27:55 +08:00
ec_lost_unfound.py qa/tasks: Add additional wait_for_clean() check in lost_unfound tasks. 2021-03-18 13:03:41 +05:30
exec_on_cleanup.py qa: fix issues in exec_on_cleanup 2021-02-09 23:39:04 +08:00
filestore_idempotent.py qa/tasks: use next(iter(..)) for accessing first element in a view 2020-04-07 20:33:47 +08:00
fs.py qa: wait for daemons to come up via cephadm 2021-03-21 10:35:07 -07:00
fwd_scrub.py qa: add run_scrub helper in filesystem class 2021-01-09 14:56:09 +08:00
immutable_object_cache_thrash.py librbd/image: fix immutable object cache read failed bug when daemon down 2020-07-01 14:19:59 -04:00
immutable_object_cache.py qa: check immutable object cache content 2020-09-21 01:36:56 +08:00
kafka.py qa/tasks: Adding RabbitMQ task for bucket notification tests 2021-06-01 23:34:31 +05:30
kclient.py qa: allow arbitrary mount options on kclient mounts 2020-12-04 18:22:28 -05:00
keycloak.py qa: STS Web Identity task files 2020-11-29 04:52:19 +05:30
keystone.py qa/tasks/keystone.py: centos-8 bindep fix 2021-01-21 13:30:55 -05:00
kubeadm.py qa/tasks/kubeadm: install kubernetes with kubeadm 2021-05-18 11:14:14 -05:00
locktest.py
logrotate.conf
lost_unfound.py qa/tasks: Add additional wait_for_clean() check in lost_unfound tasks. 2021-03-18 13:03:41 +05:30
manypools.py qa/tasks/manypools: get rid of itervalues for py3 compat 2020-03-04 13:09:17 +08:00
mds_creation_failure.py qa: import with full path 2020-03-24 18:27:55 +08:00
mds_pre_upgrade.py
mds_thrash.py qa: remove ceph file systems on completion 2021-01-08 10:44:53 -08:00
metadata.yaml
mon_clock_skew_check.py qa: import with full path 2020-03-24 18:27:55 +08:00
mon_recovery.py qa/tasks: use "a // b" instead of "a / b" 2020-04-07 20:33:47 +08:00
mon_thrash.py qa/tasks: fix import module path for py3 compat 2020-03-04 13:09:16 +08:00
multibench.py qa: import with full path 2020-03-24 18:27:55 +08:00
netem.py qa/tasks/netem: get rid of cStringIO for py3 2020-03-04 13:09:17 +08:00
netsplit.py qa: add new netsplit task 2020-07-08 04:26:03 +00:00
notification_tests.py qa/tasks: Adding RabbitMQ task for bucket notification tests 2021-06-01 23:34:31 +05:30
object_source_down.py qa: import with full path 2020-03-24 18:27:55 +08:00
omapbench.py qa/tasks: drop python2 support 2020-07-21 19:52:11 +08:00
openssl_keys.py qa/tasks/openssl_keys.py: add subjectAltName to certificates 2020-12-02 15:44:35 -05:00
osd_backfill.py qa: import with full path 2020-03-24 18:27:55 +08:00
osd_failsafe_enospc.py qa: log-whitelist -> log-ignorelist 2020-08-24 19:53:08 +00:00
osd_max_pg_per_osd.py qa/tasks/osd_max_pg_per_osd: get rid of itervalues for py3 2020-03-04 13:09:16 +08:00
osd_recovery.py qa: import with full path 2020-03-24 18:27:55 +08:00
peer.py qa: import with full path 2020-03-24 18:27:55 +08:00
peering_speed_test.py qa: import with full path 2020-03-24 18:27:55 +08:00
populate_rbd_pool.py
pykmip.py rgw/kms/kmip - correct documentation. 2021-03-03 19:14:10 -05:00
qemu.py qa/tasks/rbd: increase image size of encrypted disks 2021-01-20 16:08:17 +02:00
rabbitmq.py qa/tasks: Adding RabbitMQ task for bucket notification tests 2021-06-01 23:34:31 +05:30
rados.py qa: add missing argument parser 2021-03-29 17:15:16 +09:00
radosbench.py qa/tasks: drop python2 support 2020-07-21 19:52:11 +08:00
radosbenchsweep.py qa/tasks: drop python2 support 2020-07-21 19:52:11 +08:00
radosgw_admin_rest.py rgw: fixes BucketInfo for missing buckets 2020-10-12 13:50:06 +03:00
radosgw_admin.py qa/tasks/radosgw_admin: drop py2 support 2020-07-05 10:58:28 +08:00
ragweed.py qa/tasks/ragweed: use remote.write_file 2020-09-04 00:02:15 +02:00
rbd_fio.py qa/tasks/rbd_fio: use remote.write_file 2020-09-04 00:02:15 +02:00
rbd_fsx.py qa: move get_valgrind_args to qa 2021-03-03 09:30:08 -08:00
rbd_mirror_thrash.py qa: Enable flake8 tox and fix failures 2019-12-12 10:21:01 +01:00
rbd_mirror.py qa: move get_valgrind_args to qa 2021-03-03 09:30:08 -08:00
rbd.py qa/tasks/rbd: fix xfstests_branch/url example 2021-02-27 15:25:39 +01:00
rebuild_mondb.py qa: import with full path 2020-03-24 18:27:55 +08:00
reg11184.py qa: import with full path 2020-03-24 18:27:55 +08:00
rep_lost_unfound_delete.py qa/tasks: Add additional wait_for_clean() check in lost_unfound tasks. 2021-03-18 13:03:41 +05:30
repair_test.py qa: improve usability of do_rados helper 2021-03-21 10:35:07 -07:00
resolve_stuck_peering.py qa: import with full path 2020-03-24 18:27:55 +08:00
restart.py qa/tasks/restart: use remote.read_file instead of misc.get_file 2020-09-04 00:02:16 +02:00
rgw_logsocket.py qa: import with full path 2020-03-24 18:27:55 +08:00
rgw_multi
rgw_multisite_tests.py qa/rgw: add 'branch' option to rgw-multisite-tests task 2021-02-17 09:01:05 -05:00
rgw_multisite.py qa: import with full path 2020-03-24 18:27:55 +08:00
rgw.py rgw: remove the civetweb frontend from src and qa 2021-05-17 15:00:31 -04:00
rook-ceph.conf qa/tasks/rook: archive logs 2021-05-18 15:19:16 -05:00
rook.py qa/tasks/rook: ROOK_HOSTPATH_REQUIRES_PRIVILEGED=true on centos 2021-05-18 15:19:16 -05:00
s3a_hadoop.py qa/tasks/s3a_hadoop: use remote.write_file 2020-09-04 00:02:15 +02:00
s3tests_java.py qa/tasks/s3tests_java: use remote.write_file 2020-09-04 00:02:15 +02:00
s3tests.py rgw/kms/vault - s3tests for both old and new test logic. 2021-03-05 23:05:13 -05:00
samba.py qa/tasks/samba: use remote.write_file 2020-09-04 00:02:15 +02:00
scrub_test.py qa: improve usability of do_rados helper 2021-03-21 10:35:07 -07:00
scrub.py qa/tasks: flake8 fixes 2020-06-23 23:00:56 +08:00
systemd.py qa/tasks: flake8 fixes 2020-06-23 23:00:56 +08:00
tempest.py blacklist -> blocklist 2020-08-24 19:53:08 +00:00
teuthology_integration.py
tgt.py qa: get rid of iteritems for python3 compatibility 2019-10-15 11:27:39 +02:00
thrash_pool_snaps.py
thrasher.py qa/tasks: Fixed AttributeError: can't set attribute 2019-11-27 08:30:58 +05:30
thrashosds-health.yaml qa: log-whitelist -> log-ignorelist 2020-08-24 19:53:08 +00:00
thrashosds.py qa/tasks: logger.warn() -> logger.warning() 2020-05-13 17:18:13 -06:00
tox.py qa/tasks/tox: use the latest stable tox 2020-06-02 09:48:32 +08:00
userdata_setup.yaml qa/tasks/qemu: precise repos have been archived 2021-06-01 12:54:16 +02:00
userdata_teardown.yaml
vault.py rgw/kms/vault - s3tests for both old and new test logic. 2021-03-05 23:05:13 -05:00
vip.py qa/tasks/vip: add 'vip.exec' task 2021-05-19 08:43:14 -04:00
vstart_runner.py Merge PR #40412 into master 2021-04-15 21:05:51 -07:00
watch_notify_same_primary.py qa/tasks: drop python2 support 2020-07-21 19:52:11 +08:00
watch_notify_stress.py qa/tasks: drop python2 support 2020-07-21 19:52:11 +08:00
workunit.py qa/tasks/workunit: use remote.read_file instead of misc.get_file 2020-09-04 00:02:16 +02:00