ceph/qa/tasks/mgr
Sridhar Seshasayee 328271d587 qa/tasks: Enhance wait_until_true() to check & retry recovery progress
With mclock scheduler enabled, the recovery throughput is throttled based
on factors like the type of mclock profile enabled, the OSD capacity among
others. Due to this the recovery times may vary and therefore the existing
timeout of 120 secs may not be sufficient.

To address the above, a new method called _is_inprogress_or_complete() is
introduced in the TestProgress Class that checks if the event with the
specified 'id' is in progress by checking the 'progress' key of the
progress command response. This method also handles the corner case where
the event completes just before it's called.

The existing wait_until_true() method in the CephTestCase Class is
modified to accept another function argument called "check_fn". This is
set to the _is_inprogress_or_complete() function described earlier in the
"test_turn_off_module" test that has been observed to fail due to the
reasons already described above. A retry mechanism of a maximum of 5
attempts is introduced after the first timeout is hit. This means that
the wait can extend up to a maximum of 600 secs (120 secs * 5) as long as
there is recovery progress reported by the 'ceph progress' command result.

Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
2021-06-02 14:19:48 +05:30
..
dashboard mgr/dashboard: fix bucket versioning when locking is enabled 2021-05-05 17:24:41 +05:30
__init__.py
mgr_test_case.py mgr/dashboard:minimize console log traces of Dashboard backend API tests 2020-12-23 11:32:43 +05:30
test_crash.py
test_dashboard.py qa/tasks/mgr/test_dashboard: skip test_standby if mgr_standby_modules=false 2021-04-16 16:00:31 -04:00
test_failover.py
test_insights.py
test_module_selftest.py qa/tasks/mgr: skip test_diskprediction_local on python>=3.8 2021-04-07 21:27:44 +08:00
test_orchestrator_cli.py mgr/orch: add errno to OrchestratorError 2020-07-28 15:54:46 -06:00
test_progress.py qa/tasks: Enhance wait_until_true() to check & retry recovery progress 2021-06-02 14:19:48 +05:30
test_prometheus.py mgr/dashboard: set security headers 2021-02-19 13:18:26 +05:30