RepoMirrors/ceph
1
0
Fork 0
mirror of https://github.com/ceph/ceph synced 2025-02-22 02:27:29 +00:00
Code Issues Packages Projects Releases Wiki Activity

doc/dev: document nested subsets in teuthology

Browse Source 
See also: https://github.com/ceph/teuthology/pull/1704

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
This commit is contained in:
Patrick Donnelly 2022-02-09 21:34:22 -05:00
parent d6516fa649
commit c4d3040b70
No known key found for this signature in database
GPG Key ID: BE69BB7D36E459B4
1 changed files with 38 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

40
doc/dev/developer_guide/testing_integration_tests/tests-integration-testing-teuthology-intro.rst
View File

@ -316,8 +316,8 @@ signifies concatenation.
Convolution operator
^^^^^^^^^^^^^^^^^^^^
The convolution operator, implemented as an empty file called ``%``, tells
teuthology to construct a test matrix from yaml facets found in
The convolution operator, implemented as a (typically empty) file called ``%``,
tells teuthology to construct a test matrix from yaml facets found in
subdirectories below the directory containing the operator.
For example, the `ceph-deploy suite
@ -384,6 +384,34 @@ An individual test from the `ceph-deploy suite`_ can be run by adding the
be combined with ``--filter``. This is because the ``--suite`` option
understands POSIX relative paths only.
Nested Subsets
^^^^^^^^^^^^^^
Suites can get quite large with the combinatorial explosion of yaml
configurations. At the time of writing, the ``rados``` suite is more than
100,000 jobs. For this reason, scheduling often uses the ``--subset`` option to
only run a subset of the jobs (see also: :ref:`subset`). However, this applies
only at the top-level of the suite being run (e.g. ``fs``). That may
incidentally inflate the ratio of jobs for some larger sub-suites (like
``fs:workload``) vs. smaller but critical suites (like ``fs:volumes``).
It is therefore attractive to automatically subset some sub-suites which are
never run fully. This is done by providing an integer divisor for the ``%``
convolution operator file instead of leaving it empty. That divisor
automatically subsets the resulting matrix. For example, if the convolution
file ``%`` contains ``2``, the matrix will be divided into two using the same
logic as the ``--subset`` mechanism.
Note the numerator is not specified as with the ``--subset`` option as there is
no meaningful way to express this when there could be several layers of
nesting. Instead, a random subset is selected (1 of 2 in our example). The
choice is based off the random seed (``--seed``) used for the scheduling.
Remember that seed is saved in the results so that a ``--rerun`` of failed
tests will still preserve the correct numerator (subset of subsets).
You can disable nested subsets using the ``--no-nested-subset`` argument to
``teuthology-suite``.
Concatenation operator
^^^^^^^^^^^^^^^^^^^^^^
@ -508,6 +536,9 @@ or
Each string is looked up anywhere in the test description and has to
be an exact match: they are not regular expressions.
.. _subset:
Reducing the number of tests
----------------------------
@ -537,6 +568,11 @@ teuthology will still ensure that all files in the suite are in at
least one test. Understanding the actual logic that drives this
requires reading the teuthology source code.
Note: some suites are now using a **nested subset** feature that automatically
applies a subset to a carefully chosen set of YAML configurations. You may
disable this behavior (for some custom filtering, perhaps) using the
``--no-nested-subset`` option.
The ``--limit`` option only runs the first ``N`` tests in the suite:
this is rarely useful, however, because there is no way to control which
test will be first.