From eb99594437b5a90f8baf7575938e7f63e4cac2e4 Mon Sep 17 00:00:00 2001 From: Zac Dover Date: Fri, 20 Nov 2020 00:18:51 +1000 Subject: [PATCH] doc/dev: update unit-tests tox section This commit - alters the structure of the unit-test section of the Developer Guide that discusses the tox tool, which is used to run certain unit tests - corrects some grammar - updates the list of locations of tox.ini files - This also incorporates N. Cutler's suggestions. Signed-off-by: Zac Dover --- doc/dev/developer_guide/tests-unit-tests.rst | 129 ++++++++++++------- 1 file changed, 83 insertions(+), 46 deletions(-) diff --git a/doc/dev/developer_guide/tests-unit-tests.rst b/doc/dev/developer_guide/tests-unit-tests.rst index 2863a8f7f9a..de9d6de6fcd 100644 --- a/doc/dev/developer_guide/tests-unit-tests.rst +++ b/doc/dev/developer_guide/tests-unit-tests.rst @@ -13,12 +13,10 @@ run. What does "make check" mean? ---------------------------- -After compiling Ceph, the code can be run through a battery of tests -For historical reasons, this is -often referred to as ``make check`` even though the actual command used to run -the tests is now ``ctest``. For inclusion in this group of tests, a test -must: - +After compiling Ceph, the code can be run through a battery of tests. For +historical reasons, this is often referred to as ``make check`` even though +the actual command used to run the tests is now ``ctest``. To be included in +this group of tests, a test must: * bind ports that do not conflict with other tests * not require root access @@ -31,27 +29,27 @@ the more complex "integration tests" that are run via the `teuthology framework`_. While it is possible to run ``ctest`` directly, it can be tricky to correctly -set up your environment. Fortunately, a script is provided to make it easier -run the unit tests on your code. It can be run from the top-level directory of -the Ceph source tree by invoking: +set up your environment for it. Fortunately, there is a script that makes it +easy to run the unit tests on your code. This script can be run from the +top-level directory of the Ceph source tree by invoking: .. prompt:: bash $ ./run-make-check.sh You will need a minimum of 8GB of RAM and 32GB of free drive space for this -command to complete successfully on x86_64; other architectures may have -different requirements. Depending on your hardware, it can take from twenty -minutes to three hours to complete, but it's worth the wait. +command to complete successfully on x86_64 architectures; other architectures +may have different requirements. Depending on your hardware, it can take from +twenty minutes to three hours to complete. How unit tests are declared --------------------------- -Unit tests are declared in the ``CMakeLists.txt`` file, which is found -in the ``./src`` directory. The ``add_ceph_test`` and -``add_ceph_unittest`` CMake functions are used to declare unit tests. -``add_ceph_test`` and ``add_ceph_unittest`` are themselves defined in +Unit tests are declared in the ``CMakeLists.txt`` file, which is found in the +``./src`` directory. The ``add_ceph_test`` and ``add_ceph_unittest`` CMake +functions are used to declare unit tests. ``add_ceph_test`` and +``add_ceph_unittest`` are themselves defined in ``./cmake/modules/AddCephTest.cmake``. Some unit tests are scripts and other unit tests are binaries that are @@ -62,44 +60,82 @@ compiled during the build process. Unit testing of CLI tools ------------------------- - Some of the CLI tools are tested using special files ending with the extension ``.t`` and stored under ``./src/test/cli``. These tests are run using a tool -called `cram`_ via a shell script ``./src/test/run-cli-tests``. `cram`_ tests -that are not suitable for ``make check`` may also be run by teuthology using -the `cram task`_. +called `cram`_ via a shell script called ``./src/test/run-cli-tests``. +`cram`_ tests that are not suitable for ``make check`` can also be run by +teuthology using the `cram task`_. .. _`cram`: https://bitheap.org/cram/ .. _`cram task`: https://github.com/ceph/ceph/blob/master/qa/tasks/cram.py -Tox based testing of python modules +Tox-based testing of Python modules ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Some of the Python modules in Ceph use `tox `_ +to run their unit tests. -Most python modules can be found under ``./src/pybind/``. +Most of these Python modules can be found in the directory ``./src/pybind/``. -Many modules use **tox** to run their unit tests. -**tox** itself is a generic virtualenv management and test command line tool. +Currently (December 2020) the following modules use **tox**: -To find out quickly if **tox** can be run you can either just try to run ``tox`` -or check for the existence of a ``tox.ini`` file. +* Cephadm (``./src/cephadm/tox.ini``) +* Ceph Manager Python API (``./src/pybind/mgr``) -Currently the following modules use **tox**: + * ``./src/pybind/mgr/tox.ini`` + + * ``./src/pybind/mgr/dashboard/tox.ini`` -- Cephadm (``./src/pybind/mgr/cephadm``) -- Insights (``./src/pybind/mgr/insights``) -- Manager core (``./src/pybind/mgr``) -- Dashboard (``./src/pybind/mgr/dashboard``) -- Python common (``./src/python-common/tox.ini``) + * ``./src/pybind/tox.ini`` +* Dashboard (``./src/pybind/mgr/dashboard``) +* Python common (``./src/python-common/tox.ini``) +* CephFS (``./src/tools/cephfs/tox.ini``) +* ceph-volume -Most **tox** configurations support multiple environments and tasks. You can see -which are supported by examining the ``envlist`` assignment within ``tox.ini`` -To run **tox**, just execute ``tox`` in the directory where ``tox.ini`` is found. -If no environments are specified with e.g. ``-e $env1,$env2``, all environments -will be run. Jenkins will run ``tox`` by executing ``run_tox.sh`` which is under -``./src/script``. + * ``./src/ceph-volume/tox.ini`` -Here some examples from the Ceph Dashboard on how to specify + * ``./src/ceph-volume/plugin/zfs/tox.ini`` + + * ``./src/ceph-volume/ceph_volume/tests/functional/batch/tox.ini`` + + * ``./src/ceph-volume/ceph_volume/tests/functional/simple/tox.ini`` + + * ``./src/ceph-volume/ceph_volume/tests/functional/lvm/tox.ini`` + +Configuring Tox environments and tasks +"""""""""""""""""""""""""""""""""""""" +Most tox configurations support multiple environments and tasks. + +The list of environments and tasks that are supported is in the ``tox.ini`` +file, under ``envlist``. For example, here are the first three lines of +``./src/cephadm/tox.ini``:: + + [tox] + envlist = py3, mypy + skipsdist=true + +In this example, the ``Python 3`` and ``mypy`` environments are specified. + +The list of environments can be retrieved with the following command: + + .. prompt:: bash $ + + tox --list + +Or: + + .. prompt:: bash $ + + tox -l + +Running Tox +""""""""""" +To run **tox**, just execute ``tox`` in the directory containing +``tox.ini``. If you do not specify any environments (for example, ``-e +$env1,$env2``), then ``tox`` will run all environments. Jenkins will run +``tox`` by executing ``./src/script/run_tox.sh``. + +Here are some examples from Ceph Dashboard that show how to specify different environments and run options:: ## Run Python 2+3 tests+lint commands: @@ -108,7 +144,7 @@ environments and run options:: ## Run Python 3 tests+lint commands: $ tox -e py3,lint,check - ## To run it like Jenkins would do + ## To run it as Jenkins would: $ ../../../script/run_tox.sh --tox-env py27,py3,lint,check $ ../../../script/run_tox.sh --tox-env py3,lint,check @@ -117,18 +153,19 @@ Manager core unit tests Currently only doctests_ inside ``mgr_util.py`` are run. -To add test additional files inside the core of the manager, add -them at the end of the line that includes ``mgr_util.py`` within ``tox.ini``. +To add more files to be tested inside the core of the manager, open the +``tox.ini`` file and add the files to be tested at the end of the line that +includes ``mgr_util.py``. .. _doctests: https://docs.python.org/3/library/doctest.html Unit test caveats ----------------- -1. Unlike the various Ceph daemons and ``ceph-fuse``, unit tests - are linked against the default memory allocator (glibc) unless explicitly - linked against something else. This enables tools like **valgrind** to be used - in the tests. +#. Unlike the various Ceph daemons and ``ceph-fuse``, the unit tests are + linked against the default memory allocator (glibc) unless they are + explicitly linked against something else. This enables tools such as + **valgrind** to be used in the tests. .. _make check: .. _teuthology framework: https://github.com/ceph/teuthology