mirror of https://github.com/schoebel/mars
README
GPLed software AS IS, sponsored by 1&1 Internet AG (www.1und1.de).
Contact frank.liepold@1und1.de
The test suite is work in progress.
At the moment it is not in the state to be usable outside of the development
team.
Even this file is not complete!
=============================================================================
Contents
--------
1. Running a test
1.1. Test output
2. Naming conventions
3. Error handling
4. Signal handling
5. Configuration of the local build environment
6. Configuration of the test hosts on which the tests are running
6.1. Access requirements
6.2. Installation kernel and mars module on the test hosts
7. Configuration of logical volumes
8. Resources
9. Starting the whole test suite via cronjob
10. Concurrent test runs
11. Firewalls
12. Programming hints and conventions
1. Running a test
-----------------
Tests are executed by a call to scripts/start_test.sh from a subdirectory
of one of the both directories build_test_environment and test_cases.
The scope and configuration of a test is completely described by this
subdirectory (we call it start directory) as follows:
- All subdirectories of the start directory containing a (usually empty)
file named i_am_a_testdirectory define one test case. In the following
we call such a subdirectory a test directory.
If a test for one test directory below the start directory fails, the tests
belonging to the other test directories of the start directory will not be
executed.
Though only tests which are likely not to fail should be executed by only
one call to start_test.sh. The simple and recommended way is to call
start_test.sh for each test directory once (which means that the
start directory consists of only one test directory and coincides with it.
- The configuration of the test case is defined by a set of *.conf files which
are included by start_test.sh in the following order:
-- default-*.conf: These are the configuration files belonging to the
scripts scripts/modules/*.sh which are also included
bei start_test.sh and which define all functions which may
be used by the various tests.
-- user_modules.conf
-- <subdirname>.conf where subdirname runs top down through the parent
directories starting with the first subdirectory of a directory called
root config directory to the test directory.
This config root directory is set to the start directory (on the directory
branch start directory -> test directory) by default. It may be changed by
the option --config_root_dir=<my dir>.
If the start directory coincides with the test directory the file
<test directory name>.conf is included for conveniance (in the strict
sense there are no true subdirectories of the start directory residing
above the test directory).
The <subdirnam>.conf files may reside in the test directory or any of
it's parent directories (up to 20 levels).
Examples:
1. start directory = test_cases/perf
test directory = test_cases/perf/apply/no_parallel_writer
Option --config_root_dir not given
leads to including of apply.conf and no_parallel_writer.conf (in
this order!) which must reside as mentioned above in the leaf
directory or any of it's parent directories (up to 20 levels).
2. start directory = test_cases/perf/apply/no_parallel_writer
leaf directory = test_cases/perf/apply/no_parallel_writer
Option --config_root_dir=test_cases/perf
leads to including the same *.conf files as above.
3. start directory = test_cases/perf/apply/no_parallel_writer
leaf directory = test_cases/perf/apply/no_parallel_writer
Option --config_root_dir given or not
leads to including of no_parallel_writer.conf
- In one of the included <subdirnam>.conf files of a test one of the variables
prepare_list, setup_list, run_list, cleanup_list or finish_list must be set.
Each of these variables may be empty or contain a list of shell function
names of functions defined in scripts/modules/*.sh.
These functions are called by start_test.sh in the order they appear in the
mentioned *list variables. The *list variables are evaluated in the order
prepare_list, setup_list, run_list, cleanup_list, finish_list.
1.1. Test output
----------------
The output fo start_test.sh consists of the following sections:
- List titled "Sourcing libraries in <library directory>" of included scripts
from <libraries directory>/lib*.sh
- List titled "Sourcing modules and default configuration" of included scripts
from scripts/modules/[0-9]*.sh and corresponding included default-*.conf
files.
- Line titled "Scanning subdirectories of <start directory>" followed by a
list of ignored or skipped subdirectories
- Per leaf directory:
- Line titled "Test directory <leaf directory> <date and time>"
- List titled "Sourcing config files between <config_root_dir> and
<leaf directory>" of included *.conf files corresponding to the directory
structure.
- List titled "Configuration variables" of all configuration variables set
via all included *.conf file
- Section titled "Creating lock files"
- Section titled "Starting <leaf directory> <date and time>"
In this section lines starting with "calling ..." mark the call to one of
the functions listed in the aforementioned variables prepare_list,
setup_list, run_list, cleanup_list and finish_list.
The main part of this section consists of output lines of the function
lib_vmsg. These lines have the following format:
<date time> [[<callstack>]] <message>
The stack level which the bash call stack is printed from (<callstack>)
is configurable via variable main_min_stack_level.
- In case of an error: Subsection titled "Callstack" (to stderr)
- Subsection titled "Deleting lock files"
- Subsection titled "General checks of error and log files"
- Line titled "Failure <return code> <leaf directory> <date and time>"
(to stderr) in case of failure,
"Finished <leaf directory> <date and time>" (to stdout) otherwise
- If all tests in all leaf directories of the start directory terminated
successfully: Line titled "Finished start directory <start directory>"
2. Naming conventions
---------------------
- All functions defined in scripts/modules/*.sh have the script's name as
name prefix (e.g. resource_check_variables)
- All variables used in *.conf files should be "defined" (the shell has no
explicit definition) and explained in a default-*.conf file. If a variable is
"defined" in default-xyz.conf it's name is prefixed by xyz.
- WIP
3. Error handling
-----------------
WIP
4. Signal handling
------------------
WIP
5. Configuration of the local build environment
-----------------------------------------------
To build the mars module and userspace tools you need:
- a directory containing a git repository which contains the kernel sources
variable : checkout_mars_kernel_src_directory
to be specified in file : default-checkout_mars.conf
- the branch from which to take the kernel sources
variable : checkout_mars_kernel_git_branch
to be specified in file : default-checkout_mars.conf
- a directory containing the kernel sources against which the mars module
should be built (may be the same as checkout_mars_kernel_src_directory)
variable : make_mars_kernel_src_directory
to be specified in file : default-make_mars.conf
- a directory containing a git repository which contains the mars sources
variable : checkout_mars_src_directory
to be specified in file : default-checkout_mars.conf
- the branch from which to take the mars sources
variable : checkout_mars_git_branch
to be specified in file : default-checkout_mars.conf
- a directory containing the mars sources which is linked in to the
kernel source directory make_mars_kernel_src_directory for making the mars
module.
May coincide with checkout_mars_src_directory
variable : make_mars_src_directory
to be specified in file : default-make_mars.conf
- a directory containing a git repository which contains the userspace tool
marsview
variable : checkout_mars_contrib_src_directory
to be specified in file : default-checkout_mars.conf
- the branch from which to take marsview
variable : checkout_mars_contrib_git_branch
to be specified in file : default-checkout_mars.conf
6. Configuration of the test hosts on which the tests are running
------------------------------------------------------------
The test hosts are given in main_host_list (default-main.conf).
6.1. Access requirements
------------------------
The tests are executed by ssh commands from your work station to the
test hosts. These commands must not prompt for a password. Though a
id file (for ssh -i <id-file>) with null password must be specified in
main_ssh_idfile_opt (default-main.conf)
All test hosts must have ssh access to each other without prompt for passwords
(this is required by marsadm join-cluster).
On your work station you must have sudo permissions for "make install", "cp".
And root of your work station must have ssh access (as root) without prompt for
password to all test hosts (this is required, because local files belonging to
root must be copied to the test hosts).
6.2. Installation kernel and mars module on the test hosts
----------------------------------------------------------
This is done by executing the tests
build_test_environment/checkout
build_test_environment/make/make_mars/grub
build_test_environment/install_mars
It's a must to read default-checkout_mars.conf, default-make_mars.conf,
default-install_mars.conf and the *.conf files in the directories above
otherwise you risk damage of the boot information on your work station and/or
the test hosts.
After these three tests (which you should call one after the other!) you can
reboot the test hosts with the new kernel.
7. Configuration of logical volumes
-----------------------------------
This is done by the test
build_test_environment/lv_config
The configuration options are contained in default-lv_config.conf
8. Resources
------------
Cluster and Resources are created by
build_test_environment/cluster
build_test_environment/resource/create_resource
Initially the first host in main_host_list is the primary, the following hosts
secondaries.
If you do not change any configuration variables after the
build_test_environment/resource/create_resource Test you will have created one
resource lv-1-2 on all hosts with underlying data devices /dev/vg-mars/lv-1-2.
/mars will be mounted on /dev/vg-mars/lv-6-100.
For further information please read default-cluster.conf and
default-resource.conf.
9. Starting the whole test suite via cronjob
--------------------------------------------
This can be done mars_test_cronjob.sh. The variable tests_to_execute contains
all tests to be executed. See the documentation in the header.
10. Concurrent test runs
------------------------
To avoid concurrent run of tests on the same host each run creates temporary
files (/tmp/test-suite_on.<host>, see main_lock_file_list) and deletes them
afterwards.
11. Firewalls
-------------
Certain tests cut temporarily network connections by defining firewall rules.
Normally even in case of failure these connections are restored. To be sure
that no firewall rules prevent tests from running the flag
net_clear_iptables_in_prepare_phase is set to 1 in default-net.conf.
This flag leads to deletion of *all* iptable chains on the test hosts.
If you do not want this behaviour set net_clear_iptables_in_prepare_phase=0.
12. Programming hints and conventions
-------------------------------------
- all global variables should be defined and explained in a
default-<module_name>.conf file
- the names of all global variables resp. all functions should have as prefix the
module name of the default-*.conf file resp. of the <module>.sh file which they
are defined in.
- in case of an error certain cleanup functions may be called (e.g. if
a network connection is cut during a test case it should be restored).
The associotive array main_error_recovery_functions has function names as
index an function parameters as values. In lib_exit each function contained
in main_error_recovery_functions is called with it's corresponding arguments.