diff --git a/.gitmodules b/.gitmodules new file mode 100644 index 00000000..b8e6a0d5 --- /dev/null +++ b/.gitmodules @@ -0,0 +1,3 @@ +[submodule "big-tests"] + path = big-tests + url = git://sourceware.org/git/libabigail-tests.git diff --git a/CONTRIBUTING b/CONTRIBUTING index cbdc9843..9f653f94 100644 --- a/CONTRIBUTING +++ b/CONTRIBUTING @@ -125,54 +125,100 @@ the libabigail *library* directly (as opposed to forking libabigail command line tools) will be verified. How tests are organized ------------------------ +======================= -There are two kinds of regression tests. Those that use the -libabigail *library* directly, and those that spawn one of the -libabigail command line tools. +There are two kinds of tests in libabigail: + + * Regression tests. -Generally, both are usually made of a loop that churns through a set of input -binaries to compare. Once the comparison is done, the resulting -report is compared against a reference report that is provided. + These are in the tests/ sub-directory of the package, hosted + in the libabigail Git repository on sourceware. They are + meant to run quickly and not be too big. They are also meant + to end up in the released libabigail tarball. -Test executables have names that starts with 'runtest*'. For instance, -under /tests/ you can find tests named -runtestdiffdwarf, runtestabidiff, etc... + There are two kinds of regression tests. Those that use the + libabigail *library* directly, and those that spawn one of the + libabigail command line tools. -If a test executable is named -/tests/runtestdiffdwarf, then its source code is -tests/test-diff-dwarf.cc. Similarly, the source code of the test -/tests/runtestabidiff is tests/test-abidiff.cc. + Generally, both are usually made of a loop that churns through a set of input + binaries to compare. Once the comparison is done, the resulting + report is compared against a reference report that is provided. -The data provided for each test (for instance the input binaries to -compare and the reference report that should result from the -comparison) is to be found under tests/data. So data for the test -runtestdiffdwarf is to be found under tests/data/test-diff-dwarf. -Data for the test runtestabidiff is to be found under -tests/data/test-abidiff.cc. + Test executables have names that starts with 'runtest*'. For instance, + under /tests/ you can find tests named + runtestdiffdwarf, runtestabidiff, etc... -So adding your own tests usually just amounts to adding the right input -into the right sub-directory of tests/data/. To do so, -look at several tests/test-*.cc to see which one you'd like to add -some input binaries to be compared in. + If a test executable is named + /tests/runtestdiffdwarf, then its source code is + tests/test-diff-dwarf.cc. Similarly, the source code of the test + /tests/runtestabidiff is tests/test-abidiff.cc. -Then once you know which tests/test-*.cc you'd like to extend, and if -you added your input binaries and reference reports (maybe other -things too) to the right sub-director of tests/data/, you just need to -extend the array of input binaries/reference reports that the test -walks to perform the comparisons. It's generally a global variable -before the main() function of the test. In test-diff-dwarf.cc, for -instance, the variable name is "in_out_specs". You just have to add a -new entry to that array; that new entry contains the paths to your new -input binaries and reference reports. Just read the code in there and -use your brains. It should be straight forward. + The data provided for each test (for instance the input binaries to + compare and the reference report that should result from the + comparison) is to be found under tests/data. So data for the test + runtestdiffdwarf is to be found under tests/data/test-diff-dwarf. + Data for the test runtestabidiff is to be found under + tests/data/test-abidiff.cc. + + So adding your own tests usually just amounts to adding the right input + into the right sub-directory of tests/data/. To do so, + look at several tests/test-*.cc to see which one you'd like to add + some input binaries to be compared in. + + Then once you know which tests/test-*.cc you'd like to extend, and if + you added your input binaries and reference reports (maybe other + things too) to the right sub-director of tests/data/, you just need to + extend the array of input binaries/reference reports that the test + walks to perform the comparisons. It's generally a global variable + before the main() function of the test. In test-diff-dwarf.cc, for + instance, the variable name is "in_out_specs". You just have to add a + new entry to that array; that new entry contains the paths to your new + input binaries and reference reports. Just read the code in there and + use your brains. It should be straight forward. + + Ah, also, if you added new files for the tests, then the build system + needs to be told that those files have to be added to the distribution + tarball when we do "make dist" (or make distcheck). To do so, please + make sure to add your new test input files to the + tests/data/Makefile.am file, in the EXTRA_DIST variable. Look at how + things are organized in that file, and please do things similarly. + + * Integration tests AKA Big Tests. + + These are bigger and slower tests that would be too cumbersome + to be shipped in the libabigail tarball. They are in a + separate Git repository which is at + git://sourceware.org/git/libabigail-tests. + + That repository is meant to be checked out as a sub-module in + a libabigail tree. We affectionately call these tests "Big + Tests". + + Suppose you've checked out a libabigail tree by doing: + + $ git clone git://sourceware.org/git/libabigail + + To get the integration tests do this: + + $ cd libabigail/big-tests + $ git submodule init + $ git submodule update + + The Git repository at + git://sourceware.org/git/libabigail-tests is now checked-out + as a sub-module in the "big-tests" sub-directory of the + libabigail tree. + + To configure libabigail with the support of the Big Tests, do + this: + + $ cd + $ /path/to/libabigail/configure --enable-big-tests + + To launch the Big Tests, do this in the : + + $ make big-tests -Ah, also, if you added new files for the tests, then the build system -needs to be told that those files have to be added to the distribution -tarball when we do "make dist" (or make distcheck). To do so, please -make sure to add your new test input files to the -tests/data/Makefile.am file, in the EXTRA_DIST variable. Look at how -things are organized in that file, and please do things similarly. fedabipkgdiff testing --------------------- @@ -291,20 +337,9 @@ the commit log message for you. Modifying the website ===================== -The source code of the website of libabigail is stored in CVS (sigh, -yeah, that is so old school). You can check out that web source code -by doing: - - CVS_RSH=ssh cvs -z9 -d :ext:user@sourceware.org:/cvs/libabigail/ co htdocs - -where 'user' is your username on the sourceware system. -Alternatively, you can check out the the web source code anonymously, -if you don't have any user account on the sourceware system by doing: - - export CVSROOT=:pserver:anoncvs@cygwin.com:/cvs/libabigail - cvs login - (the CVS anonymous password to use is "anoncvs") - cvs checkout htdocs +The source code of the website of libabigail is stored in Git. You +can check out that web source code by doing: + git co git://sourceware.org/git/libabigail-htdocs.git Happy Hacking! diff --git a/Makefile.am b/Makefile.am index a0310368..72e6d1fa 100644 --- a/Makefile.am +++ b/Makefile.am @@ -30,7 +30,7 @@ libabigail.pc.in # Don't make make repeat. AM_MAKEFLAGS = --no-print-directory -.PHONY: doc +.PHONY: doc big-tests doc: html-doc man info @@ -94,3 +94,10 @@ distcheck-fast: check-self-compare: $(MAKE) -C tests check-self-compare +big-tests: + test -d big-tests && $(MAKE) -C big-tests check + +clean-big-tests: + test -d big-tests && $(MAKE) -C big-tests clean + +full-check: check check-self-compare big-tests diff --git a/big-tests b/big-tests new file mode 160000 index 00000000..8447ddfd --- /dev/null +++ b/big-tests @@ -0,0 +1 @@ +Subproject commit 8447ddfd40ee4d3dbb7b3ba8e5fae809a89334a5 diff --git a/configure.ac b/configure.ac index 5fd77821..9c8e5511 100644 --- a/configure.ac +++ b/configure.ac @@ -231,6 +231,14 @@ AC_ARG_ENABLE(btf, [disable support of btf files)]), ENABLE_BTF=$enableval, ENABLE_BTF=auto) + +ENABLE_BIG_TESTS=no +AC_ARG_ENABLE(big-tests, + AS_HELP_STRING([--enable-big-tests=yes|no], + [enable the support of libabigail big tests (in the big-tests subdir)]), + ENABLE_BIG_TESTS=$enableval, + ENABLE_BIG_TESTS=no) + dnl ************************************************* dnl check for dependencies dnl ************************************************* @@ -457,6 +465,29 @@ dnl Test if various functions and structs are present. fi fi +if test x$ENABLE_BIG_TESTS = xyes; then + AC_MSG_NOTICE([looking for $srcdir/big-tests ...]); + if test -d $srcdir/big-tests -o -L $srcdir/big-tests; then + AC_CONFIG_COMMANDS([create-big-tests-build-dir-and-configure], + [AC_MSG_NOTICE([Found $srcdir/big-tests!]) + cd ac_abs_top_srcdir/big-tests + autoreconf -i + cd $ac_abs_top_builddir + test -d big-tests || mkdir big-tests + cd big-tests + CTF_OPTS= + BTF_OPTS= + if test x$CTF_ENABLED = xyes; then CTF_OPTS="--enable-ctf"; fi; + if test x$BTF_ENABLED = xyes; then BTF_OPTS="--enable-btf"; fi; + AC_MSG_NOTICE([configuring $srcdir/big-tests --with-libabigail-tools-dir=$ac_abs_top_builddir/tools --with-libabigail-headers-dir=$ac_abs_top_srcdir/include $CTF_OPTS $BTF_OPTS]) + $ac_abs_top_srcdir/big-tests/configure --with-libabigail-tools-dir=$ac_abs_top_builddir/tools --with-libabigail-headers-dir=$ac_abs_top_srcdir/include $CTF_OPTS $BTF_OPTS], + [CTF_ENABLED=$ENABLE_CTF + BTF_ENABLED=$ENABLE_BTF]) + else + AC_MSG_ERROR([No sub-directory big-tests found. Please checkout libabigail-big-tests under a sub-directory named 'big-tests']) + fi +fi + dnl Check for dependency: libxml LIBXML2_VERSION=2.6.22 PKG_CHECK_MODULES(XML, libxml-2.0 >= $LIBXML2_VERSION) @@ -1267,6 +1298,7 @@ AC_MSG_NOTICE([ Enable python 3 : ${ENABLE_PYTHON3} Enable CTF front-end : ${ENABLE_CTF} Enable BTF front-end : ${ENABLE_BTF} + Enable Big Libabigail tests : ${ENABLE_BIG_TESTS} Enable running tests under Valgrind : ${enable_valgrind} Enable build with -fsanitize=address : ${ENABLE_ASAN} Enable build with -fsanitize=memory : ${ENABLE_MSAN} diff --git a/tests/Makefile.am b/tests/Makefile.am index 141c8768..8204bdc5 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -274,5 +274,5 @@ check-valgrind-helgrind-recursive: $(MAKE) check-valgrind-helgrind VALGRIND_FLAGS="${RECURSIVE_VALGRIND_FLAGS}" check-self-compare: - $(MAKE) check TESTS=runtestslowselfcompare.sh ENABLE_SLOW_TEST=yes + $(MAKE) check TESTS=runtestslowselfcompare.sh ENABLE_SLOW_TEST=yes AM_TESTSUITE_SUMMARY_HEADER=' for runtestslowselfcompare.sh'