configure: Support the optional 'big-tests' sub-directory
If the 'big-tests' sub-directory (or a symbolic link to it) is present then this patch configures it. By default, the patch defines a new git sub-module into the 'big-tests' sub-directory. That git sub-module is the repository git://sourceware.org/git/libabigail-tests. Users can now add the --recurse-submodules option to git clone when they clone git://sourceware.org/git/libabigail, so that it clones git big-tests submodule as well. The patch adds a new 'big-tests' target (to the top-most Makefile.am) which runs "make check" in the big-tests sub-directory. The patch also adds a new 'full-check' target that runs targets check, check-self-compare and big-tests. * Makefile.am: Add big-tests,clean-big-tests and full-check targets. * configure.ac: Add an --enable-big-tests option. For now, this option is mandatory to handle the optional 'big-tests' support. If the --enable-big-tests option is provided and if the 'big-tests' sub-directory is present, configure the big-tests/ sub-package. * .gitmodules: New sub-module configuration to get git://sourceware.org/git/libabigail-tests and stick into ... * big-tests: ... this directory as a sub-module. * CONTRIBUTING: Update this to explain how to get the Big Tests going. * tests/Makefile.am: Update the test summary for the check-self-compare target. Signed-off-by: Dodji Seketeli <dodji@redhat.com>
This commit is contained in:
parent
5fb6df40ea
commit
3e93b0f222
|
@ -0,0 +1,3 @@
|
||||||
|
[submodule "big-tests"]
|
||||||
|
path = big-tests
|
||||||
|
url = git://sourceware.org/git/libabigail-tests.git
|
143
CONTRIBUTING
143
CONTRIBUTING
|
@ -125,54 +125,100 @@ the libabigail *library* directly (as opposed to forking libabigail
|
||||||
command line tools) will be verified.
|
command line tools) will be verified.
|
||||||
|
|
||||||
How tests are organized
|
How tests are organized
|
||||||
-----------------------
|
=======================
|
||||||
|
|
||||||
There are two kinds of regression tests. Those that use the
|
There are two kinds of tests in libabigail:
|
||||||
libabigail *library* directly, and those that spawn one of the
|
|
||||||
libabigail command line tools.
|
|
||||||
|
|
||||||
Generally, both are usually made of a loop that churns through a set of input
|
* Regression tests.
|
||||||
binaries to compare. Once the comparison is done, the resulting
|
|
||||||
report is compared against a reference report that is provided.
|
|
||||||
|
|
||||||
Test executables have names that starts with 'runtest*'. For instance,
|
These are in the tests/ sub-directory of the package, hosted
|
||||||
under <build-directory>/tests/ you can find tests named
|
in the libabigail Git repository on sourceware. They are
|
||||||
runtestdiffdwarf, runtestabidiff, etc...
|
meant to run quickly and not be too big. They are also meant
|
||||||
|
to end up in the released libabigail tarball.
|
||||||
|
|
||||||
If a test executable is named
|
There are two kinds of regression tests. Those that use the
|
||||||
<build-directory>/tests/runtestdiffdwarf, then its source code is
|
libabigail *library* directly, and those that spawn one of the
|
||||||
tests/test-diff-dwarf.cc. Similarly, the source code of the test
|
libabigail command line tools.
|
||||||
<build-directory>/tests/runtestabidiff is tests/test-abidiff.cc.
|
|
||||||
|
|
||||||
The data provided for each test (for instance the input binaries to
|
Generally, both are usually made of a loop that churns through a set of input
|
||||||
compare and the reference report that should result from the
|
binaries to compare. Once the comparison is done, the resulting
|
||||||
comparison) is to be found under tests/data. So data for the test
|
report is compared against a reference report that is provided.
|
||||||
runtestdiffdwarf is to be found under tests/data/test-diff-dwarf.
|
|
||||||
Data for the test runtestabidiff is to be found under
|
|
||||||
tests/data/test-abidiff.cc.
|
|
||||||
|
|
||||||
So adding your own tests usually just amounts to adding the right input
|
Test executables have names that starts with 'runtest*'. For instance,
|
||||||
into the right sub-directory of tests/data/. To do so,
|
under <build-directory>/tests/ you can find tests named
|
||||||
look at several tests/test-*.cc to see which one you'd like to add
|
runtestdiffdwarf, runtestabidiff, etc...
|
||||||
some input binaries to be compared in.
|
|
||||||
|
|
||||||
Then once you know which tests/test-*.cc you'd like to extend, and if
|
If a test executable is named
|
||||||
you added your input binaries and reference reports (maybe other
|
<build-directory>/tests/runtestdiffdwarf, then its source code is
|
||||||
things too) to the right sub-director of tests/data/, you just need to
|
tests/test-diff-dwarf.cc. Similarly, the source code of the test
|
||||||
extend the array of input binaries/reference reports that the test
|
<build-directory>/tests/runtestabidiff is tests/test-abidiff.cc.
|
||||||
walks to perform the comparisons. It's generally a global variable
|
|
||||||
before the main() function of the test. In test-diff-dwarf.cc, for
|
The data provided for each test (for instance the input binaries to
|
||||||
instance, the variable name is "in_out_specs". You just have to add a
|
compare and the reference report that should result from the
|
||||||
new entry to that array; that new entry contains the paths to your new
|
comparison) is to be found under tests/data. So data for the test
|
||||||
input binaries and reference reports. Just read the code in there and
|
runtestdiffdwarf is to be found under tests/data/test-diff-dwarf.
|
||||||
use your brains. It should be straight forward.
|
Data for the test runtestabidiff is to be found under
|
||||||
|
tests/data/test-abidiff.cc.
|
||||||
|
|
||||||
|
So adding your own tests usually just amounts to adding the right input
|
||||||
|
into the right sub-directory of tests/data/. To do so,
|
||||||
|
look at several tests/test-*.cc to see which one you'd like to add
|
||||||
|
some input binaries to be compared in.
|
||||||
|
|
||||||
|
Then once you know which tests/test-*.cc you'd like to extend, and if
|
||||||
|
you added your input binaries and reference reports (maybe other
|
||||||
|
things too) to the right sub-director of tests/data/, you just need to
|
||||||
|
extend the array of input binaries/reference reports that the test
|
||||||
|
walks to perform the comparisons. It's generally a global variable
|
||||||
|
before the main() function of the test. In test-diff-dwarf.cc, for
|
||||||
|
instance, the variable name is "in_out_specs". You just have to add a
|
||||||
|
new entry to that array; that new entry contains the paths to your new
|
||||||
|
input binaries and reference reports. Just read the code in there and
|
||||||
|
use your brains. It should be straight forward.
|
||||||
|
|
||||||
|
Ah, also, if you added new files for the tests, then the build system
|
||||||
|
needs to be told that those files have to be added to the distribution
|
||||||
|
tarball when we do "make dist" (or make distcheck). To do so, please
|
||||||
|
make sure to add your new test input files to the
|
||||||
|
tests/data/Makefile.am file, in the EXTRA_DIST variable. Look at how
|
||||||
|
things are organized in that file, and please do things similarly.
|
||||||
|
|
||||||
|
* Integration tests AKA Big Tests.
|
||||||
|
|
||||||
|
These are bigger and slower tests that would be too cumbersome
|
||||||
|
to be shipped in the libabigail tarball. They are in a
|
||||||
|
separate Git repository which is at
|
||||||
|
git://sourceware.org/git/libabigail-tests.
|
||||||
|
|
||||||
|
That repository is meant to be checked out as a sub-module in
|
||||||
|
a libabigail tree. We affectionately call these tests "Big
|
||||||
|
Tests".
|
||||||
|
|
||||||
|
Suppose you've checked out a libabigail tree by doing:
|
||||||
|
|
||||||
|
$ git clone git://sourceware.org/git/libabigail
|
||||||
|
|
||||||
|
To get the integration tests do this:
|
||||||
|
|
||||||
|
$ cd libabigail/big-tests
|
||||||
|
$ git submodule init
|
||||||
|
$ git submodule update
|
||||||
|
|
||||||
|
The Git repository at
|
||||||
|
git://sourceware.org/git/libabigail-tests is now checked-out
|
||||||
|
as a sub-module in the "big-tests" sub-directory of the
|
||||||
|
libabigail tree.
|
||||||
|
|
||||||
|
To configure libabigail with the support of the Big Tests, do
|
||||||
|
this:
|
||||||
|
|
||||||
|
$ cd <build-directory>
|
||||||
|
$ /path/to/libabigail/configure --enable-big-tests
|
||||||
|
|
||||||
|
To launch the Big Tests, do this in the <build-directory>:
|
||||||
|
|
||||||
|
$ make big-tests
|
||||||
|
|
||||||
Ah, also, if you added new files for the tests, then the build system
|
|
||||||
needs to be told that those files have to be added to the distribution
|
|
||||||
tarball when we do "make dist" (or make distcheck). To do so, please
|
|
||||||
make sure to add your new test input files to the
|
|
||||||
tests/data/Makefile.am file, in the EXTRA_DIST variable. Look at how
|
|
||||||
things are organized in that file, and please do things similarly.
|
|
||||||
|
|
||||||
fedabipkgdiff testing
|
fedabipkgdiff testing
|
||||||
---------------------
|
---------------------
|
||||||
|
@ -291,20 +337,9 @@ the commit log message for you.
|
||||||
Modifying the website
|
Modifying the website
|
||||||
=====================
|
=====================
|
||||||
|
|
||||||
The source code of the website of libabigail is stored in CVS (sigh,
|
The source code of the website of libabigail is stored in Git. You
|
||||||
yeah, that is so old school). You can check out that web source code
|
can check out that web source code by doing:
|
||||||
by doing:
|
|
||||||
|
|
||||||
CVS_RSH=ssh cvs -z9 -d :ext:user@sourceware.org:/cvs/libabigail/ co htdocs
|
|
||||||
|
|
||||||
where 'user' is your username on the sourceware system.
|
|
||||||
Alternatively, you can check out the the web source code anonymously,
|
|
||||||
if you don't have any user account on the sourceware system by doing:
|
|
||||||
|
|
||||||
export CVSROOT=:pserver:anoncvs@cygwin.com:/cvs/libabigail
|
|
||||||
cvs login
|
|
||||||
(the CVS anonymous password to use is "anoncvs")
|
|
||||||
cvs checkout htdocs
|
|
||||||
|
|
||||||
|
git co git://sourceware.org/git/libabigail-htdocs.git
|
||||||
|
|
||||||
Happy Hacking!
|
Happy Hacking!
|
||||||
|
|
|
@ -30,7 +30,7 @@ libabigail.pc.in
|
||||||
# Don't make make repeat.
|
# Don't make make repeat.
|
||||||
AM_MAKEFLAGS = --no-print-directory
|
AM_MAKEFLAGS = --no-print-directory
|
||||||
|
|
||||||
.PHONY: doc
|
.PHONY: doc big-tests
|
||||||
|
|
||||||
doc: html-doc man info
|
doc: html-doc man info
|
||||||
|
|
||||||
|
@ -94,3 +94,10 @@ distcheck-fast:
|
||||||
check-self-compare:
|
check-self-compare:
|
||||||
$(MAKE) -C tests check-self-compare
|
$(MAKE) -C tests check-self-compare
|
||||||
|
|
||||||
|
big-tests:
|
||||||
|
test -d big-tests && $(MAKE) -C big-tests check
|
||||||
|
|
||||||
|
clean-big-tests:
|
||||||
|
test -d big-tests && $(MAKE) -C big-tests clean
|
||||||
|
|
||||||
|
full-check: check check-self-compare big-tests
|
||||||
|
|
|
@ -0,0 +1 @@
|
||||||
|
Subproject commit 8447ddfd40ee4d3dbb7b3ba8e5fae809a89334a5
|
32
configure.ac
32
configure.ac
|
@ -231,6 +231,14 @@ AC_ARG_ENABLE(btf,
|
||||||
[disable support of btf files)]),
|
[disable support of btf files)]),
|
||||||
ENABLE_BTF=$enableval,
|
ENABLE_BTF=$enableval,
|
||||||
ENABLE_BTF=auto)
|
ENABLE_BTF=auto)
|
||||||
|
|
||||||
|
ENABLE_BIG_TESTS=no
|
||||||
|
AC_ARG_ENABLE(big-tests,
|
||||||
|
AS_HELP_STRING([--enable-big-tests=yes|no],
|
||||||
|
[enable the support of libabigail big tests (in the big-tests subdir)]),
|
||||||
|
ENABLE_BIG_TESTS=$enableval,
|
||||||
|
ENABLE_BIG_TESTS=no)
|
||||||
|
|
||||||
dnl *************************************************
|
dnl *************************************************
|
||||||
dnl check for dependencies
|
dnl check for dependencies
|
||||||
dnl *************************************************
|
dnl *************************************************
|
||||||
|
@ -457,6 +465,29 @@ dnl Test if various functions and structs are present.
|
||||||
fi
|
fi
|
||||||
fi
|
fi
|
||||||
|
|
||||||
|
if test x$ENABLE_BIG_TESTS = xyes; then
|
||||||
|
AC_MSG_NOTICE([looking for $srcdir/big-tests ...]);
|
||||||
|
if test -d $srcdir/big-tests -o -L $srcdir/big-tests; then
|
||||||
|
AC_CONFIG_COMMANDS([create-big-tests-build-dir-and-configure],
|
||||||
|
[AC_MSG_NOTICE([Found $srcdir/big-tests!])
|
||||||
|
cd ac_abs_top_srcdir/big-tests
|
||||||
|
autoreconf -i
|
||||||
|
cd $ac_abs_top_builddir
|
||||||
|
test -d big-tests || mkdir big-tests
|
||||||
|
cd big-tests
|
||||||
|
CTF_OPTS=
|
||||||
|
BTF_OPTS=
|
||||||
|
if test x$CTF_ENABLED = xyes; then CTF_OPTS="--enable-ctf"; fi;
|
||||||
|
if test x$BTF_ENABLED = xyes; then BTF_OPTS="--enable-btf"; fi;
|
||||||
|
AC_MSG_NOTICE([configuring $srcdir/big-tests --with-libabigail-tools-dir=$ac_abs_top_builddir/tools --with-libabigail-headers-dir=$ac_abs_top_srcdir/include $CTF_OPTS $BTF_OPTS])
|
||||||
|
$ac_abs_top_srcdir/big-tests/configure --with-libabigail-tools-dir=$ac_abs_top_builddir/tools --with-libabigail-headers-dir=$ac_abs_top_srcdir/include $CTF_OPTS $BTF_OPTS],
|
||||||
|
[CTF_ENABLED=$ENABLE_CTF
|
||||||
|
BTF_ENABLED=$ENABLE_BTF])
|
||||||
|
else
|
||||||
|
AC_MSG_ERROR([No sub-directory big-tests found. Please checkout libabigail-big-tests under a sub-directory named 'big-tests'])
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
dnl Check for dependency: libxml
|
dnl Check for dependency: libxml
|
||||||
LIBXML2_VERSION=2.6.22
|
LIBXML2_VERSION=2.6.22
|
||||||
PKG_CHECK_MODULES(XML, libxml-2.0 >= $LIBXML2_VERSION)
|
PKG_CHECK_MODULES(XML, libxml-2.0 >= $LIBXML2_VERSION)
|
||||||
|
@ -1267,6 +1298,7 @@ AC_MSG_NOTICE([
|
||||||
Enable python 3 : ${ENABLE_PYTHON3}
|
Enable python 3 : ${ENABLE_PYTHON3}
|
||||||
Enable CTF front-end : ${ENABLE_CTF}
|
Enable CTF front-end : ${ENABLE_CTF}
|
||||||
Enable BTF front-end : ${ENABLE_BTF}
|
Enable BTF front-end : ${ENABLE_BTF}
|
||||||
|
Enable Big Libabigail tests : ${ENABLE_BIG_TESTS}
|
||||||
Enable running tests under Valgrind : ${enable_valgrind}
|
Enable running tests under Valgrind : ${enable_valgrind}
|
||||||
Enable build with -fsanitize=address : ${ENABLE_ASAN}
|
Enable build with -fsanitize=address : ${ENABLE_ASAN}
|
||||||
Enable build with -fsanitize=memory : ${ENABLE_MSAN}
|
Enable build with -fsanitize=memory : ${ENABLE_MSAN}
|
||||||
|
|
|
@ -274,5 +274,5 @@ check-valgrind-helgrind-recursive:
|
||||||
$(MAKE) check-valgrind-helgrind VALGRIND_FLAGS="${RECURSIVE_VALGRIND_FLAGS}"
|
$(MAKE) check-valgrind-helgrind VALGRIND_FLAGS="${RECURSIVE_VALGRIND_FLAGS}"
|
||||||
|
|
||||||
check-self-compare:
|
check-self-compare:
|
||||||
$(MAKE) check TESTS=runtestslowselfcompare.sh ENABLE_SLOW_TEST=yes
|
$(MAKE) check TESTS=runtestslowselfcompare.sh ENABLE_SLOW_TEST=yes AM_TESTSUITE_SUMMARY_HEADER=' for runtestslowselfcompare.sh'
|
||||||
|
|
||||||
|
|
Loading…
Reference in New Issue