configure: Support the optional 'big-tests' sub-directory
If the 'big-tests' sub-directory (or a symbolic link to it) is present
then this patch configures it.
By default, the patch defines a new git sub-module into the
'big-tests' sub-directory. That git sub-module is the repository
git://sourceware.org/git/libabigail-tests. Users can now add the
--recurse-submodules option to git clone when they clone
git://sourceware.org/git/libabigail, so that it clones git big-tests
submodule as well.
The patch adds a new 'big-tests' target (to the top-most Makefile.am)
which runs "make check" in the big-tests sub-directory.
The patch also adds a new 'full-check' target that runs targets check,
check-self-compare and big-tests.
* Makefile.am: Add big-tests,clean-big-tests and full-check
targets.
* configure.ac: Add an --enable-big-tests option. For now, this
option is mandatory to handle the optional 'big-tests' support.
If the --enable-big-tests option is provided and if the
'big-tests' sub-directory is present, configure the big-tests/
sub-package.
* .gitmodules: New sub-module configuration to get
git://sourceware.org/git/libabigail-tests and stick into ...
* big-tests: ... this directory as a sub-module.
* CONTRIBUTING: Update this to explain how to get the Big Tests
going.
* tests/Makefile.am: Update the test summary for the
check-self-compare target.
Signed-off-by: Dodji Seketeli <dodji@redhat.com>
This commit is contained in:
Dodji Seketeli
parent
5fb6df40ea
commit
3e93b0f222
|
|
@ -0,0 +1,3 @@
|
|||
[submodule "big-tests"]
|
||||
path = big-tests
|
||||
url = git://sourceware.org/git/libabigail-tests.git
|
||||
143
CONTRIBUTING
143
CONTRIBUTING
|
|
@ -125,54 +125,100 @@ the libabigail *library* directly (as opposed to forking libabigail
|
|||
command line tools) will be verified.
|
||||
|
||||
How tests are organized
|
||||
-----------------------
|
||||
=======================
|
||||
|
||||
There are two kinds of regression tests. Those that use the
|
||||
libabigail *library* directly, and those that spawn one of the
|
||||
libabigail command line tools.
|
||||
There are two kinds of tests in libabigail:
|
||||
|
||||
Generally, both are usually made of a loop that churns through a set of input
|
||||
binaries to compare. Once the comparison is done, the resulting
|
||||
report is compared against a reference report that is provided.
|
||||
* Regression tests.
|
||||
|
||||
Test executables have names that starts with 'runtest*'. For instance,
|
||||
under <build-directory>/tests/ you can find tests named
|
||||
runtestdiffdwarf, runtestabidiff, etc...
|
||||
These are in the tests/ sub-directory of the package, hosted
|
||||
in the libabigail Git repository on sourceware. They are
|
||||
meant to run quickly and not be too big. They are also meant
|
||||
to end up in the released libabigail tarball.
|
||||
|
||||
If a test executable is named
|
||||
<build-directory>/tests/runtestdiffdwarf, then its source code is
|
||||
tests/test-diff-dwarf.cc. Similarly, the source code of the test
|
||||
<build-directory>/tests/runtestabidiff is tests/test-abidiff.cc.
|
||||
There are two kinds of regression tests. Those that use the
|
||||
libabigail *library* directly, and those that spawn one of the
|
||||
libabigail command line tools.
|
||||
|
||||
The data provided for each test (for instance the input binaries to
|
||||
compare and the reference report that should result from the
|
||||
comparison) is to be found under tests/data. So data for the test
|
||||
runtestdiffdwarf is to be found under tests/data/test-diff-dwarf.
|
||||
Data for the test runtestabidiff is to be found under
|
||||
tests/data/test-abidiff.cc.
|
||||
Generally, both are usually made of a loop that churns through a set of input
|
||||
binaries to compare. Once the comparison is done, the resulting
|
||||
report is compared against a reference report that is provided.
|
||||
|
||||
So adding your own tests usually just amounts to adding the right input
|
||||
into the right sub-directory of tests/data/. To do so,
|
||||
look at several tests/test-*.cc to see which one you'd like to add
|
||||
some input binaries to be compared in.
|
||||
Test executables have names that starts with 'runtest*'. For instance,
|
||||
under <build-directory>/tests/ you can find tests named
|
||||
runtestdiffdwarf, runtestabidiff, etc...
|
||||
|
||||
Then once you know which tests/test-*.cc you'd like to extend, and if
|
||||
you added your input binaries and reference reports (maybe other
|
||||
things too) to the right sub-director of tests/data/, you just need to
|
||||
extend the array of input binaries/reference reports that the test
|
||||
walks to perform the comparisons. It's generally a global variable
|
||||
before the main() function of the test. In test-diff-dwarf.cc, for
|
||||
instance, the variable name is "in_out_specs". You just have to add a
|
||||
new entry to that array; that new entry contains the paths to your new
|
||||
input binaries and reference reports. Just read the code in there and
|
||||
use your brains. It should be straight forward.
|
||||
If a test executable is named
|
||||
<build-directory>/tests/runtestdiffdwarf, then its source code is
|
||||
tests/test-diff-dwarf.cc. Similarly, the source code of the test
|
||||
<build-directory>/tests/runtestabidiff is tests/test-abidiff.cc.
|
||||
|
||||
The data provided for each test (for instance the input binaries to
|
||||
compare and the reference report that should result from the
|
||||
comparison) is to be found under tests/data. So data for the test
|
||||
runtestdiffdwarf is to be found under tests/data/test-diff-dwarf.
|
||||
Data for the test runtestabidiff is to be found under
|
||||
tests/data/test-abidiff.cc.
|
||||
|
||||
So adding your own tests usually just amounts to adding the right input
|
||||
into the right sub-directory of tests/data/. To do so,
|
||||
look at several tests/test-*.cc to see which one you'd like to add
|
||||
some input binaries to be compared in.
|
||||
|
||||
Then once you know which tests/test-*.cc you'd like to extend, and if
|
||||
you added your input binaries and reference reports (maybe other
|
||||
things too) to the right sub-director of tests/data/, you just need to
|
||||
extend the array of input binaries/reference reports that the test
|
||||
walks to perform the comparisons. It's generally a global variable
|
||||
before the main() function of the test. In test-diff-dwarf.cc, for
|
||||
instance, the variable name is "in_out_specs". You just have to add a
|
||||
new entry to that array; that new entry contains the paths to your new
|
||||
input binaries and reference reports. Just read the code in there and
|
||||
use your brains. It should be straight forward.
|
||||
|
||||
Ah, also, if you added new files for the tests, then the build system
|
||||
needs to be told that those files have to be added to the distribution
|
||||
tarball when we do "make dist" (or make distcheck). To do so, please
|
||||
make sure to add your new test input files to the
|
||||
tests/data/Makefile.am file, in the EXTRA_DIST variable. Look at how
|
||||
things are organized in that file, and please do things similarly.
|
||||
|
||||
* Integration tests AKA Big Tests.
|
||||
|
||||
These are bigger and slower tests that would be too cumbersome
|
||||
to be shipped in the libabigail tarball. They are in a
|
||||
separate Git repository which is at
|
||||
git://sourceware.org/git/libabigail-tests.
|
||||
|
||||
That repository is meant to be checked out as a sub-module in
|
||||
a libabigail tree. We affectionately call these tests "Big
|
||||
Tests".
|
||||
|
||||
Suppose you've checked out a libabigail tree by doing:
|
||||
|
||||
$ git clone git://sourceware.org/git/libabigail
|
||||
|
||||
To get the integration tests do this:
|
||||
|
||||
$ cd libabigail/big-tests
|
||||
$ git submodule init
|
||||
$ git submodule update
|
||||
|
||||
The Git repository at
|
||||
git://sourceware.org/git/libabigail-tests is now checked-out
|
||||
as a sub-module in the "big-tests" sub-directory of the
|
||||
libabigail tree.
|
||||
|
||||
To configure libabigail with the support of the Big Tests, do
|
||||
this:
|
||||
|
||||
$ cd <build-directory>
|
||||
$ /path/to/libabigail/configure --enable-big-tests
|
||||
|
||||
To launch the Big Tests, do this in the <build-directory>:
|
||||
|
||||
$ make big-tests
|
||||
|
||||
Ah, also, if you added new files for the tests, then the build system
|
||||
needs to be told that those files have to be added to the distribution
|
||||
tarball when we do "make dist" (or make distcheck). To do so, please
|
||||
make sure to add your new test input files to the
|
||||
tests/data/Makefile.am file, in the EXTRA_DIST variable. Look at how
|
||||
things are organized in that file, and please do things similarly.
|
||||
|
||||
fedabipkgdiff testing
|
||||
---------------------
|
||||
|
|
@ -291,20 +337,9 @@ the commit log message for you.
|
|||
Modifying the website
|
||||
=====================
|
||||
|
||||
The source code of the website of libabigail is stored in CVS (sigh,
|
||||
yeah, that is so old school). You can check out that web source code
|
||||
by doing:
|
||||
|
||||
CVS_RSH=ssh cvs -z9 -d :ext:user@sourceware.org:/cvs/libabigail/ co htdocs
|
||||
|
||||
where 'user' is your username on the sourceware system.
|
||||
Alternatively, you can check out the the web source code anonymously,
|
||||
if you don't have any user account on the sourceware system by doing:
|
||||
|
||||
export CVSROOT=:pserver:anoncvs@cygwin.com:/cvs/libabigail
|
||||
cvs login
|
||||
(the CVS anonymous password to use is "anoncvs")
|
||||
cvs checkout htdocs
|
||||
The source code of the website of libabigail is stored in Git. You
|
||||
can check out that web source code by doing:
|
||||
|
||||
git co git://sourceware.org/git/libabigail-htdocs.git
|
||||
|
||||
Happy Hacking!
|
||||
|
|
|
|||
|
|
@ -30,7 +30,7 @@ libabigail.pc.in
|
|||
# Don't make make repeat.
|
||||
AM_MAKEFLAGS = --no-print-directory
|
||||
|
||||
.PHONY: doc
|
||||
.PHONY: doc big-tests
|
||||
|
||||
doc: html-doc man info
|
||||
|
||||
|
|
@ -94,3 +94,10 @@ distcheck-fast:
|
|||
check-self-compare:
|
||||
$(MAKE) -C tests check-self-compare
|
||||
|
||||
big-tests:
|
||||
test -d big-tests && $(MAKE) -C big-tests check
|
||||
|
||||
clean-big-tests:
|
||||
test -d big-tests && $(MAKE) -C big-tests clean
|
||||
|
||||
full-check: check check-self-compare big-tests
|
||||
|
|
|
|||
|
|
@ -0,0 +1 @@
|
|||
Subproject commit 8447ddfd40ee4d3dbb7b3ba8e5fae809a89334a5
|
||||
32
configure.ac
32
configure.ac
|
|
@ -231,6 +231,14 @@ AC_ARG_ENABLE(btf,
|
|||
[disable support of btf files)]),
|
||||
ENABLE_BTF=$enableval,
|
||||
ENABLE_BTF=auto)
|
||||
|
||||
ENABLE_BIG_TESTS=no
|
||||
AC_ARG_ENABLE(big-tests,
|
||||
AS_HELP_STRING([--enable-big-tests=yes|no],
|
||||
[enable the support of libabigail big tests (in the big-tests subdir)]),
|
||||
ENABLE_BIG_TESTS=$enableval,
|
||||
ENABLE_BIG_TESTS=no)
|
||||
|
||||
dnl *************************************************
|
||||
dnl check for dependencies
|
||||
dnl *************************************************
|
||||
|
|
@ -457,6 +465,29 @@ dnl Test if various functions and structs are present.
|
|||
fi
|
||||
fi
|
||||
|
||||
if test x$ENABLE_BIG_TESTS = xyes; then
|
||||
AC_MSG_NOTICE([looking for $srcdir/big-tests ...]);
|
||||
if test -d $srcdir/big-tests -o -L $srcdir/big-tests; then
|
||||
AC_CONFIG_COMMANDS([create-big-tests-build-dir-and-configure],
|
||||
[AC_MSG_NOTICE([Found $srcdir/big-tests!])
|
||||
cd ac_abs_top_srcdir/big-tests
|
||||
autoreconf -i
|
||||
cd $ac_abs_top_builddir
|
||||
test -d big-tests || mkdir big-tests
|
||||
cd big-tests
|
||||
CTF_OPTS=
|
||||
BTF_OPTS=
|
||||
if test x$CTF_ENABLED = xyes; then CTF_OPTS="--enable-ctf"; fi;
|
||||
if test x$BTF_ENABLED = xyes; then BTF_OPTS="--enable-btf"; fi;
|
||||
AC_MSG_NOTICE([configuring $srcdir/big-tests --with-libabigail-tools-dir=$ac_abs_top_builddir/tools --with-libabigail-headers-dir=$ac_abs_top_srcdir/include $CTF_OPTS $BTF_OPTS])
|
||||
$ac_abs_top_srcdir/big-tests/configure --with-libabigail-tools-dir=$ac_abs_top_builddir/tools --with-libabigail-headers-dir=$ac_abs_top_srcdir/include $CTF_OPTS $BTF_OPTS],
|
||||
[CTF_ENABLED=$ENABLE_CTF
|
||||
BTF_ENABLED=$ENABLE_BTF])
|
||||
else
|
||||
AC_MSG_ERROR([No sub-directory big-tests found. Please checkout libabigail-big-tests under a sub-directory named 'big-tests'])
|
||||
fi
|
||||
fi
|
||||
|
||||
dnl Check for dependency: libxml
|
||||
LIBXML2_VERSION=2.6.22
|
||||
PKG_CHECK_MODULES(XML, libxml-2.0 >= $LIBXML2_VERSION)
|
||||
|
|
@ -1267,6 +1298,7 @@ AC_MSG_NOTICE([
|
|||
Enable python 3 : ${ENABLE_PYTHON3}
|
||||
Enable CTF front-end : ${ENABLE_CTF}
|
||||
Enable BTF front-end : ${ENABLE_BTF}
|
||||
Enable Big Libabigail tests : ${ENABLE_BIG_TESTS}
|
||||
Enable running tests under Valgrind : ${enable_valgrind}
|
||||
Enable build with -fsanitize=address : ${ENABLE_ASAN}
|
||||
Enable build with -fsanitize=memory : ${ENABLE_MSAN}
|
||||
|
|
|
|||
|
|
@ -274,5 +274,5 @@ check-valgrind-helgrind-recursive:
|
|||
$(MAKE) check-valgrind-helgrind VALGRIND_FLAGS="${RECURSIVE_VALGRIND_FLAGS}"
|
||||
|
||||
check-self-compare:
|
||||
$(MAKE) check TESTS=runtestslowselfcompare.sh ENABLE_SLOW_TEST=yes
|
||||
$(MAKE) check TESTS=runtestslowselfcompare.sh ENABLE_SLOW_TEST=yes AM_TESTSUITE_SUMMARY_HEADER=' for runtestslowselfcompare.sh'
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue