RepoMirrors/libabigail
1
0
Fork 0
mirror of git://sourceware.org/git/libabigail.git synced 2025-03-02 12:50:25 +00:00
Code Issues Packages Projects Releases Wiki Activity

configure: Support the optional 'big-tests' sub-directory

Browse Source 
If the 'big-tests' sub-directory (or a symbolic link to it) is present
then this patch configures it.

By default, the patch defines a new git sub-module into the
'big-tests' sub-directory.  That git sub-module is the repository
git://sourceware.org/git/libabigail-tests.  Users can now add the
--recurse-submodules option to git clone when they clone
git://sourceware.org/git/libabigail, so that it clones git big-tests
submodule as well.

The patch adds a new 'big-tests' target (to the top-most Makefile.am)
which runs "make check" in the big-tests sub-directory.

The patch also adds a new 'full-check' target that runs targets check,
check-self-compare and big-tests.

        * Makefile.am: Add big-tests,clean-big-tests and full-check
	targets.
        * configure.ac: Add an --enable-big-tests option.  For now, this
	option is mandatory to handle the optional 'big-tests' support.
	If the --enable-big-tests option is provided and if the
	'big-tests' sub-directory is present, configure the big-tests/
	sub-package.
	* .gitmodules: New sub-module configuration to get
	git://sourceware.org/git/libabigail-tests and stick into ...
	* big-tests: ... this directory as a sub-module.
	* CONTRIBUTING: Update this to explain how to get the Big Tests
	going.
	* tests/Makefile.am: Update the test summary for the
	check-self-compare target.

Signed-off-by: Dodji Seketeli <dodji@redhat.com>
This commit is contained in:
Dodji Seketeli 2024-07-04 14:19:08 +02:00
parent 5fb6df40ea
commit 3e93b0f222
6 changed files with 134 additions and 56 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
.gitmodules vendored Normal file
View File

@ -0,0 +1,3 @@
[submodule "big-tests"]
path = big-tests
url = git://sourceware.org/git/libabigail-tests.git

143
CONTRIBUTING
View File

@ -125,54 +125,100 @@ the libabigail *library* directly (as opposed to forking libabigail
command line tools) will be verified.
How tests are organized
-----------------------
=======================
There are two kinds of regression tests. Those that use the
libabigail *library* directly, and those that spawn one of the
libabigail command line tools.
There are two kinds of tests in libabigail:
* Regression tests.
Generally, both are usually made of a loop that churns through a set of input
binaries to compare. Once the comparison is done, the resulting
report is compared against a reference report that is provided.
These are in the tests/ sub-directory of the package, hosted
in the libabigail Git repository on sourceware. They are
meant to run quickly and not be too big. They are also meant
to end up in the released libabigail tarball.
Test executables have names that starts with 'runtest*'. For instance,
under <build-directory>/tests/ you can find tests named
runtestdiffdwarf, runtestabidiff, etc...
There are two kinds of regression tests. Those that use the
libabigail *library* directly, and those that spawn one of the
libabigail command line tools.
If a test executable is named
<build-directory>/tests/runtestdiffdwarf, then its source code is
tests/test-diff-dwarf.cc. Similarly, the source code of the test
<build-directory>/tests/runtestabidiff is tests/test-abidiff.cc.
Generally, both are usually made of a loop that churns through a set of input
binaries to compare. Once the comparison is done, the resulting
report is compared against a reference report that is provided.
The data provided for each test (for instance the input binaries to
compare and the reference report that should result from the
comparison) is to be found under tests/data. So data for the test
runtestdiffdwarf is to be found under tests/data/test-diff-dwarf.
Data for the test runtestabidiff is to be found under
tests/data/test-abidiff.cc.
Test executables have names that starts with 'runtest*'. For instance,
under <build-directory>/tests/ you can find tests named
runtestdiffdwarf, runtestabidiff, etc...
So adding your own tests usually just amounts to adding the right input
into the right sub-directory of tests/data/. To do so,
look at several tests/test-*.cc to see which one you'd like to add
some input binaries to be compared in.
If a test executable is named
<build-directory>/tests/runtestdiffdwarf, then its source code is
tests/test-diff-dwarf.cc. Similarly, the source code of the test
<build-directory>/tests/runtestabidiff is tests/test-abidiff.cc.
Then once you know which tests/test-*.cc you'd like to extend, and if
you added your input binaries and reference reports (maybe other
things too) to the right sub-director of tests/data/, you just need to
extend the array of input binaries/reference reports that the test
walks to perform the comparisons. It's generally a global variable
before the main() function of the test. In test-diff-dwarf.cc, for
instance, the variable name is "in_out_specs". You just have to add a
new entry to that array; that new entry contains the paths to your new
input binaries and reference reports. Just read the code in there and
use your brains. It should be straight forward.
The data provided for each test (for instance the input binaries to
compare and the reference report that should result from the
comparison) is to be found under tests/data. So data for the test
runtestdiffdwarf is to be found under tests/data/test-diff-dwarf.
Data for the test runtestabidiff is to be found under
tests/data/test-abidiff.cc.
So adding your own tests usually just amounts to adding the right input
into the right sub-directory of tests/data/. To do so,
look at several tests/test-*.cc to see which one you'd like to add
some input binaries to be compared in.
Then once you know which tests/test-*.cc you'd like to extend, and if
you added your input binaries and reference reports (maybe other
things too) to the right sub-director of tests/data/, you just need to
extend the array of input binaries/reference reports that the test
walks to perform the comparisons. It's generally a global variable
before the main() function of the test. In test-diff-dwarf.cc, for
instance, the variable name is "in_out_specs". You just have to add a
new entry to that array; that new entry contains the paths to your new
input binaries and reference reports. Just read the code in there and
use your brains. It should be straight forward.
Ah, also, if you added new files for the tests, then the build system
needs to be told that those files have to be added to the distribution
tarball when we do "make dist" (or make distcheck). To do so, please
make sure to add your new test input files to the
tests/data/Makefile.am file, in the EXTRA_DIST variable. Look at how
things are organized in that file, and please do things similarly.
* Integration tests AKA Big Tests.
These are bigger and slower tests that would be too cumbersome
to be shipped in the libabigail tarball. They are in a
separate Git repository which is at
git://sourceware.org/git/libabigail-tests.
That repository is meant to be checked out as a sub-module in
a libabigail tree. We affectionately call these tests "Big
Tests".
Suppose you've checked out a libabigail tree by doing:
$ git clone git://sourceware.org/git/libabigail
To get the integration tests do this:
$ cd libabigail/big-tests
$ git submodule init
$ git submodule update
The Git repository at
git://sourceware.org/git/libabigail-tests is now checked-out
as a sub-module in the "big-tests" sub-directory of the
libabigail tree.
To configure libabigail with the support of the Big Tests, do
this:
$ cd <build-directory>
$ /path/to/libabigail/configure --enable-big-tests
To launch the Big Tests, do this in the <build-directory>:
$ make big-tests
Ah, also, if you added new files for the tests, then the build system
needs to be told that those files have to be added to the distribution
tarball when we do "make dist" (or make distcheck). To do so, please
make sure to add your new test input files to the
tests/data/Makefile.am file, in the EXTRA_DIST variable. Look at how
things are organized in that file, and please do things similarly.
fedabipkgdiff testing
---------------------
@ -291,20 +337,9 @@ the commit log message for you.
Modifying the website
=====================
The source code of the website of libabigail is stored in CVS (sigh,
yeah, that is so old school). You can check out that web source code
by doing:
CVS_RSH=ssh cvs -z9 -d :ext:user@sourceware.org:/cvs/libabigail/ co htdocs
where 'user' is your username on the sourceware system.
Alternatively, you can check out the the web source code anonymously,
if you don't have any user account on the sourceware system by doing:
export CVSROOT=:pserver:anoncvs@cygwin.com:/cvs/libabigail
cvs login
(the CVS anonymous password to use is "anoncvs")
cvs checkout htdocs
The source code of the website of libabigail is stored in Git. You
can check out that web source code by doing:
git co git://sourceware.org/git/libabigail-htdocs.git
Happy Hacking!

9
Makefile.am
View File

@ -30,7 +30,7 @@ libabigail.pc.in
# Don't make make repeat.
AM_MAKEFLAGS = --no-print-directory
.PHONY: doc
.PHONY: doc big-tests
doc: html-doc man info
@ -94,3 +94,10 @@ distcheck-fast:
check-self-compare:
$(MAKE) -C tests check-self-compare
big-tests:
test -d big-tests && $(MAKE) -C big-tests check
clean-big-tests:
test -d big-tests && $(MAKE) -C big-tests clean
full-check: check check-self-compare big-tests

1
big-tests Submodule

@ -0,0 +1 @@
Subproject commit 8447ddfd40ee4d3dbb7b3ba8e5fae809a89334a5

32
configure.ac
View File

@ -231,6 +231,14 @@ AC_ARG_ENABLE(btf,
[disable support of btf files)]),
ENABLE_BTF=$enableval,
ENABLE_BTF=auto)
ENABLE_BIG_TESTS=no
AC_ARG_ENABLE(big-tests,
AS_HELP_STRING([--enable-big-tests=yes|no],
[enable the support of libabigail big tests (in the big-tests subdir)]),
ENABLE_BIG_TESTS=$enableval,
ENABLE_BIG_TESTS=no)
dnl *************************************************
dnl check for dependencies
dnl *************************************************
@ -457,6 +465,29 @@ dnl Test if various functions and structs are present.
fi
fi
if test x$ENABLE_BIG_TESTS = xyes; then
AC_MSG_NOTICE([looking for $srcdir/big-tests ...]);
if test -d $srcdir/big-tests -o -L $srcdir/big-tests; then
AC_CONFIG_COMMANDS([create-big-tests-build-dir-and-configure],
[AC_MSG_NOTICE([Found $srcdir/big-tests!])
cd ac_abs_top_srcdir/big-tests
autoreconf -i
cd $ac_abs_top_builddir
test -d big-tests || mkdir big-tests
cd big-tests
CTF_OPTS=
BTF_OPTS=
if test x$CTF_ENABLED = xyes; then CTF_OPTS="--enable-ctf"; fi;
if test x$BTF_ENABLED = xyes; then BTF_OPTS="--enable-btf"; fi;
AC_MSG_NOTICE([configuring $srcdir/big-tests --with-libabigail-tools-dir=$ac_abs_top_builddir/tools --with-libabigail-headers-dir=$ac_abs_top_srcdir/include $CTF_OPTS $BTF_OPTS])
$ac_abs_top_srcdir/big-tests/configure --with-libabigail-tools-dir=$ac_abs_top_builddir/tools --with-libabigail-headers-dir=$ac_abs_top_srcdir/include $CTF_OPTS $BTF_OPTS],
[CTF_ENABLED=$ENABLE_CTF
BTF_ENABLED=$ENABLE_BTF])
else
AC_MSG_ERROR([No sub-directory big-tests found. Please checkout libabigail-big-tests under a sub-directory named 'big-tests'])
fi
fi
dnl Check for dependency: libxml
LIBXML2_VERSION=2.6.22
PKG_CHECK_MODULES(XML, libxml-2.0 >= $LIBXML2_VERSION)
@ -1267,6 +1298,7 @@ AC_MSG_NOTICE([
Enable python 3 : ${ENABLE_PYTHON3}
Enable CTF front-end : ${ENABLE_CTF}
Enable BTF front-end : ${ENABLE_BTF}
Enable Big Libabigail tests : ${ENABLE_BIG_TESTS}
Enable running tests under Valgrind : ${enable_valgrind}
Enable build with -fsanitize=address : ${ENABLE_ASAN}
Enable build with -fsanitize=memory : ${ENABLE_MSAN}

2
tests/Makefile.am
View File

@ -274,5 +274,5 @@ check-valgrind-helgrind-recursive:
$(MAKE) check-valgrind-helgrind VALGRIND_FLAGS="${RECURSIVE_VALGRIND_FLAGS}"
check-self-compare:
$(MAKE) check TESTS=runtestslowselfcompare.sh ENABLE_SLOW_TEST=yes
$(MAKE) check TESTS=runtestslowselfcompare.sh ENABLE_SLOW_TEST=yes AM_TESTSUITE_SUMMARY_HEADER=' for runtestslowselfcompare.sh'