Make fedabipkgdiff consistent with Libabigail's other tests
In the libabigail project, almost all the tests for the tools follow a
similar pattern. The test contains a set of input files to compare.
The result of the comparison is thus compared to a set of reference
comparison result.
This approach is already documented in the CONTRIBUTING file.
There are several interesting properties with this scheme. First, it
capture the behaviour of the tools, including what is shown to the
user.
Second, it eases the job of a hacker who wants to add a new test for a
new behaviour of a given tool. The user just has to provide:
1/ A new reference output of the new use case of the tool (that is easily
constructed by using the tool itself and saving its output) and add an
entry to array of entries that describe what to compare
2/ A new set of inputs to the tool
And voila.
Unfortunately, fedabipkgdiff tests don't follow this scheme. That
make them surprising to hackers who read the source code of the
existing tests, at very least. Also, the fedabipkgdiff tests were
only unit tests. They were not testing the tool as used by users in
general.
This patch makes the fedabipkgdiff tests follow the general approach
of the tests of the other Libabigail tools.
The patch first craetes a program names tests/mockfedabipkgdiff. It's
a wrapper around tools/fedabipkgdiff. It overloads the Koji client of
fedabipkgdiff with a fake Koji client that gets the Fedora packages
locally, from tests/data/test-fedabipkgdiff/packages. In other words,
mockfedabipkgdiff is fedabipkgdiff without going to the network.
I believe that in the future, tests/mockfedabipkgdiff should be killed
when fedabipkgdiff is able to cache a local description of a local
partial view of a Koji repository, along with the build packages that
were already downloaded.
That way, calling fedabipkgdiff twice with the same set of option
should make the tool perform correctly without going to the netword on
the second invocation. We should be able to save the local partial
view of the Koji repository under e.g,
tests/data/test-fedabipkgdiff/local-koji and tell fedabipkgdiff to use
that, instead of using the network. But we are not there yet. So for
now, I am using mockfedabipkgdiff.
Then, tests/runtestfedabipkdiff.py.in has been re-written to use
tests/mockfedabipkgdiff and to look like what is described in
CONTRIBUTING as far as how Libabigail tools' tests are organized:
mockfedabipkgdiff is called to perform a comparison. The result of
the comparison is then compared (using GNU diff) to a reference
comparison result file.
Please note that tests/runtestfedabipkdiff.py is relatively fast for
the moment. But when it contains much more tests and start becoming
slow, then we'll need to change the code to run several comparisons in
parallel, just like we do today in tests/test-diff-filter.cc. At that
point, I believe I'll just re-write this in C++, just like
tests/test-diff-filter.cc because that will allow us to have true
concurrent code based on the abigail:workers API. For now, I am
keeping this in Python also because I think that keeps Chenxiong happy
;-)
To be sure that fedabipkgdiff (and its underlying abipkgdiff) are
really comparing all the packages they are supposed to compare and
also all the binaries in those packages, I have added a new
--show-identical-binaries to abipkgdiff and fedabipkgdiff. That
option shows the name of the binaries (and packages) that are
compared, *even if* the ABI of those binaries are identical. Because
otherwise, today, if two binaries (or packages) have the same ABI,
nothing is displayed.
For instance, here is the result of comparing a package against
itself, using this new --show-identical-binaries options:
dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ./build/tools/abipkgdiff --show-identical-binaries dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm
Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm:
================ changes of 'dbus-binding-tool'===============
No ABI change detected
================ end of changes of 'dbus-binding-tool'===============
================ changes of 'libdbus-glib-1.so.2.1.0'===============
No ABI change detected
================ end of changes of 'libdbus-glib-1.so.2.1.0'===============
dodji@adjoa:patches$
And here is what this command before that patch would do:
dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ../master/build/tools/abipkgdiff dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm
Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm:
dodji@adjoa:patches$
The rest of the patch is mostly new test inputs material and the
necessary adjustments following all the changes above.
* configure.ac: Do not require Python dependencies itertools,
unittest and StringIO anymore as they are not used anymore.
Require new module tempfile now. Generate new executable script
tests/mockfedabipkgdiff from tests/mockfedabipkgdiff.in.
* doc/manuals/abipkgdiff.rst: Add doc for new option
--show-identical-binaries to abipkgdiff
* doc/manuals/fedabipkgdiff.rst: Add doc for new options
--show-identical-binaries to fedabipkgdiff.
* tools/abipkgdiff.cc (options::show_identical_binaries): New data
member.
(options::options): Initialize new data member.
(display_usage): Add a new help string for the new
--show-identical-binaries option.
(parse_command_line): Parse the newq --show-identical-binaries
command line switch.
(pthread_routine_compare): When the comparison of two binaries is
empty, if --show-identical-binaries was provided, then emit some
output saying the comparison did yield the empty set.
* tools/fedabipkgdiff (DEFAULT_ABIPKGDIFF): Store the default path
to abipkgdiff in this new global variable. Naming this default
path is useful because it can then be cleanly overloaded when
using mock.patch.
(build_path_to_abipkgdiff): Return the new DEFAULT_ABIPKGDIFF
global variable.
(cmd): Parse the new --show-identical-binaries command line
switch.
* tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt:
New reference output.
* tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt:
Likewise.
* tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt:
Likewise.
* tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt:
Likewise.
* tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt:
Likewise.
* tests/mockfedabipkgdiff.in: New uninstalled script template.
* tests/runtestfedabipkgdiff.py.in (counter)
(temp_file_or_dir_prefix, UtilsTest, RPMTest, LocalRPMTest)
(RunAbipkgdiffTest, GetPackageLatestBuildTest, DownloadRPMTest)
(BrewListRPMsTest, AssertionHelper, MockGlobalConfig)
(BUILT_ABIPKGDIFF, CompareABIFromCommandLineTest): Remove these
classes, global variables and functions.
(FEDABIPKGDIFF, TEST_SRC_DIR, TEST_BUILD_DIR, INPUT_DIR)
(OUTPUT_DIR, FEDABIPKGDIFF_TEST_SPECS): New global variables.
(ensure_output_dir_created, run_fedabipkgdiff_tests, main): New
functions.
* tests/test-diff-pkg.cc (in_out_specs): Add
tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt
to the set of reference outputs to consider.
* tests/Makefile.am: Add non-installed script mockfedabipkgdiff to
source distribution. Also added
tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt,
tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt,
tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt,
tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt
and
tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt
to source distribution.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2016-07-06 09:51:13 +00:00
|
|
|
#!/usr/bin/python
|
|
|
|
# -*- coding: utf-8 -*-
|
|
|
|
# -*- Mode: Python
|
|
|
|
#
|
|
|
|
# This file is part of the GNU Application Binary Interface Generic
|
|
|
|
# Analysis and Instrumentation Library (libabigail). This library is
|
|
|
|
# free software; you can redistribute it and/or modify it under the
|
|
|
|
# terms of the GNU General Public License as published by the
|
|
|
|
# Free Software Foundation; either version 3, or (at your option) any
|
|
|
|
# later version.
|
|
|
|
#
|
|
|
|
# This library is distributed in the hope that it will be useful, but
|
|
|
|
# WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
|
|
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
|
|
|
|
# General Public License for more details.
|
|
|
|
#
|
|
|
|
# You should have received a copy of the GNU General Public
|
|
|
|
# License along with this program; see the file COPYING-GPLV3. If
|
|
|
|
# not, see <http:#www.gnu.org/licenses/>.
|
|
|
|
#
|
|
|
|
# Author: Dodji Seketeli
|
|
|
|
#
|
|
|
|
# Based on some preliminary work from Chenxiong Qi, posted to
|
|
|
|
# https://sourceware.org/ml/libabigail/2016-q3/msg00031.html.
|
|
|
|
|
|
|
|
|
|
|
|
'''A local-only version of the fedabipkgdiff program
|
|
|
|
|
|
|
|
This program behaves exactly like the fedabipkgdiff tool, except
|
|
|
|
that it doesn't use the network (via the Koji client) to get Fedora
|
|
|
|
packages. The Koji client is the interface that is used to access
|
|
|
|
the Fedora build system to get Fedora binary packages.
|
|
|
|
|
|
|
|
This program thus loads the fedabipkgdiff tool and overloads
|
|
|
|
(replaces) its Koji Client session with a fake (also known as a
|
|
|
|
mock) Koji client named MockClientSession that gets the packages
|
|
|
|
locally. Packages are stored under the directory
|
|
|
|
tests/data/test-fedabipkgdiff/packages. This program then invokes
|
|
|
|
the fedabipkgdiff program just as if the user invoked it from the
|
|
|
|
command line.
|
|
|
|
|
|
|
|
This program is useful for tests that can be called from "make
|
|
|
|
check" because those must be able to perform in environments where
|
|
|
|
no network is avaible.
|
|
|
|
|
|
|
|
Please note that the descriptions of the builds of each package are
|
|
|
|
stored in at least three global variables below: packages, builds
|
|
|
|
and rpms.
|
|
|
|
|
|
|
|
Whenever a user wants to add a new build to the local set of builds
|
|
|
|
locally available via MockClientSession, she must update these three
|
|
|
|
variables.
|
|
|
|
'''
|
|
|
|
|
|
|
|
import os
|
|
|
|
import tempfile
|
|
|
|
import imp
|
|
|
|
|
|
|
|
try:
|
|
|
|
from mock import patch
|
|
|
|
except ImportError:
|
|
|
|
import sys
|
|
|
|
print >>sys.stderr, \
|
|
|
|
'mock is required to run tests. Please install before running tests.'
|
|
|
|
sys.exit(1)
|
|
|
|
|
|
|
|
ABIPKGDIFF = '@abs_top_builddir@/tools/abipkgdiff'
|
|
|
|
FEDABIPKGDIFF = '@abs_top_srcdir@/tools/fedabipkgdiff'
|
|
|
|
INPUT_DIR = '@abs_top_srcdir@/tests/data/test-fedabipkgdiff'
|
|
|
|
OUTPUT_DIR = '@abs_top_builddir@/tests/output/test-fedabipkgdiff'
|
|
|
|
DOWNLOAD_CACHE_DIR_PREFIX=os.path.join(OUTPUT_DIR, 'download-cache')
|
|
|
|
TEST_TOPDIR = 'file://{0}'.format(INPUT_DIR)
|
|
|
|
|
|
|
|
def get_download_dir():
|
|
|
|
|
|
|
|
if not os.path.exists(DOWNLOAD_CACHE_DIR_PREFIX):
|
|
|
|
try:
|
|
|
|
os.makedirs(DOWNLOAD_CACHE_DIR_PREFIX)
|
|
|
|
except:
|
|
|
|
pass
|
|
|
|
if not os.path.exists(DOWNLOAD_CACHE_DIR_PREFIX):
|
|
|
|
sys.exit(1);
|
|
|
|
|
|
|
|
return tempfile.mkdtemp(dir=DOWNLOAD_CACHE_DIR_PREFIX)
|
|
|
|
|
|
|
|
DOWNLOAD_CACHE_DIR = get_download_dir()
|
|
|
|
|
|
|
|
def mock_get_download_dir():
|
|
|
|
return DOWNLOAD_CACHE_DIR
|
|
|
|
|
|
|
|
|
|
|
|
# Import the fedabipkgdiff program file from the source directory.
|
|
|
|
fedabipkgdiff_mod = imp.load_source('fedabipkgdiff', FEDABIPKGDIFF)
|
|
|
|
|
|
|
|
|
|
|
|
# ----------------- Koji resource storage begins ------------------
|
|
|
|
#
|
|
|
|
# List all necessary Koji resources for running tests within this test
|
|
|
|
# module. Currently, packages, builds, and rpms are listed here, and their
|
|
|
|
# relationship is maintained well. At the same time, all information including
|
|
|
|
# the ID for each package, build and rpm is real and can be queried from Koji,
|
|
|
|
# that is for convenience once developers need this.
|
|
|
|
#
|
|
|
|
# When additional packages, builds and rpms are required for test cases, here
|
|
|
|
# is the right place to add them. Just think them as a super simple in-memory
|
|
|
|
# database, and methods of MockClientSession knows well how to read them.
|
|
|
|
|
|
|
|
packages = [
|
|
|
|
{'id': 286, 'name': 'gnutls'},
|
|
|
|
{'id': 612, 'name': 'dbus-glib'},
|
|
|
|
{'id': 9041, 'name': 'nss-util'},
|
2017-01-23 22:28:12 +00:00
|
|
|
{'id': 18494, 'name': 'vte291'},
|
Make fedabipkgdiff consistent with Libabigail's other tests
In the libabigail project, almost all the tests for the tools follow a
similar pattern. The test contains a set of input files to compare.
The result of the comparison is thus compared to a set of reference
comparison result.
This approach is already documented in the CONTRIBUTING file.
There are several interesting properties with this scheme. First, it
capture the behaviour of the tools, including what is shown to the
user.
Second, it eases the job of a hacker who wants to add a new test for a
new behaviour of a given tool. The user just has to provide:
1/ A new reference output of the new use case of the tool (that is easily
constructed by using the tool itself and saving its output) and add an
entry to array of entries that describe what to compare
2/ A new set of inputs to the tool
And voila.
Unfortunately, fedabipkgdiff tests don't follow this scheme. That
make them surprising to hackers who read the source code of the
existing tests, at very least. Also, the fedabipkgdiff tests were
only unit tests. They were not testing the tool as used by users in
general.
This patch makes the fedabipkgdiff tests follow the general approach
of the tests of the other Libabigail tools.
The patch first craetes a program names tests/mockfedabipkgdiff. It's
a wrapper around tools/fedabipkgdiff. It overloads the Koji client of
fedabipkgdiff with a fake Koji client that gets the Fedora packages
locally, from tests/data/test-fedabipkgdiff/packages. In other words,
mockfedabipkgdiff is fedabipkgdiff without going to the network.
I believe that in the future, tests/mockfedabipkgdiff should be killed
when fedabipkgdiff is able to cache a local description of a local
partial view of a Koji repository, along with the build packages that
were already downloaded.
That way, calling fedabipkgdiff twice with the same set of option
should make the tool perform correctly without going to the netword on
the second invocation. We should be able to save the local partial
view of the Koji repository under e.g,
tests/data/test-fedabipkgdiff/local-koji and tell fedabipkgdiff to use
that, instead of using the network. But we are not there yet. So for
now, I am using mockfedabipkgdiff.
Then, tests/runtestfedabipkdiff.py.in has been re-written to use
tests/mockfedabipkgdiff and to look like what is described in
CONTRIBUTING as far as how Libabigail tools' tests are organized:
mockfedabipkgdiff is called to perform a comparison. The result of
the comparison is then compared (using GNU diff) to a reference
comparison result file.
Please note that tests/runtestfedabipkdiff.py is relatively fast for
the moment. But when it contains much more tests and start becoming
slow, then we'll need to change the code to run several comparisons in
parallel, just like we do today in tests/test-diff-filter.cc. At that
point, I believe I'll just re-write this in C++, just like
tests/test-diff-filter.cc because that will allow us to have true
concurrent code based on the abigail:workers API. For now, I am
keeping this in Python also because I think that keeps Chenxiong happy
;-)
To be sure that fedabipkgdiff (and its underlying abipkgdiff) are
really comparing all the packages they are supposed to compare and
also all the binaries in those packages, I have added a new
--show-identical-binaries to abipkgdiff and fedabipkgdiff. That
option shows the name of the binaries (and packages) that are
compared, *even if* the ABI of those binaries are identical. Because
otherwise, today, if two binaries (or packages) have the same ABI,
nothing is displayed.
For instance, here is the result of comparing a package against
itself, using this new --show-identical-binaries options:
dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ./build/tools/abipkgdiff --show-identical-binaries dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm
Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm:
================ changes of 'dbus-binding-tool'===============
No ABI change detected
================ end of changes of 'dbus-binding-tool'===============
================ changes of 'libdbus-glib-1.so.2.1.0'===============
No ABI change detected
================ end of changes of 'libdbus-glib-1.so.2.1.0'===============
dodji@adjoa:patches$
And here is what this command before that patch would do:
dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ../master/build/tools/abipkgdiff dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm
Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm:
dodji@adjoa:patches$
The rest of the patch is mostly new test inputs material and the
necessary adjustments following all the changes above.
* configure.ac: Do not require Python dependencies itertools,
unittest and StringIO anymore as they are not used anymore.
Require new module tempfile now. Generate new executable script
tests/mockfedabipkgdiff from tests/mockfedabipkgdiff.in.
* doc/manuals/abipkgdiff.rst: Add doc for new option
--show-identical-binaries to abipkgdiff
* doc/manuals/fedabipkgdiff.rst: Add doc for new options
--show-identical-binaries to fedabipkgdiff.
* tools/abipkgdiff.cc (options::show_identical_binaries): New data
member.
(options::options): Initialize new data member.
(display_usage): Add a new help string for the new
--show-identical-binaries option.
(parse_command_line): Parse the newq --show-identical-binaries
command line switch.
(pthread_routine_compare): When the comparison of two binaries is
empty, if --show-identical-binaries was provided, then emit some
output saying the comparison did yield the empty set.
* tools/fedabipkgdiff (DEFAULT_ABIPKGDIFF): Store the default path
to abipkgdiff in this new global variable. Naming this default
path is useful because it can then be cleanly overloaded when
using mock.patch.
(build_path_to_abipkgdiff): Return the new DEFAULT_ABIPKGDIFF
global variable.
(cmd): Parse the new --show-identical-binaries command line
switch.
* tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt:
New reference output.
* tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt:
Likewise.
* tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt:
Likewise.
* tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt:
Likewise.
* tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt:
Likewise.
* tests/mockfedabipkgdiff.in: New uninstalled script template.
* tests/runtestfedabipkgdiff.py.in (counter)
(temp_file_or_dir_prefix, UtilsTest, RPMTest, LocalRPMTest)
(RunAbipkgdiffTest, GetPackageLatestBuildTest, DownloadRPMTest)
(BrewListRPMsTest, AssertionHelper, MockGlobalConfig)
(BUILT_ABIPKGDIFF, CompareABIFromCommandLineTest): Remove these
classes, global variables and functions.
(FEDABIPKGDIFF, TEST_SRC_DIR, TEST_BUILD_DIR, INPUT_DIR)
(OUTPUT_DIR, FEDABIPKGDIFF_TEST_SPECS): New global variables.
(ensure_output_dir_created, run_fedabipkgdiff_tests, main): New
functions.
* tests/test-diff-pkg.cc (in_out_specs): Add
tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt
to the set of reference outputs to consider.
* tests/Makefile.am: Add non-installed script mockfedabipkgdiff to
source distribution. Also added
tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt,
tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt,
tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt,
tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt
and
tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt
to source distribution.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2016-07-06 09:51:13 +00:00
|
|
|
]
|
|
|
|
|
|
|
|
builds = [
|
|
|
|
{'build_id': 428835, 'nvr': 'dbus-glib-0.100-4.fc20',
|
|
|
|
'name': 'dbus-glib', 'release': '4.fc20', 'version': '0.100',
|
|
|
|
'package_id': 612, 'package_name': 'dbus-glib', 'state': 1,
|
|
|
|
},
|
|
|
|
{'build_id': 430720, 'nvr': 'dbus-glib-0.100.2-1.fc20',
|
|
|
|
'name': 'dbus-glib', 'release': '1.fc20', 'version': '0.100.2',
|
|
|
|
'package_id': 612, 'package_name': 'dbus-glib', 'state': 1,
|
|
|
|
},
|
|
|
|
{'build_id': 442366, 'nvr': 'dbus-glib-0.100.2-2.fc20',
|
|
|
|
'name': 'dbus-glib', 'release': '2.fc20', 'version': '0.100.2',
|
|
|
|
'package_id': 612, 'package_name': 'dbus-glib', 'state': 1,
|
|
|
|
},
|
|
|
|
{'build_id': 715478, 'nvr': 'dbus-glib-0.106-1.fc23',
|
|
|
|
'name': 'dbus-glib', 'release': '1.fc23', 'version': '0.106',
|
|
|
|
'package_id': 612, 'package_name': 'dbus-glib', 'state': 1,
|
|
|
|
},
|
|
|
|
{'build_id': 648058, 'nvr': 'dbus-glib-0.104-3.fc23',
|
|
|
|
'name': 'dbus-glib', 'release': '3.fc23', 'version': '0.104',
|
|
|
|
'package_id': 612, 'package_name': 'dbus-glib', 'state': 1,
|
|
|
|
},
|
|
|
|
{'build_id': 613769, 'nvr': 'dbus-glib-0.104-2.fc23',
|
|
|
|
'name': 'dbus-glib', 'release': '2.fc23', 'version': '0.104',
|
|
|
|
'package_id': 612, 'package_name': 'dbus-glib', 'state': 1,
|
|
|
|
},
|
|
|
|
|
|
|
|
{'build_id': 160295, 'nvr': 'nss-util-3.12.6-1.fc14',
|
|
|
|
'name': 'nss-util', 'version': '3.12.6', 'release': '1.fc14',
|
|
|
|
'package_id': 9041, 'package_name': 'nss-util', 'state': 1,
|
|
|
|
},
|
|
|
|
{'build_id': 767978, 'nvr': 'nss-util-3.24.0-2.0.fc25',
|
|
|
|
'name': 'nss-util', 'version': '3.24.0', 'release': '2.0.fc25',
|
|
|
|
'package_id': 9041, 'package_name': 'nss-util', 'state': 1,
|
|
|
|
},
|
|
|
|
|
|
|
|
# builds of package gnutls
|
|
|
|
{'build_id': 767306, 'nvr': 'gnutls-3.4.12-1.fc23',
|
|
|
|
'name': 'gnutls', 'release': '1.fc23', 'version': '3.4.12',
|
|
|
|
'package_id': 286, 'package_name': 'gnutls', 'state': 1,
|
|
|
|
},
|
|
|
|
{'build_id': 770965, 'nvr': 'gnutls-3.4.13-1.fc23',
|
|
|
|
'name': 'gnutls', 'release': '1.fc23', 'version': '3.4.13',
|
|
|
|
'package_id': 286, 'package_name': 'gnutls', 'state': 1,
|
|
|
|
},
|
|
|
|
{'build_id': 649701, 'nvr': 'gnutls-3.4.2-1.fc23',
|
|
|
|
'name': 'gnutls', 'release': '1.fc23', 'version': '3.4.2',
|
|
|
|
'package_id': 286, 'package_name': 'gnutls', 'state': 1,
|
|
|
|
},
|
2017-01-23 22:28:12 +00:00
|
|
|
|
|
|
|
# builds of package vte291
|
|
|
|
{'build_id': 600011, 'nvr': 'vte291-0.39.1-1.fc22',
|
|
|
|
'name': 'vte291', 'version': '0.39.1', 'release': '1.fc22',
|
|
|
|
'package_id': 18494, 'package_name': 'vte291', 'state': 1,
|
|
|
|
},
|
|
|
|
{'build_id': 612610, 'nvr': 'vte291-0.39.90-1.fc22',
|
|
|
|
'name': 'vte291', 'version': '0.39.90', 'release': '1.fc22',
|
|
|
|
'package_id': 18494, 'package_name': 'vte291', 'state': 1,
|
|
|
|
},
|
Make fedabipkgdiff consistent with Libabigail's other tests
In the libabigail project, almost all the tests for the tools follow a
similar pattern. The test contains a set of input files to compare.
The result of the comparison is thus compared to a set of reference
comparison result.
This approach is already documented in the CONTRIBUTING file.
There are several interesting properties with this scheme. First, it
capture the behaviour of the tools, including what is shown to the
user.
Second, it eases the job of a hacker who wants to add a new test for a
new behaviour of a given tool. The user just has to provide:
1/ A new reference output of the new use case of the tool (that is easily
constructed by using the tool itself and saving its output) and add an
entry to array of entries that describe what to compare
2/ A new set of inputs to the tool
And voila.
Unfortunately, fedabipkgdiff tests don't follow this scheme. That
make them surprising to hackers who read the source code of the
existing tests, at very least. Also, the fedabipkgdiff tests were
only unit tests. They were not testing the tool as used by users in
general.
This patch makes the fedabipkgdiff tests follow the general approach
of the tests of the other Libabigail tools.
The patch first craetes a program names tests/mockfedabipkgdiff. It's
a wrapper around tools/fedabipkgdiff. It overloads the Koji client of
fedabipkgdiff with a fake Koji client that gets the Fedora packages
locally, from tests/data/test-fedabipkgdiff/packages. In other words,
mockfedabipkgdiff is fedabipkgdiff without going to the network.
I believe that in the future, tests/mockfedabipkgdiff should be killed
when fedabipkgdiff is able to cache a local description of a local
partial view of a Koji repository, along with the build packages that
were already downloaded.
That way, calling fedabipkgdiff twice with the same set of option
should make the tool perform correctly without going to the netword on
the second invocation. We should be able to save the local partial
view of the Koji repository under e.g,
tests/data/test-fedabipkgdiff/local-koji and tell fedabipkgdiff to use
that, instead of using the network. But we are not there yet. So for
now, I am using mockfedabipkgdiff.
Then, tests/runtestfedabipkdiff.py.in has been re-written to use
tests/mockfedabipkgdiff and to look like what is described in
CONTRIBUTING as far as how Libabigail tools' tests are organized:
mockfedabipkgdiff is called to perform a comparison. The result of
the comparison is then compared (using GNU diff) to a reference
comparison result file.
Please note that tests/runtestfedabipkdiff.py is relatively fast for
the moment. But when it contains much more tests and start becoming
slow, then we'll need to change the code to run several comparisons in
parallel, just like we do today in tests/test-diff-filter.cc. At that
point, I believe I'll just re-write this in C++, just like
tests/test-diff-filter.cc because that will allow us to have true
concurrent code based on the abigail:workers API. For now, I am
keeping this in Python also because I think that keeps Chenxiong happy
;-)
To be sure that fedabipkgdiff (and its underlying abipkgdiff) are
really comparing all the packages they are supposed to compare and
also all the binaries in those packages, I have added a new
--show-identical-binaries to abipkgdiff and fedabipkgdiff. That
option shows the name of the binaries (and packages) that are
compared, *even if* the ABI of those binaries are identical. Because
otherwise, today, if two binaries (or packages) have the same ABI,
nothing is displayed.
For instance, here is the result of comparing a package against
itself, using this new --show-identical-binaries options:
dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ./build/tools/abipkgdiff --show-identical-binaries dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm
Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm:
================ changes of 'dbus-binding-tool'===============
No ABI change detected
================ end of changes of 'dbus-binding-tool'===============
================ changes of 'libdbus-glib-1.so.2.1.0'===============
No ABI change detected
================ end of changes of 'libdbus-glib-1.so.2.1.0'===============
dodji@adjoa:patches$
And here is what this command before that patch would do:
dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ../master/build/tools/abipkgdiff dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm
Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm:
dodji@adjoa:patches$
The rest of the patch is mostly new test inputs material and the
necessary adjustments following all the changes above.
* configure.ac: Do not require Python dependencies itertools,
unittest and StringIO anymore as they are not used anymore.
Require new module tempfile now. Generate new executable script
tests/mockfedabipkgdiff from tests/mockfedabipkgdiff.in.
* doc/manuals/abipkgdiff.rst: Add doc for new option
--show-identical-binaries to abipkgdiff
* doc/manuals/fedabipkgdiff.rst: Add doc for new options
--show-identical-binaries to fedabipkgdiff.
* tools/abipkgdiff.cc (options::show_identical_binaries): New data
member.
(options::options): Initialize new data member.
(display_usage): Add a new help string for the new
--show-identical-binaries option.
(parse_command_line): Parse the newq --show-identical-binaries
command line switch.
(pthread_routine_compare): When the comparison of two binaries is
empty, if --show-identical-binaries was provided, then emit some
output saying the comparison did yield the empty set.
* tools/fedabipkgdiff (DEFAULT_ABIPKGDIFF): Store the default path
to abipkgdiff in this new global variable. Naming this default
path is useful because it can then be cleanly overloaded when
using mock.patch.
(build_path_to_abipkgdiff): Return the new DEFAULT_ABIPKGDIFF
global variable.
(cmd): Parse the new --show-identical-binaries command line
switch.
* tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt:
New reference output.
* tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt:
Likewise.
* tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt:
Likewise.
* tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt:
Likewise.
* tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt:
Likewise.
* tests/mockfedabipkgdiff.in: New uninstalled script template.
* tests/runtestfedabipkgdiff.py.in (counter)
(temp_file_or_dir_prefix, UtilsTest, RPMTest, LocalRPMTest)
(RunAbipkgdiffTest, GetPackageLatestBuildTest, DownloadRPMTest)
(BrewListRPMsTest, AssertionHelper, MockGlobalConfig)
(BUILT_ABIPKGDIFF, CompareABIFromCommandLineTest): Remove these
classes, global variables and functions.
(FEDABIPKGDIFF, TEST_SRC_DIR, TEST_BUILD_DIR, INPUT_DIR)
(OUTPUT_DIR, FEDABIPKGDIFF_TEST_SPECS): New global variables.
(ensure_output_dir_created, run_fedabipkgdiff_tests, main): New
functions.
* tests/test-diff-pkg.cc (in_out_specs): Add
tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt
to the set of reference outputs to consider.
* tests/Makefile.am: Add non-installed script mockfedabipkgdiff to
source distribution. Also added
tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt,
tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt,
tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt,
tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt
and
tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt
to source distribution.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2016-07-06 09:51:13 +00:00
|
|
|
]
|
|
|
|
|
|
|
|
rpms = [
|
|
|
|
{'build_id': 442366,
|
|
|
|
'name': 'dbus-glib', 'release': '2.fc20', 'version': '0.100.2',
|
|
|
|
'arch': 'x86_64', 'nvr': 'dbus-glib-0.100.2-2.fc20',
|
|
|
|
},
|
|
|
|
{'build_id': 442366,
|
|
|
|
'name': 'dbus-glib-devel', 'release': '2.fc20', 'version': '0.100.2',
|
|
|
|
'arch': 'x86_64', 'nvr': 'dbus-glib-devel-0.100.2-2.fc20',
|
|
|
|
},
|
|
|
|
{'build_id': 442366,
|
|
|
|
'name': 'dbus-glib-debuginfo', 'release': '2.fc20', 'version': '0.100.2',
|
|
|
|
'arch': 'x86_64', 'nvr': 'dbus-glib-debuginfo-0.100.2-2.fc20',
|
|
|
|
},
|
|
|
|
{'build_id': 442366,
|
|
|
|
'name': 'dbus-glib-devel', 'release': '2.fc20', 'version': '0.100.2',
|
|
|
|
'arch': 'i686', 'nvr': 'dbus-glib-devel-0.100.2-2.fc20',
|
|
|
|
},
|
|
|
|
{'build_id': 442366,
|
|
|
|
'name': 'dbus-glib-debuginfo', 'release': '2.fc20', 'version': '0.100.2',
|
|
|
|
'arch': 'i686', 'nvr': 'dbus-glib-debuginfo-0.100.2-2.fc20',
|
|
|
|
},
|
|
|
|
{'build_id': 442366,
|
|
|
|
'name': 'dbus-glib', 'version': '0.100.2', 'release': '2.fc20',
|
|
|
|
'arch': 'i686', 'nvr': 'dbus-glib-0.100.2-2.fc20',
|
|
|
|
},
|
|
|
|
|
|
|
|
{'build_id': 715478,
|
|
|
|
'name': 'dbus-glib-debuginfo', 'version': '0.106', 'release': '1.fc23',
|
|
|
|
'arch': 'i686', 'nvr': 'dbus-glib-debuginfo-0.106-1.fc23',
|
|
|
|
},
|
|
|
|
{'build_id': 715478,
|
|
|
|
'name': 'dbus-glib', 'version': '0.106', 'release': '1.fc23',
|
|
|
|
'arch': 'i686', 'nvr': 'dbus-glib-0.106-1.fc23',
|
|
|
|
},
|
|
|
|
{'build_id': 715478,
|
|
|
|
'name': 'dbus-glib-devel', 'version': '0.106', 'release': '1.fc23',
|
|
|
|
'arch': 'i686', 'nvr': 'dbus-glib-devel-0.106-1.fc23',
|
|
|
|
},
|
|
|
|
{'build_id': 715478,
|
|
|
|
'name': 'dbus-glib', 'version': '0.106', 'release': '1.fc23',
|
|
|
|
'arch': 'x86_64', 'nvr': 'dbus-glib-0.106-1.fc23',
|
|
|
|
},
|
|
|
|
{'build_id': 715478,
|
|
|
|
'name': 'dbus-glib-debuginfo', 'version': '0.106', 'release': '1.fc23',
|
|
|
|
'arch': 'x86_64', 'nvr': 'dbus-glib-debuginfo-0.106-1.fc23',
|
|
|
|
},
|
|
|
|
{'build_id': 715478,
|
|
|
|
'name': 'dbus-glib-devel', 'release': '1.fc23', 'version': '0.106',
|
|
|
|
'arch': 'x86_64', 'nvr': 'dbus-glib-devel-0.106-1.fc23',
|
|
|
|
},
|
|
|
|
|
|
|
|
# RPMs of build nss-util-3.12.6-1.fc14
|
|
|
|
{'build_id': 160295,
|
|
|
|
'name': 'nss-util', 'release': '1.fc14', 'version': '3.12.6',
|
|
|
|
'arch': 'x86_64', 'nvr': 'nss-util-3.12.6-1.fc14',
|
|
|
|
},
|
|
|
|
{'build_id': 160295,
|
|
|
|
'name': 'nss-util-devel', 'release': '1.fc14', 'version': '3.12.6',
|
|
|
|
'arch': 'x86_64', 'nvr': 'nss-util-devel-3.12.6-1.fc14',
|
|
|
|
},
|
|
|
|
{'build_id': 160295,
|
|
|
|
'name': 'nss-util-debuginfo', 'release': '1.fc14', 'version': '3.12.6',
|
|
|
|
'arch': 'x86_64', 'nvr': 'nss-util-debuginfo-3.12.6-1.fc14',
|
|
|
|
},
|
|
|
|
|
|
|
|
# RPMs of build nss-util-3.24.0-2.0.fc25
|
|
|
|
{'build_id': 767978,
|
|
|
|
'name': 'nss-util-debuginfo', 'release': '2.0.fc25', 'version': '3.24.0',
|
|
|
|
'arch': 'x86_64', 'nvr': 'nss-util-debuginfo-3.24.0-2.0.fc25',
|
|
|
|
},
|
|
|
|
{'build_id': 767978,
|
|
|
|
'name': 'nss-util', 'release': '2.0.fc25', 'version': '3.24.0',
|
|
|
|
'arch': 'x86_64', 'nvr': 'nss-util-3.24.0-2.0.fc25',
|
|
|
|
},
|
|
|
|
{'build_id': 767978,
|
|
|
|
'name': 'nss-util-devel', 'release': '2.0.fc25', 'version': '3.24.0',
|
|
|
|
'arch': 'x86_64', 'nvr': 'nss-util-devel-3.24.0-2.0.fc25',
|
|
|
|
},
|
2017-01-23 22:28:12 +00:00
|
|
|
# RPMs of build vte291-0.39.1-1.fc22
|
|
|
|
{'build_id': 600011,
|
|
|
|
'name': 'vte291', 'version': '0.39.1', 'release': '1.fc22',
|
|
|
|
'arch': 'x86_64', 'nvr': 'vte291-0.39.1-1.fc22',
|
|
|
|
},
|
|
|
|
{'build_id': 600011,
|
|
|
|
'name': 'vte291-devel', 'version': '0.39.1', 'release': '1.fc22',
|
|
|
|
'arch': 'x86_64', 'nvr': 'vte291-0.39.1-1.fc22',
|
|
|
|
},
|
|
|
|
{'build_id': 600011,
|
|
|
|
'name': 'vte291-debuginfo', 'version': '0.39.1', 'release': '1.fc22',
|
|
|
|
'arch': 'x86_64', 'nvr': 'vte291-0.39.1-1.fc22',
|
|
|
|
},
|
|
|
|
|
|
|
|
# RPMs of build vte291-0.39.90-1.fc22
|
|
|
|
{'build_id': 612610,
|
|
|
|
'name': 'vte291', 'version': '0.39.90', 'release': '1.fc22',
|
|
|
|
'arch': 'x86_64', 'nvr': 'vte291-0.39.90-1.fc22',
|
|
|
|
},
|
|
|
|
{'build_id': 612610,
|
|
|
|
'name': 'vte291-devel', 'version': '0.39.90', 'release': '1.fc22',
|
|
|
|
'arch': 'x86_64', 'nvr': 'vte291-0.39.90-1.fc22',
|
|
|
|
},
|
|
|
|
{'build_id': 612610,
|
|
|
|
'name': 'vte291-debuginfo', 'version': '0.39.90', 'release': '1.fc22',
|
|
|
|
'arch': 'x86_64', 'nvr': 'vte291-0.39.90-1.fc22',
|
|
|
|
},
|
|
|
|
]
|
Make fedabipkgdiff consistent with Libabigail's other tests
In the libabigail project, almost all the tests for the tools follow a
similar pattern. The test contains a set of input files to compare.
The result of the comparison is thus compared to a set of reference
comparison result.
This approach is already documented in the CONTRIBUTING file.
There are several interesting properties with this scheme. First, it
capture the behaviour of the tools, including what is shown to the
user.
Second, it eases the job of a hacker who wants to add a new test for a
new behaviour of a given tool. The user just has to provide:
1/ A new reference output of the new use case of the tool (that is easily
constructed by using the tool itself and saving its output) and add an
entry to array of entries that describe what to compare
2/ A new set of inputs to the tool
And voila.
Unfortunately, fedabipkgdiff tests don't follow this scheme. That
make them surprising to hackers who read the source code of the
existing tests, at very least. Also, the fedabipkgdiff tests were
only unit tests. They were not testing the tool as used by users in
general.
This patch makes the fedabipkgdiff tests follow the general approach
of the tests of the other Libabigail tools.
The patch first craetes a program names tests/mockfedabipkgdiff. It's
a wrapper around tools/fedabipkgdiff. It overloads the Koji client of
fedabipkgdiff with a fake Koji client that gets the Fedora packages
locally, from tests/data/test-fedabipkgdiff/packages. In other words,
mockfedabipkgdiff is fedabipkgdiff without going to the network.
I believe that in the future, tests/mockfedabipkgdiff should be killed
when fedabipkgdiff is able to cache a local description of a local
partial view of a Koji repository, along with the build packages that
were already downloaded.
That way, calling fedabipkgdiff twice with the same set of option
should make the tool perform correctly without going to the netword on
the second invocation. We should be able to save the local partial
view of the Koji repository under e.g,
tests/data/test-fedabipkgdiff/local-koji and tell fedabipkgdiff to use
that, instead of using the network. But we are not there yet. So for
now, I am using mockfedabipkgdiff.
Then, tests/runtestfedabipkdiff.py.in has been re-written to use
tests/mockfedabipkgdiff and to look like what is described in
CONTRIBUTING as far as how Libabigail tools' tests are organized:
mockfedabipkgdiff is called to perform a comparison. The result of
the comparison is then compared (using GNU diff) to a reference
comparison result file.
Please note that tests/runtestfedabipkdiff.py is relatively fast for
the moment. But when it contains much more tests and start becoming
slow, then we'll need to change the code to run several comparisons in
parallel, just like we do today in tests/test-diff-filter.cc. At that
point, I believe I'll just re-write this in C++, just like
tests/test-diff-filter.cc because that will allow us to have true
concurrent code based on the abigail:workers API. For now, I am
keeping this in Python also because I think that keeps Chenxiong happy
;-)
To be sure that fedabipkgdiff (and its underlying abipkgdiff) are
really comparing all the packages they are supposed to compare and
also all the binaries in those packages, I have added a new
--show-identical-binaries to abipkgdiff and fedabipkgdiff. That
option shows the name of the binaries (and packages) that are
compared, *even if* the ABI of those binaries are identical. Because
otherwise, today, if two binaries (or packages) have the same ABI,
nothing is displayed.
For instance, here is the result of comparing a package against
itself, using this new --show-identical-binaries options:
dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ./build/tools/abipkgdiff --show-identical-binaries dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm
Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm:
================ changes of 'dbus-binding-tool'===============
No ABI change detected
================ end of changes of 'dbus-binding-tool'===============
================ changes of 'libdbus-glib-1.so.2.1.0'===============
No ABI change detected
================ end of changes of 'libdbus-glib-1.so.2.1.0'===============
dodji@adjoa:patches$
And here is what this command before that patch would do:
dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ../master/build/tools/abipkgdiff dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm
Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm:
dodji@adjoa:patches$
The rest of the patch is mostly new test inputs material and the
necessary adjustments following all the changes above.
* configure.ac: Do not require Python dependencies itertools,
unittest and StringIO anymore as they are not used anymore.
Require new module tempfile now. Generate new executable script
tests/mockfedabipkgdiff from tests/mockfedabipkgdiff.in.
* doc/manuals/abipkgdiff.rst: Add doc for new option
--show-identical-binaries to abipkgdiff
* doc/manuals/fedabipkgdiff.rst: Add doc for new options
--show-identical-binaries to fedabipkgdiff.
* tools/abipkgdiff.cc (options::show_identical_binaries): New data
member.
(options::options): Initialize new data member.
(display_usage): Add a new help string for the new
--show-identical-binaries option.
(parse_command_line): Parse the newq --show-identical-binaries
command line switch.
(pthread_routine_compare): When the comparison of two binaries is
empty, if --show-identical-binaries was provided, then emit some
output saying the comparison did yield the empty set.
* tools/fedabipkgdiff (DEFAULT_ABIPKGDIFF): Store the default path
to abipkgdiff in this new global variable. Naming this default
path is useful because it can then be cleanly overloaded when
using mock.patch.
(build_path_to_abipkgdiff): Return the new DEFAULT_ABIPKGDIFF
global variable.
(cmd): Parse the new --show-identical-binaries command line
switch.
* tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt:
New reference output.
* tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt:
Likewise.
* tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt:
Likewise.
* tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt:
Likewise.
* tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt:
Likewise.
* tests/mockfedabipkgdiff.in: New uninstalled script template.
* tests/runtestfedabipkgdiff.py.in (counter)
(temp_file_or_dir_prefix, UtilsTest, RPMTest, LocalRPMTest)
(RunAbipkgdiffTest, GetPackageLatestBuildTest, DownloadRPMTest)
(BrewListRPMsTest, AssertionHelper, MockGlobalConfig)
(BUILT_ABIPKGDIFF, CompareABIFromCommandLineTest): Remove these
classes, global variables and functions.
(FEDABIPKGDIFF, TEST_SRC_DIR, TEST_BUILD_DIR, INPUT_DIR)
(OUTPUT_DIR, FEDABIPKGDIFF_TEST_SPECS): New global variables.
(ensure_output_dir_created, run_fedabipkgdiff_tests, main): New
functions.
* tests/test-diff-pkg.cc (in_out_specs): Add
tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt
to the set of reference outputs to consider.
* tests/Makefile.am: Add non-installed script mockfedabipkgdiff to
source distribution. Also added
tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt,
tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt,
tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt,
tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt
and
tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt
to source distribution.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2016-07-06 09:51:13 +00:00
|
|
|
|
|
|
|
# ----------------- End of Koji resource storage ------------------
|
|
|
|
|
|
|
|
|
|
|
|
class MockClientSession(object):
|
|
|
|
"""Mock koji.ClientSession
|
|
|
|
|
|
|
|
This mock ClientSession aims to avoid touching a real Koji instance to
|
|
|
|
interact with XMLRPC APIs required by fedabipkgdiff.
|
|
|
|
|
|
|
|
For the tests within this module, methods do not necessarily to return
|
|
|
|
complete RPM and build information. So, if you need more additional
|
|
|
|
information, here is the right place to add them.
|
|
|
|
"""
|
|
|
|
|
|
|
|
def __init__(self, baseurl):
|
|
|
|
"""Initialize a mock ClientSession
|
|
|
|
|
|
|
|
:param str baseurl: the URL to remote kojihub service. As of writing
|
|
|
|
this mock class, `baseurl` is not used at all, just keep it here if
|
|
|
|
it's useful in the future.
|
|
|
|
|
|
|
|
All mock methods have same signature as corresponding kojihub.*
|
|
|
|
methods, and type of parameters may be different and only satify
|
|
|
|
fedabipkgdiff requirement.
|
|
|
|
"""
|
|
|
|
self.baseurl = baseurl
|
|
|
|
|
|
|
|
def getPackage(self, name):
|
|
|
|
"""Mock kojihub.getPackage
|
|
|
|
|
|
|
|
:param str name: name of package to find and return
|
|
|
|
:return: the found package
|
|
|
|
:rtype: dict
|
|
|
|
"""
|
|
|
|
assert isinstance(name, basestring)
|
|
|
|
|
|
|
|
def selector(package):
|
|
|
|
return package['name'] == name
|
|
|
|
|
|
|
|
return filter(selector, packages)[0]
|
|
|
|
|
|
|
|
def getBuild(self, build_id):
|
|
|
|
"""Mock kojihub.getBuild
|
|
|
|
|
|
|
|
:param int build_id: ID of build to find and return
|
|
|
|
:return: the found build
|
|
|
|
:rtype: dict
|
|
|
|
"""
|
|
|
|
assert isinstance(build_id, int)
|
|
|
|
|
|
|
|
def selector(build):
|
|
|
|
return build['build_id'] == build_id
|
|
|
|
|
|
|
|
return filter(selector, builds)[0]
|
|
|
|
|
|
|
|
def listBuilds(self, packageID, state=None):
|
|
|
|
"""Mock kojihub.listBuilds
|
|
|
|
|
|
|
|
:param int packageID: ID of package whose builds is found and returned
|
|
|
|
:param state: build state. If state is omitted, all builds of a package
|
|
|
|
are returned
|
|
|
|
:type state: int or None
|
|
|
|
"""
|
|
|
|
assert isinstance(packageID, int)
|
|
|
|
if state is not None:
|
|
|
|
assert isinstance(state, int)
|
|
|
|
|
|
|
|
def selector(build):
|
|
|
|
selected = build['package_id'] == packageID
|
|
|
|
if state is not None:
|
|
|
|
selected = selected and build['state'] == state
|
|
|
|
return selected
|
|
|
|
|
|
|
|
return filter(selector, builds)
|
|
|
|
|
|
|
|
def getRPM(self, rpminfo):
|
|
|
|
"""Mock kojihub.getRPM
|
|
|
|
|
|
|
|
:param dict rpminfo: a mapping containing rpm information, at least,
|
|
|
|
it contains name, version, release, and arch.
|
|
|
|
"""
|
|
|
|
assert isinstance(rpminfo, dict)
|
|
|
|
|
|
|
|
def selector(rpm):
|
|
|
|
return rpm['name'] == rpminfo['name'] and \
|
|
|
|
rpm['version'] == rpminfo['version'] and \
|
|
|
|
rpm['release'] == rpminfo['release'] and \
|
|
|
|
rpm['arch'] == rpminfo['arch']
|
|
|
|
|
|
|
|
return filter(selector, rpms)[0]
|
|
|
|
|
|
|
|
def listRPMs(self, buildID, arches=None):
|
|
|
|
"""Mock kojihub.listRPMs
|
|
|
|
|
|
|
|
:param int buildID: ID of build from which to list rpms
|
|
|
|
:param arches: to list rpms built for specific arches. If arches is
|
|
|
|
omitted, rpms of all arches will be listed.
|
|
|
|
:type arches: list, tuple, str, or None
|
|
|
|
:return: list of rpms
|
|
|
|
:rtype: list
|
|
|
|
"""
|
|
|
|
assert isinstance(buildID, int)
|
|
|
|
if arches is not None:
|
|
|
|
assert isinstance(arches, (tuple, list, basestring))
|
|
|
|
|
|
|
|
if arches is not None and isinstance(arches, basestring):
|
|
|
|
arches = [arches]
|
|
|
|
|
|
|
|
def selector(rpm):
|
|
|
|
selected = rpm['build_id'] == buildID
|
|
|
|
if arches is not None:
|
|
|
|
selected = selected and rpm['arch'] in arches
|
|
|
|
return selected
|
|
|
|
|
|
|
|
return filter(selector, rpms)
|
|
|
|
|
|
|
|
|
|
|
|
@patch('koji.ClientSession', new=MockClientSession)
|
2017-01-23 23:37:22 +00:00
|
|
|
@patch('fedabipkgdiff.DEFAULT_KOJI_TOPURL', new=TEST_TOPDIR)
|
Make fedabipkgdiff consistent with Libabigail's other tests
In the libabigail project, almost all the tests for the tools follow a
similar pattern. The test contains a set of input files to compare.
The result of the comparison is thus compared to a set of reference
comparison result.
This approach is already documented in the CONTRIBUTING file.
There are several interesting properties with this scheme. First, it
capture the behaviour of the tools, including what is shown to the
user.
Second, it eases the job of a hacker who wants to add a new test for a
new behaviour of a given tool. The user just has to provide:
1/ A new reference output of the new use case of the tool (that is easily
constructed by using the tool itself and saving its output) and add an
entry to array of entries that describe what to compare
2/ A new set of inputs to the tool
And voila.
Unfortunately, fedabipkgdiff tests don't follow this scheme. That
make them surprising to hackers who read the source code of the
existing tests, at very least. Also, the fedabipkgdiff tests were
only unit tests. They were not testing the tool as used by users in
general.
This patch makes the fedabipkgdiff tests follow the general approach
of the tests of the other Libabigail tools.
The patch first craetes a program names tests/mockfedabipkgdiff. It's
a wrapper around tools/fedabipkgdiff. It overloads the Koji client of
fedabipkgdiff with a fake Koji client that gets the Fedora packages
locally, from tests/data/test-fedabipkgdiff/packages. In other words,
mockfedabipkgdiff is fedabipkgdiff without going to the network.
I believe that in the future, tests/mockfedabipkgdiff should be killed
when fedabipkgdiff is able to cache a local description of a local
partial view of a Koji repository, along with the build packages that
were already downloaded.
That way, calling fedabipkgdiff twice with the same set of option
should make the tool perform correctly without going to the netword on
the second invocation. We should be able to save the local partial
view of the Koji repository under e.g,
tests/data/test-fedabipkgdiff/local-koji and tell fedabipkgdiff to use
that, instead of using the network. But we are not there yet. So for
now, I am using mockfedabipkgdiff.
Then, tests/runtestfedabipkdiff.py.in has been re-written to use
tests/mockfedabipkgdiff and to look like what is described in
CONTRIBUTING as far as how Libabigail tools' tests are organized:
mockfedabipkgdiff is called to perform a comparison. The result of
the comparison is then compared (using GNU diff) to a reference
comparison result file.
Please note that tests/runtestfedabipkdiff.py is relatively fast for
the moment. But when it contains much more tests and start becoming
slow, then we'll need to change the code to run several comparisons in
parallel, just like we do today in tests/test-diff-filter.cc. At that
point, I believe I'll just re-write this in C++, just like
tests/test-diff-filter.cc because that will allow us to have true
concurrent code based on the abigail:workers API. For now, I am
keeping this in Python also because I think that keeps Chenxiong happy
;-)
To be sure that fedabipkgdiff (and its underlying abipkgdiff) are
really comparing all the packages they are supposed to compare and
also all the binaries in those packages, I have added a new
--show-identical-binaries to abipkgdiff and fedabipkgdiff. That
option shows the name of the binaries (and packages) that are
compared, *even if* the ABI of those binaries are identical. Because
otherwise, today, if two binaries (or packages) have the same ABI,
nothing is displayed.
For instance, here is the result of comparing a package against
itself, using this new --show-identical-binaries options:
dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ./build/tools/abipkgdiff --show-identical-binaries dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm
Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm:
================ changes of 'dbus-binding-tool'===============
No ABI change detected
================ end of changes of 'dbus-binding-tool'===============
================ changes of 'libdbus-glib-1.so.2.1.0'===============
No ABI change detected
================ end of changes of 'libdbus-glib-1.so.2.1.0'===============
dodji@adjoa:patches$
And here is what this command before that patch would do:
dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ../master/build/tools/abipkgdiff dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm
Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm:
dodji@adjoa:patches$
The rest of the patch is mostly new test inputs material and the
necessary adjustments following all the changes above.
* configure.ac: Do not require Python dependencies itertools,
unittest and StringIO anymore as they are not used anymore.
Require new module tempfile now. Generate new executable script
tests/mockfedabipkgdiff from tests/mockfedabipkgdiff.in.
* doc/manuals/abipkgdiff.rst: Add doc for new option
--show-identical-binaries to abipkgdiff
* doc/manuals/fedabipkgdiff.rst: Add doc for new options
--show-identical-binaries to fedabipkgdiff.
* tools/abipkgdiff.cc (options::show_identical_binaries): New data
member.
(options::options): Initialize new data member.
(display_usage): Add a new help string for the new
--show-identical-binaries option.
(parse_command_line): Parse the newq --show-identical-binaries
command line switch.
(pthread_routine_compare): When the comparison of two binaries is
empty, if --show-identical-binaries was provided, then emit some
output saying the comparison did yield the empty set.
* tools/fedabipkgdiff (DEFAULT_ABIPKGDIFF): Store the default path
to abipkgdiff in this new global variable. Naming this default
path is useful because it can then be cleanly overloaded when
using mock.patch.
(build_path_to_abipkgdiff): Return the new DEFAULT_ABIPKGDIFF
global variable.
(cmd): Parse the new --show-identical-binaries command line
switch.
* tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt:
New reference output.
* tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt:
Likewise.
* tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt:
Likewise.
* tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt:
Likewise.
* tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt:
Likewise.
* tests/mockfedabipkgdiff.in: New uninstalled script template.
* tests/runtestfedabipkgdiff.py.in (counter)
(temp_file_or_dir_prefix, UtilsTest, RPMTest, LocalRPMTest)
(RunAbipkgdiffTest, GetPackageLatestBuildTest, DownloadRPMTest)
(BrewListRPMsTest, AssertionHelper, MockGlobalConfig)
(BUILT_ABIPKGDIFF, CompareABIFromCommandLineTest): Remove these
classes, global variables and functions.
(FEDABIPKGDIFF, TEST_SRC_DIR, TEST_BUILD_DIR, INPUT_DIR)
(OUTPUT_DIR, FEDABIPKGDIFF_TEST_SPECS): New global variables.
(ensure_output_dir_created, run_fedabipkgdiff_tests, main): New
functions.
* tests/test-diff-pkg.cc (in_out_specs): Add
tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt
to the set of reference outputs to consider.
* tests/Makefile.am: Add non-installed script mockfedabipkgdiff to
source distribution. Also added
tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt,
tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt,
tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt,
tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt
and
tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt
to source distribution.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2016-07-06 09:51:13 +00:00
|
|
|
@patch('fedabipkgdiff.DEFAULT_ABIPKGDIFF', new=ABIPKGDIFF)
|
|
|
|
@patch('fedabipkgdiff.get_download_dir', side_effect=mock_get_download_dir)
|
|
|
|
def run_fedabipkgdiff(get_download_dir):
|
|
|
|
return fedabipkgdiff_mod.main()
|
|
|
|
|
|
|
|
def do_main():
|
|
|
|
run_fedabipkgdiff()
|
|
|
|
|
|
|
|
if __name__ == '__main__':
|
|
|
|
do_main()
|