libabigail/tests/runtestfedabipkgdiff.py.in
Chenxiong Qi 783099cd25 Bug 20380 - Compare two local RPMs
Bug 20270 is also fixed.

This patch allows developer to compare two local RPMs in form

    fedabipkgdiff some/place/foo.rpm another/place/bar.rpm

But, network is still needed to talk with Koji.

This patch also introduces new terms for libabigail, that is the
subject, ancillary package, and comparison half. Subject represents a
package that is subject of the ABI comparison, a subject could be a RPM
and maybe it would be a DEB or some other kind of "package". A subject
may have several ancillary packages that are used to compare the
subject.  Generally, a subject may have devel, debuginfo, or both.

	* configure.ac: add dependent mimetype module.
	* doc/manuals/fedabipkgdiff.rst: Update to add document for the
	new use case of comparing two local RPMs.
	* tests/data/Makefile.am: Include new RPMs for tests.
	* tests/data/test-fedabipkgdiff/dbus-glib/dbus-glib-0.100.2-2.fc20.x86_64.rpm:
	New RPM for running test.
	* tests/data/test-fedabipkgdiff/dbus-glib/dbus-glib-0.106-1.fc23.x86_64.rpm:
	Likewise.
	* tests/data/test-fedabipkgdiff/nss-util/nss-util-3.12.6-1.fc14.x86_64.rpm:
	Likewise.
	* tests/data/test-fedabipkgdiff/nss-util/nss-util-3.24.0-2.0.fc25.x86_64.rpm:
	Likewise.
	* tests/data/test-fedabipkgdiff/nss-util/nss-util-devel-3.24.0-2.0.fc25.x86_64.rpm:
	Likewise.
	* tests/data/test-fedabipkgdiff/test4-glib-0.100.2-2.fc20.x86_64.rpm-glib-0.106-1.fc23.x86_64.rpm-report-0.txt:
	Rename filename by adding .rpm extension.
	* tests/data/test-fedabipkgdiff/test5-same-dir-dbus-glib-0.100.2-2.fc20.x86_64--dbus-glib-0.106-1.fc23.x86_64-report-0.txt:
	New reference output for testing comparing local RPMs.
	* tests/data/test-fedabipkgdiff/test6-missing-devel-debuginfo-nss-util-3.12.6-1.fc14.x86_64--nss-util-3.24.0-2.0.fc25.x86_64-report-0.txt:
	New reference output for testing comparison without non-existent
	debuginfo or development package.
	* tests/runtestfedabipkgdiff.py.in (FEDABIPKGDIFF_TEST_SPECS):
	Rename filename for test4. Add two new test cases.
	(run_fedabipkgdiff_tests): Remove semicolon and trailing
	whitespaces.
	(main): Likewise.
	(ensure_output_dir_created): Likewise.
	* tools/fedabipkgdiff: Require some new modules.
	Fix of return code.
	(PkgInfo): Renamed to ComparisonHalf.
	(match_nvr): New method to determine if a string matches format
	of N-V-R.
	(match_nvra): New method to determine if a string matches format
	of N-V-R.A.
	(is_rpm_file): New method to guess if a file is a RPM file.
	(RPM.is_peer): New method to determine if current RPM is a peer
	of another.
	(RPM.filename): Use Koji module API to construct the filename.
	(RPM.nvra): Get nvra from filename instead of constructing
	manually that is duplicated with Koji module API.
	(RPMCollection): New class to represent a set of RPMs.
	(generate_pkg_info_pair_for_abipkgdiff): New method working as a
	generator to yeild comparison halves for running abipkgdiff.
	(Brew.getRPM): Fix string format with incorrect argument.
	(Brew.select_rpms_from_a_build): Return instance of
	RPMCollection.
	(abipkgdiff): If there is no debuginfo or development package,
	just ignore it and leave a warning. If --error-on-warning is
	specified, raise an exception instead. Arguments are modified
	to represent the new name ComparisonHalf, and relative docstring
	is also updated.
	(magic_construct): Removed.
	(run_abipkgdiff): Rewrite.
	(make_rpms_usable_for_abipkgdiff): Removed.
	(diff_local_rpm_with_latest_rpm_from_koji): Rewrite by using
	RPMCollection.
	(diff_latest_rpms_based_on_distros): Likewise.
	(diff_two_nvras_from_koji): Likewise.
	(diff_from_two_rpm_files): New method to compare two local RPMs.
	(build_commandline_args_parser): Add new option
	--error-on-warning.
	(main): Add support to compare local RPMs.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2016-12-12 15:21:10 +01:00

165 lines
6.5 KiB
Python
Executable File

#!/usr/bin/python
# -*- coding: utf-8 -*-
# -*- Mode: Python
#
# This file is part of the GNU Application Binary Interface Generic
# Analysis and Instrumentation Library (libabigail). This library is
# free software; you can redistribute it and/or modify it under the
# terms of the GNU General Public License as published by the
# Free Software Foundation; either version 3, or (at your option) any
# later version.
#
# This library is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
# General Public License for more details.
#
# You should have received a copy of the GNU General Public
# License along with this program; see the file COPYING-GPLV3. If
# not, see <http:#www.gnu.org/licenses/>.
#
# Author: Chenxiong Qi
'''Runs tests for the fedabipkgdiff program
This program runs the fedabipkgdiff tool (via the mockfedabipkgdiff
tool) repeatedly and compares its output against a set of reference
output files.
'''
import sys
import os
import subprocess
FEDABIPKGDIFF = '@abs_top_builddir@/tests/mockfedabipkgdiff'
TEST_SRC_DIR = '@abs_top_srcdir@/tests'
TEST_BUILD_DIR = '@abs_top_builddir@/tests'
INPUT_DIR = '@abs_top_srcdir@/tests/data/test-fedabipkgdiff'
OUTPUT_DIR = '@abs_top_builddir@/tests/output/test-fedabipkgdiff'
# This variable is a list of 3-uples. Here is the meaning of
# the elements of each 3-uples:
#
# (arguments to pass to fedabipkgdiff,
# reference output file,
# where to store the result of the comparison)
#
# Note that after fedabipkgdiff is run (using the arguments that are
# the first element of the tuple) the result of the comparison is
# going to be compared against the reference output file, and both
# must be equal.
#
# If a user wants to add a new test, she must add a new tuple to the
# variable below, and also add a new reference output to the source
# distribution.
#
FEDABIPKGDIFF_TEST_SPECS = [
(['--from', 'fc20', '--to', 'fc23', 'dbus-glib'],
'data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt',
'output/test-fedabipkgdiff/test0-type-suppr-report-0.txt'),
(['--from', 'fc20', os.path.join(INPUT_DIR,
'packages/dbus-glib/0.106/1.fc23/x86_64/'
'dbus-glib-0.106-1.fc23.x86_64.rpm')],
'data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt',
'output/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt'),
(['dbus-glib-0.100.2-2.fc20', 'dbus-glib-0.106-1.fc23'],
'data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt',
'output/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt'),
(['dbus-glib-0.100.2-2.fc20.i686', 'dbus-glib-0.106-1.fc23.i686'],
'data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt',
'output/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt'),
([os.path.join(INPUT_DIR, 'packages/dbus-glib/0.100.2/2.fc20/x86_64/dbus-glib-0.100.2-2.fc20.x86_64.rpm'),
os.path.join(INPUT_DIR, 'packages/dbus-glib/0.106/1.fc23/x86_64/dbus-glib-0.106-1.fc23.x86_64.rpm')],
'data/test-fedabipkgdiff/test4-glib-0.100.2-2.fc20.x86_64.rpm-glib-0.106-1.fc23.x86_64.rpm-report-0.txt',
'output/test-fedabipkgdiff/test4-glib-0.100.2-2.fc20.x86_64.rpm-glib-0.106-1.fc23.x86_64.rpm-report-0.txt'),
([os.path.join(INPUT_DIR, 'dbus-glib/dbus-glib-0.100.2-2.fc20.x86_64.rpm'),
os.path.join(INPUT_DIR, 'dbus-glib/dbus-glib-0.106-1.fc23.x86_64.rpm')],
'data/test-fedabipkgdiff/test5-same-dir-dbus-glib-0.100.2-2.fc20.x86_64--dbus-glib-0.106-1.fc23.x86_64-report-0.txt',
'output/test-fedabipkgdiff/test5-same-dir-dbus-glib-0.100.2-2.fc20.x86_64--dbus-glib-0.106-1.fc23.x86_64-report-0.txt'),
([os.path.join(INPUT_DIR, 'nss-util/nss-util-3.12.6-1.fc14.x86_64.rpm'),
os.path.join(INPUT_DIR, 'nss-util/nss-util-3.24.0-2.0.fc25.x86_64.rpm')],
'data/test-fedabipkgdiff/test6-missing-devel-debuginfo-nss-util-3.12.6-1.fc14.x86_64--nss-util-3.24.0-2.0.fc25.x86_64-report-0.txt',
'output/test-fedabipkgdiff/test6-missing-devel-debuginfo-nss-util-3.12.6-1.fc14.x86_64--nss-util-3.24.0-2.0.fc25.x86_64-report-0.txt'),
]
def ensure_output_dir_created():
'''Create output dir if it's not already created.'''
try:
os.makedirs(OUTPUT_DIR)
except:
pass
if not os.path.exists(OUTPUT_DIR):
sys.exit(1)
def run_fedabipkgdiff_tests():
"""Run the fedabipkgdiff tests
Loop through the test inputs in the FEDABIPKGDIFF_TEST_SPECS global
variable and for each of the test input, launch a comparison using
mockfedabipkgdiff, which calls fedabipkgdiff by making sure it
doesn't use the network.
The result of the comparison is stored in an output file, and that
output file is compared (using GNU diff) against the reference
output file specified in the FEDABIPKGDIFF_TEST_SPECS global
variable.
This function returns True iff all comparison specified by
FEDABIPKGDIFF_TEST_SPECS succeed.
"""
result = True
for args, reference_report_path, output_path in FEDABIPKGDIFF_TEST_SPECS:
reference_report_path = os.path.join(TEST_SRC_DIR, reference_report_path)
output_path = os.path.join(TEST_BUILD_DIR, output_path)
cmd = [FEDABIPKGDIFF,
'--no-default-suppression',
'--show-identical-binaries'] + args
with open(output_path, 'w') as out_file:
subprocess.call(cmd, stdout=out_file)
if not os.path.exists(reference_report_path):
sys.stderr.write('file not found: ' + reference_report_path)
return_code = -1
else:
diffcmd = ['diff', '-u', reference_report_path, output_path]
return_code = subprocess.call(diffcmd)
if return_code:
sys.stderr.write('fedabipkgdiff test failed for ' +
reference_report_path + '\n')
result &= not return_code
return result
def main():
"""The main entry point of this program.
This creates the output directory and launches the tests for the
fedabipkgdiff program.
"""
ensure_output_dir_created()
result = 0
result = run_fedabipkgdiff_tests()
if not result:
return result
if __name__ == '__main__':
exit_code = main()
sys.exit(exit_code)