libabigail/tests/mockfedabipkgdiff.in

393 lines
14 KiB
Plaintext
Raw Normal View History

Bug 22722 - Make fedabipkgdiff and its tests support both python 3 and 2 This patch makes fedabipkgdiff Python 3 compatible. All tests written in Python are updated and compatible with Python 3 as well. The patch looks for a Python 3 interperter. If it finds one then it runs the tests using that interpreter. Otherwise it just tries to use the Python 2 interpreter. This behaviour can be disabled by the new --disable-python3 option. * configure.ac: Add new option --enable-python3. Add new test runner file tests/runtestdefaultsupprs-py3 and tests/runtestfedabipkgdiffpy3.sh. Add required six Python module. * tests/Makefile.am: Add new test files tests/runtestdefaultsupprspy3.sh and tests/runtestfedabipkgdiffpy3.sh accordingly. * tests/mockfedabipkgdiff.in: Convert print statement to six.print_. Replace call to function filter with list comprehension. Replace basestring with six.string_types. * tests/runtestdefaultsupprspy3.sh.in: New shell script to run test runtestdefaultsupprs with Python 3. * tests/runtestdefaultsupprs.py.in: Repalce a few tabs with proper number of spaces which is detected by Python 3 interpreter. * tests/runtestfedabipkgdiffpy3.sh.in: New shell script to run test runtestfedabipkgdiff with Python 3. * tests/runtestfedabipkgdiff.py.in: Use python from env in shebang instead of a fixed path to a Python interpreter. * tools/fedabipkgdiff: Globally replace print statement with a function call to print which is available by importing print_function from __future__ module. Use six.print_ to output string to stderr instead. Convert function call to map to for-loop. (cmp_nvr): Change argument to handle a Koji build mapping instead of only the nvr. (Brew.listBuilds): use the new cmp_nvr to sort builds. Signed-off-by: Chenxiong Qi <cqi@redhat.com> Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2018-03-25 07:34:59 +00:00
#!/usr/bin/env python
Re-license the project to Apache v2 With LLVM Exception Thanks to the previous work done, changing the license is just a matter of changing the SPDX identifer from "LGPL-3.0-or-later" to "Apache-2.0 WITH LLVM-exception". Note that for the abigail.m4, tests/test-dot.cc and tests/test-svg.cc the change was from "GPL-3.0-or-later WITH GCC-exception-3.1" to "Apache-2.0 WITH LLVM-exception". include/abg-cxx-compat.h was changed from "LGPL-2.0-or-later" to "Apache-2.0 WITH LLVM-exception". Source code of programs (as opposed to source code of the library) where generally licensed under GPL-3.0-or-later; they are also now licensed "Apache-2.0 WITH LLVM-exception". This is what this patch does. * abigail.m4: Change the SPDX identifier from "GPL-3.0-or-later WITH GCC-exception-3.1" to "Apache-2.0 WITH LLVM-exception" * include/abg-cxx-compat.h: Change the SPDX identifier from "LGPL-2.0-or-later" to "Apache-2.0 WITH LLVM-exception". * .clang-format: Change the SPDX identifier from "LGPL-3.0-or-later" to "Apache-2.0 WITH LLVM-exception". * Makefile.am: Likewise. * bash-completion/Makefile.am: Likewise. * bash-completion/abicompat: Likewise. * bash-completion/abidiff: Likewise. * bash-completion/abidw: Likewise. * bash-completion/abilint: Likewise. * bash-completion/abinilint: Likewise. * bash-completion/abipkgdiff: Likewise. * bash-completion/abisym: Likewise. * bash-completion/fedabipkgdiff: Likewise. * configure.ac: Likewise. * default.abignore: Likewise. * doc/Makefile.am: Likewise. * doc/api/libabigail.doxy: Likewise. * doc/manuals/Makefile.am: Likewise. * doc/website/libabigail-website.doxy: Likewise. * include/Makefile.am: Likewise. * include/abg-comp-filter.h: Likewise. * include/abg-comparison.h: Likewise. * include/abg-config.h: Likewise. * include/abg-corpus.h: Likewise. * include/abg-diff-utils.h: Likewise. * include/abg-dwarf-reader.h: Likewise. * include/abg-fwd.h: Likewise. * include/abg-hash.h: Likewise. * include/abg-ini.h: Likewise. * include/abg-interned-str.h: Likewise. * include/abg-ir.h: Likewise. * include/abg-libxml-utils.h: Likewise. * include/abg-libzip-utils.h: Likewise. * include/abg-reader.h: Likewise. * include/abg-regex.h: Likewise. * include/abg-reporter.h: Likewise. * include/abg-sptr-utils.h: Likewise. * include/abg-suppression.h: Likewise. * include/abg-tools-utils.h: Likewise. * include/abg-traverse.h: Likewise. * include/abg-version.h.in: Likewise. * include/abg-viz-common.h: Likewise. * include/abg-viz-dot.h: Likewise. * include/abg-viz-svg.h: Likewise. * include/abg-workers.h: Likewise. * include/abg-writer.h: Likewise. * scripts/dot_to_png.sh: Likewise. * scripts/dot_to_svg.sh: Likewise. * scripts/make-verbose.sh: Likewise. * scripts/svg_to_plain_svg.sh: Likewise. * scripts/svg_to_png_and_pdf.sh: Likewise. * src/Makefile.am: Likewise. * src/abg-comp-filter.cc: Likewise. * src/abg-comparison-priv.h: Likewise. * src/abg-comparison.cc: Likewise. * src/abg-config.cc: Likewise. * src/abg-corpus-priv.h: Likewise. * src/abg-corpus.cc: Likewise. * src/abg-default-reporter.cc: Likewise. * src/abg-diff-utils.cc: Likewise. * src/abg-dwarf-reader.cc: Likewise. * src/abg-elf-helpers.cc: Likewise. * src/abg-elf-helpers.h: Likewise. * src/abg-hash.cc: Likewise. * src/abg-ini.cc: Likewise. * src/abg-internal.h: Likewise. * src/abg-ir-priv.h: Likewise. * src/abg-ir.cc: Likewise. * src/abg-leaf-reporter.cc: Likewise. * src/abg-libxml-utils.cc: Likewise. * src/abg-libzip-utils.cc: Likewise. * src/abg-reader.cc: Likewise. * src/abg-regex.cc: Likewise. * src/abg-reporter-priv.cc: Likewise. * src/abg-reporter-priv.h: Likewise. * src/abg-suppression-priv.h: Likewise. * src/abg-suppression.cc: Likewise. * src/abg-tools-utils.cc: Likewise. * src/abg-traverse.cc: Likewise. * src/abg-viz-common.cc: Likewise. * src/abg-viz-dot.cc: Likewise. * src/abg-viz-svg.cc: Likewise. * src/abg-workers.cc: Likewise. * src/abg-writer.cc: Likewise. * tests/Makefile.am: Likewise. * tests/data/Makefile.am: Likewise. * tests/lib/catch.cc: Likewise. * tests/mockfedabipkgdiff.in: Likewise. * tests/print-diff-tree.cc: Likewise. * tests/runtestcanonicalizetypes.sh.in: Likewise. * tests/runtestdefaultsupprs.py.in: Likewise. * tests/runtestdefaultsupprspy3.sh.in: Likewise. * tests/runtestfedabipkgdiff.py.in: Likewise. * tests/runtestfedabipkgdiffpy3.sh.in: Likewise. * tests/test-abicompat.cc: Likewise. * tests/test-abidiff-exit.cc: Likewise. * tests/test-abidiff.cc: Likewise. * tests/test-alt-dwarf-file.cc: Likewise. * tests/test-annotate.cc: Likewise. * tests/test-core-diff.cc: Likewise. * tests/test-cxx-compat.cc: Likewise. * tests/test-diff-dwarf-abixml.cc: Likewise. * tests/test-diff-dwarf.cc: Likewise. * tests/test-diff-filter.cc: Likewise. * tests/test-diff-pkg.cc: Likewise. * tests/test-diff-suppr.cc: Likewise. * tests/test-diff2.cc: Likewise. * tests/test-dot.cc: Change the SPDX identifier from "GPL-3.0-or-later WITH GCC-exception-3.1" to "Apache-2.0 WITH LLVM-exception" * tests/test-elf-helpers.cc: Change the SPDX identifier from "LGPL-3.0-or-later" to "Apache-2.0 WITH LLVM-exception" * tests/test-ini.cc: Likewise. * tests/test-ir-walker.cc: Likewise. * tests/test-kmi-whitelist.cc: Likewise. * tests/test-lookup-syms.cc: Likewise. * tests/test-read-dwarf.cc: Likewise. * tests/test-read-write.cc: Likewise. * tests/test-svg.cc: Change the SPDX identifier from "GPL-3.0-or-later WITH GCC-exception-3.1" to "Apache-2.0 WITH LLVM-exception". * tests/test-symtab.cc: Change the SPDX identifier from "LGPL-3.0-or-later" to "Apache-2.0 WITH LLVM-exception" * tests/test-tools-utils.cc: Likewise. * tests/test-types-stability.cc: Likewise. * tests/test-utils.cc: Likewise. * tests/test-utils.h: Likewise. * tests/test-write-read-archive.cc: Likewise. * tests/update-test-output.py: Likewise. * tools/Makefile.am: Likewise. * tools/abiar.cc: Likewise. * tools/abicompat.cc: Likewise. * tools/abidiff.cc: Likewise. * tools/abidw.cc: Likewise. * tools/abilint.cc: Likewise. * tools/abipkgdiff.cc: Likewise. * tools/abisym.cc: Likewise. * tools/binilint.cc: Likewise. * tools/fedabipkgdiff: Likewise. * tools/kmidiff.cc: Likewise. * update-copyright.sh: Likewise. Signed-off-by: Benjamin De Kosnik <bkoz@gnu.org> Signed-off-by: Ben Woodard <woodard@redhat.com> Signed-off-by: Chenxiong Qi <cqi@redhat.com> Signed-off-by: Dodji Seketeli <dodji@redhat.com> Signed-off-by: Giuliano Procida <gprocida@google.com> Signed-off-by: Jan Engelhardt <jengelh@inai.de> Signed-off-by: Jessica Yu <jeyu@kernel.org> Signed-off-by: Jonathan Wakely <jwakely@redhat.com> Signed-off-by: Mark Wielaard <mark@klomp.org> Signed-off-by: Matthias Klose <doko@ubuntu.com> Signed-off-by: Matthias Maennich <maennich@google.com> Signed-off-by: Ondrej Oprala <ondrej.oprala@gmail.com> Signed-off-by: Roland McGrath <roland@hack.frob.com> Signed-off-by: Sinny Kumari <ksinny@gmail.com> Signed-off-by: Slava Barinov <v.barinov@samsung.com>
2020-05-29 14:26:04 +00:00
# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
Make fedabipkgdiff consistent with Libabigail's other tests In the libabigail project, almost all the tests for the tools follow a similar pattern. The test contains a set of input files to compare. The result of the comparison is thus compared to a set of reference comparison result. This approach is already documented in the CONTRIBUTING file. There are several interesting properties with this scheme. First, it capture the behaviour of the tools, including what is shown to the user. Second, it eases the job of a hacker who wants to add a new test for a new behaviour of a given tool. The user just has to provide: 1/ A new reference output of the new use case of the tool (that is easily constructed by using the tool itself and saving its output) and add an entry to array of entries that describe what to compare 2/ A new set of inputs to the tool And voila. Unfortunately, fedabipkgdiff tests don't follow this scheme. That make them surprising to hackers who read the source code of the existing tests, at very least. Also, the fedabipkgdiff tests were only unit tests. They were not testing the tool as used by users in general. This patch makes the fedabipkgdiff tests follow the general approach of the tests of the other Libabigail tools. The patch first craetes a program names tests/mockfedabipkgdiff. It's a wrapper around tools/fedabipkgdiff. It overloads the Koji client of fedabipkgdiff with a fake Koji client that gets the Fedora packages locally, from tests/data/test-fedabipkgdiff/packages. In other words, mockfedabipkgdiff is fedabipkgdiff without going to the network. I believe that in the future, tests/mockfedabipkgdiff should be killed when fedabipkgdiff is able to cache a local description of a local partial view of a Koji repository, along with the build packages that were already downloaded. That way, calling fedabipkgdiff twice with the same set of option should make the tool perform correctly without going to the netword on the second invocation. We should be able to save the local partial view of the Koji repository under e.g, tests/data/test-fedabipkgdiff/local-koji and tell fedabipkgdiff to use that, instead of using the network. But we are not there yet. So for now, I am using mockfedabipkgdiff. Then, tests/runtestfedabipkdiff.py.in has been re-written to use tests/mockfedabipkgdiff and to look like what is described in CONTRIBUTING as far as how Libabigail tools' tests are organized: mockfedabipkgdiff is called to perform a comparison. The result of the comparison is then compared (using GNU diff) to a reference comparison result file. Please note that tests/runtestfedabipkdiff.py is relatively fast for the moment. But when it contains much more tests and start becoming slow, then we'll need to change the code to run several comparisons in parallel, just like we do today in tests/test-diff-filter.cc. At that point, I believe I'll just re-write this in C++, just like tests/test-diff-filter.cc because that will allow us to have true concurrent code based on the abigail:workers API. For now, I am keeping this in Python also because I think that keeps Chenxiong happy ;-) To be sure that fedabipkgdiff (and its underlying abipkgdiff) are really comparing all the packages they are supposed to compare and also all the binaries in those packages, I have added a new --show-identical-binaries to abipkgdiff and fedabipkgdiff. That option shows the name of the binaries (and packages) that are compared, *even if* the ABI of those binaries are identical. Because otherwise, today, if two binaries (or packages) have the same ABI, nothing is displayed. For instance, here is the result of comparing a package against itself, using this new --show-identical-binaries options: dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ./build/tools/abipkgdiff --show-identical-binaries dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm: ================ changes of 'dbus-binding-tool'=============== No ABI change detected ================ end of changes of 'dbus-binding-tool'=============== ================ changes of 'libdbus-glib-1.so.2.1.0'=============== No ABI change detected ================ end of changes of 'libdbus-glib-1.so.2.1.0'=============== dodji@adjoa:patches$ And here is what this command before that patch would do: dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ../master/build/tools/abipkgdiff dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm: dodji@adjoa:patches$ The rest of the patch is mostly new test inputs material and the necessary adjustments following all the changes above. * configure.ac: Do not require Python dependencies itertools, unittest and StringIO anymore as they are not used anymore. Require new module tempfile now. Generate new executable script tests/mockfedabipkgdiff from tests/mockfedabipkgdiff.in. * doc/manuals/abipkgdiff.rst: Add doc for new option --show-identical-binaries to abipkgdiff * doc/manuals/fedabipkgdiff.rst: Add doc for new options --show-identical-binaries to fedabipkgdiff. * tools/abipkgdiff.cc (options::show_identical_binaries): New data member. (options::options): Initialize new data member. (display_usage): Add a new help string for the new --show-identical-binaries option. (parse_command_line): Parse the newq --show-identical-binaries command line switch. (pthread_routine_compare): When the comparison of two binaries is empty, if --show-identical-binaries was provided, then emit some output saying the comparison did yield the empty set. * tools/fedabipkgdiff (DEFAULT_ABIPKGDIFF): Store the default path to abipkgdiff in this new global variable. Naming this default path is useful because it can then be cleanly overloaded when using mock.patch. (build_path_to_abipkgdiff): Return the new DEFAULT_ABIPKGDIFF global variable. (cmd): Parse the new --show-identical-binaries command line switch. * tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt: New reference output. * tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt: Likewise. * tests/mockfedabipkgdiff.in: New uninstalled script template. * tests/runtestfedabipkgdiff.py.in (counter) (temp_file_or_dir_prefix, UtilsTest, RPMTest, LocalRPMTest) (RunAbipkgdiffTest, GetPackageLatestBuildTest, DownloadRPMTest) (BrewListRPMsTest, AssertionHelper, MockGlobalConfig) (BUILT_ABIPKGDIFF, CompareABIFromCommandLineTest): Remove these classes, global variables and functions. (FEDABIPKGDIFF, TEST_SRC_DIR, TEST_BUILD_DIR, INPUT_DIR) (OUTPUT_DIR, FEDABIPKGDIFF_TEST_SPECS): New global variables. (ensure_output_dir_created, run_fedabipkgdiff_tests, main): New functions. * tests/test-diff-pkg.cc (in_out_specs): Add tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt to the set of reference outputs to consider. * tests/Makefile.am: Add non-installed script mockfedabipkgdiff to source distribution. Also added tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt, tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt, tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt, tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt and tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt to source distribution. Signed-off-by: Chenxiong Qi <cqi@redhat.com> Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2016-07-06 09:51:13 +00:00
# -*- coding: utf-8 -*-
# -*- Mode: Python
#
# Author: Dodji Seketeli
#
# Based on some preliminary work from Chenxiong Qi, posted to
# https://sourceware.org/ml/libabigail/2016-q3/msg00031.html.
'''A local-only version of the fedabipkgdiff program
This program behaves exactly like the fedabipkgdiff tool, except
that it doesn't use the network (via the Koji client) to get Fedora
packages. The Koji client is the interface that is used to access
the Fedora build system to get Fedora binary packages.
This program thus loads the fedabipkgdiff tool and overloads
(replaces) its Koji Client session with a fake (also known as a
mock) Koji client named MockClientSession that gets the packages
locally. Packages are stored under the directory
tests/data/test-fedabipkgdiff/packages. This program then invokes
the fedabipkgdiff program just as if the user invoked it from the
command line.
This program is useful for tests that can be called from "make
check" because those must be able to perform in environments where
no network is avaible.
Please note that the descriptions of the builds of each package are
stored in at least three global variables below: packages, builds
and rpms.
Whenever a user wants to add a new build to the local set of builds
locally available via MockClientSession, she must update these three
variables.
'''
import os
import tempfile
import imp
Bug 22722 - Make fedabipkgdiff and its tests support both python 3 and 2 This patch makes fedabipkgdiff Python 3 compatible. All tests written in Python are updated and compatible with Python 3 as well. The patch looks for a Python 3 interperter. If it finds one then it runs the tests using that interpreter. Otherwise it just tries to use the Python 2 interpreter. This behaviour can be disabled by the new --disable-python3 option. * configure.ac: Add new option --enable-python3. Add new test runner file tests/runtestdefaultsupprs-py3 and tests/runtestfedabipkgdiffpy3.sh. Add required six Python module. * tests/Makefile.am: Add new test files tests/runtestdefaultsupprspy3.sh and tests/runtestfedabipkgdiffpy3.sh accordingly. * tests/mockfedabipkgdiff.in: Convert print statement to six.print_. Replace call to function filter with list comprehension. Replace basestring with six.string_types. * tests/runtestdefaultsupprspy3.sh.in: New shell script to run test runtestdefaultsupprs with Python 3. * tests/runtestdefaultsupprs.py.in: Repalce a few tabs with proper number of spaces which is detected by Python 3 interpreter. * tests/runtestfedabipkgdiffpy3.sh.in: New shell script to run test runtestfedabipkgdiff with Python 3. * tests/runtestfedabipkgdiff.py.in: Use python from env in shebang instead of a fixed path to a Python interpreter. * tools/fedabipkgdiff: Globally replace print statement with a function call to print which is available by importing print_function from __future__ module. Use six.print_ to output string to stderr instead. Convert function call to map to for-loop. (cmp_nvr): Change argument to handle a Koji build mapping instead of only the nvr. (Brew.listBuilds): use the new cmp_nvr to sort builds. Signed-off-by: Chenxiong Qi <cqi@redhat.com> Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2018-03-25 07:34:59 +00:00
import six
Make fedabipkgdiff consistent with Libabigail's other tests In the libabigail project, almost all the tests for the tools follow a similar pattern. The test contains a set of input files to compare. The result of the comparison is thus compared to a set of reference comparison result. This approach is already documented in the CONTRIBUTING file. There are several interesting properties with this scheme. First, it capture the behaviour of the tools, including what is shown to the user. Second, it eases the job of a hacker who wants to add a new test for a new behaviour of a given tool. The user just has to provide: 1/ A new reference output of the new use case of the tool (that is easily constructed by using the tool itself and saving its output) and add an entry to array of entries that describe what to compare 2/ A new set of inputs to the tool And voila. Unfortunately, fedabipkgdiff tests don't follow this scheme. That make them surprising to hackers who read the source code of the existing tests, at very least. Also, the fedabipkgdiff tests were only unit tests. They were not testing the tool as used by users in general. This patch makes the fedabipkgdiff tests follow the general approach of the tests of the other Libabigail tools. The patch first craetes a program names tests/mockfedabipkgdiff. It's a wrapper around tools/fedabipkgdiff. It overloads the Koji client of fedabipkgdiff with a fake Koji client that gets the Fedora packages locally, from tests/data/test-fedabipkgdiff/packages. In other words, mockfedabipkgdiff is fedabipkgdiff without going to the network. I believe that in the future, tests/mockfedabipkgdiff should be killed when fedabipkgdiff is able to cache a local description of a local partial view of a Koji repository, along with the build packages that were already downloaded. That way, calling fedabipkgdiff twice with the same set of option should make the tool perform correctly without going to the netword on the second invocation. We should be able to save the local partial view of the Koji repository under e.g, tests/data/test-fedabipkgdiff/local-koji and tell fedabipkgdiff to use that, instead of using the network. But we are not there yet. So for now, I am using mockfedabipkgdiff. Then, tests/runtestfedabipkdiff.py.in has been re-written to use tests/mockfedabipkgdiff and to look like what is described in CONTRIBUTING as far as how Libabigail tools' tests are organized: mockfedabipkgdiff is called to perform a comparison. The result of the comparison is then compared (using GNU diff) to a reference comparison result file. Please note that tests/runtestfedabipkdiff.py is relatively fast for the moment. But when it contains much more tests and start becoming slow, then we'll need to change the code to run several comparisons in parallel, just like we do today in tests/test-diff-filter.cc. At that point, I believe I'll just re-write this in C++, just like tests/test-diff-filter.cc because that will allow us to have true concurrent code based on the abigail:workers API. For now, I am keeping this in Python also because I think that keeps Chenxiong happy ;-) To be sure that fedabipkgdiff (and its underlying abipkgdiff) are really comparing all the packages they are supposed to compare and also all the binaries in those packages, I have added a new --show-identical-binaries to abipkgdiff and fedabipkgdiff. That option shows the name of the binaries (and packages) that are compared, *even if* the ABI of those binaries are identical. Because otherwise, today, if two binaries (or packages) have the same ABI, nothing is displayed. For instance, here is the result of comparing a package against itself, using this new --show-identical-binaries options: dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ./build/tools/abipkgdiff --show-identical-binaries dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm: ================ changes of 'dbus-binding-tool'=============== No ABI change detected ================ end of changes of 'dbus-binding-tool'=============== ================ changes of 'libdbus-glib-1.so.2.1.0'=============== No ABI change detected ================ end of changes of 'libdbus-glib-1.so.2.1.0'=============== dodji@adjoa:patches$ And here is what this command before that patch would do: dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ../master/build/tools/abipkgdiff dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm: dodji@adjoa:patches$ The rest of the patch is mostly new test inputs material and the necessary adjustments following all the changes above. * configure.ac: Do not require Python dependencies itertools, unittest and StringIO anymore as they are not used anymore. Require new module tempfile now. Generate new executable script tests/mockfedabipkgdiff from tests/mockfedabipkgdiff.in. * doc/manuals/abipkgdiff.rst: Add doc for new option --show-identical-binaries to abipkgdiff * doc/manuals/fedabipkgdiff.rst: Add doc for new options --show-identical-binaries to fedabipkgdiff. * tools/abipkgdiff.cc (options::show_identical_binaries): New data member. (options::options): Initialize new data member. (display_usage): Add a new help string for the new --show-identical-binaries option. (parse_command_line): Parse the newq --show-identical-binaries command line switch. (pthread_routine_compare): When the comparison of two binaries is empty, if --show-identical-binaries was provided, then emit some output saying the comparison did yield the empty set. * tools/fedabipkgdiff (DEFAULT_ABIPKGDIFF): Store the default path to abipkgdiff in this new global variable. Naming this default path is useful because it can then be cleanly overloaded when using mock.patch. (build_path_to_abipkgdiff): Return the new DEFAULT_ABIPKGDIFF global variable. (cmd): Parse the new --show-identical-binaries command line switch. * tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt: New reference output. * tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt: Likewise. * tests/mockfedabipkgdiff.in: New uninstalled script template. * tests/runtestfedabipkgdiff.py.in (counter) (temp_file_or_dir_prefix, UtilsTest, RPMTest, LocalRPMTest) (RunAbipkgdiffTest, GetPackageLatestBuildTest, DownloadRPMTest) (BrewListRPMsTest, AssertionHelper, MockGlobalConfig) (BUILT_ABIPKGDIFF, CompareABIFromCommandLineTest): Remove these classes, global variables and functions. (FEDABIPKGDIFF, TEST_SRC_DIR, TEST_BUILD_DIR, INPUT_DIR) (OUTPUT_DIR, FEDABIPKGDIFF_TEST_SPECS): New global variables. (ensure_output_dir_created, run_fedabipkgdiff_tests, main): New functions. * tests/test-diff-pkg.cc (in_out_specs): Add tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt to the set of reference outputs to consider. * tests/Makefile.am: Add non-installed script mockfedabipkgdiff to source distribution. Also added tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt, tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt, tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt, tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt and tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt to source distribution. Signed-off-by: Chenxiong Qi <cqi@redhat.com> Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2016-07-06 09:51:13 +00:00
try:
from mock import patch
except ImportError:
import sys
Bug 22722 - Make fedabipkgdiff and its tests support both python 3 and 2 This patch makes fedabipkgdiff Python 3 compatible. All tests written in Python are updated and compatible with Python 3 as well. The patch looks for a Python 3 interperter. If it finds one then it runs the tests using that interpreter. Otherwise it just tries to use the Python 2 interpreter. This behaviour can be disabled by the new --disable-python3 option. * configure.ac: Add new option --enable-python3. Add new test runner file tests/runtestdefaultsupprs-py3 and tests/runtestfedabipkgdiffpy3.sh. Add required six Python module. * tests/Makefile.am: Add new test files tests/runtestdefaultsupprspy3.sh and tests/runtestfedabipkgdiffpy3.sh accordingly. * tests/mockfedabipkgdiff.in: Convert print statement to six.print_. Replace call to function filter with list comprehension. Replace basestring with six.string_types. * tests/runtestdefaultsupprspy3.sh.in: New shell script to run test runtestdefaultsupprs with Python 3. * tests/runtestdefaultsupprs.py.in: Repalce a few tabs with proper number of spaces which is detected by Python 3 interpreter. * tests/runtestfedabipkgdiffpy3.sh.in: New shell script to run test runtestfedabipkgdiff with Python 3. * tests/runtestfedabipkgdiff.py.in: Use python from env in shebang instead of a fixed path to a Python interpreter. * tools/fedabipkgdiff: Globally replace print statement with a function call to print which is available by importing print_function from __future__ module. Use six.print_ to output string to stderr instead. Convert function call to map to for-loop. (cmp_nvr): Change argument to handle a Koji build mapping instead of only the nvr. (Brew.listBuilds): use the new cmp_nvr to sort builds. Signed-off-by: Chenxiong Qi <cqi@redhat.com> Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2018-03-25 07:34:59 +00:00
six.print_('mock is required to run tests. Please install before running'
' tests.', file=sys.stderr)
Make fedabipkgdiff consistent with Libabigail's other tests In the libabigail project, almost all the tests for the tools follow a similar pattern. The test contains a set of input files to compare. The result of the comparison is thus compared to a set of reference comparison result. This approach is already documented in the CONTRIBUTING file. There are several interesting properties with this scheme. First, it capture the behaviour of the tools, including what is shown to the user. Second, it eases the job of a hacker who wants to add a new test for a new behaviour of a given tool. The user just has to provide: 1/ A new reference output of the new use case of the tool (that is easily constructed by using the tool itself and saving its output) and add an entry to array of entries that describe what to compare 2/ A new set of inputs to the tool And voila. Unfortunately, fedabipkgdiff tests don't follow this scheme. That make them surprising to hackers who read the source code of the existing tests, at very least. Also, the fedabipkgdiff tests were only unit tests. They were not testing the tool as used by users in general. This patch makes the fedabipkgdiff tests follow the general approach of the tests of the other Libabigail tools. The patch first craetes a program names tests/mockfedabipkgdiff. It's a wrapper around tools/fedabipkgdiff. It overloads the Koji client of fedabipkgdiff with a fake Koji client that gets the Fedora packages locally, from tests/data/test-fedabipkgdiff/packages. In other words, mockfedabipkgdiff is fedabipkgdiff without going to the network. I believe that in the future, tests/mockfedabipkgdiff should be killed when fedabipkgdiff is able to cache a local description of a local partial view of a Koji repository, along with the build packages that were already downloaded. That way, calling fedabipkgdiff twice with the same set of option should make the tool perform correctly without going to the netword on the second invocation. We should be able to save the local partial view of the Koji repository under e.g, tests/data/test-fedabipkgdiff/local-koji and tell fedabipkgdiff to use that, instead of using the network. But we are not there yet. So for now, I am using mockfedabipkgdiff. Then, tests/runtestfedabipkdiff.py.in has been re-written to use tests/mockfedabipkgdiff and to look like what is described in CONTRIBUTING as far as how Libabigail tools' tests are organized: mockfedabipkgdiff is called to perform a comparison. The result of the comparison is then compared (using GNU diff) to a reference comparison result file. Please note that tests/runtestfedabipkdiff.py is relatively fast for the moment. But when it contains much more tests and start becoming slow, then we'll need to change the code to run several comparisons in parallel, just like we do today in tests/test-diff-filter.cc. At that point, I believe I'll just re-write this in C++, just like tests/test-diff-filter.cc because that will allow us to have true concurrent code based on the abigail:workers API. For now, I am keeping this in Python also because I think that keeps Chenxiong happy ;-) To be sure that fedabipkgdiff (and its underlying abipkgdiff) are really comparing all the packages they are supposed to compare and also all the binaries in those packages, I have added a new --show-identical-binaries to abipkgdiff and fedabipkgdiff. That option shows the name of the binaries (and packages) that are compared, *even if* the ABI of those binaries are identical. Because otherwise, today, if two binaries (or packages) have the same ABI, nothing is displayed. For instance, here is the result of comparing a package against itself, using this new --show-identical-binaries options: dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ./build/tools/abipkgdiff --show-identical-binaries dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm: ================ changes of 'dbus-binding-tool'=============== No ABI change detected ================ end of changes of 'dbus-binding-tool'=============== ================ changes of 'libdbus-glib-1.so.2.1.0'=============== No ABI change detected ================ end of changes of 'libdbus-glib-1.so.2.1.0'=============== dodji@adjoa:patches$ And here is what this command before that patch would do: dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ../master/build/tools/abipkgdiff dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm: dodji@adjoa:patches$ The rest of the patch is mostly new test inputs material and the necessary adjustments following all the changes above. * configure.ac: Do not require Python dependencies itertools, unittest and StringIO anymore as they are not used anymore. Require new module tempfile now. Generate new executable script tests/mockfedabipkgdiff from tests/mockfedabipkgdiff.in. * doc/manuals/abipkgdiff.rst: Add doc for new option --show-identical-binaries to abipkgdiff * doc/manuals/fedabipkgdiff.rst: Add doc for new options --show-identical-binaries to fedabipkgdiff. * tools/abipkgdiff.cc (options::show_identical_binaries): New data member. (options::options): Initialize new data member. (display_usage): Add a new help string for the new --show-identical-binaries option. (parse_command_line): Parse the newq --show-identical-binaries command line switch. (pthread_routine_compare): When the comparison of two binaries is empty, if --show-identical-binaries was provided, then emit some output saying the comparison did yield the empty set. * tools/fedabipkgdiff (DEFAULT_ABIPKGDIFF): Store the default path to abipkgdiff in this new global variable. Naming this default path is useful because it can then be cleanly overloaded when using mock.patch. (build_path_to_abipkgdiff): Return the new DEFAULT_ABIPKGDIFF global variable. (cmd): Parse the new --show-identical-binaries command line switch. * tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt: New reference output. * tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt: Likewise. * tests/mockfedabipkgdiff.in: New uninstalled script template. * tests/runtestfedabipkgdiff.py.in (counter) (temp_file_or_dir_prefix, UtilsTest, RPMTest, LocalRPMTest) (RunAbipkgdiffTest, GetPackageLatestBuildTest, DownloadRPMTest) (BrewListRPMsTest, AssertionHelper, MockGlobalConfig) (BUILT_ABIPKGDIFF, CompareABIFromCommandLineTest): Remove these classes, global variables and functions. (FEDABIPKGDIFF, TEST_SRC_DIR, TEST_BUILD_DIR, INPUT_DIR) (OUTPUT_DIR, FEDABIPKGDIFF_TEST_SPECS): New global variables. (ensure_output_dir_created, run_fedabipkgdiff_tests, main): New functions. * tests/test-diff-pkg.cc (in_out_specs): Add tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt to the set of reference outputs to consider. * tests/Makefile.am: Add non-installed script mockfedabipkgdiff to source distribution. Also added tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt, tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt, tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt, tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt and tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt to source distribution. Signed-off-by: Chenxiong Qi <cqi@redhat.com> Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2016-07-06 09:51:13 +00:00
sys.exit(1)
ABIPKGDIFF = '@abs_top_builddir@/tools/abipkgdiff'
FEDABIPKGDIFF = '@abs_top_srcdir@/tools/fedabipkgdiff'
INPUT_DIR = '@abs_top_srcdir@/tests/data/test-fedabipkgdiff'
OUTPUT_DIR = '@abs_top_builddir@/tests/output/test-fedabipkgdiff'
TEST_TOPDIR = 'file://{0}'.format(INPUT_DIR)
DOWNLOAD_CACHE_DIR_PREFIX=os.path.join(OUTPUT_DIR, 'download-cache')
if not os.path.exists(DOWNLOAD_CACHE_DIR_PREFIX):
os.makedirs(DOWNLOAD_CACHE_DIR_PREFIX)
DOWNLOAD_CACHE_DIR=tempfile.mkdtemp(dir=DOWNLOAD_CACHE_DIR_PREFIX)
Make fedabipkgdiff consistent with Libabigail's other tests In the libabigail project, almost all the tests for the tools follow a similar pattern. The test contains a set of input files to compare. The result of the comparison is thus compared to a set of reference comparison result. This approach is already documented in the CONTRIBUTING file. There are several interesting properties with this scheme. First, it capture the behaviour of the tools, including what is shown to the user. Second, it eases the job of a hacker who wants to add a new test for a new behaviour of a given tool. The user just has to provide: 1/ A new reference output of the new use case of the tool (that is easily constructed by using the tool itself and saving its output) and add an entry to array of entries that describe what to compare 2/ A new set of inputs to the tool And voila. Unfortunately, fedabipkgdiff tests don't follow this scheme. That make them surprising to hackers who read the source code of the existing tests, at very least. Also, the fedabipkgdiff tests were only unit tests. They were not testing the tool as used by users in general. This patch makes the fedabipkgdiff tests follow the general approach of the tests of the other Libabigail tools. The patch first craetes a program names tests/mockfedabipkgdiff. It's a wrapper around tools/fedabipkgdiff. It overloads the Koji client of fedabipkgdiff with a fake Koji client that gets the Fedora packages locally, from tests/data/test-fedabipkgdiff/packages. In other words, mockfedabipkgdiff is fedabipkgdiff without going to the network. I believe that in the future, tests/mockfedabipkgdiff should be killed when fedabipkgdiff is able to cache a local description of a local partial view of a Koji repository, along with the build packages that were already downloaded. That way, calling fedabipkgdiff twice with the same set of option should make the tool perform correctly without going to the netword on the second invocation. We should be able to save the local partial view of the Koji repository under e.g, tests/data/test-fedabipkgdiff/local-koji and tell fedabipkgdiff to use that, instead of using the network. But we are not there yet. So for now, I am using mockfedabipkgdiff. Then, tests/runtestfedabipkdiff.py.in has been re-written to use tests/mockfedabipkgdiff and to look like what is described in CONTRIBUTING as far as how Libabigail tools' tests are organized: mockfedabipkgdiff is called to perform a comparison. The result of the comparison is then compared (using GNU diff) to a reference comparison result file. Please note that tests/runtestfedabipkdiff.py is relatively fast for the moment. But when it contains much more tests and start becoming slow, then we'll need to change the code to run several comparisons in parallel, just like we do today in tests/test-diff-filter.cc. At that point, I believe I'll just re-write this in C++, just like tests/test-diff-filter.cc because that will allow us to have true concurrent code based on the abigail:workers API. For now, I am keeping this in Python also because I think that keeps Chenxiong happy ;-) To be sure that fedabipkgdiff (and its underlying abipkgdiff) are really comparing all the packages they are supposed to compare and also all the binaries in those packages, I have added a new --show-identical-binaries to abipkgdiff and fedabipkgdiff. That option shows the name of the binaries (and packages) that are compared, *even if* the ABI of those binaries are identical. Because otherwise, today, if two binaries (or packages) have the same ABI, nothing is displayed. For instance, here is the result of comparing a package against itself, using this new --show-identical-binaries options: dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ./build/tools/abipkgdiff --show-identical-binaries dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm: ================ changes of 'dbus-binding-tool'=============== No ABI change detected ================ end of changes of 'dbus-binding-tool'=============== ================ changes of 'libdbus-glib-1.so.2.1.0'=============== No ABI change detected ================ end of changes of 'libdbus-glib-1.so.2.1.0'=============== dodji@adjoa:patches$ And here is what this command before that patch would do: dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ../master/build/tools/abipkgdiff dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm: dodji@adjoa:patches$ The rest of the patch is mostly new test inputs material and the necessary adjustments following all the changes above. * configure.ac: Do not require Python dependencies itertools, unittest and StringIO anymore as they are not used anymore. Require new module tempfile now. Generate new executable script tests/mockfedabipkgdiff from tests/mockfedabipkgdiff.in. * doc/manuals/abipkgdiff.rst: Add doc for new option --show-identical-binaries to abipkgdiff * doc/manuals/fedabipkgdiff.rst: Add doc for new options --show-identical-binaries to fedabipkgdiff. * tools/abipkgdiff.cc (options::show_identical_binaries): New data member. (options::options): Initialize new data member. (display_usage): Add a new help string for the new --show-identical-binaries option. (parse_command_line): Parse the newq --show-identical-binaries command line switch. (pthread_routine_compare): When the comparison of two binaries is empty, if --show-identical-binaries was provided, then emit some output saying the comparison did yield the empty set. * tools/fedabipkgdiff (DEFAULT_ABIPKGDIFF): Store the default path to abipkgdiff in this new global variable. Naming this default path is useful because it can then be cleanly overloaded when using mock.patch. (build_path_to_abipkgdiff): Return the new DEFAULT_ABIPKGDIFF global variable. (cmd): Parse the new --show-identical-binaries command line switch. * tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt: New reference output. * tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt: Likewise. * tests/mockfedabipkgdiff.in: New uninstalled script template. * tests/runtestfedabipkgdiff.py.in (counter) (temp_file_or_dir_prefix, UtilsTest, RPMTest, LocalRPMTest) (RunAbipkgdiffTest, GetPackageLatestBuildTest, DownloadRPMTest) (BrewListRPMsTest, AssertionHelper, MockGlobalConfig) (BUILT_ABIPKGDIFF, CompareABIFromCommandLineTest): Remove these classes, global variables and functions. (FEDABIPKGDIFF, TEST_SRC_DIR, TEST_BUILD_DIR, INPUT_DIR) (OUTPUT_DIR, FEDABIPKGDIFF_TEST_SPECS): New global variables. (ensure_output_dir_created, run_fedabipkgdiff_tests, main): New functions. * tests/test-diff-pkg.cc (in_out_specs): Add tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt to the set of reference outputs to consider. * tests/Makefile.am: Add non-installed script mockfedabipkgdiff to source distribution. Also added tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt, tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt, tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt, tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt and tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt to source distribution. Signed-off-by: Chenxiong Qi <cqi@redhat.com> Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2016-07-06 09:51:13 +00:00
def get_download_dir():
"""Mock get_download_dir"""
if not os.path.exists(DOWNLOAD_CACHE_DIR):
os.makedirs(DOWNLOAD_CACHE_DIR)
Make fedabipkgdiff consistent with Libabigail's other tests In the libabigail project, almost all the tests for the tools follow a similar pattern. The test contains a set of input files to compare. The result of the comparison is thus compared to a set of reference comparison result. This approach is already documented in the CONTRIBUTING file. There are several interesting properties with this scheme. First, it capture the behaviour of the tools, including what is shown to the user. Second, it eases the job of a hacker who wants to add a new test for a new behaviour of a given tool. The user just has to provide: 1/ A new reference output of the new use case of the tool (that is easily constructed by using the tool itself and saving its output) and add an entry to array of entries that describe what to compare 2/ A new set of inputs to the tool And voila. Unfortunately, fedabipkgdiff tests don't follow this scheme. That make them surprising to hackers who read the source code of the existing tests, at very least. Also, the fedabipkgdiff tests were only unit tests. They were not testing the tool as used by users in general. This patch makes the fedabipkgdiff tests follow the general approach of the tests of the other Libabigail tools. The patch first craetes a program names tests/mockfedabipkgdiff. It's a wrapper around tools/fedabipkgdiff. It overloads the Koji client of fedabipkgdiff with a fake Koji client that gets the Fedora packages locally, from tests/data/test-fedabipkgdiff/packages. In other words, mockfedabipkgdiff is fedabipkgdiff without going to the network. I believe that in the future, tests/mockfedabipkgdiff should be killed when fedabipkgdiff is able to cache a local description of a local partial view of a Koji repository, along with the build packages that were already downloaded. That way, calling fedabipkgdiff twice with the same set of option should make the tool perform correctly without going to the netword on the second invocation. We should be able to save the local partial view of the Koji repository under e.g, tests/data/test-fedabipkgdiff/local-koji and tell fedabipkgdiff to use that, instead of using the network. But we are not there yet. So for now, I am using mockfedabipkgdiff. Then, tests/runtestfedabipkdiff.py.in has been re-written to use tests/mockfedabipkgdiff and to look like what is described in CONTRIBUTING as far as how Libabigail tools' tests are organized: mockfedabipkgdiff is called to perform a comparison. The result of the comparison is then compared (using GNU diff) to a reference comparison result file. Please note that tests/runtestfedabipkdiff.py is relatively fast for the moment. But when it contains much more tests and start becoming slow, then we'll need to change the code to run several comparisons in parallel, just like we do today in tests/test-diff-filter.cc. At that point, I believe I'll just re-write this in C++, just like tests/test-diff-filter.cc because that will allow us to have true concurrent code based on the abigail:workers API. For now, I am keeping this in Python also because I think that keeps Chenxiong happy ;-) To be sure that fedabipkgdiff (and its underlying abipkgdiff) are really comparing all the packages they are supposed to compare and also all the binaries in those packages, I have added a new --show-identical-binaries to abipkgdiff and fedabipkgdiff. That option shows the name of the binaries (and packages) that are compared, *even if* the ABI of those binaries are identical. Because otherwise, today, if two binaries (or packages) have the same ABI, nothing is displayed. For instance, here is the result of comparing a package against itself, using this new --show-identical-binaries options: dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ./build/tools/abipkgdiff --show-identical-binaries dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm: ================ changes of 'dbus-binding-tool'=============== No ABI change detected ================ end of changes of 'dbus-binding-tool'=============== ================ changes of 'libdbus-glib-1.so.2.1.0'=============== No ABI change detected ================ end of changes of 'libdbus-glib-1.so.2.1.0'=============== dodji@adjoa:patches$ And here is what this command before that patch would do: dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ../master/build/tools/abipkgdiff dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm: dodji@adjoa:patches$ The rest of the patch is mostly new test inputs material and the necessary adjustments following all the changes above. * configure.ac: Do not require Python dependencies itertools, unittest and StringIO anymore as they are not used anymore. Require new module tempfile now. Generate new executable script tests/mockfedabipkgdiff from tests/mockfedabipkgdiff.in. * doc/manuals/abipkgdiff.rst: Add doc for new option --show-identical-binaries to abipkgdiff * doc/manuals/fedabipkgdiff.rst: Add doc for new options --show-identical-binaries to fedabipkgdiff. * tools/abipkgdiff.cc (options::show_identical_binaries): New data member. (options::options): Initialize new data member. (display_usage): Add a new help string for the new --show-identical-binaries option. (parse_command_line): Parse the newq --show-identical-binaries command line switch. (pthread_routine_compare): When the comparison of two binaries is empty, if --show-identical-binaries was provided, then emit some output saying the comparison did yield the empty set. * tools/fedabipkgdiff (DEFAULT_ABIPKGDIFF): Store the default path to abipkgdiff in this new global variable. Naming this default path is useful because it can then be cleanly overloaded when using mock.patch. (build_path_to_abipkgdiff): Return the new DEFAULT_ABIPKGDIFF global variable. (cmd): Parse the new --show-identical-binaries command line switch. * tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt: New reference output. * tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt: Likewise. * tests/mockfedabipkgdiff.in: New uninstalled script template. * tests/runtestfedabipkgdiff.py.in (counter) (temp_file_or_dir_prefix, UtilsTest, RPMTest, LocalRPMTest) (RunAbipkgdiffTest, GetPackageLatestBuildTest, DownloadRPMTest) (BrewListRPMsTest, AssertionHelper, MockGlobalConfig) (BUILT_ABIPKGDIFF, CompareABIFromCommandLineTest): Remove these classes, global variables and functions. (FEDABIPKGDIFF, TEST_SRC_DIR, TEST_BUILD_DIR, INPUT_DIR) (OUTPUT_DIR, FEDABIPKGDIFF_TEST_SPECS): New global variables. (ensure_output_dir_created, run_fedabipkgdiff_tests, main): New functions. * tests/test-diff-pkg.cc (in_out_specs): Add tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt to the set of reference outputs to consider. * tests/Makefile.am: Add non-installed script mockfedabipkgdiff to source distribution. Also added tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt, tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt, tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt, tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt and tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt to source distribution. Signed-off-by: Chenxiong Qi <cqi@redhat.com> Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2016-07-06 09:51:13 +00:00
return DOWNLOAD_CACHE_DIR
# Import the fedabipkgdiff program file from the source directory.
fedabipkgdiff_mod = imp.load_source('fedabipkgdiff', FEDABIPKGDIFF)
# ----------------- Koji resource storage begins ------------------
#
# List all necessary Koji resources for running tests within this test
# module. Currently, packages, builds, and rpms are listed here, and their
# relationship is maintained well. At the same time, all information including
# the ID for each package, build and rpm is real and can be queried from Koji,
# that is for convenience once developers need this.
#
# When additional packages, builds and rpms are required for test cases, here
# is the right place to add them. Just think them as a super simple in-memory
# database, and methods of MockClientSession knows well how to read them.
packages = [
{'id': 286, 'name': 'gnutls'},
{'id': 612, 'name': 'dbus-glib'},
{'id': 9041, 'name': 'nss-util'},
fedabipkgdiff refuses to compare packages with the same release number I tried to run: fedabipkgdiff vte291-0.39.1-1.fc22.x86_64 vte291-0.39.90-1.fc22.x86_64 And it wouldn't work :-( The program considers the two packages as not being "peers". This has to do with the RPM.is_peer method which considers the two package as not being "peers", just because they have the same release number (1.fc22). They should be considered peers, though, because they have the same name but different {version, release} pairs. This patch fixes the RPM.is_peer method and adds the aforementioned packages to the regression test suite for good measure. * tools/fedabipkgdiff (RPM.is_peer): Update comment. Fix logic. * tests/data/test-fedabipkgdiff/packages/vte291/0.39.1/1.fc22/x86_64/vte291-0.39.1-1.fc22.x86_64.rpm: New test input file. * tests/data/test-fedabipkgdiff/packages/vte291/0.39.1/1.fc22/x86_64/vte291-debuginfo-0.39.1-1.fc22.x86_64.rpm: Likewise. * tests/data/test-fedabipkgdiff/packages/vte291/0.39.1/1.fc22/x86_64/vte291-devel-0.39.1-1.fc22.x86_64.rpm: Likewise. * tests/data/test-fedabipkgdiff/packages/vte291/0.39.90/1.fc22/x86_64/vte291-0.39.90-1.fc22.x86_64.rpm: Likewise. * tests/data/test-fedabipkgdiff/packages/vte291/0.39.90/1.fc22/x86_64/vte291-debuginfo-0.39.90-1.fc22.x86_64.rpm: Likewise. * tests/data/test-fedabipkgdiff/packages/vte291/0.39.90/1.fc22/x86_64/vte291-devel-0.39.90-1.fc22.x86_64.rpm: Likewise. * tests/data/test-fedabipkgdiff/vte291-0.39.1-1.fc22.x86_64--vte291-0.39.90-1.fc22.x86_64-report-0.txt: Likewise. * tests/data/Makefile.am: Add the new test input data to source distribution. * tests/mockfedabipkgdiff.in: Update the package and build information to add the new vte291-0.39.1-1.fc22.x86_64.rpm and vte291-0.39.90-1.fc22.x86_64.rpm packages (as well as their devel and debuginfo packages) into the "mock" Koji build database. * tests/runtestfedabipkgdiff.py.in: Make this test harness run over the two aforementioned packages. Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2017-01-23 22:28:12 +00:00
{'id': 18494, 'name': 'vte291'},
Make fedabipkgdiff consistent with Libabigail's other tests In the libabigail project, almost all the tests for the tools follow a similar pattern. The test contains a set of input files to compare. The result of the comparison is thus compared to a set of reference comparison result. This approach is already documented in the CONTRIBUTING file. There are several interesting properties with this scheme. First, it capture the behaviour of the tools, including what is shown to the user. Second, it eases the job of a hacker who wants to add a new test for a new behaviour of a given tool. The user just has to provide: 1/ A new reference output of the new use case of the tool (that is easily constructed by using the tool itself and saving its output) and add an entry to array of entries that describe what to compare 2/ A new set of inputs to the tool And voila. Unfortunately, fedabipkgdiff tests don't follow this scheme. That make them surprising to hackers who read the source code of the existing tests, at very least. Also, the fedabipkgdiff tests were only unit tests. They were not testing the tool as used by users in general. This patch makes the fedabipkgdiff tests follow the general approach of the tests of the other Libabigail tools. The patch first craetes a program names tests/mockfedabipkgdiff. It's a wrapper around tools/fedabipkgdiff. It overloads the Koji client of fedabipkgdiff with a fake Koji client that gets the Fedora packages locally, from tests/data/test-fedabipkgdiff/packages. In other words, mockfedabipkgdiff is fedabipkgdiff without going to the network. I believe that in the future, tests/mockfedabipkgdiff should be killed when fedabipkgdiff is able to cache a local description of a local partial view of a Koji repository, along with the build packages that were already downloaded. That way, calling fedabipkgdiff twice with the same set of option should make the tool perform correctly without going to the netword on the second invocation. We should be able to save the local partial view of the Koji repository under e.g, tests/data/test-fedabipkgdiff/local-koji and tell fedabipkgdiff to use that, instead of using the network. But we are not there yet. So for now, I am using mockfedabipkgdiff. Then, tests/runtestfedabipkdiff.py.in has been re-written to use tests/mockfedabipkgdiff and to look like what is described in CONTRIBUTING as far as how Libabigail tools' tests are organized: mockfedabipkgdiff is called to perform a comparison. The result of the comparison is then compared (using GNU diff) to a reference comparison result file. Please note that tests/runtestfedabipkdiff.py is relatively fast for the moment. But when it contains much more tests and start becoming slow, then we'll need to change the code to run several comparisons in parallel, just like we do today in tests/test-diff-filter.cc. At that point, I believe I'll just re-write this in C++, just like tests/test-diff-filter.cc because that will allow us to have true concurrent code based on the abigail:workers API. For now, I am keeping this in Python also because I think that keeps Chenxiong happy ;-) To be sure that fedabipkgdiff (and its underlying abipkgdiff) are really comparing all the packages they are supposed to compare and also all the binaries in those packages, I have added a new --show-identical-binaries to abipkgdiff and fedabipkgdiff. That option shows the name of the binaries (and packages) that are compared, *even if* the ABI of those binaries are identical. Because otherwise, today, if two binaries (or packages) have the same ABI, nothing is displayed. For instance, here is the result of comparing a package against itself, using this new --show-identical-binaries options: dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ./build/tools/abipkgdiff --show-identical-binaries dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm: ================ changes of 'dbus-binding-tool'=============== No ABI change detected ================ end of changes of 'dbus-binding-tool'=============== ================ changes of 'libdbus-glib-1.so.2.1.0'=============== No ABI change detected ================ end of changes of 'libdbus-glib-1.so.2.1.0'=============== dodji@adjoa:patches$ And here is what this command before that patch would do: dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ../master/build/tools/abipkgdiff dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm: dodji@adjoa:patches$ The rest of the patch is mostly new test inputs material and the necessary adjustments following all the changes above. * configure.ac: Do not require Python dependencies itertools, unittest and StringIO anymore as they are not used anymore. Require new module tempfile now. Generate new executable script tests/mockfedabipkgdiff from tests/mockfedabipkgdiff.in. * doc/manuals/abipkgdiff.rst: Add doc for new option --show-identical-binaries to abipkgdiff * doc/manuals/fedabipkgdiff.rst: Add doc for new options --show-identical-binaries to fedabipkgdiff. * tools/abipkgdiff.cc (options::show_identical_binaries): New data member. (options::options): Initialize new data member. (display_usage): Add a new help string for the new --show-identical-binaries option. (parse_command_line): Parse the newq --show-identical-binaries command line switch. (pthread_routine_compare): When the comparison of two binaries is empty, if --show-identical-binaries was provided, then emit some output saying the comparison did yield the empty set. * tools/fedabipkgdiff (DEFAULT_ABIPKGDIFF): Store the default path to abipkgdiff in this new global variable. Naming this default path is useful because it can then be cleanly overloaded when using mock.patch. (build_path_to_abipkgdiff): Return the new DEFAULT_ABIPKGDIFF global variable. (cmd): Parse the new --show-identical-binaries command line switch. * tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt: New reference output. * tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt: Likewise. * tests/mockfedabipkgdiff.in: New uninstalled script template. * tests/runtestfedabipkgdiff.py.in (counter) (temp_file_or_dir_prefix, UtilsTest, RPMTest, LocalRPMTest) (RunAbipkgdiffTest, GetPackageLatestBuildTest, DownloadRPMTest) (BrewListRPMsTest, AssertionHelper, MockGlobalConfig) (BUILT_ABIPKGDIFF, CompareABIFromCommandLineTest): Remove these classes, global variables and functions. (FEDABIPKGDIFF, TEST_SRC_DIR, TEST_BUILD_DIR, INPUT_DIR) (OUTPUT_DIR, FEDABIPKGDIFF_TEST_SPECS): New global variables. (ensure_output_dir_created, run_fedabipkgdiff_tests, main): New functions. * tests/test-diff-pkg.cc (in_out_specs): Add tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt to the set of reference outputs to consider. * tests/Makefile.am: Add non-installed script mockfedabipkgdiff to source distribution. Also added tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt, tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt, tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt, tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt and tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt to source distribution. Signed-off-by: Chenxiong Qi <cqi@redhat.com> Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2016-07-06 09:51:13 +00:00
]
builds = [
{'build_id': 428835, 'nvr': 'dbus-glib-0.100-4.fc20',
'name': 'dbus-glib', 'release': '4.fc20', 'version': '0.100',
'package_id': 612, 'package_name': 'dbus-glib', 'state': 1,
},
{'build_id': 430720, 'nvr': 'dbus-glib-0.100.2-1.fc20',
'name': 'dbus-glib', 'release': '1.fc20', 'version': '0.100.2',
'package_id': 612, 'package_name': 'dbus-glib', 'state': 1,
},
{'build_id': 442366, 'nvr': 'dbus-glib-0.100.2-2.fc20',
'name': 'dbus-glib', 'release': '2.fc20', 'version': '0.100.2',
'package_id': 612, 'package_name': 'dbus-glib', 'state': 1,
},
{'build_id': 715478, 'nvr': 'dbus-glib-0.106-1.fc23',
'name': 'dbus-glib', 'release': '1.fc23', 'version': '0.106',
'package_id': 612, 'package_name': 'dbus-glib', 'state': 1,
},
{'build_id': 648058, 'nvr': 'dbus-glib-0.104-3.fc23',
'name': 'dbus-glib', 'release': '3.fc23', 'version': '0.104',
'package_id': 612, 'package_name': 'dbus-glib', 'state': 1,
},
{'build_id': 613769, 'nvr': 'dbus-glib-0.104-2.fc23',
'name': 'dbus-glib', 'release': '2.fc23', 'version': '0.104',
'package_id': 612, 'package_name': 'dbus-glib', 'state': 1,
},
{'build_id': 160295, 'nvr': 'nss-util-3.12.6-1.fc14',
'name': 'nss-util', 'version': '3.12.6', 'release': '1.fc14',
'package_id': 9041, 'package_name': 'nss-util', 'state': 1,
},
{'build_id': 767978, 'nvr': 'nss-util-3.24.0-2.0.fc25',
'name': 'nss-util', 'version': '3.24.0', 'release': '2.0.fc25',
'package_id': 9041, 'package_name': 'nss-util', 'state': 1,
},
# builds of package gnutls
{'build_id': 767306, 'nvr': 'gnutls-3.4.12-1.fc23',
'name': 'gnutls', 'release': '1.fc23', 'version': '3.4.12',
'package_id': 286, 'package_name': 'gnutls', 'state': 1,
},
{'build_id': 770965, 'nvr': 'gnutls-3.4.13-1.fc23',
'name': 'gnutls', 'release': '1.fc23', 'version': '3.4.13',
'package_id': 286, 'package_name': 'gnutls', 'state': 1,
},
{'build_id': 649701, 'nvr': 'gnutls-3.4.2-1.fc23',
'name': 'gnutls', 'release': '1.fc23', 'version': '3.4.2',
'package_id': 286, 'package_name': 'gnutls', 'state': 1,
},
fedabipkgdiff refuses to compare packages with the same release number I tried to run: fedabipkgdiff vte291-0.39.1-1.fc22.x86_64 vte291-0.39.90-1.fc22.x86_64 And it wouldn't work :-( The program considers the two packages as not being "peers". This has to do with the RPM.is_peer method which considers the two package as not being "peers", just because they have the same release number (1.fc22). They should be considered peers, though, because they have the same name but different {version, release} pairs. This patch fixes the RPM.is_peer method and adds the aforementioned packages to the regression test suite for good measure. * tools/fedabipkgdiff (RPM.is_peer): Update comment. Fix logic. * tests/data/test-fedabipkgdiff/packages/vte291/0.39.1/1.fc22/x86_64/vte291-0.39.1-1.fc22.x86_64.rpm: New test input file. * tests/data/test-fedabipkgdiff/packages/vte291/0.39.1/1.fc22/x86_64/vte291-debuginfo-0.39.1-1.fc22.x86_64.rpm: Likewise. * tests/data/test-fedabipkgdiff/packages/vte291/0.39.1/1.fc22/x86_64/vte291-devel-0.39.1-1.fc22.x86_64.rpm: Likewise. * tests/data/test-fedabipkgdiff/packages/vte291/0.39.90/1.fc22/x86_64/vte291-0.39.90-1.fc22.x86_64.rpm: Likewise. * tests/data/test-fedabipkgdiff/packages/vte291/0.39.90/1.fc22/x86_64/vte291-debuginfo-0.39.90-1.fc22.x86_64.rpm: Likewise. * tests/data/test-fedabipkgdiff/packages/vte291/0.39.90/1.fc22/x86_64/vte291-devel-0.39.90-1.fc22.x86_64.rpm: Likewise. * tests/data/test-fedabipkgdiff/vte291-0.39.1-1.fc22.x86_64--vte291-0.39.90-1.fc22.x86_64-report-0.txt: Likewise. * tests/data/Makefile.am: Add the new test input data to source distribution. * tests/mockfedabipkgdiff.in: Update the package and build information to add the new vte291-0.39.1-1.fc22.x86_64.rpm and vte291-0.39.90-1.fc22.x86_64.rpm packages (as well as their devel and debuginfo packages) into the "mock" Koji build database. * tests/runtestfedabipkgdiff.py.in: Make this test harness run over the two aforementioned packages. Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2017-01-23 22:28:12 +00:00
# builds of package vte291
{'build_id': 600011, 'nvr': 'vte291-0.39.1-1.fc22',
'name': 'vte291', 'version': '0.39.1', 'release': '1.fc22',
'package_id': 18494, 'package_name': 'vte291', 'state': 1,
},
{'build_id': 612610, 'nvr': 'vte291-0.39.90-1.fc22',
'name': 'vte291', 'version': '0.39.90', 'release': '1.fc22',
'package_id': 18494, 'package_name': 'vte291', 'state': 1,
},
Make fedabipkgdiff consistent with Libabigail's other tests In the libabigail project, almost all the tests for the tools follow a similar pattern. The test contains a set of input files to compare. The result of the comparison is thus compared to a set of reference comparison result. This approach is already documented in the CONTRIBUTING file. There are several interesting properties with this scheme. First, it capture the behaviour of the tools, including what is shown to the user. Second, it eases the job of a hacker who wants to add a new test for a new behaviour of a given tool. The user just has to provide: 1/ A new reference output of the new use case of the tool (that is easily constructed by using the tool itself and saving its output) and add an entry to array of entries that describe what to compare 2/ A new set of inputs to the tool And voila. Unfortunately, fedabipkgdiff tests don't follow this scheme. That make them surprising to hackers who read the source code of the existing tests, at very least. Also, the fedabipkgdiff tests were only unit tests. They were not testing the tool as used by users in general. This patch makes the fedabipkgdiff tests follow the general approach of the tests of the other Libabigail tools. The patch first craetes a program names tests/mockfedabipkgdiff. It's a wrapper around tools/fedabipkgdiff. It overloads the Koji client of fedabipkgdiff with a fake Koji client that gets the Fedora packages locally, from tests/data/test-fedabipkgdiff/packages. In other words, mockfedabipkgdiff is fedabipkgdiff without going to the network. I believe that in the future, tests/mockfedabipkgdiff should be killed when fedabipkgdiff is able to cache a local description of a local partial view of a Koji repository, along with the build packages that were already downloaded. That way, calling fedabipkgdiff twice with the same set of option should make the tool perform correctly without going to the netword on the second invocation. We should be able to save the local partial view of the Koji repository under e.g, tests/data/test-fedabipkgdiff/local-koji and tell fedabipkgdiff to use that, instead of using the network. But we are not there yet. So for now, I am using mockfedabipkgdiff. Then, tests/runtestfedabipkdiff.py.in has been re-written to use tests/mockfedabipkgdiff and to look like what is described in CONTRIBUTING as far as how Libabigail tools' tests are organized: mockfedabipkgdiff is called to perform a comparison. The result of the comparison is then compared (using GNU diff) to a reference comparison result file. Please note that tests/runtestfedabipkdiff.py is relatively fast for the moment. But when it contains much more tests and start becoming slow, then we'll need to change the code to run several comparisons in parallel, just like we do today in tests/test-diff-filter.cc. At that point, I believe I'll just re-write this in C++, just like tests/test-diff-filter.cc because that will allow us to have true concurrent code based on the abigail:workers API. For now, I am keeping this in Python also because I think that keeps Chenxiong happy ;-) To be sure that fedabipkgdiff (and its underlying abipkgdiff) are really comparing all the packages they are supposed to compare and also all the binaries in those packages, I have added a new --show-identical-binaries to abipkgdiff and fedabipkgdiff. That option shows the name of the binaries (and packages) that are compared, *even if* the ABI of those binaries are identical. Because otherwise, today, if two binaries (or packages) have the same ABI, nothing is displayed. For instance, here is the result of comparing a package against itself, using this new --show-identical-binaries options: dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ./build/tools/abipkgdiff --show-identical-binaries dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm: ================ changes of 'dbus-binding-tool'=============== No ABI change detected ================ end of changes of 'dbus-binding-tool'=============== ================ changes of 'libdbus-glib-1.so.2.1.0'=============== No ABI change detected ================ end of changes of 'libdbus-glib-1.so.2.1.0'=============== dodji@adjoa:patches$ And here is what this command before that patch would do: dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ../master/build/tools/abipkgdiff dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm: dodji@adjoa:patches$ The rest of the patch is mostly new test inputs material and the necessary adjustments following all the changes above. * configure.ac: Do not require Python dependencies itertools, unittest and StringIO anymore as they are not used anymore. Require new module tempfile now. Generate new executable script tests/mockfedabipkgdiff from tests/mockfedabipkgdiff.in. * doc/manuals/abipkgdiff.rst: Add doc for new option --show-identical-binaries to abipkgdiff * doc/manuals/fedabipkgdiff.rst: Add doc for new options --show-identical-binaries to fedabipkgdiff. * tools/abipkgdiff.cc (options::show_identical_binaries): New data member. (options::options): Initialize new data member. (display_usage): Add a new help string for the new --show-identical-binaries option. (parse_command_line): Parse the newq --show-identical-binaries command line switch. (pthread_routine_compare): When the comparison of two binaries is empty, if --show-identical-binaries was provided, then emit some output saying the comparison did yield the empty set. * tools/fedabipkgdiff (DEFAULT_ABIPKGDIFF): Store the default path to abipkgdiff in this new global variable. Naming this default path is useful because it can then be cleanly overloaded when using mock.patch. (build_path_to_abipkgdiff): Return the new DEFAULT_ABIPKGDIFF global variable. (cmd): Parse the new --show-identical-binaries command line switch. * tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt: New reference output. * tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt: Likewise. * tests/mockfedabipkgdiff.in: New uninstalled script template. * tests/runtestfedabipkgdiff.py.in (counter) (temp_file_or_dir_prefix, UtilsTest, RPMTest, LocalRPMTest) (RunAbipkgdiffTest, GetPackageLatestBuildTest, DownloadRPMTest) (BrewListRPMsTest, AssertionHelper, MockGlobalConfig) (BUILT_ABIPKGDIFF, CompareABIFromCommandLineTest): Remove these classes, global variables and functions. (FEDABIPKGDIFF, TEST_SRC_DIR, TEST_BUILD_DIR, INPUT_DIR) (OUTPUT_DIR, FEDABIPKGDIFF_TEST_SPECS): New global variables. (ensure_output_dir_created, run_fedabipkgdiff_tests, main): New functions. * tests/test-diff-pkg.cc (in_out_specs): Add tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt to the set of reference outputs to consider. * tests/Makefile.am: Add non-installed script mockfedabipkgdiff to source distribution. Also added tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt, tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt, tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt, tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt and tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt to source distribution. Signed-off-by: Chenxiong Qi <cqi@redhat.com> Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2016-07-06 09:51:13 +00:00
]
rpms = [
{'build_id': 442366,
'name': 'dbus-glib', 'release': '2.fc20', 'version': '0.100.2',
'arch': 'x86_64', 'nvr': 'dbus-glib-0.100.2-2.fc20',
},
{'build_id': 442366,
'name': 'dbus-glib-devel', 'release': '2.fc20', 'version': '0.100.2',
'arch': 'x86_64', 'nvr': 'dbus-glib-devel-0.100.2-2.fc20',
},
{'build_id': 442366,
'name': 'dbus-glib-debuginfo', 'release': '2.fc20', 'version': '0.100.2',
'arch': 'x86_64', 'nvr': 'dbus-glib-debuginfo-0.100.2-2.fc20',
},
{'build_id': 442366,
'name': 'dbus-glib-devel', 'release': '2.fc20', 'version': '0.100.2',
'arch': 'i686', 'nvr': 'dbus-glib-devel-0.100.2-2.fc20',
},
{'build_id': 442366,
'name': 'dbus-glib-debuginfo', 'release': '2.fc20', 'version': '0.100.2',
'arch': 'i686', 'nvr': 'dbus-glib-debuginfo-0.100.2-2.fc20',
},
{'build_id': 442366,
'name': 'dbus-glib', 'version': '0.100.2', 'release': '2.fc20',
'arch': 'i686', 'nvr': 'dbus-glib-0.100.2-2.fc20',
},
{'build_id': 715478,
'name': 'dbus-glib-debuginfo', 'version': '0.106', 'release': '1.fc23',
'arch': 'i686', 'nvr': 'dbus-glib-debuginfo-0.106-1.fc23',
},
{'build_id': 715478,
'name': 'dbus-glib', 'version': '0.106', 'release': '1.fc23',
'arch': 'i686', 'nvr': 'dbus-glib-0.106-1.fc23',
},
{'build_id': 715478,
'name': 'dbus-glib-devel', 'version': '0.106', 'release': '1.fc23',
'arch': 'i686', 'nvr': 'dbus-glib-devel-0.106-1.fc23',
},
{'build_id': 715478,
'name': 'dbus-glib', 'version': '0.106', 'release': '1.fc23',
'arch': 'x86_64', 'nvr': 'dbus-glib-0.106-1.fc23',
},
{'build_id': 715478,
'name': 'dbus-glib-debuginfo', 'version': '0.106', 'release': '1.fc23',
'arch': 'x86_64', 'nvr': 'dbus-glib-debuginfo-0.106-1.fc23',
},
{'build_id': 715478,
'name': 'dbus-glib-devel', 'release': '1.fc23', 'version': '0.106',
'arch': 'x86_64', 'nvr': 'dbus-glib-devel-0.106-1.fc23',
},
# RPMs of build nss-util-3.12.6-1.fc14
{'build_id': 160295,
'name': 'nss-util', 'release': '1.fc14', 'version': '3.12.6',
'arch': 'x86_64', 'nvr': 'nss-util-3.12.6-1.fc14',
},
{'build_id': 160295,
'name': 'nss-util-devel', 'release': '1.fc14', 'version': '3.12.6',
'arch': 'x86_64', 'nvr': 'nss-util-devel-3.12.6-1.fc14',
},
{'build_id': 160295,
'name': 'nss-util-debuginfo', 'release': '1.fc14', 'version': '3.12.6',
'arch': 'x86_64', 'nvr': 'nss-util-debuginfo-3.12.6-1.fc14',
},
# RPMs of build nss-util-3.24.0-2.0.fc25
{'build_id': 767978,
'name': 'nss-util-debuginfo', 'release': '2.0.fc25', 'version': '3.24.0',
'arch': 'x86_64', 'nvr': 'nss-util-debuginfo-3.24.0-2.0.fc25',
},
{'build_id': 767978,
'name': 'nss-util', 'release': '2.0.fc25', 'version': '3.24.0',
'arch': 'x86_64', 'nvr': 'nss-util-3.24.0-2.0.fc25',
},
{'build_id': 767978,
'name': 'nss-util-devel', 'release': '2.0.fc25', 'version': '3.24.0',
'arch': 'x86_64', 'nvr': 'nss-util-devel-3.24.0-2.0.fc25',
},
fedabipkgdiff refuses to compare packages with the same release number I tried to run: fedabipkgdiff vte291-0.39.1-1.fc22.x86_64 vte291-0.39.90-1.fc22.x86_64 And it wouldn't work :-( The program considers the two packages as not being "peers". This has to do with the RPM.is_peer method which considers the two package as not being "peers", just because they have the same release number (1.fc22). They should be considered peers, though, because they have the same name but different {version, release} pairs. This patch fixes the RPM.is_peer method and adds the aforementioned packages to the regression test suite for good measure. * tools/fedabipkgdiff (RPM.is_peer): Update comment. Fix logic. * tests/data/test-fedabipkgdiff/packages/vte291/0.39.1/1.fc22/x86_64/vte291-0.39.1-1.fc22.x86_64.rpm: New test input file. * tests/data/test-fedabipkgdiff/packages/vte291/0.39.1/1.fc22/x86_64/vte291-debuginfo-0.39.1-1.fc22.x86_64.rpm: Likewise. * tests/data/test-fedabipkgdiff/packages/vte291/0.39.1/1.fc22/x86_64/vte291-devel-0.39.1-1.fc22.x86_64.rpm: Likewise. * tests/data/test-fedabipkgdiff/packages/vte291/0.39.90/1.fc22/x86_64/vte291-0.39.90-1.fc22.x86_64.rpm: Likewise. * tests/data/test-fedabipkgdiff/packages/vte291/0.39.90/1.fc22/x86_64/vte291-debuginfo-0.39.90-1.fc22.x86_64.rpm: Likewise. * tests/data/test-fedabipkgdiff/packages/vte291/0.39.90/1.fc22/x86_64/vte291-devel-0.39.90-1.fc22.x86_64.rpm: Likewise. * tests/data/test-fedabipkgdiff/vte291-0.39.1-1.fc22.x86_64--vte291-0.39.90-1.fc22.x86_64-report-0.txt: Likewise. * tests/data/Makefile.am: Add the new test input data to source distribution. * tests/mockfedabipkgdiff.in: Update the package and build information to add the new vte291-0.39.1-1.fc22.x86_64.rpm and vte291-0.39.90-1.fc22.x86_64.rpm packages (as well as their devel and debuginfo packages) into the "mock" Koji build database. * tests/runtestfedabipkgdiff.py.in: Make this test harness run over the two aforementioned packages. Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2017-01-23 22:28:12 +00:00
# RPMs of build vte291-0.39.1-1.fc22
{'build_id': 600011,
'name': 'vte291', 'version': '0.39.1', 'release': '1.fc22',
'arch': 'x86_64', 'nvr': 'vte291-0.39.1-1.fc22',
},
{'build_id': 600011,
'name': 'vte291-devel', 'version': '0.39.1', 'release': '1.fc22',
'arch': 'x86_64', 'nvr': 'vte291-0.39.1-1.fc22',
},
{'build_id': 600011,
'name': 'vte291-debuginfo', 'version': '0.39.1', 'release': '1.fc22',
'arch': 'x86_64', 'nvr': 'vte291-0.39.1-1.fc22',
},
# RPMs of build vte291-0.39.90-1.fc22
{'build_id': 612610,
'name': 'vte291', 'version': '0.39.90', 'release': '1.fc22',
'arch': 'x86_64', 'nvr': 'vte291-0.39.90-1.fc22',
},
{'build_id': 612610,
'name': 'vte291-devel', 'version': '0.39.90', 'release': '1.fc22',
'arch': 'x86_64', 'nvr': 'vte291-0.39.90-1.fc22',
},
{'build_id': 612610,
'name': 'vte291-debuginfo', 'version': '0.39.90', 'release': '1.fc22',
'arch': 'x86_64', 'nvr': 'vte291-0.39.90-1.fc22',
},
]
Make fedabipkgdiff consistent with Libabigail's other tests In the libabigail project, almost all the tests for the tools follow a similar pattern. The test contains a set of input files to compare. The result of the comparison is thus compared to a set of reference comparison result. This approach is already documented in the CONTRIBUTING file. There are several interesting properties with this scheme. First, it capture the behaviour of the tools, including what is shown to the user. Second, it eases the job of a hacker who wants to add a new test for a new behaviour of a given tool. The user just has to provide: 1/ A new reference output of the new use case of the tool (that is easily constructed by using the tool itself and saving its output) and add an entry to array of entries that describe what to compare 2/ A new set of inputs to the tool And voila. Unfortunately, fedabipkgdiff tests don't follow this scheme. That make them surprising to hackers who read the source code of the existing tests, at very least. Also, the fedabipkgdiff tests were only unit tests. They were not testing the tool as used by users in general. This patch makes the fedabipkgdiff tests follow the general approach of the tests of the other Libabigail tools. The patch first craetes a program names tests/mockfedabipkgdiff. It's a wrapper around tools/fedabipkgdiff. It overloads the Koji client of fedabipkgdiff with a fake Koji client that gets the Fedora packages locally, from tests/data/test-fedabipkgdiff/packages. In other words, mockfedabipkgdiff is fedabipkgdiff without going to the network. I believe that in the future, tests/mockfedabipkgdiff should be killed when fedabipkgdiff is able to cache a local description of a local partial view of a Koji repository, along with the build packages that were already downloaded. That way, calling fedabipkgdiff twice with the same set of option should make the tool perform correctly without going to the netword on the second invocation. We should be able to save the local partial view of the Koji repository under e.g, tests/data/test-fedabipkgdiff/local-koji and tell fedabipkgdiff to use that, instead of using the network. But we are not there yet. So for now, I am using mockfedabipkgdiff. Then, tests/runtestfedabipkdiff.py.in has been re-written to use tests/mockfedabipkgdiff and to look like what is described in CONTRIBUTING as far as how Libabigail tools' tests are organized: mockfedabipkgdiff is called to perform a comparison. The result of the comparison is then compared (using GNU diff) to a reference comparison result file. Please note that tests/runtestfedabipkdiff.py is relatively fast for the moment. But when it contains much more tests and start becoming slow, then we'll need to change the code to run several comparisons in parallel, just like we do today in tests/test-diff-filter.cc. At that point, I believe I'll just re-write this in C++, just like tests/test-diff-filter.cc because that will allow us to have true concurrent code based on the abigail:workers API. For now, I am keeping this in Python also because I think that keeps Chenxiong happy ;-) To be sure that fedabipkgdiff (and its underlying abipkgdiff) are really comparing all the packages they are supposed to compare and also all the binaries in those packages, I have added a new --show-identical-binaries to abipkgdiff and fedabipkgdiff. That option shows the name of the binaries (and packages) that are compared, *even if* the ABI of those binaries are identical. Because otherwise, today, if two binaries (or packages) have the same ABI, nothing is displayed. For instance, here is the result of comparing a package against itself, using this new --show-identical-binaries options: dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ./build/tools/abipkgdiff --show-identical-binaries dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm: ================ changes of 'dbus-binding-tool'=============== No ABI change detected ================ end of changes of 'dbus-binding-tool'=============== ================ changes of 'libdbus-glib-1.so.2.1.0'=============== No ABI change detected ================ end of changes of 'libdbus-glib-1.so.2.1.0'=============== dodji@adjoa:patches$ And here is what this command before that patch would do: dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ../master/build/tools/abipkgdiff dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm: dodji@adjoa:patches$ The rest of the patch is mostly new test inputs material and the necessary adjustments following all the changes above. * configure.ac: Do not require Python dependencies itertools, unittest and StringIO anymore as they are not used anymore. Require new module tempfile now. Generate new executable script tests/mockfedabipkgdiff from tests/mockfedabipkgdiff.in. * doc/manuals/abipkgdiff.rst: Add doc for new option --show-identical-binaries to abipkgdiff * doc/manuals/fedabipkgdiff.rst: Add doc for new options --show-identical-binaries to fedabipkgdiff. * tools/abipkgdiff.cc (options::show_identical_binaries): New data member. (options::options): Initialize new data member. (display_usage): Add a new help string for the new --show-identical-binaries option. (parse_command_line): Parse the newq --show-identical-binaries command line switch. (pthread_routine_compare): When the comparison of two binaries is empty, if --show-identical-binaries was provided, then emit some output saying the comparison did yield the empty set. * tools/fedabipkgdiff (DEFAULT_ABIPKGDIFF): Store the default path to abipkgdiff in this new global variable. Naming this default path is useful because it can then be cleanly overloaded when using mock.patch. (build_path_to_abipkgdiff): Return the new DEFAULT_ABIPKGDIFF global variable. (cmd): Parse the new --show-identical-binaries command line switch. * tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt: New reference output. * tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt: Likewise. * tests/mockfedabipkgdiff.in: New uninstalled script template. * tests/runtestfedabipkgdiff.py.in (counter) (temp_file_or_dir_prefix, UtilsTest, RPMTest, LocalRPMTest) (RunAbipkgdiffTest, GetPackageLatestBuildTest, DownloadRPMTest) (BrewListRPMsTest, AssertionHelper, MockGlobalConfig) (BUILT_ABIPKGDIFF, CompareABIFromCommandLineTest): Remove these classes, global variables and functions. (FEDABIPKGDIFF, TEST_SRC_DIR, TEST_BUILD_DIR, INPUT_DIR) (OUTPUT_DIR, FEDABIPKGDIFF_TEST_SPECS): New global variables. (ensure_output_dir_created, run_fedabipkgdiff_tests, main): New functions. * tests/test-diff-pkg.cc (in_out_specs): Add tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt to the set of reference outputs to consider. * tests/Makefile.am: Add non-installed script mockfedabipkgdiff to source distribution. Also added tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt, tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt, tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt, tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt and tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt to source distribution. Signed-off-by: Chenxiong Qi <cqi@redhat.com> Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2016-07-06 09:51:13 +00:00
# ----------------- End of Koji resource storage ------------------
class MockClientSession(object):
"""Mock koji.ClientSession
This mock ClientSession aims to avoid touching a real Koji instance to
interact with XMLRPC APIs required by fedabipkgdiff.
For the tests within this module, methods do not necessarily to return
complete RPM and build information. So, if you need more additional
information, here is the right place to add them.
"""
def __init__(self, baseurl):
"""Initialize a mock ClientSession
:param str baseurl: the URL to remote kojihub service. As of writing
this mock class, `baseurl` is not used at all, just keep it here if
it's useful in the future.
All mock methods have same signature as corresponding kojihub.*
methods, and type of parameters may be different and only satify
fedabipkgdiff requirement.
"""
self.baseurl = baseurl
def getPackage(self, name):
"""Mock kojihub.getPackage
:param str name: name of package to find and return
:return: the found package
:rtype: dict
"""
Bug 22722 - Make fedabipkgdiff and its tests support both python 3 and 2 This patch makes fedabipkgdiff Python 3 compatible. All tests written in Python are updated and compatible with Python 3 as well. The patch looks for a Python 3 interperter. If it finds one then it runs the tests using that interpreter. Otherwise it just tries to use the Python 2 interpreter. This behaviour can be disabled by the new --disable-python3 option. * configure.ac: Add new option --enable-python3. Add new test runner file tests/runtestdefaultsupprs-py3 and tests/runtestfedabipkgdiffpy3.sh. Add required six Python module. * tests/Makefile.am: Add new test files tests/runtestdefaultsupprspy3.sh and tests/runtestfedabipkgdiffpy3.sh accordingly. * tests/mockfedabipkgdiff.in: Convert print statement to six.print_. Replace call to function filter with list comprehension. Replace basestring with six.string_types. * tests/runtestdefaultsupprspy3.sh.in: New shell script to run test runtestdefaultsupprs with Python 3. * tests/runtestdefaultsupprs.py.in: Repalce a few tabs with proper number of spaces which is detected by Python 3 interpreter. * tests/runtestfedabipkgdiffpy3.sh.in: New shell script to run test runtestfedabipkgdiff with Python 3. * tests/runtestfedabipkgdiff.py.in: Use python from env in shebang instead of a fixed path to a Python interpreter. * tools/fedabipkgdiff: Globally replace print statement with a function call to print which is available by importing print_function from __future__ module. Use six.print_ to output string to stderr instead. Convert function call to map to for-loop. (cmp_nvr): Change argument to handle a Koji build mapping instead of only the nvr. (Brew.listBuilds): use the new cmp_nvr to sort builds. Signed-off-by: Chenxiong Qi <cqi@redhat.com> Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2018-03-25 07:34:59 +00:00
assert isinstance(name, six.string_types)
Make fedabipkgdiff consistent with Libabigail's other tests In the libabigail project, almost all the tests for the tools follow a similar pattern. The test contains a set of input files to compare. The result of the comparison is thus compared to a set of reference comparison result. This approach is already documented in the CONTRIBUTING file. There are several interesting properties with this scheme. First, it capture the behaviour of the tools, including what is shown to the user. Second, it eases the job of a hacker who wants to add a new test for a new behaviour of a given tool. The user just has to provide: 1/ A new reference output of the new use case of the tool (that is easily constructed by using the tool itself and saving its output) and add an entry to array of entries that describe what to compare 2/ A new set of inputs to the tool And voila. Unfortunately, fedabipkgdiff tests don't follow this scheme. That make them surprising to hackers who read the source code of the existing tests, at very least. Also, the fedabipkgdiff tests were only unit tests. They were not testing the tool as used by users in general. This patch makes the fedabipkgdiff tests follow the general approach of the tests of the other Libabigail tools. The patch first craetes a program names tests/mockfedabipkgdiff. It's a wrapper around tools/fedabipkgdiff. It overloads the Koji client of fedabipkgdiff with a fake Koji client that gets the Fedora packages locally, from tests/data/test-fedabipkgdiff/packages. In other words, mockfedabipkgdiff is fedabipkgdiff without going to the network. I believe that in the future, tests/mockfedabipkgdiff should be killed when fedabipkgdiff is able to cache a local description of a local partial view of a Koji repository, along with the build packages that were already downloaded. That way, calling fedabipkgdiff twice with the same set of option should make the tool perform correctly without going to the netword on the second invocation. We should be able to save the local partial view of the Koji repository under e.g, tests/data/test-fedabipkgdiff/local-koji and tell fedabipkgdiff to use that, instead of using the network. But we are not there yet. So for now, I am using mockfedabipkgdiff. Then, tests/runtestfedabipkdiff.py.in has been re-written to use tests/mockfedabipkgdiff and to look like what is described in CONTRIBUTING as far as how Libabigail tools' tests are organized: mockfedabipkgdiff is called to perform a comparison. The result of the comparison is then compared (using GNU diff) to a reference comparison result file. Please note that tests/runtestfedabipkdiff.py is relatively fast for the moment. But when it contains much more tests and start becoming slow, then we'll need to change the code to run several comparisons in parallel, just like we do today in tests/test-diff-filter.cc. At that point, I believe I'll just re-write this in C++, just like tests/test-diff-filter.cc because that will allow us to have true concurrent code based on the abigail:workers API. For now, I am keeping this in Python also because I think that keeps Chenxiong happy ;-) To be sure that fedabipkgdiff (and its underlying abipkgdiff) are really comparing all the packages they are supposed to compare and also all the binaries in those packages, I have added a new --show-identical-binaries to abipkgdiff and fedabipkgdiff. That option shows the name of the binaries (and packages) that are compared, *even if* the ABI of those binaries are identical. Because otherwise, today, if two binaries (or packages) have the same ABI, nothing is displayed. For instance, here is the result of comparing a package against itself, using this new --show-identical-binaries options: dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ./build/tools/abipkgdiff --show-identical-binaries dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm: ================ changes of 'dbus-binding-tool'=============== No ABI change detected ================ end of changes of 'dbus-binding-tool'=============== ================ changes of 'libdbus-glib-1.so.2.1.0'=============== No ABI change detected ================ end of changes of 'libdbus-glib-1.so.2.1.0'=============== dodji@adjoa:patches$ And here is what this command before that patch would do: dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ../master/build/tools/abipkgdiff dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm: dodji@adjoa:patches$ The rest of the patch is mostly new test inputs material and the necessary adjustments following all the changes above. * configure.ac: Do not require Python dependencies itertools, unittest and StringIO anymore as they are not used anymore. Require new module tempfile now. Generate new executable script tests/mockfedabipkgdiff from tests/mockfedabipkgdiff.in. * doc/manuals/abipkgdiff.rst: Add doc for new option --show-identical-binaries to abipkgdiff * doc/manuals/fedabipkgdiff.rst: Add doc for new options --show-identical-binaries to fedabipkgdiff. * tools/abipkgdiff.cc (options::show_identical_binaries): New data member. (options::options): Initialize new data member. (display_usage): Add a new help string for the new --show-identical-binaries option. (parse_command_line): Parse the newq --show-identical-binaries command line switch. (pthread_routine_compare): When the comparison of two binaries is empty, if --show-identical-binaries was provided, then emit some output saying the comparison did yield the empty set. * tools/fedabipkgdiff (DEFAULT_ABIPKGDIFF): Store the default path to abipkgdiff in this new global variable. Naming this default path is useful because it can then be cleanly overloaded when using mock.patch. (build_path_to_abipkgdiff): Return the new DEFAULT_ABIPKGDIFF global variable. (cmd): Parse the new --show-identical-binaries command line switch. * tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt: New reference output. * tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt: Likewise. * tests/mockfedabipkgdiff.in: New uninstalled script template. * tests/runtestfedabipkgdiff.py.in (counter) (temp_file_or_dir_prefix, UtilsTest, RPMTest, LocalRPMTest) (RunAbipkgdiffTest, GetPackageLatestBuildTest, DownloadRPMTest) (BrewListRPMsTest, AssertionHelper, MockGlobalConfig) (BUILT_ABIPKGDIFF, CompareABIFromCommandLineTest): Remove these classes, global variables and functions. (FEDABIPKGDIFF, TEST_SRC_DIR, TEST_BUILD_DIR, INPUT_DIR) (OUTPUT_DIR, FEDABIPKGDIFF_TEST_SPECS): New global variables. (ensure_output_dir_created, run_fedabipkgdiff_tests, main): New functions. * tests/test-diff-pkg.cc (in_out_specs): Add tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt to the set of reference outputs to consider. * tests/Makefile.am: Add non-installed script mockfedabipkgdiff to source distribution. Also added tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt, tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt, tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt, tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt and tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt to source distribution. Signed-off-by: Chenxiong Qi <cqi@redhat.com> Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2016-07-06 09:51:13 +00:00
def selector(package):
return package['name'] == name
Bug 22722 - Make fedabipkgdiff and its tests support both python 3 and 2 This patch makes fedabipkgdiff Python 3 compatible. All tests written in Python are updated and compatible with Python 3 as well. The patch looks for a Python 3 interperter. If it finds one then it runs the tests using that interpreter. Otherwise it just tries to use the Python 2 interpreter. This behaviour can be disabled by the new --disable-python3 option. * configure.ac: Add new option --enable-python3. Add new test runner file tests/runtestdefaultsupprs-py3 and tests/runtestfedabipkgdiffpy3.sh. Add required six Python module. * tests/Makefile.am: Add new test files tests/runtestdefaultsupprspy3.sh and tests/runtestfedabipkgdiffpy3.sh accordingly. * tests/mockfedabipkgdiff.in: Convert print statement to six.print_. Replace call to function filter with list comprehension. Replace basestring with six.string_types. * tests/runtestdefaultsupprspy3.sh.in: New shell script to run test runtestdefaultsupprs with Python 3. * tests/runtestdefaultsupprs.py.in: Repalce a few tabs with proper number of spaces which is detected by Python 3 interpreter. * tests/runtestfedabipkgdiffpy3.sh.in: New shell script to run test runtestfedabipkgdiff with Python 3. * tests/runtestfedabipkgdiff.py.in: Use python from env in shebang instead of a fixed path to a Python interpreter. * tools/fedabipkgdiff: Globally replace print statement with a function call to print which is available by importing print_function from __future__ module. Use six.print_ to output string to stderr instead. Convert function call to map to for-loop. (cmp_nvr): Change argument to handle a Koji build mapping instead of only the nvr. (Brew.listBuilds): use the new cmp_nvr to sort builds. Signed-off-by: Chenxiong Qi <cqi@redhat.com> Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2018-03-25 07:34:59 +00:00
return [p for p in packages if selector(p)][0]
Make fedabipkgdiff consistent with Libabigail's other tests In the libabigail project, almost all the tests for the tools follow a similar pattern. The test contains a set of input files to compare. The result of the comparison is thus compared to a set of reference comparison result. This approach is already documented in the CONTRIBUTING file. There are several interesting properties with this scheme. First, it capture the behaviour of the tools, including what is shown to the user. Second, it eases the job of a hacker who wants to add a new test for a new behaviour of a given tool. The user just has to provide: 1/ A new reference output of the new use case of the tool (that is easily constructed by using the tool itself and saving its output) and add an entry to array of entries that describe what to compare 2/ A new set of inputs to the tool And voila. Unfortunately, fedabipkgdiff tests don't follow this scheme. That make them surprising to hackers who read the source code of the existing tests, at very least. Also, the fedabipkgdiff tests were only unit tests. They were not testing the tool as used by users in general. This patch makes the fedabipkgdiff tests follow the general approach of the tests of the other Libabigail tools. The patch first craetes a program names tests/mockfedabipkgdiff. It's a wrapper around tools/fedabipkgdiff. It overloads the Koji client of fedabipkgdiff with a fake Koji client that gets the Fedora packages locally, from tests/data/test-fedabipkgdiff/packages. In other words, mockfedabipkgdiff is fedabipkgdiff without going to the network. I believe that in the future, tests/mockfedabipkgdiff should be killed when fedabipkgdiff is able to cache a local description of a local partial view of a Koji repository, along with the build packages that were already downloaded. That way, calling fedabipkgdiff twice with the same set of option should make the tool perform correctly without going to the netword on the second invocation. We should be able to save the local partial view of the Koji repository under e.g, tests/data/test-fedabipkgdiff/local-koji and tell fedabipkgdiff to use that, instead of using the network. But we are not there yet. So for now, I am using mockfedabipkgdiff. Then, tests/runtestfedabipkdiff.py.in has been re-written to use tests/mockfedabipkgdiff and to look like what is described in CONTRIBUTING as far as how Libabigail tools' tests are organized: mockfedabipkgdiff is called to perform a comparison. The result of the comparison is then compared (using GNU diff) to a reference comparison result file. Please note that tests/runtestfedabipkdiff.py is relatively fast for the moment. But when it contains much more tests and start becoming slow, then we'll need to change the code to run several comparisons in parallel, just like we do today in tests/test-diff-filter.cc. At that point, I believe I'll just re-write this in C++, just like tests/test-diff-filter.cc because that will allow us to have true concurrent code based on the abigail:workers API. For now, I am keeping this in Python also because I think that keeps Chenxiong happy ;-) To be sure that fedabipkgdiff (and its underlying abipkgdiff) are really comparing all the packages they are supposed to compare and also all the binaries in those packages, I have added a new --show-identical-binaries to abipkgdiff and fedabipkgdiff. That option shows the name of the binaries (and packages) that are compared, *even if* the ABI of those binaries are identical. Because otherwise, today, if two binaries (or packages) have the same ABI, nothing is displayed. For instance, here is the result of comparing a package against itself, using this new --show-identical-binaries options: dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ./build/tools/abipkgdiff --show-identical-binaries dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm: ================ changes of 'dbus-binding-tool'=============== No ABI change detected ================ end of changes of 'dbus-binding-tool'=============== ================ changes of 'libdbus-glib-1.so.2.1.0'=============== No ABI change detected ================ end of changes of 'libdbus-glib-1.so.2.1.0'=============== dodji@adjoa:patches$ And here is what this command before that patch would do: dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ../master/build/tools/abipkgdiff dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm: dodji@adjoa:patches$ The rest of the patch is mostly new test inputs material and the necessary adjustments following all the changes above. * configure.ac: Do not require Python dependencies itertools, unittest and StringIO anymore as they are not used anymore. Require new module tempfile now. Generate new executable script tests/mockfedabipkgdiff from tests/mockfedabipkgdiff.in. * doc/manuals/abipkgdiff.rst: Add doc for new option --show-identical-binaries to abipkgdiff * doc/manuals/fedabipkgdiff.rst: Add doc for new options --show-identical-binaries to fedabipkgdiff. * tools/abipkgdiff.cc (options::show_identical_binaries): New data member. (options::options): Initialize new data member. (display_usage): Add a new help string for the new --show-identical-binaries option. (parse_command_line): Parse the newq --show-identical-binaries command line switch. (pthread_routine_compare): When the comparison of two binaries is empty, if --show-identical-binaries was provided, then emit some output saying the comparison did yield the empty set. * tools/fedabipkgdiff (DEFAULT_ABIPKGDIFF): Store the default path to abipkgdiff in this new global variable. Naming this default path is useful because it can then be cleanly overloaded when using mock.patch. (build_path_to_abipkgdiff): Return the new DEFAULT_ABIPKGDIFF global variable. (cmd): Parse the new --show-identical-binaries command line switch. * tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt: New reference output. * tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt: Likewise. * tests/mockfedabipkgdiff.in: New uninstalled script template. * tests/runtestfedabipkgdiff.py.in (counter) (temp_file_or_dir_prefix, UtilsTest, RPMTest, LocalRPMTest) (RunAbipkgdiffTest, GetPackageLatestBuildTest, DownloadRPMTest) (BrewListRPMsTest, AssertionHelper, MockGlobalConfig) (BUILT_ABIPKGDIFF, CompareABIFromCommandLineTest): Remove these classes, global variables and functions. (FEDABIPKGDIFF, TEST_SRC_DIR, TEST_BUILD_DIR, INPUT_DIR) (OUTPUT_DIR, FEDABIPKGDIFF_TEST_SPECS): New global variables. (ensure_output_dir_created, run_fedabipkgdiff_tests, main): New functions. * tests/test-diff-pkg.cc (in_out_specs): Add tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt to the set of reference outputs to consider. * tests/Makefile.am: Add non-installed script mockfedabipkgdiff to source distribution. Also added tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt, tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt, tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt, tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt and tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt to source distribution. Signed-off-by: Chenxiong Qi <cqi@redhat.com> Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2016-07-06 09:51:13 +00:00
def getBuild(self, build_id):
"""Mock kojihub.getBuild
:param int build_id: ID of build to find and return
:return: the found build
:rtype: dict
"""
assert isinstance(build_id, int)
def selector(build):
return build['build_id'] == build_id
Bug 22722 - Make fedabipkgdiff and its tests support both python 3 and 2 This patch makes fedabipkgdiff Python 3 compatible. All tests written in Python are updated and compatible with Python 3 as well. The patch looks for a Python 3 interperter. If it finds one then it runs the tests using that interpreter. Otherwise it just tries to use the Python 2 interpreter. This behaviour can be disabled by the new --disable-python3 option. * configure.ac: Add new option --enable-python3. Add new test runner file tests/runtestdefaultsupprs-py3 and tests/runtestfedabipkgdiffpy3.sh. Add required six Python module. * tests/Makefile.am: Add new test files tests/runtestdefaultsupprspy3.sh and tests/runtestfedabipkgdiffpy3.sh accordingly. * tests/mockfedabipkgdiff.in: Convert print statement to six.print_. Replace call to function filter with list comprehension. Replace basestring with six.string_types. * tests/runtestdefaultsupprspy3.sh.in: New shell script to run test runtestdefaultsupprs with Python 3. * tests/runtestdefaultsupprs.py.in: Repalce a few tabs with proper number of spaces which is detected by Python 3 interpreter. * tests/runtestfedabipkgdiffpy3.sh.in: New shell script to run test runtestfedabipkgdiff with Python 3. * tests/runtestfedabipkgdiff.py.in: Use python from env in shebang instead of a fixed path to a Python interpreter. * tools/fedabipkgdiff: Globally replace print statement with a function call to print which is available by importing print_function from __future__ module. Use six.print_ to output string to stderr instead. Convert function call to map to for-loop. (cmp_nvr): Change argument to handle a Koji build mapping instead of only the nvr. (Brew.listBuilds): use the new cmp_nvr to sort builds. Signed-off-by: Chenxiong Qi <cqi@redhat.com> Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2018-03-25 07:34:59 +00:00
return [b for b in builds if selector(b)][0]
Make fedabipkgdiff consistent with Libabigail's other tests In the libabigail project, almost all the tests for the tools follow a similar pattern. The test contains a set of input files to compare. The result of the comparison is thus compared to a set of reference comparison result. This approach is already documented in the CONTRIBUTING file. There are several interesting properties with this scheme. First, it capture the behaviour of the tools, including what is shown to the user. Second, it eases the job of a hacker who wants to add a new test for a new behaviour of a given tool. The user just has to provide: 1/ A new reference output of the new use case of the tool (that is easily constructed by using the tool itself and saving its output) and add an entry to array of entries that describe what to compare 2/ A new set of inputs to the tool And voila. Unfortunately, fedabipkgdiff tests don't follow this scheme. That make them surprising to hackers who read the source code of the existing tests, at very least. Also, the fedabipkgdiff tests were only unit tests. They were not testing the tool as used by users in general. This patch makes the fedabipkgdiff tests follow the general approach of the tests of the other Libabigail tools. The patch first craetes a program names tests/mockfedabipkgdiff. It's a wrapper around tools/fedabipkgdiff. It overloads the Koji client of fedabipkgdiff with a fake Koji client that gets the Fedora packages locally, from tests/data/test-fedabipkgdiff/packages. In other words, mockfedabipkgdiff is fedabipkgdiff without going to the network. I believe that in the future, tests/mockfedabipkgdiff should be killed when fedabipkgdiff is able to cache a local description of a local partial view of a Koji repository, along with the build packages that were already downloaded. That way, calling fedabipkgdiff twice with the same set of option should make the tool perform correctly without going to the netword on the second invocation. We should be able to save the local partial view of the Koji repository under e.g, tests/data/test-fedabipkgdiff/local-koji and tell fedabipkgdiff to use that, instead of using the network. But we are not there yet. So for now, I am using mockfedabipkgdiff. Then, tests/runtestfedabipkdiff.py.in has been re-written to use tests/mockfedabipkgdiff and to look like what is described in CONTRIBUTING as far as how Libabigail tools' tests are organized: mockfedabipkgdiff is called to perform a comparison. The result of the comparison is then compared (using GNU diff) to a reference comparison result file. Please note that tests/runtestfedabipkdiff.py is relatively fast for the moment. But when it contains much more tests and start becoming slow, then we'll need to change the code to run several comparisons in parallel, just like we do today in tests/test-diff-filter.cc. At that point, I believe I'll just re-write this in C++, just like tests/test-diff-filter.cc because that will allow us to have true concurrent code based on the abigail:workers API. For now, I am keeping this in Python also because I think that keeps Chenxiong happy ;-) To be sure that fedabipkgdiff (and its underlying abipkgdiff) are really comparing all the packages they are supposed to compare and also all the binaries in those packages, I have added a new --show-identical-binaries to abipkgdiff and fedabipkgdiff. That option shows the name of the binaries (and packages) that are compared, *even if* the ABI of those binaries are identical. Because otherwise, today, if two binaries (or packages) have the same ABI, nothing is displayed. For instance, here is the result of comparing a package against itself, using this new --show-identical-binaries options: dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ./build/tools/abipkgdiff --show-identical-binaries dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm: ================ changes of 'dbus-binding-tool'=============== No ABI change detected ================ end of changes of 'dbus-binding-tool'=============== ================ changes of 'libdbus-glib-1.so.2.1.0'=============== No ABI change detected ================ end of changes of 'libdbus-glib-1.so.2.1.0'=============== dodji@adjoa:patches$ And here is what this command before that patch would do: dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ../master/build/tools/abipkgdiff dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm: dodji@adjoa:patches$ The rest of the patch is mostly new test inputs material and the necessary adjustments following all the changes above. * configure.ac: Do not require Python dependencies itertools, unittest and StringIO anymore as they are not used anymore. Require new module tempfile now. Generate new executable script tests/mockfedabipkgdiff from tests/mockfedabipkgdiff.in. * doc/manuals/abipkgdiff.rst: Add doc for new option --show-identical-binaries to abipkgdiff * doc/manuals/fedabipkgdiff.rst: Add doc for new options --show-identical-binaries to fedabipkgdiff. * tools/abipkgdiff.cc (options::show_identical_binaries): New data member. (options::options): Initialize new data member. (display_usage): Add a new help string for the new --show-identical-binaries option. (parse_command_line): Parse the newq --show-identical-binaries command line switch. (pthread_routine_compare): When the comparison of two binaries is empty, if --show-identical-binaries was provided, then emit some output saying the comparison did yield the empty set. * tools/fedabipkgdiff (DEFAULT_ABIPKGDIFF): Store the default path to abipkgdiff in this new global variable. Naming this default path is useful because it can then be cleanly overloaded when using mock.patch. (build_path_to_abipkgdiff): Return the new DEFAULT_ABIPKGDIFF global variable. (cmd): Parse the new --show-identical-binaries command line switch. * tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt: New reference output. * tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt: Likewise. * tests/mockfedabipkgdiff.in: New uninstalled script template. * tests/runtestfedabipkgdiff.py.in (counter) (temp_file_or_dir_prefix, UtilsTest, RPMTest, LocalRPMTest) (RunAbipkgdiffTest, GetPackageLatestBuildTest, DownloadRPMTest) (BrewListRPMsTest, AssertionHelper, MockGlobalConfig) (BUILT_ABIPKGDIFF, CompareABIFromCommandLineTest): Remove these classes, global variables and functions. (FEDABIPKGDIFF, TEST_SRC_DIR, TEST_BUILD_DIR, INPUT_DIR) (OUTPUT_DIR, FEDABIPKGDIFF_TEST_SPECS): New global variables. (ensure_output_dir_created, run_fedabipkgdiff_tests, main): New functions. * tests/test-diff-pkg.cc (in_out_specs): Add tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt to the set of reference outputs to consider. * tests/Makefile.am: Add non-installed script mockfedabipkgdiff to source distribution. Also added tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt, tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt, tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt, tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt and tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt to source distribution. Signed-off-by: Chenxiong Qi <cqi@redhat.com> Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2016-07-06 09:51:13 +00:00
def listBuilds(self, packageID, state=None):
"""Mock kojihub.listBuilds
:param int packageID: ID of package whose builds is found and returned
:param state: build state. If state is omitted, all builds of a package
are returned
:type state: int or None
"""
assert isinstance(packageID, int)
if state is not None:
assert isinstance(state, int)
def selector(build):
selected = build['package_id'] == packageID
if state is not None:
selected = selected and build['state'] == state
return selected
return filter(selector, builds)
def getRPM(self, rpminfo):
"""Mock kojihub.getRPM
:param dict rpminfo: a mapping containing rpm information, at least,
it contains name, version, release, and arch.
"""
assert isinstance(rpminfo, dict)
def selector(rpm):
return rpm['name'] == rpminfo['name'] and \
rpm['version'] == rpminfo['version'] and \
rpm['release'] == rpminfo['release'] and \
rpm['arch'] == rpminfo['arch']
Bug 22722 - Make fedabipkgdiff and its tests support both python 3 and 2 This patch makes fedabipkgdiff Python 3 compatible. All tests written in Python are updated and compatible with Python 3 as well. The patch looks for a Python 3 interperter. If it finds one then it runs the tests using that interpreter. Otherwise it just tries to use the Python 2 interpreter. This behaviour can be disabled by the new --disable-python3 option. * configure.ac: Add new option --enable-python3. Add new test runner file tests/runtestdefaultsupprs-py3 and tests/runtestfedabipkgdiffpy3.sh. Add required six Python module. * tests/Makefile.am: Add new test files tests/runtestdefaultsupprspy3.sh and tests/runtestfedabipkgdiffpy3.sh accordingly. * tests/mockfedabipkgdiff.in: Convert print statement to six.print_. Replace call to function filter with list comprehension. Replace basestring with six.string_types. * tests/runtestdefaultsupprspy3.sh.in: New shell script to run test runtestdefaultsupprs with Python 3. * tests/runtestdefaultsupprs.py.in: Repalce a few tabs with proper number of spaces which is detected by Python 3 interpreter. * tests/runtestfedabipkgdiffpy3.sh.in: New shell script to run test runtestfedabipkgdiff with Python 3. * tests/runtestfedabipkgdiff.py.in: Use python from env in shebang instead of a fixed path to a Python interpreter. * tools/fedabipkgdiff: Globally replace print statement with a function call to print which is available by importing print_function from __future__ module. Use six.print_ to output string to stderr instead. Convert function call to map to for-loop. (cmp_nvr): Change argument to handle a Koji build mapping instead of only the nvr. (Brew.listBuilds): use the new cmp_nvr to sort builds. Signed-off-by: Chenxiong Qi <cqi@redhat.com> Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2018-03-25 07:34:59 +00:00
return [rpm for rpm in rpms if selector(rpm)][0]
Make fedabipkgdiff consistent with Libabigail's other tests In the libabigail project, almost all the tests for the tools follow a similar pattern. The test contains a set of input files to compare. The result of the comparison is thus compared to a set of reference comparison result. This approach is already documented in the CONTRIBUTING file. There are several interesting properties with this scheme. First, it capture the behaviour of the tools, including what is shown to the user. Second, it eases the job of a hacker who wants to add a new test for a new behaviour of a given tool. The user just has to provide: 1/ A new reference output of the new use case of the tool (that is easily constructed by using the tool itself and saving its output) and add an entry to array of entries that describe what to compare 2/ A new set of inputs to the tool And voila. Unfortunately, fedabipkgdiff tests don't follow this scheme. That make them surprising to hackers who read the source code of the existing tests, at very least. Also, the fedabipkgdiff tests were only unit tests. They were not testing the tool as used by users in general. This patch makes the fedabipkgdiff tests follow the general approach of the tests of the other Libabigail tools. The patch first craetes a program names tests/mockfedabipkgdiff. It's a wrapper around tools/fedabipkgdiff. It overloads the Koji client of fedabipkgdiff with a fake Koji client that gets the Fedora packages locally, from tests/data/test-fedabipkgdiff/packages. In other words, mockfedabipkgdiff is fedabipkgdiff without going to the network. I believe that in the future, tests/mockfedabipkgdiff should be killed when fedabipkgdiff is able to cache a local description of a local partial view of a Koji repository, along with the build packages that were already downloaded. That way, calling fedabipkgdiff twice with the same set of option should make the tool perform correctly without going to the netword on the second invocation. We should be able to save the local partial view of the Koji repository under e.g, tests/data/test-fedabipkgdiff/local-koji and tell fedabipkgdiff to use that, instead of using the network. But we are not there yet. So for now, I am using mockfedabipkgdiff. Then, tests/runtestfedabipkdiff.py.in has been re-written to use tests/mockfedabipkgdiff and to look like what is described in CONTRIBUTING as far as how Libabigail tools' tests are organized: mockfedabipkgdiff is called to perform a comparison. The result of the comparison is then compared (using GNU diff) to a reference comparison result file. Please note that tests/runtestfedabipkdiff.py is relatively fast for the moment. But when it contains much more tests and start becoming slow, then we'll need to change the code to run several comparisons in parallel, just like we do today in tests/test-diff-filter.cc. At that point, I believe I'll just re-write this in C++, just like tests/test-diff-filter.cc because that will allow us to have true concurrent code based on the abigail:workers API. For now, I am keeping this in Python also because I think that keeps Chenxiong happy ;-) To be sure that fedabipkgdiff (and its underlying abipkgdiff) are really comparing all the packages they are supposed to compare and also all the binaries in those packages, I have added a new --show-identical-binaries to abipkgdiff and fedabipkgdiff. That option shows the name of the binaries (and packages) that are compared, *even if* the ABI of those binaries are identical. Because otherwise, today, if two binaries (or packages) have the same ABI, nothing is displayed. For instance, here is the result of comparing a package against itself, using this new --show-identical-binaries options: dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ./build/tools/abipkgdiff --show-identical-binaries dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm: ================ changes of 'dbus-binding-tool'=============== No ABI change detected ================ end of changes of 'dbus-binding-tool'=============== ================ changes of 'libdbus-glib-1.so.2.1.0'=============== No ABI change detected ================ end of changes of 'libdbus-glib-1.so.2.1.0'=============== dodji@adjoa:patches$ And here is what this command before that patch would do: dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ../master/build/tools/abipkgdiff dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm: dodji@adjoa:patches$ The rest of the patch is mostly new test inputs material and the necessary adjustments following all the changes above. * configure.ac: Do not require Python dependencies itertools, unittest and StringIO anymore as they are not used anymore. Require new module tempfile now. Generate new executable script tests/mockfedabipkgdiff from tests/mockfedabipkgdiff.in. * doc/manuals/abipkgdiff.rst: Add doc for new option --show-identical-binaries to abipkgdiff * doc/manuals/fedabipkgdiff.rst: Add doc for new options --show-identical-binaries to fedabipkgdiff. * tools/abipkgdiff.cc (options::show_identical_binaries): New data member. (options::options): Initialize new data member. (display_usage): Add a new help string for the new --show-identical-binaries option. (parse_command_line): Parse the newq --show-identical-binaries command line switch. (pthread_routine_compare): When the comparison of two binaries is empty, if --show-identical-binaries was provided, then emit some output saying the comparison did yield the empty set. * tools/fedabipkgdiff (DEFAULT_ABIPKGDIFF): Store the default path to abipkgdiff in this new global variable. Naming this default path is useful because it can then be cleanly overloaded when using mock.patch. (build_path_to_abipkgdiff): Return the new DEFAULT_ABIPKGDIFF global variable. (cmd): Parse the new --show-identical-binaries command line switch. * tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt: New reference output. * tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt: Likewise. * tests/mockfedabipkgdiff.in: New uninstalled script template. * tests/runtestfedabipkgdiff.py.in (counter) (temp_file_or_dir_prefix, UtilsTest, RPMTest, LocalRPMTest) (RunAbipkgdiffTest, GetPackageLatestBuildTest, DownloadRPMTest) (BrewListRPMsTest, AssertionHelper, MockGlobalConfig) (BUILT_ABIPKGDIFF, CompareABIFromCommandLineTest): Remove these classes, global variables and functions. (FEDABIPKGDIFF, TEST_SRC_DIR, TEST_BUILD_DIR, INPUT_DIR) (OUTPUT_DIR, FEDABIPKGDIFF_TEST_SPECS): New global variables. (ensure_output_dir_created, run_fedabipkgdiff_tests, main): New functions. * tests/test-diff-pkg.cc (in_out_specs): Add tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt to the set of reference outputs to consider. * tests/Makefile.am: Add non-installed script mockfedabipkgdiff to source distribution. Also added tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt, tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt, tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt, tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt and tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt to source distribution. Signed-off-by: Chenxiong Qi <cqi@redhat.com> Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2016-07-06 09:51:13 +00:00
def listRPMs(self, buildID, arches=None):
"""Mock kojihub.listRPMs
:param int buildID: ID of build from which to list rpms
:param arches: to list rpms built for specific arches. If arches is
omitted, rpms of all arches will be listed.
:type arches: list, tuple, str, or None
:return: list of rpms
:rtype: list
"""
assert isinstance(buildID, int)
if arches is not None:
Bug 22722 - Make fedabipkgdiff and its tests support both python 3 and 2 This patch makes fedabipkgdiff Python 3 compatible. All tests written in Python are updated and compatible with Python 3 as well. The patch looks for a Python 3 interperter. If it finds one then it runs the tests using that interpreter. Otherwise it just tries to use the Python 2 interpreter. This behaviour can be disabled by the new --disable-python3 option. * configure.ac: Add new option --enable-python3. Add new test runner file tests/runtestdefaultsupprs-py3 and tests/runtestfedabipkgdiffpy3.sh. Add required six Python module. * tests/Makefile.am: Add new test files tests/runtestdefaultsupprspy3.sh and tests/runtestfedabipkgdiffpy3.sh accordingly. * tests/mockfedabipkgdiff.in: Convert print statement to six.print_. Replace call to function filter with list comprehension. Replace basestring with six.string_types. * tests/runtestdefaultsupprspy3.sh.in: New shell script to run test runtestdefaultsupprs with Python 3. * tests/runtestdefaultsupprs.py.in: Repalce a few tabs with proper number of spaces which is detected by Python 3 interpreter. * tests/runtestfedabipkgdiffpy3.sh.in: New shell script to run test runtestfedabipkgdiff with Python 3. * tests/runtestfedabipkgdiff.py.in: Use python from env in shebang instead of a fixed path to a Python interpreter. * tools/fedabipkgdiff: Globally replace print statement with a function call to print which is available by importing print_function from __future__ module. Use six.print_ to output string to stderr instead. Convert function call to map to for-loop. (cmp_nvr): Change argument to handle a Koji build mapping instead of only the nvr. (Brew.listBuilds): use the new cmp_nvr to sort builds. Signed-off-by: Chenxiong Qi <cqi@redhat.com> Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2018-03-25 07:34:59 +00:00
assert isinstance(arches, (tuple, list, six.string_types))
Make fedabipkgdiff consistent with Libabigail's other tests In the libabigail project, almost all the tests for the tools follow a similar pattern. The test contains a set of input files to compare. The result of the comparison is thus compared to a set of reference comparison result. This approach is already documented in the CONTRIBUTING file. There are several interesting properties with this scheme. First, it capture the behaviour of the tools, including what is shown to the user. Second, it eases the job of a hacker who wants to add a new test for a new behaviour of a given tool. The user just has to provide: 1/ A new reference output of the new use case of the tool (that is easily constructed by using the tool itself and saving its output) and add an entry to array of entries that describe what to compare 2/ A new set of inputs to the tool And voila. Unfortunately, fedabipkgdiff tests don't follow this scheme. That make them surprising to hackers who read the source code of the existing tests, at very least. Also, the fedabipkgdiff tests were only unit tests. They were not testing the tool as used by users in general. This patch makes the fedabipkgdiff tests follow the general approach of the tests of the other Libabigail tools. The patch first craetes a program names tests/mockfedabipkgdiff. It's a wrapper around tools/fedabipkgdiff. It overloads the Koji client of fedabipkgdiff with a fake Koji client that gets the Fedora packages locally, from tests/data/test-fedabipkgdiff/packages. In other words, mockfedabipkgdiff is fedabipkgdiff without going to the network. I believe that in the future, tests/mockfedabipkgdiff should be killed when fedabipkgdiff is able to cache a local description of a local partial view of a Koji repository, along with the build packages that were already downloaded. That way, calling fedabipkgdiff twice with the same set of option should make the tool perform correctly without going to the netword on the second invocation. We should be able to save the local partial view of the Koji repository under e.g, tests/data/test-fedabipkgdiff/local-koji and tell fedabipkgdiff to use that, instead of using the network. But we are not there yet. So for now, I am using mockfedabipkgdiff. Then, tests/runtestfedabipkdiff.py.in has been re-written to use tests/mockfedabipkgdiff and to look like what is described in CONTRIBUTING as far as how Libabigail tools' tests are organized: mockfedabipkgdiff is called to perform a comparison. The result of the comparison is then compared (using GNU diff) to a reference comparison result file. Please note that tests/runtestfedabipkdiff.py is relatively fast for the moment. But when it contains much more tests and start becoming slow, then we'll need to change the code to run several comparisons in parallel, just like we do today in tests/test-diff-filter.cc. At that point, I believe I'll just re-write this in C++, just like tests/test-diff-filter.cc because that will allow us to have true concurrent code based on the abigail:workers API. For now, I am keeping this in Python also because I think that keeps Chenxiong happy ;-) To be sure that fedabipkgdiff (and its underlying abipkgdiff) are really comparing all the packages they are supposed to compare and also all the binaries in those packages, I have added a new --show-identical-binaries to abipkgdiff and fedabipkgdiff. That option shows the name of the binaries (and packages) that are compared, *even if* the ABI of those binaries are identical. Because otherwise, today, if two binaries (or packages) have the same ABI, nothing is displayed. For instance, here is the result of comparing a package against itself, using this new --show-identical-binaries options: dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ./build/tools/abipkgdiff --show-identical-binaries dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm: ================ changes of 'dbus-binding-tool'=============== No ABI change detected ================ end of changes of 'dbus-binding-tool'=============== ================ changes of 'libdbus-glib-1.so.2.1.0'=============== No ABI change detected ================ end of changes of 'libdbus-glib-1.so.2.1.0'=============== dodji@adjoa:patches$ And here is what this command before that patch would do: dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ../master/build/tools/abipkgdiff dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm: dodji@adjoa:patches$ The rest of the patch is mostly new test inputs material and the necessary adjustments following all the changes above. * configure.ac: Do not require Python dependencies itertools, unittest and StringIO anymore as they are not used anymore. Require new module tempfile now. Generate new executable script tests/mockfedabipkgdiff from tests/mockfedabipkgdiff.in. * doc/manuals/abipkgdiff.rst: Add doc for new option --show-identical-binaries to abipkgdiff * doc/manuals/fedabipkgdiff.rst: Add doc for new options --show-identical-binaries to fedabipkgdiff. * tools/abipkgdiff.cc (options::show_identical_binaries): New data member. (options::options): Initialize new data member. (display_usage): Add a new help string for the new --show-identical-binaries option. (parse_command_line): Parse the newq --show-identical-binaries command line switch. (pthread_routine_compare): When the comparison of two binaries is empty, if --show-identical-binaries was provided, then emit some output saying the comparison did yield the empty set. * tools/fedabipkgdiff (DEFAULT_ABIPKGDIFF): Store the default path to abipkgdiff in this new global variable. Naming this default path is useful because it can then be cleanly overloaded when using mock.patch. (build_path_to_abipkgdiff): Return the new DEFAULT_ABIPKGDIFF global variable. (cmd): Parse the new --show-identical-binaries command line switch. * tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt: New reference output. * tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt: Likewise. * tests/mockfedabipkgdiff.in: New uninstalled script template. * tests/runtestfedabipkgdiff.py.in (counter) (temp_file_or_dir_prefix, UtilsTest, RPMTest, LocalRPMTest) (RunAbipkgdiffTest, GetPackageLatestBuildTest, DownloadRPMTest) (BrewListRPMsTest, AssertionHelper, MockGlobalConfig) (BUILT_ABIPKGDIFF, CompareABIFromCommandLineTest): Remove these classes, global variables and functions. (FEDABIPKGDIFF, TEST_SRC_DIR, TEST_BUILD_DIR, INPUT_DIR) (OUTPUT_DIR, FEDABIPKGDIFF_TEST_SPECS): New global variables. (ensure_output_dir_created, run_fedabipkgdiff_tests, main): New functions. * tests/test-diff-pkg.cc (in_out_specs): Add tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt to the set of reference outputs to consider. * tests/Makefile.am: Add non-installed script mockfedabipkgdiff to source distribution. Also added tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt, tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt, tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt, tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt and tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt to source distribution. Signed-off-by: Chenxiong Qi <cqi@redhat.com> Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2016-07-06 09:51:13 +00:00
Bug 22722 - Make fedabipkgdiff and its tests support both python 3 and 2 This patch makes fedabipkgdiff Python 3 compatible. All tests written in Python are updated and compatible with Python 3 as well. The patch looks for a Python 3 interperter. If it finds one then it runs the tests using that interpreter. Otherwise it just tries to use the Python 2 interpreter. This behaviour can be disabled by the new --disable-python3 option. * configure.ac: Add new option --enable-python3. Add new test runner file tests/runtestdefaultsupprs-py3 and tests/runtestfedabipkgdiffpy3.sh. Add required six Python module. * tests/Makefile.am: Add new test files tests/runtestdefaultsupprspy3.sh and tests/runtestfedabipkgdiffpy3.sh accordingly. * tests/mockfedabipkgdiff.in: Convert print statement to six.print_. Replace call to function filter with list comprehension. Replace basestring with six.string_types. * tests/runtestdefaultsupprspy3.sh.in: New shell script to run test runtestdefaultsupprs with Python 3. * tests/runtestdefaultsupprs.py.in: Repalce a few tabs with proper number of spaces which is detected by Python 3 interpreter. * tests/runtestfedabipkgdiffpy3.sh.in: New shell script to run test runtestfedabipkgdiff with Python 3. * tests/runtestfedabipkgdiff.py.in: Use python from env in shebang instead of a fixed path to a Python interpreter. * tools/fedabipkgdiff: Globally replace print statement with a function call to print which is available by importing print_function from __future__ module. Use six.print_ to output string to stderr instead. Convert function call to map to for-loop. (cmp_nvr): Change argument to handle a Koji build mapping instead of only the nvr. (Brew.listBuilds): use the new cmp_nvr to sort builds. Signed-off-by: Chenxiong Qi <cqi@redhat.com> Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2018-03-25 07:34:59 +00:00
if arches is not None and isinstance(arches, six.string_types):
Make fedabipkgdiff consistent with Libabigail's other tests In the libabigail project, almost all the tests for the tools follow a similar pattern. The test contains a set of input files to compare. The result of the comparison is thus compared to a set of reference comparison result. This approach is already documented in the CONTRIBUTING file. There are several interesting properties with this scheme. First, it capture the behaviour of the tools, including what is shown to the user. Second, it eases the job of a hacker who wants to add a new test for a new behaviour of a given tool. The user just has to provide: 1/ A new reference output of the new use case of the tool (that is easily constructed by using the tool itself and saving its output) and add an entry to array of entries that describe what to compare 2/ A new set of inputs to the tool And voila. Unfortunately, fedabipkgdiff tests don't follow this scheme. That make them surprising to hackers who read the source code of the existing tests, at very least. Also, the fedabipkgdiff tests were only unit tests. They were not testing the tool as used by users in general. This patch makes the fedabipkgdiff tests follow the general approach of the tests of the other Libabigail tools. The patch first craetes a program names tests/mockfedabipkgdiff. It's a wrapper around tools/fedabipkgdiff. It overloads the Koji client of fedabipkgdiff with a fake Koji client that gets the Fedora packages locally, from tests/data/test-fedabipkgdiff/packages. In other words, mockfedabipkgdiff is fedabipkgdiff without going to the network. I believe that in the future, tests/mockfedabipkgdiff should be killed when fedabipkgdiff is able to cache a local description of a local partial view of a Koji repository, along with the build packages that were already downloaded. That way, calling fedabipkgdiff twice with the same set of option should make the tool perform correctly without going to the netword on the second invocation. We should be able to save the local partial view of the Koji repository under e.g, tests/data/test-fedabipkgdiff/local-koji and tell fedabipkgdiff to use that, instead of using the network. But we are not there yet. So for now, I am using mockfedabipkgdiff. Then, tests/runtestfedabipkdiff.py.in has been re-written to use tests/mockfedabipkgdiff and to look like what is described in CONTRIBUTING as far as how Libabigail tools' tests are organized: mockfedabipkgdiff is called to perform a comparison. The result of the comparison is then compared (using GNU diff) to a reference comparison result file. Please note that tests/runtestfedabipkdiff.py is relatively fast for the moment. But when it contains much more tests and start becoming slow, then we'll need to change the code to run several comparisons in parallel, just like we do today in tests/test-diff-filter.cc. At that point, I believe I'll just re-write this in C++, just like tests/test-diff-filter.cc because that will allow us to have true concurrent code based on the abigail:workers API. For now, I am keeping this in Python also because I think that keeps Chenxiong happy ;-) To be sure that fedabipkgdiff (and its underlying abipkgdiff) are really comparing all the packages they are supposed to compare and also all the binaries in those packages, I have added a new --show-identical-binaries to abipkgdiff and fedabipkgdiff. That option shows the name of the binaries (and packages) that are compared, *even if* the ABI of those binaries are identical. Because otherwise, today, if two binaries (or packages) have the same ABI, nothing is displayed. For instance, here is the result of comparing a package against itself, using this new --show-identical-binaries options: dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ./build/tools/abipkgdiff --show-identical-binaries dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm: ================ changes of 'dbus-binding-tool'=============== No ABI change detected ================ end of changes of 'dbus-binding-tool'=============== ================ changes of 'libdbus-glib-1.so.2.1.0'=============== No ABI change detected ================ end of changes of 'libdbus-glib-1.so.2.1.0'=============== dodji@adjoa:patches$ And here is what this command before that patch would do: dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ../master/build/tools/abipkgdiff dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm: dodji@adjoa:patches$ The rest of the patch is mostly new test inputs material and the necessary adjustments following all the changes above. * configure.ac: Do not require Python dependencies itertools, unittest and StringIO anymore as they are not used anymore. Require new module tempfile now. Generate new executable script tests/mockfedabipkgdiff from tests/mockfedabipkgdiff.in. * doc/manuals/abipkgdiff.rst: Add doc for new option --show-identical-binaries to abipkgdiff * doc/manuals/fedabipkgdiff.rst: Add doc for new options --show-identical-binaries to fedabipkgdiff. * tools/abipkgdiff.cc (options::show_identical_binaries): New data member. (options::options): Initialize new data member. (display_usage): Add a new help string for the new --show-identical-binaries option. (parse_command_line): Parse the newq --show-identical-binaries command line switch. (pthread_routine_compare): When the comparison of two binaries is empty, if --show-identical-binaries was provided, then emit some output saying the comparison did yield the empty set. * tools/fedabipkgdiff (DEFAULT_ABIPKGDIFF): Store the default path to abipkgdiff in this new global variable. Naming this default path is useful because it can then be cleanly overloaded when using mock.patch. (build_path_to_abipkgdiff): Return the new DEFAULT_ABIPKGDIFF global variable. (cmd): Parse the new --show-identical-binaries command line switch. * tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt: New reference output. * tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt: Likewise. * tests/mockfedabipkgdiff.in: New uninstalled script template. * tests/runtestfedabipkgdiff.py.in (counter) (temp_file_or_dir_prefix, UtilsTest, RPMTest, LocalRPMTest) (RunAbipkgdiffTest, GetPackageLatestBuildTest, DownloadRPMTest) (BrewListRPMsTest, AssertionHelper, MockGlobalConfig) (BUILT_ABIPKGDIFF, CompareABIFromCommandLineTest): Remove these classes, global variables and functions. (FEDABIPKGDIFF, TEST_SRC_DIR, TEST_BUILD_DIR, INPUT_DIR) (OUTPUT_DIR, FEDABIPKGDIFF_TEST_SPECS): New global variables. (ensure_output_dir_created, run_fedabipkgdiff_tests, main): New functions. * tests/test-diff-pkg.cc (in_out_specs): Add tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt to the set of reference outputs to consider. * tests/Makefile.am: Add non-installed script mockfedabipkgdiff to source distribution. Also added tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt, tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt, tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt, tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt and tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt to source distribution. Signed-off-by: Chenxiong Qi <cqi@redhat.com> Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2016-07-06 09:51:13 +00:00
arches = [arches]
def selector(rpm):
selected = rpm['build_id'] == buildID
if arches is not None:
selected = selected and rpm['arch'] in arches
return selected
Bug 22722 - Make fedabipkgdiff and its tests support both python 3 and 2 This patch makes fedabipkgdiff Python 3 compatible. All tests written in Python are updated and compatible with Python 3 as well. The patch looks for a Python 3 interperter. If it finds one then it runs the tests using that interpreter. Otherwise it just tries to use the Python 2 interpreter. This behaviour can be disabled by the new --disable-python3 option. * configure.ac: Add new option --enable-python3. Add new test runner file tests/runtestdefaultsupprs-py3 and tests/runtestfedabipkgdiffpy3.sh. Add required six Python module. * tests/Makefile.am: Add new test files tests/runtestdefaultsupprspy3.sh and tests/runtestfedabipkgdiffpy3.sh accordingly. * tests/mockfedabipkgdiff.in: Convert print statement to six.print_. Replace call to function filter with list comprehension. Replace basestring with six.string_types. * tests/runtestdefaultsupprspy3.sh.in: New shell script to run test runtestdefaultsupprs with Python 3. * tests/runtestdefaultsupprs.py.in: Repalce a few tabs with proper number of spaces which is detected by Python 3 interpreter. * tests/runtestfedabipkgdiffpy3.sh.in: New shell script to run test runtestfedabipkgdiff with Python 3. * tests/runtestfedabipkgdiff.py.in: Use python from env in shebang instead of a fixed path to a Python interpreter. * tools/fedabipkgdiff: Globally replace print statement with a function call to print which is available by importing print_function from __future__ module. Use six.print_ to output string to stderr instead. Convert function call to map to for-loop. (cmp_nvr): Change argument to handle a Koji build mapping instead of only the nvr. (Brew.listBuilds): use the new cmp_nvr to sort builds. Signed-off-by: Chenxiong Qi <cqi@redhat.com> Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2018-03-25 07:34:59 +00:00
return [rpm for rpm in rpms if selector(rpm)]
Make fedabipkgdiff consistent with Libabigail's other tests In the libabigail project, almost all the tests for the tools follow a similar pattern. The test contains a set of input files to compare. The result of the comparison is thus compared to a set of reference comparison result. This approach is already documented in the CONTRIBUTING file. There are several interesting properties with this scheme. First, it capture the behaviour of the tools, including what is shown to the user. Second, it eases the job of a hacker who wants to add a new test for a new behaviour of a given tool. The user just has to provide: 1/ A new reference output of the new use case of the tool (that is easily constructed by using the tool itself and saving its output) and add an entry to array of entries that describe what to compare 2/ A new set of inputs to the tool And voila. Unfortunately, fedabipkgdiff tests don't follow this scheme. That make them surprising to hackers who read the source code of the existing tests, at very least. Also, the fedabipkgdiff tests were only unit tests. They were not testing the tool as used by users in general. This patch makes the fedabipkgdiff tests follow the general approach of the tests of the other Libabigail tools. The patch first craetes a program names tests/mockfedabipkgdiff. It's a wrapper around tools/fedabipkgdiff. It overloads the Koji client of fedabipkgdiff with a fake Koji client that gets the Fedora packages locally, from tests/data/test-fedabipkgdiff/packages. In other words, mockfedabipkgdiff is fedabipkgdiff without going to the network. I believe that in the future, tests/mockfedabipkgdiff should be killed when fedabipkgdiff is able to cache a local description of a local partial view of a Koji repository, along with the build packages that were already downloaded. That way, calling fedabipkgdiff twice with the same set of option should make the tool perform correctly without going to the netword on the second invocation. We should be able to save the local partial view of the Koji repository under e.g, tests/data/test-fedabipkgdiff/local-koji and tell fedabipkgdiff to use that, instead of using the network. But we are not there yet. So for now, I am using mockfedabipkgdiff. Then, tests/runtestfedabipkdiff.py.in has been re-written to use tests/mockfedabipkgdiff and to look like what is described in CONTRIBUTING as far as how Libabigail tools' tests are organized: mockfedabipkgdiff is called to perform a comparison. The result of the comparison is then compared (using GNU diff) to a reference comparison result file. Please note that tests/runtestfedabipkdiff.py is relatively fast for the moment. But when it contains much more tests and start becoming slow, then we'll need to change the code to run several comparisons in parallel, just like we do today in tests/test-diff-filter.cc. At that point, I believe I'll just re-write this in C++, just like tests/test-diff-filter.cc because that will allow us to have true concurrent code based on the abigail:workers API. For now, I am keeping this in Python also because I think that keeps Chenxiong happy ;-) To be sure that fedabipkgdiff (and its underlying abipkgdiff) are really comparing all the packages they are supposed to compare and also all the binaries in those packages, I have added a new --show-identical-binaries to abipkgdiff and fedabipkgdiff. That option shows the name of the binaries (and packages) that are compared, *even if* the ABI of those binaries are identical. Because otherwise, today, if two binaries (or packages) have the same ABI, nothing is displayed. For instance, here is the result of comparing a package against itself, using this new --show-identical-binaries options: dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ./build/tools/abipkgdiff --show-identical-binaries dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm: ================ changes of 'dbus-binding-tool'=============== No ABI change detected ================ end of changes of 'dbus-binding-tool'=============== ================ changes of 'libdbus-glib-1.so.2.1.0'=============== No ABI change detected ================ end of changes of 'libdbus-glib-1.so.2.1.0'=============== dodji@adjoa:patches$ And here is what this command before that patch would do: dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ../master/build/tools/abipkgdiff dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm: dodji@adjoa:patches$ The rest of the patch is mostly new test inputs material and the necessary adjustments following all the changes above. * configure.ac: Do not require Python dependencies itertools, unittest and StringIO anymore as they are not used anymore. Require new module tempfile now. Generate new executable script tests/mockfedabipkgdiff from tests/mockfedabipkgdiff.in. * doc/manuals/abipkgdiff.rst: Add doc for new option --show-identical-binaries to abipkgdiff * doc/manuals/fedabipkgdiff.rst: Add doc for new options --show-identical-binaries to fedabipkgdiff. * tools/abipkgdiff.cc (options::show_identical_binaries): New data member. (options::options): Initialize new data member. (display_usage): Add a new help string for the new --show-identical-binaries option. (parse_command_line): Parse the newq --show-identical-binaries command line switch. (pthread_routine_compare): When the comparison of two binaries is empty, if --show-identical-binaries was provided, then emit some output saying the comparison did yield the empty set. * tools/fedabipkgdiff (DEFAULT_ABIPKGDIFF): Store the default path to abipkgdiff in this new global variable. Naming this default path is useful because it can then be cleanly overloaded when using mock.patch. (build_path_to_abipkgdiff): Return the new DEFAULT_ABIPKGDIFF global variable. (cmd): Parse the new --show-identical-binaries command line switch. * tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt: New reference output. * tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt: Likewise. * tests/mockfedabipkgdiff.in: New uninstalled script template. * tests/runtestfedabipkgdiff.py.in (counter) (temp_file_or_dir_prefix, UtilsTest, RPMTest, LocalRPMTest) (RunAbipkgdiffTest, GetPackageLatestBuildTest, DownloadRPMTest) (BrewListRPMsTest, AssertionHelper, MockGlobalConfig) (BUILT_ABIPKGDIFF, CompareABIFromCommandLineTest): Remove these classes, global variables and functions. (FEDABIPKGDIFF, TEST_SRC_DIR, TEST_BUILD_DIR, INPUT_DIR) (OUTPUT_DIR, FEDABIPKGDIFF_TEST_SPECS): New global variables. (ensure_output_dir_created, run_fedabipkgdiff_tests, main): New functions. * tests/test-diff-pkg.cc (in_out_specs): Add tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt to the set of reference outputs to consider. * tests/Makefile.am: Add non-installed script mockfedabipkgdiff to source distribution. Also added tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt, tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt, tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt, tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt and tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt to source distribution. Signed-off-by: Chenxiong Qi <cqi@redhat.com> Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2016-07-06 09:51:13 +00:00
@patch('koji.ClientSession', new=MockClientSession)
@patch('fedabipkgdiff.DEFAULT_KOJI_TOPURL', new=TEST_TOPDIR)
Make fedabipkgdiff consistent with Libabigail's other tests In the libabigail project, almost all the tests for the tools follow a similar pattern. The test contains a set of input files to compare. The result of the comparison is thus compared to a set of reference comparison result. This approach is already documented in the CONTRIBUTING file. There are several interesting properties with this scheme. First, it capture the behaviour of the tools, including what is shown to the user. Second, it eases the job of a hacker who wants to add a new test for a new behaviour of a given tool. The user just has to provide: 1/ A new reference output of the new use case of the tool (that is easily constructed by using the tool itself and saving its output) and add an entry to array of entries that describe what to compare 2/ A new set of inputs to the tool And voila. Unfortunately, fedabipkgdiff tests don't follow this scheme. That make them surprising to hackers who read the source code of the existing tests, at very least. Also, the fedabipkgdiff tests were only unit tests. They were not testing the tool as used by users in general. This patch makes the fedabipkgdiff tests follow the general approach of the tests of the other Libabigail tools. The patch first craetes a program names tests/mockfedabipkgdiff. It's a wrapper around tools/fedabipkgdiff. It overloads the Koji client of fedabipkgdiff with a fake Koji client that gets the Fedora packages locally, from tests/data/test-fedabipkgdiff/packages. In other words, mockfedabipkgdiff is fedabipkgdiff without going to the network. I believe that in the future, tests/mockfedabipkgdiff should be killed when fedabipkgdiff is able to cache a local description of a local partial view of a Koji repository, along with the build packages that were already downloaded. That way, calling fedabipkgdiff twice with the same set of option should make the tool perform correctly without going to the netword on the second invocation. We should be able to save the local partial view of the Koji repository under e.g, tests/data/test-fedabipkgdiff/local-koji and tell fedabipkgdiff to use that, instead of using the network. But we are not there yet. So for now, I am using mockfedabipkgdiff. Then, tests/runtestfedabipkdiff.py.in has been re-written to use tests/mockfedabipkgdiff and to look like what is described in CONTRIBUTING as far as how Libabigail tools' tests are organized: mockfedabipkgdiff is called to perform a comparison. The result of the comparison is then compared (using GNU diff) to a reference comparison result file. Please note that tests/runtestfedabipkdiff.py is relatively fast for the moment. But when it contains much more tests and start becoming slow, then we'll need to change the code to run several comparisons in parallel, just like we do today in tests/test-diff-filter.cc. At that point, I believe I'll just re-write this in C++, just like tests/test-diff-filter.cc because that will allow us to have true concurrent code based on the abigail:workers API. For now, I am keeping this in Python also because I think that keeps Chenxiong happy ;-) To be sure that fedabipkgdiff (and its underlying abipkgdiff) are really comparing all the packages they are supposed to compare and also all the binaries in those packages, I have added a new --show-identical-binaries to abipkgdiff and fedabipkgdiff. That option shows the name of the binaries (and packages) that are compared, *even if* the ABI of those binaries are identical. Because otherwise, today, if two binaries (or packages) have the same ABI, nothing is displayed. For instance, here is the result of comparing a package against itself, using this new --show-identical-binaries options: dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ./build/tools/abipkgdiff --show-identical-binaries dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm: ================ changes of 'dbus-binding-tool'=============== No ABI change detected ================ end of changes of 'dbus-binding-tool'=============== ================ changes of 'libdbus-glib-1.so.2.1.0'=============== No ABI change detected ================ end of changes of 'libdbus-glib-1.so.2.1.0'=============== dodji@adjoa:patches$ And here is what this command before that patch would do: dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ../master/build/tools/abipkgdiff dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm: dodji@adjoa:patches$ The rest of the patch is mostly new test inputs material and the necessary adjustments following all the changes above. * configure.ac: Do not require Python dependencies itertools, unittest and StringIO anymore as they are not used anymore. Require new module tempfile now. Generate new executable script tests/mockfedabipkgdiff from tests/mockfedabipkgdiff.in. * doc/manuals/abipkgdiff.rst: Add doc for new option --show-identical-binaries to abipkgdiff * doc/manuals/fedabipkgdiff.rst: Add doc for new options --show-identical-binaries to fedabipkgdiff. * tools/abipkgdiff.cc (options::show_identical_binaries): New data member. (options::options): Initialize new data member. (display_usage): Add a new help string for the new --show-identical-binaries option. (parse_command_line): Parse the newq --show-identical-binaries command line switch. (pthread_routine_compare): When the comparison of two binaries is empty, if --show-identical-binaries was provided, then emit some output saying the comparison did yield the empty set. * tools/fedabipkgdiff (DEFAULT_ABIPKGDIFF): Store the default path to abipkgdiff in this new global variable. Naming this default path is useful because it can then be cleanly overloaded when using mock.patch. (build_path_to_abipkgdiff): Return the new DEFAULT_ABIPKGDIFF global variable. (cmd): Parse the new --show-identical-binaries command line switch. * tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt: New reference output. * tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt: Likewise. * tests/mockfedabipkgdiff.in: New uninstalled script template. * tests/runtestfedabipkgdiff.py.in (counter) (temp_file_or_dir_prefix, UtilsTest, RPMTest, LocalRPMTest) (RunAbipkgdiffTest, GetPackageLatestBuildTest, DownloadRPMTest) (BrewListRPMsTest, AssertionHelper, MockGlobalConfig) (BUILT_ABIPKGDIFF, CompareABIFromCommandLineTest): Remove these classes, global variables and functions. (FEDABIPKGDIFF, TEST_SRC_DIR, TEST_BUILD_DIR, INPUT_DIR) (OUTPUT_DIR, FEDABIPKGDIFF_TEST_SPECS): New global variables. (ensure_output_dir_created, run_fedabipkgdiff_tests, main): New functions. * tests/test-diff-pkg.cc (in_out_specs): Add tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt to the set of reference outputs to consider. * tests/Makefile.am: Add non-installed script mockfedabipkgdiff to source distribution. Also added tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt, tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt, tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt, tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt and tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt to source distribution. Signed-off-by: Chenxiong Qi <cqi@redhat.com> Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2016-07-06 09:51:13 +00:00
@patch('fedabipkgdiff.DEFAULT_ABIPKGDIFF', new=ABIPKGDIFF)
@patch('fedabipkgdiff.get_download_dir', new=get_download_dir)
def run_fedabipkgdiff():
Make fedabipkgdiff consistent with Libabigail's other tests In the libabigail project, almost all the tests for the tools follow a similar pattern. The test contains a set of input files to compare. The result of the comparison is thus compared to a set of reference comparison result. This approach is already documented in the CONTRIBUTING file. There are several interesting properties with this scheme. First, it capture the behaviour of the tools, including what is shown to the user. Second, it eases the job of a hacker who wants to add a new test for a new behaviour of a given tool. The user just has to provide: 1/ A new reference output of the new use case of the tool (that is easily constructed by using the tool itself and saving its output) and add an entry to array of entries that describe what to compare 2/ A new set of inputs to the tool And voila. Unfortunately, fedabipkgdiff tests don't follow this scheme. That make them surprising to hackers who read the source code of the existing tests, at very least. Also, the fedabipkgdiff tests were only unit tests. They were not testing the tool as used by users in general. This patch makes the fedabipkgdiff tests follow the general approach of the tests of the other Libabigail tools. The patch first craetes a program names tests/mockfedabipkgdiff. It's a wrapper around tools/fedabipkgdiff. It overloads the Koji client of fedabipkgdiff with a fake Koji client that gets the Fedora packages locally, from tests/data/test-fedabipkgdiff/packages. In other words, mockfedabipkgdiff is fedabipkgdiff without going to the network. I believe that in the future, tests/mockfedabipkgdiff should be killed when fedabipkgdiff is able to cache a local description of a local partial view of a Koji repository, along with the build packages that were already downloaded. That way, calling fedabipkgdiff twice with the same set of option should make the tool perform correctly without going to the netword on the second invocation. We should be able to save the local partial view of the Koji repository under e.g, tests/data/test-fedabipkgdiff/local-koji and tell fedabipkgdiff to use that, instead of using the network. But we are not there yet. So for now, I am using mockfedabipkgdiff. Then, tests/runtestfedabipkdiff.py.in has been re-written to use tests/mockfedabipkgdiff and to look like what is described in CONTRIBUTING as far as how Libabigail tools' tests are organized: mockfedabipkgdiff is called to perform a comparison. The result of the comparison is then compared (using GNU diff) to a reference comparison result file. Please note that tests/runtestfedabipkdiff.py is relatively fast for the moment. But when it contains much more tests and start becoming slow, then we'll need to change the code to run several comparisons in parallel, just like we do today in tests/test-diff-filter.cc. At that point, I believe I'll just re-write this in C++, just like tests/test-diff-filter.cc because that will allow us to have true concurrent code based on the abigail:workers API. For now, I am keeping this in Python also because I think that keeps Chenxiong happy ;-) To be sure that fedabipkgdiff (and its underlying abipkgdiff) are really comparing all the packages they are supposed to compare and also all the binaries in those packages, I have added a new --show-identical-binaries to abipkgdiff and fedabipkgdiff. That option shows the name of the binaries (and packages) that are compared, *even if* the ABI of those binaries are identical. Because otherwise, today, if two binaries (or packages) have the same ABI, nothing is displayed. For instance, here is the result of comparing a package against itself, using this new --show-identical-binaries options: dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ./build/tools/abipkgdiff --show-identical-binaries dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm: ================ changes of 'dbus-binding-tool'=============== No ABI change detected ================ end of changes of 'dbus-binding-tool'=============== ================ changes of 'libdbus-glib-1.so.2.1.0'=============== No ABI change detected ================ end of changes of 'libdbus-glib-1.so.2.1.0'=============== dodji@adjoa:patches$ And here is what this command before that patch would do: dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ../master/build/tools/abipkgdiff dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm: dodji@adjoa:patches$ The rest of the patch is mostly new test inputs material and the necessary adjustments following all the changes above. * configure.ac: Do not require Python dependencies itertools, unittest and StringIO anymore as they are not used anymore. Require new module tempfile now. Generate new executable script tests/mockfedabipkgdiff from tests/mockfedabipkgdiff.in. * doc/manuals/abipkgdiff.rst: Add doc for new option --show-identical-binaries to abipkgdiff * doc/manuals/fedabipkgdiff.rst: Add doc for new options --show-identical-binaries to fedabipkgdiff. * tools/abipkgdiff.cc (options::show_identical_binaries): New data member. (options::options): Initialize new data member. (display_usage): Add a new help string for the new --show-identical-binaries option. (parse_command_line): Parse the newq --show-identical-binaries command line switch. (pthread_routine_compare): When the comparison of two binaries is empty, if --show-identical-binaries was provided, then emit some output saying the comparison did yield the empty set. * tools/fedabipkgdiff (DEFAULT_ABIPKGDIFF): Store the default path to abipkgdiff in this new global variable. Naming this default path is useful because it can then be cleanly overloaded when using mock.patch. (build_path_to_abipkgdiff): Return the new DEFAULT_ABIPKGDIFF global variable. (cmd): Parse the new --show-identical-binaries command line switch. * tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt: New reference output. * tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt: Likewise. * tests/mockfedabipkgdiff.in: New uninstalled script template. * tests/runtestfedabipkgdiff.py.in (counter) (temp_file_or_dir_prefix, UtilsTest, RPMTest, LocalRPMTest) (RunAbipkgdiffTest, GetPackageLatestBuildTest, DownloadRPMTest) (BrewListRPMsTest, AssertionHelper, MockGlobalConfig) (BUILT_ABIPKGDIFF, CompareABIFromCommandLineTest): Remove these classes, global variables and functions. (FEDABIPKGDIFF, TEST_SRC_DIR, TEST_BUILD_DIR, INPUT_DIR) (OUTPUT_DIR, FEDABIPKGDIFF_TEST_SPECS): New global variables. (ensure_output_dir_created, run_fedabipkgdiff_tests, main): New functions. * tests/test-diff-pkg.cc (in_out_specs): Add tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt to the set of reference outputs to consider. * tests/Makefile.am: Add non-installed script mockfedabipkgdiff to source distribution. Also added tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt, tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt, tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt, tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt and tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt to source distribution. Signed-off-by: Chenxiong Qi <cqi@redhat.com> Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2016-07-06 09:51:13 +00:00
return fedabipkgdiff_mod.main()
Make fedabipkgdiff consistent with Libabigail's other tests In the libabigail project, almost all the tests for the tools follow a similar pattern. The test contains a set of input files to compare. The result of the comparison is thus compared to a set of reference comparison result. This approach is already documented in the CONTRIBUTING file. There are several interesting properties with this scheme. First, it capture the behaviour of the tools, including what is shown to the user. Second, it eases the job of a hacker who wants to add a new test for a new behaviour of a given tool. The user just has to provide: 1/ A new reference output of the new use case of the tool (that is easily constructed by using the tool itself and saving its output) and add an entry to array of entries that describe what to compare 2/ A new set of inputs to the tool And voila. Unfortunately, fedabipkgdiff tests don't follow this scheme. That make them surprising to hackers who read the source code of the existing tests, at very least. Also, the fedabipkgdiff tests were only unit tests. They were not testing the tool as used by users in general. This patch makes the fedabipkgdiff tests follow the general approach of the tests of the other Libabigail tools. The patch first craetes a program names tests/mockfedabipkgdiff. It's a wrapper around tools/fedabipkgdiff. It overloads the Koji client of fedabipkgdiff with a fake Koji client that gets the Fedora packages locally, from tests/data/test-fedabipkgdiff/packages. In other words, mockfedabipkgdiff is fedabipkgdiff without going to the network. I believe that in the future, tests/mockfedabipkgdiff should be killed when fedabipkgdiff is able to cache a local description of a local partial view of a Koji repository, along with the build packages that were already downloaded. That way, calling fedabipkgdiff twice with the same set of option should make the tool perform correctly without going to the netword on the second invocation. We should be able to save the local partial view of the Koji repository under e.g, tests/data/test-fedabipkgdiff/local-koji and tell fedabipkgdiff to use that, instead of using the network. But we are not there yet. So for now, I am using mockfedabipkgdiff. Then, tests/runtestfedabipkdiff.py.in has been re-written to use tests/mockfedabipkgdiff and to look like what is described in CONTRIBUTING as far as how Libabigail tools' tests are organized: mockfedabipkgdiff is called to perform a comparison. The result of the comparison is then compared (using GNU diff) to a reference comparison result file. Please note that tests/runtestfedabipkdiff.py is relatively fast for the moment. But when it contains much more tests and start becoming slow, then we'll need to change the code to run several comparisons in parallel, just like we do today in tests/test-diff-filter.cc. At that point, I believe I'll just re-write this in C++, just like tests/test-diff-filter.cc because that will allow us to have true concurrent code based on the abigail:workers API. For now, I am keeping this in Python also because I think that keeps Chenxiong happy ;-) To be sure that fedabipkgdiff (and its underlying abipkgdiff) are really comparing all the packages they are supposed to compare and also all the binaries in those packages, I have added a new --show-identical-binaries to abipkgdiff and fedabipkgdiff. That option shows the name of the binaries (and packages) that are compared, *even if* the ABI of those binaries are identical. Because otherwise, today, if two binaries (or packages) have the same ABI, nothing is displayed. For instance, here is the result of comparing a package against itself, using this new --show-identical-binaries options: dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ./build/tools/abipkgdiff --show-identical-binaries dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm: ================ changes of 'dbus-binding-tool'=============== No ABI change detected ================ end of changes of 'dbus-binding-tool'=============== ================ changes of 'libdbus-glib-1.so.2.1.0'=============== No ABI change detected ================ end of changes of 'libdbus-glib-1.so.2.1.0'=============== dodji@adjoa:patches$ And here is what this command before that patch would do: dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ../master/build/tools/abipkgdiff dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm: dodji@adjoa:patches$ The rest of the patch is mostly new test inputs material and the necessary adjustments following all the changes above. * configure.ac: Do not require Python dependencies itertools, unittest and StringIO anymore as they are not used anymore. Require new module tempfile now. Generate new executable script tests/mockfedabipkgdiff from tests/mockfedabipkgdiff.in. * doc/manuals/abipkgdiff.rst: Add doc for new option --show-identical-binaries to abipkgdiff * doc/manuals/fedabipkgdiff.rst: Add doc for new options --show-identical-binaries to fedabipkgdiff. * tools/abipkgdiff.cc (options::show_identical_binaries): New data member. (options::options): Initialize new data member. (display_usage): Add a new help string for the new --show-identical-binaries option. (parse_command_line): Parse the newq --show-identical-binaries command line switch. (pthread_routine_compare): When the comparison of two binaries is empty, if --show-identical-binaries was provided, then emit some output saying the comparison did yield the empty set. * tools/fedabipkgdiff (DEFAULT_ABIPKGDIFF): Store the default path to abipkgdiff in this new global variable. Naming this default path is useful because it can then be cleanly overloaded when using mock.patch. (build_path_to_abipkgdiff): Return the new DEFAULT_ABIPKGDIFF global variable. (cmd): Parse the new --show-identical-binaries command line switch. * tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt: New reference output. * tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt: Likewise. * tests/mockfedabipkgdiff.in: New uninstalled script template. * tests/runtestfedabipkgdiff.py.in (counter) (temp_file_or_dir_prefix, UtilsTest, RPMTest, LocalRPMTest) (RunAbipkgdiffTest, GetPackageLatestBuildTest, DownloadRPMTest) (BrewListRPMsTest, AssertionHelper, MockGlobalConfig) (BUILT_ABIPKGDIFF, CompareABIFromCommandLineTest): Remove these classes, global variables and functions. (FEDABIPKGDIFF, TEST_SRC_DIR, TEST_BUILD_DIR, INPUT_DIR) (OUTPUT_DIR, FEDABIPKGDIFF_TEST_SPECS): New global variables. (ensure_output_dir_created, run_fedabipkgdiff_tests, main): New functions. * tests/test-diff-pkg.cc (in_out_specs): Add tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt to the set of reference outputs to consider. * tests/Makefile.am: Add non-installed script mockfedabipkgdiff to source distribution. Also added tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt, tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt, tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt, tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt and tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt to source distribution. Signed-off-by: Chenxiong Qi <cqi@redhat.com> Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2016-07-06 09:51:13 +00:00
def do_main():
run_fedabipkgdiff()
Make fedabipkgdiff consistent with Libabigail's other tests In the libabigail project, almost all the tests for the tools follow a similar pattern. The test contains a set of input files to compare. The result of the comparison is thus compared to a set of reference comparison result. This approach is already documented in the CONTRIBUTING file. There are several interesting properties with this scheme. First, it capture the behaviour of the tools, including what is shown to the user. Second, it eases the job of a hacker who wants to add a new test for a new behaviour of a given tool. The user just has to provide: 1/ A new reference output of the new use case of the tool (that is easily constructed by using the tool itself and saving its output) and add an entry to array of entries that describe what to compare 2/ A new set of inputs to the tool And voila. Unfortunately, fedabipkgdiff tests don't follow this scheme. That make them surprising to hackers who read the source code of the existing tests, at very least. Also, the fedabipkgdiff tests were only unit tests. They were not testing the tool as used by users in general. This patch makes the fedabipkgdiff tests follow the general approach of the tests of the other Libabigail tools. The patch first craetes a program names tests/mockfedabipkgdiff. It's a wrapper around tools/fedabipkgdiff. It overloads the Koji client of fedabipkgdiff with a fake Koji client that gets the Fedora packages locally, from tests/data/test-fedabipkgdiff/packages. In other words, mockfedabipkgdiff is fedabipkgdiff without going to the network. I believe that in the future, tests/mockfedabipkgdiff should be killed when fedabipkgdiff is able to cache a local description of a local partial view of a Koji repository, along with the build packages that were already downloaded. That way, calling fedabipkgdiff twice with the same set of option should make the tool perform correctly without going to the netword on the second invocation. We should be able to save the local partial view of the Koji repository under e.g, tests/data/test-fedabipkgdiff/local-koji and tell fedabipkgdiff to use that, instead of using the network. But we are not there yet. So for now, I am using mockfedabipkgdiff. Then, tests/runtestfedabipkdiff.py.in has been re-written to use tests/mockfedabipkgdiff and to look like what is described in CONTRIBUTING as far as how Libabigail tools' tests are organized: mockfedabipkgdiff is called to perform a comparison. The result of the comparison is then compared (using GNU diff) to a reference comparison result file. Please note that tests/runtestfedabipkdiff.py is relatively fast for the moment. But when it contains much more tests and start becoming slow, then we'll need to change the code to run several comparisons in parallel, just like we do today in tests/test-diff-filter.cc. At that point, I believe I'll just re-write this in C++, just like tests/test-diff-filter.cc because that will allow us to have true concurrent code based on the abigail:workers API. For now, I am keeping this in Python also because I think that keeps Chenxiong happy ;-) To be sure that fedabipkgdiff (and its underlying abipkgdiff) are really comparing all the packages they are supposed to compare and also all the binaries in those packages, I have added a new --show-identical-binaries to abipkgdiff and fedabipkgdiff. That option shows the name of the binaries (and packages) that are compared, *even if* the ABI of those binaries are identical. Because otherwise, today, if two binaries (or packages) have the same ABI, nothing is displayed. For instance, here is the result of comparing a package against itself, using this new --show-identical-binaries options: dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ./build/tools/abipkgdiff --show-identical-binaries dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm: ================ changes of 'dbus-binding-tool'=============== No ABI change detected ================ end of changes of 'dbus-binding-tool'=============== ================ changes of 'libdbus-glib-1.so.2.1.0'=============== No ABI change detected ================ end of changes of 'libdbus-glib-1.so.2.1.0'=============== dodji@adjoa:patches$ And here is what this command before that patch would do: dodji@adjoa:patches$ ./tools/fedabipkgdiff --abipkgdiff ../master/build/tools/abipkgdiff dbus-glib-0.80-3.fc12.x86_64.rpm dbus-glib-0.80-3.fc12.x86_64.rpm Comparing the ABI of binaries between dbus-glib-0.80-3.fc12.x86_64.rpm and dbus-glib-0.80-3.fc12.x86_64.rpm: dodji@adjoa:patches$ The rest of the patch is mostly new test inputs material and the necessary adjustments following all the changes above. * configure.ac: Do not require Python dependencies itertools, unittest and StringIO anymore as they are not used anymore. Require new module tempfile now. Generate new executable script tests/mockfedabipkgdiff from tests/mockfedabipkgdiff.in. * doc/manuals/abipkgdiff.rst: Add doc for new option --show-identical-binaries to abipkgdiff * doc/manuals/fedabipkgdiff.rst: Add doc for new options --show-identical-binaries to fedabipkgdiff. * tools/abipkgdiff.cc (options::show_identical_binaries): New data member. (options::options): Initialize new data member. (display_usage): Add a new help string for the new --show-identical-binaries option. (parse_command_line): Parse the newq --show-identical-binaries command line switch. (pthread_routine_compare): When the comparison of two binaries is empty, if --show-identical-binaries was provided, then emit some output saying the comparison did yield the empty set. * tools/fedabipkgdiff (DEFAULT_ABIPKGDIFF): Store the default path to abipkgdiff in this new global variable. Naming this default path is useful because it can then be cleanly overloaded when using mock.patch. (build_path_to_abipkgdiff): Return the new DEFAULT_ABIPKGDIFF global variable. (cmd): Parse the new --show-identical-binaries command line switch. * tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt: New reference output. * tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt: Likewise. * tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt: Likewise. * tests/mockfedabipkgdiff.in: New uninstalled script template. * tests/runtestfedabipkgdiff.py.in (counter) (temp_file_or_dir_prefix, UtilsTest, RPMTest, LocalRPMTest) (RunAbipkgdiffTest, GetPackageLatestBuildTest, DownloadRPMTest) (BrewListRPMsTest, AssertionHelper, MockGlobalConfig) (BUILT_ABIPKGDIFF, CompareABIFromCommandLineTest): Remove these classes, global variables and functions. (FEDABIPKGDIFF, TEST_SRC_DIR, TEST_BUILD_DIR, INPUT_DIR) (OUTPUT_DIR, FEDABIPKGDIFF_TEST_SPECS): New global variables. (ensure_output_dir_created, run_fedabipkgdiff_tests, main): New functions. * tests/test-diff-pkg.cc (in_out_specs): Add tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt to the set of reference outputs to consider. * tests/Makefile.am: Add non-installed script mockfedabipkgdiff to source distribution. Also added tests/data/test-diff-pkg/test-dbus-glib-0.80-3.fc12.x86_64-report-0.txt, tests/data/test-fedabipkgdiff/test0-from-fc20-to-fc23-dbus-glib-report-0.txt, tests/data/test-fedabipkgdiff/test1-from-fc20-to-dbus-glib-0.106-1.fc23.x86_64-report-0.txt, tests/data/test-fedabipkgdiff/test2-dbus-glib-0.100.2-2.fc20--dbus-glib-0.106-1.fc23-report-0.txt and tests/data/test-fedabipkgdiff/test3-dbus-glib-0.100.2-2.fc20.i686--dbus-glib-0.106-1.fc23.i686-report-0.txt to source distribution. Signed-off-by: Chenxiong Qi <cqi@redhat.com> Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2016-07-06 09:51:13 +00:00
if __name__ == '__main__':
do_main()